An insurance company offers annual motor-car insurance based on a “no claims discount” system with levels of discount 0%, 30% and 60%. A policyholder who makes no claims during the year either moves to the next higher level of discount or remains at the top level. If there is exactly one claim during the year, the policyholder either moves down one level or stays at the bottom level (0%). If there is more than one claim during the year, the policyholder either moves down to or stays at the bottom level. For a particular policyholder, it may be assumed that claims arise in a Poisson process at rate λ > 0. Explain why the situation described above is suitable for modelling in terms of a Markov chain with three states, and write down the transition probability matrix in terms of λ.

This situation can be modeled as a 3-state Markov chain, where the states represent the 3 discount levels:

0% discount = State 1

30% discount = State 2  

60% discount = State 3

The transitions between states depend on the number of claims in a year. Let's define the transition probability matrix as:

P = [p(i,j)]

where p(i,j) is the probability of moving from state i to state j.

We can write the transition probabilities as:

p(1,1) = e^-λ  //probability of 0 claims, so stay in state 1

p(1,2) = 1 - e^-λ //probability of 1 claim, so move from state 1 to 2

p(2,1) = e^-λ //probability of 0 claims, so move from state 2 to 1

p(2,2) = 1 - e^-λ //probability of 1 claim, so stay in state 2

p(2,3) = 0 //cannot move from state 2 to 3 in 1 transition

p(3,2) = e^-λ //probability of 0 claims, so move from state 3 to 2

p(3,3) = 1 - e^-λ //probability of 1 claim, so stay in state 3

Therefore, the transition probability matrix is:

P = [[e^-λ, 1 - e^-λ, 0],

    [e^-λ, 1 - e^-λ, 0],

    [0, e^-λ, 1 - e^-λ]]

A person buys stock of a certain company at a price of 3 cents a share. The investor decides to sell her shares if their value decreases to 1 cent or becomes greater that or equal to 5 cents. Let Xn be the value of the shares (for the investor) after n days. We suppose {xn, n = 0, 1, · · · } is a Markov chain having a state space {0, 1, · · · , 7}, and for which rows 3 to 5 (corresponding to states 2,3,4) of the one-step transition probability matrix P are the following: P =   1/8 1/4 1/4 1/8 0 0 0 1/12 1/7 1/4 1/21 1/4 1/7 1/12 0 0 1/12 1/7 1/4 1/21 1/4 1/7 1/12 