MATH2012/6128 Stochastic Processes Coursework 2022-23
This coursework is 20% of the assessment for the module.
Please upload your answer, as one single pdf file, by the deadline
UK time 23:59pm Friday 24/2/2023, in math2012 Blackboard.
Please submit no more than 6 A4-size pages; your work beyond
the 6-page limit will not be marked.
Please write your student id on the top-right of each page.
You are reminded the university policy on academic integrity.
A possible simple illness-death model can be represented by a four-state Markov chain in which state “1”
is a state in which an individual is free of a particular disease, “2” is a state in which the individual has
the disease, and “3” and “4” are respectively death states arising as a result of the disease, or from other
causes. Assume that the one-step transition probabilities between the four states in an annual cycle are
given by the following transition probability matrix P
P =
0
BBB@
1 2 34
1 1/2, 1/4, 0, 1/4
2 1/4, 1/2, 1/8, 1/8
3 0, 0, 1, 0
4 0, 0, 0, 1
1
CCCA .
(a) [3 marks] Draw the transition diagram of this Markov chain. Discuss briefly in which states this
Markov chain will end up eventually.
(b) [5 marks] Consider the initial distribution p(0) = (1, 0, 0, 0). Write down the R code to specify the
initial distribution at step 0, p0, as given by p(0) above, and the 1-step transition probability matrix,
trans, as given by P above.
Continue from your R code in Part (b) with
transita <- function(char,trans) {
sample(colnames(trans),1,prob=trans[char,])
}
transita(’1’,trans)
(c) [5 marks] Explain what the result of the code transita(’1’,trans) gives.
Continue from the R code above with
simula <- function(p0,trans,N) {
sim <- character(N+1)
sim[1] <- sample(colnames(trans),1, prob=p0)
for (i in 2:(N+1)) {
sim[i] <- transita(sim[i-1],trans)
1
}
sim
}
(d) (1) [5 marks] Give the value of MCstates from running the code below, and explain what is given
by MCstates
set.seed(200)
N <- 4
MCstates <- simula(p0,trans,N)
(2) [5 marks] Explain what the code set.seed(200) above is for.
Continue from the R code above with
simulaDist <- function(p0,trans,N, Repl) {
simState <- character(Repl)
for (i in 1:Repl) {
simState[i] <- simula(p0,trans,N)[N+1]
}
Dist <- as.matrix(table(simState))/Repl
Dist
}
(e) (1) [1 marks] Give the value of Distribution from running the code
set.seed(200)
N <- 4
Repl <- 100000
Distribution <- simulaDist(p0,trans,N, Repl)
(2) [4 marks] Explain what is given by Distribution above.
Continue from the R code above with
N1 <- 38
ExactDist1 <- matrix(p0, nrow=1)
for (i in 1:N1) {
ExactDist1 <- ExactDist1 %*% trans
}
ExactDist1
N2 <- 39
ExactDist2 <- matrix(p0, nrow=1)
for (i in 1:N2) {
ExactDist2 <- ExactDist2 %*% trans
}
ExactDist2
(f) (1) [4 marks] Explain what the value of ExactDist1 above gives.
(2) [4 marks] What conclusion you can draw by comparing the values of ExactDist1 and ExactDist2
above.
2
(3) [4 marks] What are the long-term probabilities that the Markov chain will be in the four states
if the Markov chains starts from state “1”
(g) [5 marks] Let pi denote the probability that the Markov chain will be absorbed by state “4” if the
Markov chain starts from state “i”, i = 1, 2, 3, 4. Explain carefully why p1 = 1
2 p1 + 4
1 p2 + 1
4 . By
writing down a similar equation for p2, or otherwise, find p1.
(h) [5 marks] Let qi denote the probability that the Markov chain will be absorbed by state “3” if the
Markov chain starts from state “i”, i = 1, 2, 3, 4. Find q2.
3
