STA 4001 Stochastic Processes Fall 2024 Homework 2Python

Java Python STA 4001

Stochastic Processes

Fall 2024

Homework 2

Due Oct 8-th Midnight

1.  A total of m white and m blackballs are distributed among two urns, with each urn containing m balls. At each stage, a ball is randomly selected from each urn and the two selected balls are interchanged.  Let Xn denote the number of blackballs in urn 1 after thenth interchange.

(a)  Give the transition probabilities of the Markov chain Xn , n ≥ 0.

(b)  Find the limiting probabilities and show that the stationary chain is time reversible.

2.  Consider a Markov chain {Xn , n ≥ 0} on the state space E = {1, 2, 3} with transition probability matrix given by

 

Assume that the (initial) distribution of X0 is the uniform. distribution on E. (a)  Calculate P(X3  = 2, X1  = 3).

(b)  Calculate P(X4  = 1