Markov Chain Steady State Calculator

What Is a Markov Chain? A Markov chain is a stochastic process where the next state depends only on the current state (not history). Named after Andrei Markov (Russia, 1906). Described by a transition matrix P where P[i][j] = probability of moving from state i to state j. Each row of P must sum to 1 (stochastic matrix).

Steady-State Distribution The steady-state (stationary) distribution π satisfies: π × P = π (π is unchanged after one step) Sum of all π_i = 1 This represents the long-run proportion of time spent in each state.

For a 2-State Chain Given: P = [[p, 1-p], [q, 1-q]] Steady state: π₁ = q / (q + (1-p)); π₂ = (1-p) / (q + (1-p)) Or: π₁ = q / ((1-p) + q), π₂ = (1-p) / ((1-p) + q)

Solving for 3-State Chains Set up the system: πP = π with constraint Σπ = 1. Replace one equation with Σπ_i = 1 and solve the linear system.

Convergence, and the two chains where it fails An ergodic chain is one that is both irreducible (you can get from any state to any other) and aperiodic (it does not march in a fixed cycle). Only ergodic chains converge, and both halves of that condition matter.

Take P = [[0, 1], [1, 0]], a chain that flips state every single step. Solve for the stationary distribution and you get a perfectly respectable answer, 50% and 50%. But start it in state 1 and it goes 1, 2, 1, 2 forever. It never settles anywhere near 50/50. The stationary distribution exists and describes the long-run average, but the chain does not converge to it. That is periodicity, and the calculator now flags it instead of drawing a graph that claims otherwise.

The other failure is an absorbing state, a row like [1, 0] where the chain enters and never leaves. Two absorbing states means there is no single answer: whichever one you start in is where you stay, so the long-run distribution depends on the starting point and the usual formula divides by zero.

Convergence speed, for chains that do converge, comes from the second-largest eigenvalue. The closer it sits to zero, the faster the chain forgets where it started.

Mean recurrence time Once you have the stationary distribution, 1/πᵢ is the average number of steps between successive visits to state i. If a machine is broken 4% of the time, it breaks roughly once every 25 steps.

Real-World Examples Web page rank (Google PageRank uses Markov chains). Customer churn: loyal/at-risk/churned states. Weather: sunny/cloudy/rainy transitions. Gene regulatory networks, queuing theory, inventory models.