$\begingroup$

What's the difference between stationary and invariant distribution of Markov chain?

Since if the stationary distribution $\pi$ is defined as

$$\pi=\pi P$$

for transition matrix $P$. Then by definition $\pi$ is invariant. But what's the difference then?

$\endgroup$ 1

1 Answer

$\begingroup$

Usually, these are just terms used by different people; some will call a vector $\pi$ with $\pi P = \pi$ and $\sum_i \pi_i = 1$ a stationary distribution, others will call it an invariant distribution.

However, there are some closely related concepts that are different:

• An invariant measure (or maybe stationary measure) is sometimes a vector $\pi$ that satisfies $\pi P = \pi$, but not necessarily $\sum_i \pi_i = 1$. (This makes a difference for infinite Markov chains, where we can't necessarily divide by $\sum_i \pi_i$ to normalize.) But some sources like Wikipedia use this synonymously with a stationary distribution.
• A limiting distribution is a stationary distribution $\pi$ with the property that for any distribution $\rho$, $\lim_{n \to \infty} \rho P^n = \pi$: after taking lots of steps starting at $\rho$, we converge to the distribution $\pi$. These do not necessarily exist for all Markov chains, or are dependent on $\rho$.
• A time-average distribution $\pi$ is defined by letting $\pi_i$ be the average fraction of time spent in state $i$ over $n$ steps, in the limit as $n \to \infty$. This is sometimes dependent on the initial state. It is equal to the limiting distribution if that exists, but the time-average distribution exists in slightly more general cases.

$\endgroup$ 4