ipynb/daisuke_oyama-markov_chain_ex01_py.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "# Finite Markov Chains: Examples"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "**Daisuke Oyama**\n",
                "\n",
                "*Faculty of Economics, University of Tokyo*"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "This notebook demonstrates how to analyze finite-state Markov chains\n",
                "with the `MarkovChain` class.\n",
                "For basic concepts and properties on Markov chains, see\n",
                "\n",
                "* [the lecture on finite Markov chains](https://lectures.quantecon.org/py/finite_markov.html)\n",
                "  in Quantitative Economics, and\n",
                "* [the documentation for `MarkovChain`](http://quanteconpy.readthedocs.org/en/stable/markov/core.html).\n",
                "\n",
                "For algorithmic issues in detecting reducibility and periodicity of a Markov chain,\n",
                "see, for example,\n",
                "\n",
                "* J. P. Jarvis and D. R. Shier,\n",
                "  \"[Graph-Theoretic Analysis of Finite Markov Chains](http://www.ces.clemson.edu/~shierd/Shier/markov.pdf),\"\n",
                "\n",
                "from which we draw some examples below."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [],
            "source": [
                "%matplotlib inline"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "metadata": {},
            "outputs": [],
            "source": [
                "import numpy as np\n",
                "import matplotlib.pyplot as plt\n",
                "from quantecon.markov import MarkovChain"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "collapsed": true
            },
            "source": [
                "## Example 1: Reducible chain"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Consider the Markov chain given by the following stochastic matrix,\n",
                "taken from Exercise 3 in Jarvis and Shier\n",
                "(where the actual values of non-zero probabilities are not important):"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {},
            "outputs": [],
            "source": [
                "P = np.zeros((6, 6))\n",
                "P[0, 0] = 1\n",
                "P[1, 4] = 1\n",
                "P[2, [2, 3, 4]] = 1/3\n",
                "P[3, [0, 5]] = 1/2\n",
                "P[4, [1, 4]] = 1/2\n",
                "P[5, [0, 3]] = 1/2"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.         0.         0.         1.         0.        ]\n",
                        " [0.         0.         0.33333333 0.33333333 0.33333333 0.        ]\n",
                        " [0.5        0.         0.         0.         0.         0.5       ]\n",
                        " [0.         0.5        0.         0.         0.5        0.        ]\n",
                        " [0.5        0.         0.         0.5        0.         0.        ]]\n"
                    ]
                }
            ],
            "source": [
                "print(P)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Create a MarkovChain instance:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {},
            "outputs": [],
            "source": [
                "mc1 = MarkovChain(P)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Classification of states"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "This Markov chain is reducible:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "False"
                        ]
                    },
                    "execution_count": 6,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.is_irreducible"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "4"
                        ]
                    },
                    "execution_count": 7,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.num_communication_classes"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Determine the communication classes:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "[array([0]), array([1, 4]), array([3, 5]), array([2])]"
                        ]
                    },
                    "execution_count": 8,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.communication_classes"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Classify the states of this Markov chain:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "[array([0]), array([1, 4])]"
                        ]
                    },
                    "execution_count": 9,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.recurrent_classes"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Obtain a list of the recurrent states:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[0 1 4]\n"
                    ]
                }
            ],
            "source": [
                "recurrent_states = np.concatenate(mc1.recurrent_classes)\n",
                "print(recurrent_states)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Obtain a list of the transient states:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[2 3 5]\n"
                    ]
                }
            ],
            "source": [
                "transient_states = np.setdiff1d(np.arange(mc1.n), recurrent_states)\n",
                "print(transient_states)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "A Markov chain is reducible (i.e., its directed graph is not strongly connected)\n",
                "if and only if by symmetric permulations of rows and columns,\n",
                "its transition probability matrix is written in the form (\"canonical form\")\n",
                "$$\n",
                "\\begin{pmatrix}\n",
                "U & 0 \\\\\n",
                "W & V\n",
                "\\end{pmatrix},\n",
                "$$\n",
                "where $U$ and $W$ are square matrices.\n",
                "\n",
                "Such a form for `mc1` is obtained by the following:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {
                "scrolled": false
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.         1.         0.         0.         0.        ]\n",
                        " [0.         0.5        0.5        0.         0.         0.        ]\n",
                        " [0.         0.         0.33333333 0.33333333 0.33333333 0.        ]\n",
                        " [0.5        0.         0.         0.         0.         0.5       ]\n",
                        " [0.5        0.         0.         0.         0.5        0.        ]]\n"
                    ]
                }
            ],
            "source": [
                "permutation = np.concatenate([recurrent_states, transient_states])\n",
                "print(mc1.P[np.ix_(permutation, permutation)])"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "This Markov chain is aperiodic\n",
                "(i.e., the least common multiple of the periods of the recurrent sub-chains is one):"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "True"
                        ]
                    },
                    "execution_count": 13,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.is_aperiodic"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Indeed, each of the sub-chains corresponding to the recurrent classes has period $1$,\n",
                "i.e., every recurrent state is aperiodic:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Period of the sub-chain\n",
                        "[[1.]]\n",
                        " = 1\n",
                        "Period of the sub-chain\n",
                        "[[0.  1. ]\n",
                        " [0.5 0.5]]\n",
                        " = 1\n"
                    ]
                }
            ],
            "source": [
                "for recurrent_class in mc1.recurrent_classes:\n",
                "    sub_matrix = P[np.ix_(recurrent_class, recurrent_class)]\n",
                "    d = MarkovChain(sub_matrix).period\n",
                "    print('Period of the sub-chain\\n{0}\\n = {1}'.format(sub_matrix, d))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Stationary distributions"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "For each recurrent class $C$, there is a unique stationary distribution $\\psi^C$\n",
                "such that $\\psi^C_i > 0$ for all $i \\in C$ and $\\psi^C_i = 0$ otherwise.\n",
                "`MarkovChain.stationary_distributions` returns\n",
                "these unique stationary distributions for the recurrent classes.\n",
                "Any stationary distribution is written as a convex combination of these distributions."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33333333 0.         0.         0.66666667 0.        ]]\n"
                    ]
                }
            ],
            "source": [
                "print(mc1.stationary_distributions)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "These are indeed stationary distributions:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33333333 0.         0.         0.66666667 0.        ]]\n"
                    ]
                }
            ],
            "source": [
                "print(mc1.stationary_distributions.dot(mc1.P))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Plot these distributions."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {},
            "outputs": [],
            "source": [
                "def draw_histogram(distribution, ax=None, figsize=None,\n",
                "                   title=None, xlabel=None, ylabel=None, ylim=(0, 1)):\n",
                "    \"\"\"\n",
                "    Plot the given distribution.\n",
                "    \n",
                "    \"\"\"\n",
                "    if ax is None:\n",
                "        fig, ax = plt.subplots(figsize=figsize)\n",
                "    n = len(distribution)\n",
                "    ax.bar(np.arange(n), distribution, align='center')\n",
                "    ax.set_xlim(-0.5, (n-1)+0.5)\n",
                "    ax.set_ylim(*ylim)\n",
                "    ax.set_xticks(range(n))\n",
                "    if title:\n",
                "        ax.set_title(title)\n",
                "    if xlabel:\n",
                "        ax.set_xlabel(xlabel)\n",
                "    if ylabel:\n",
                "        ax.set_ylabel(ylabel)\n",
                "    if ax is None:\n",
                "        plt.show()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAE3CAYAAACpToeUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xu4XGV59/Hvz0SInAXiARIIraDGE9oIKrbFSi0nQV+8LKgo9YC2UmvFKq0WgValvq1trVRLPaCgIFrRVOMLVgEFQQmIYEA00mBCBALI+Qz3+8daGybD7L1nTybZOzvfz3Xta+9Z65ln3c+amXvf86w1a1JVSJIkSRu6x0x2AJIkSdJUYGEsSZIkYWEsSZIkARbGkiRJEmBhLEmSJAEWxpIkSRKwARbGSV6b5KzJjmOikhyW5LyO23ck+a0h9f03ST7V/j0vSSWZOaS+d2hjnTGM/rr63iPJL9r+XzHs/tttLEuy19roe0PV/VxeB9urJHcm+WCf7Y9t2w/tdaD+mJ979m1+Hn0b5uchm+r5eYJ9b9w+/+5P8vf93m/KFcZJXpzkB0luTXJzkvOTPL9dN6EHrFcSqaovVNXL1kbs61JVbVZVV4/VJsmeSVb00deHqurNw4irO1FV1a/aWB8cRv9djgM+3vb/tTXtLMlJE3nxbMiSHJPklMmOYwKeU1XvG7mRZNckFye5q/2968i6qvoA8IxJiXKKMz/3x/wMmJ8nzTTIzycmuSrJQ0kO66eDJBsl+Vnna6qq7q2qzYAvTCSYKVUYJ9kC+Abwb8DWwPbAscC9kxnXMKyNd+TDsJ7PiO0ILBnkjpM97nW5/V7bmuzxT6YkGwFfB04BHg98Dvh6u1yjMD+ve+v569T8POC2Jnv8U8BPgD8DLpnAff4KuGEoW6+qKfMDLABuGWXd04F7gAeBO0baAfsBPwZuA5YDx3Tc51dAte3vAF4IHAac19HmRcBFwK3t7xd1rDsH+DvgfOB24Cxg2471Xwaua+/7PeAZHetOAj4BLALubB+064GZHW0OAi4dZbzbAAvbcf2ojaMz7gKe0v69L3BFG+O1wLuBTYG7gYc6xr8dcAzwFZqi4Dbgze2yU9q+5rV9Hw6sBH4NHNk1rr/vuL0nsKL9++R2e3e323tPR38z2zbbteO6GVgKvKWjr2OA04HPt2NZAiwYZf/8smtbG/fR92rj7urvcOB+4L62v/9uly9r9+dl7eP8JWBWx/32By4FbgF+ADx7jOd3AW8HfgH8b7vsacC325ivAl7d0f5xwD8B17TbPq9d9vA+72i7DNhrtLGOsuwxwFHtvryp3fdbdz0P3kDzOroReF+7bu92P93f7qufjDLeucBXgVVt/x9vlx/G6s/lf6V57d4GXAz8bse63YDF7brrgY+2y2e1Y7mp3fcXAU8cY78/peP2y2heJ+nKFXt33B4Z/8xefW6IP5ifzc/mZ/PzWs7PXevOAw7rIzftBFwJ7NO973u9Lsbtb20k0EF/gC3aHfm5doCP71q/2gPW8cJ/Vvskenb74Lyi68kzs1cfNLMevwEOBWYCh7S3t2nXn9M+KXdpn/DnAMd39PVGYHOaF/2/0JFE2wfiVmCPNrZZNMlxn442Z9CR1LrGdVr7QtgUeCZNQh0t8f565MlKMwP2vI590/0CPYbmBfOKNq7H0Tvxntpu+1k0L5y9OsbVM/F2J4BejwFwLvDv7f7Yte37pR2x3UPzj2QG8GHgwjGeL93bGq/v1cbdz4un3caPaJL61jQvvre1655H8w519zbeN7TtNx4jAXy77edx7f5dDvwJzfPveTQJ7hlt+xNonnPbt/2/iOa51utxfXhfjPEYdy97J3AhMKft9z+AU7set/9s2z6HZmbw6R3bOGWMx2YGzbv+f27HOQt4ca/XMfA6mkJjJnAkTTEzq113AXBo+/dmwAvav98K/DewSbut3wG2GGO/dxbGfwl8q6vNN1i9wBgZv4XxI/vE/PzIOvOz+dn8vBbyc9e6fgvjbwCv7LXvR3vujPUzpU6lqKrbgBfzyAO+KsnCJE8c4z7nVNXlVfVQVV1GkzB+v89N7gf8oqpOrqoHqupU4GfAyzvafLaqfl5Vd9Mkws5zET9TVbdX1b00T8TnJNmy475fr6rz29juofmH8jqAJFsDfwR8sTuo9rDeQcDRVXVnVf20ve9o7gfmJ9miqn5TVeMdfrigqr7WxnX3KG2Obbd9OfBZmn9KayTJXJrH971VdU9VXQp8iuYf34jzqmpRNee8nUzzgh9W3/2Mu5ePVdXKqrqZ5sU+8hx4C/AfVfXDqnqwqj5Hk5xeMEZfH66qm9vt7w8sq6rPts+/S4D/Al6V5DE0/9j/oqqubfv/Qftc60evsXYveyvNLMOKjufwq7oO4x1bVXdX1U9oEmlfjwfNTMJ2wF+1z6N7qqrn+adVdUpV3dTug3+i+Sfw1Hb1/cBTkmxbVXdU1YUdy7ehSagPVtXFbf7ox2Y0RVGnW2mKKI3C/NwwP5ufzc9rNT9PSJJX0ryxO2NYfU6pwhigqq6sqsOqag7NO/HtaN7t95Rk9yRnJ1mV5FbgbcC2fW5uO5rDIJ2uoXkHOOK6jr/vovmnSpIZSY5P8sskt9G8I6Rr28u7+j4FeHmSzYBXA9+vql/3iGs2zbuzzvt3x9npIJp38dckOTfJC8do2yuu8dpcQ7Ov1tR2wM1VdXtX32Pt71l9nm/VT9/9jLuXns8BmnPojkxyy8gPzeGpsfZVZww7Art33f+1wJNonkezaGbEBtFrrN3LdgTO6Nj2lTSHwjsLndHGPp65wDVV9cB4DZMcmeTK9gNdtwBb8sjr6E00M4I/S3JRkv3b5ScDZwKnJVmZ5CNJHttnbHfQzH522oLm8LDGYH4GzM8jzM/m57WRn/uWZFPgI8CfD7PfKVcYd6qqn9FMgT9zZFGPZl+kOW9pblVtCXwSyBjtO62kefJ12oHmsNh4XgMcCOxF80SZ1y5PR5vVtl9V19IcenglzTvlk0fpexXwAM2TtzOunqrqoqo6EHgC8DWamZNHbX+0uEbRve2V7d930hweGfGkCfS9Etg6SefMXL/7ezz99D3euPvZL52WAx+sqq06fjZpZ7b62cZy4Nyu+29WVX9Kc8juHuC3e/Sx2mPQzmDN7mMs3cuW0xw67tz+rPZ5Op7x9tVyYIfx/mkm+V3gvTSFyOOraiua2dsAVNUvquoQmuf2PwBfSbJpVd1fVcdW1XyaQ5j7A6/vI25ozo18dpLO1+qzGfCDQhsq87P5eQLMz+OPxfw8cTvTvLa/n+Q6mnOmn5zkuiTzBu10ShXGSZ7WvjuZ096eS3OIaGR6/npgTtenxzeneSd6T5LdaBLiiFU0HwAY7XqSi4BdkrwmycwkfwzMpzlfZTyb0xyWuYnmRfChvgbZfHDhPTTnhvWc+m8PU30VOCbJJknm05wf9ShpLlHy2iRbVtX9NCfBj1x653pgm67Dh/3623bbz6A5x+pL7fJLgX2TbJ3kSTTnQXW6nlH2d1Utp/kAxIeTzErybJp3nBO6lMpa7HvU2Efxn8Db2lmxJNk0yX5dyX8s36B5/h2a5LHtz/OTPL2qHgI+A3w0yXbtDNgLk2wM/Jxmpma/9l34+2kOb03UJ4EPJtkRIMnsJAf2ed/rgXlpDin28iOacyuPb/fLrCR79Gi3OU2RsQqYmeRoOmZzk7wuyex2f9zSLn4wyUuSPKv9p3MbzaG7fi85dU7b9h1prnN5RLv8u33ef4Nkfm6YnyfO/Gx+pv/8PPK6mUVTgD+2ja/XWH5K8yZx1/bnze3Yd2XwIxBTqzCmOZS5O/DDJHfSJNyf0pzwDc0/riXAdUlubJf9GXBcktuBo3nk3ThVdRfwQeD8NIcjVju3qKpuonkncyRNAn0PsH9V3cj4Pk9zKOhamg9tXDh284edQXuIpKruHKPdETSHRa6jmZX57BhtDwWWpTlk+Dba8+TaGZ1Tgavb8U/kcNu5NJ8c/g7wj1U1ctH9k2nOZVpG8ynwL3Xd78PA+9vtvbtHv4fQvMNbSbMvPlBV355AXGNZ074/TXMu4C1Jxr3uZlUtpjmP7eM0HwpaSvPBhb60hxVfBhzcxnwdzbvukST6buBymk/03tyue0xV3UrzvP8UzfPvTmDc66H28K80s3lnta+fC2lef/34cvv7piSPOmeyLR5eDjyF5lPTK4A/7tHPmcC3aP6ZXEMzC9OZ0PYGliS5o4334GrOB30Szae4b6M5xHguzaHwcVXVfTQfcnk9TTJ/I80Hwu7r5/4bMPPzI8zPE2d+npgNMj+3zqK5osmLgBPbv3+vxzgeqKrrRn5oHoeH2tsDX5s7VRM9OqE1leSXwFur6n8mOxZpuktyD83s4ceq6m/7aP8B4F00/wA3XZMEq/WP+VladyaanyfY98Y0M8iPBT5SVcf2dT8L43UryUE07yx3aQ8/SJKmAPOzpHFPpUjymSQ3JPnpKOuT5GNJlia5LMnzhh/m9JDkHJqLyr/dpCtpbTFvT5z5WRL0d47xSTTnkYxmH5pPBu5M8+00n1jzsKanqtqzqp5QVWdOdiySprWTMG9PiPlZEvRRGFfV92hOaB7NgcDnq3EhsFWSJw8rQEnSxJi3JWkww7gqxfas/inFFax+0W5J0tRi3pakHvr5xprxpMeynp/oS3I4zWE7Nt1009952tOettr6y6/t/pbWqedZ2w9yyUlJ08nFF198Y1V1X7R/fdJX3h4vZ0vS+qLfvD2MwngFq38Lzxwe+Rae1VTViTTXpGPBggW1ePHi1dbPO+qbQwhn7Vp8/H6THYKkSZZkrK8AXh/0lbfHy9mStL7oN28P41SKhcDr2085vwC4tXp/v7wkaWowb0tSD+POGCc5FdgT2DbJCuADNBdLpqo+SfO1nfvSfKvMXTRfTylJmiTmbUkazLiFcVUdMs76At4+tIgkSWvEvC1JgxnGqRSSJEnSes/CWJIkScLCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkC+iyMk+yd5KokS5Mc1WP9DknOTvLjJJcl2Xf4oUqS+mHOlqTBjFsYJ5kBnADsA8wHDkkyv6vZ+4HTq+q5wMHAvw87UEnS+MzZkjS4fmaMdwOWVtXVVXUfcBpwYFebArZo/94SWDm8ECVJE2DOlqQB9VMYbw8s77i9ol3W6RjgdUlWAIuAP+/VUZLDkyxOsnjVqlUDhCtJGoc5W5IG1E9hnB7Lquv2IcBJVTUH2Bc4Ocmj+q6qE6tqQVUtmD179sSjlSSNx5wtSQPqpzBeAcztuD2HRx92exNwOkBVXQDMArYdRoCSpAkxZ0vSgPopjC8Cdk6yU5KNaD6osbCrza+AlwIkeTpNkvW4mySte+ZsSRrQuIVxVT0AHAGcCVxJ80nmJUmOS3JA2+xI4C1JfgKcChxWVd2H7iRJa5k5W5IGN7OfRlW1iOYDGp3Lju74+wpgj+GGJkkahDlbkgbjN99JkiRJWBhLkiRJgIWxJEmSBFgYS5IkSYCFsSRJkgRYGEuSJEmAhbEkSZIEWBhLkiRJgIWxJEmSBFgYS5IkSYCFsSRJkgRYGEuSJEmAhbEkSZIEWBhLkiRJgIWxJEmSBFgYS5IkSYCFsSRJkgRYGEuSJEmAhbEkSZIEWBhLkiRJgIWxJEmSBFgYS5IkSYCFsSRJkgTAzMkOQJIkaTqbd9Q3JzuEcS07fr/JDmFKcMZYkiRJwsJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQL6LIyT7J3kqiRLkxw1SptXJ7kiyZIkXxxumJKkfpmzJWkwM8drkGQGcALwh8AK4KIkC6vqio42OwN/DexRVb9J8oS1FbAkaXTmbEkaXD8zxrsBS6vq6qq6DzgNOLCrzVuAE6rqNwBVdcNww5Qk9cmcLUkD6qcw3h5Y3nF7Rbus0y7ALknOT3Jhkr2HFaAkaULM2ZI0oHFPpQDSY1n16GdnYE9gDvD9JM+sqltW6yg5HDgcYIcddphwsJKkcZmzJWlA/cwYrwDmdtyeA6zs0ebrVXV/Vf0vcBVN0l1NVZ1YVQuqasHs2bMHjVmSNDpztiQNqJ/C+CJg5yQ7JdkIOBhY2NXma8BLAJJsS3OY7uphBipJ6os5W5IGNG5hXFUPAEcAZwJXAqdX1ZIkxyU5oG12JnBTkiuAs4G/qqqb1lbQkqTezNmSNLh+zjGmqhYBi7qWHd3xdwHvan8kSZPInC1Jg/Gb7yRJkiQsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAvosjJPsneSqJEuTHDVGu1clqSQLhheiJGkizNmSNJhxC+MkM4ATgH2A+cAhSeb3aLc58A7gh8MOUpLUH3O2JA2unxnj3YClVXV1Vd0HnAYc2KPd3wEfAe4ZYnySpIkxZ0vSgPopjLcHlnfcXtEue1iS5wJzq+obY3WU5PAki5MsXrVq1YSDlSSNy5wtSQPqpzBOj2X18MrkMcA/A0eO11FVnVhVC6pqwezZs/uPUpLUL3O2JA2on8J4BTC34/YcYGXH7c2BZwLnJFkGvABY6Ic5JGlSmLMlaUD9FMYXATsn2SnJRsDBwMKRlVV1a1VtW1XzqmoecCFwQFUtXisRS5LGYs6WpAGNWxhX1QPAEcCZwJXA6VW1JMlxSQ5Y2wFKkvpnzpakwc3sp1FVLQIWdS07epS2e655WJKkQZmzJWkwfvOdJEmShIWxJEmSBFgYS5IkSYCFsSRJkgRYGEuSJEmAhbEkSZIEWBhLkiRJgIWxJEmSBFgYS5IkSUCf33wnSdJ0Nu+ob052CONadvx+kx2CNO05YyxJkiRhYSxJkiQBFsaSJEkSYGEsSZIkARbGkiRJEmBhLEmSJAEWxpIkSRJgYSxJkiQBFsaSJEkSYGEsSZIkARbGkiRJEmBhLEmSJAEWxpIkSRJgYSxJkiQBFsaSJEkSYGEsSZIkARbGkiRJEmBhLEmSJAEWxpIkSRJgYSxJkiQBFsaSJEkSYGEsSZIkARbGkiRJEmBhLEmSJAF9FsZJ9k5yVZKlSY7qsf5dSa5IclmS7yTZcfihSpL6Yc6WpMGMWxgnmQGcAOwDzAcOSTK/q9mPgQVV9WzgK8BHhh2oJGl85mxJGlw/M8a7AUur6uqqug84DTiws0FVnV1Vd7U3LwTmDDdMSVKfzNmSNKB+CuPtgeUdt1e0y0bzJuBbaxKUJGlg5mxJGtDMPtqkx7Lq2TB5HbAA+P1R1h8OHA6www479BmiJGkCzNmSNKB+ZoxXAHM7bs8BVnY3SrIX8D7ggKq6t1dHVXViVS2oqgWzZ88eJF5J0tjM2ZI0oH4K44uAnZPslGQj4GBgYWeDJM8F/oMmwd4w/DAlSX0yZ0vSgMYtjKvqAeAI4EzgSuD0qlqS5LgkB7TN/i+wGfDlJJcmWThKd5KktcicLUmD6+ccY6pqEbCoa9nRHX/vNeS4JEkDMmdL0mD85jtJkiQJC2NJkiQJsDCWJEmSAAtjSZIkCbAwliRJkgALY0mSJAmwMJYkSZIAC2NJkiQJsDCWJEmSAAtjSZIkCbAwliRJkgALY0mSJAmwMJYkSZIAC2NJkiQJsDCWJEmSAAtjSZIkCbAwliRJkgALY0mSJAmwMJYkSZIAC2NJkiQJsDCWJEmSAAtjSZIkCbAwliRJkgALY0mSJAmwMJYkSZIAC2NJkiQJsDCWJEmSAAtjSZIkCbAwliRJkgALY0mSJAmwMJYkSZIAC2NJkiQJsDCWJEmSAAtjSZIkCbAwliRJkoA+C+Mkeye5KsnSJEf1WL9xki+163+YZN6wA5Uk9cecLUmDGbcwTjIDOAHYB5gPHJJkflezNwG/qaqnAP8M/MOwA5Ukjc+cLUmD62fGeDdgaVVdXVX3AacBB3a1ORD4XPv3V4CXJsnwwpQk9cmcLUkDmtlHm+2B5R23VwC7j9amqh5IciuwDXBjZ6MkhwOHtzfvSHLVIEFPwLbdMaypTN68ytDHMokcy9QzXcYB62YsO67l/tfE+pyzYfo8F/3/MzU5llFsAM+vvvJ2P4Vxr1mEGqANVXUicGIf2xyKJIurasG62t7a5FimpukylukyDpheYxnQepuzYfo8ftNlHOBYpqrpMpapNo5+TqVYAcztuD0HWDlamyQzgS2Bm4cRoCRpQszZkjSgfgrji4Cdk+yUZCPgYGBhV5uFwBvav18FfLeqHjX7IEla68zZkjSgcU+laM8/OwI4E5gBfKaqliQ5DlhcVQuBTwMnJ1lKM+tw8NoMegLW6SHAtcyxTE3TZSzTZRwwvcYyYet5zobp8/hNl3GAY5mqpstYptQ44iSBJEmS5DffSZIkSYCFsSRJkgRM48J4vK9EXV8k+UySG5L8dLJjWRNJ5iY5O8mVSZYk+YvJjmlQSWYl+VGSn7RjOXayY1pTSWYk+XGSb0x2LGsiybIklye5NMniyY5H/TNnTz3TJW+bs6euqZizp+U5xu1Xov4c+EOayxJdBBxSVVdMamADSPJ7wB3A56vqmZMdz6CSPBl4clVdkmRz4GLgFevpYxJg06q6I8ljgfOAv6iqCyc5tIEleRewANiiqvaf7HgGlWQZsKCqpssF/DcI5uypabrkbXP21DUVc/Z0nTHu5ytR1wtV9T2mwfVFq+rXVXVJ+/ftwJU037613qnGHe3Nx7Y/6+07zCRzgP2AT012LNpgmbOnoOmSt83ZmojpWhj3+krU9e7FPF0lmQc8F/jh5EYyuPYw1qXADcC3q2q9HQvwL8B7gIcmO5AhKOCsJBe3X2es9YM5e4pb3/O2OXvKmnI5e7oWxn193anWvSSbAf8FvLOqbpvseAZVVQ9W1a403yq2W5L18pBpkv2BG6rq4smOZUj2qKrnAfsAb28Pa2vqM2dPYdMhb5uzp6wpl7Ona2Hcz1eiah1rz+36L+ALVfXVyY5nGKrqFuAcYO9JDmVQewAHtOd5nQb8QZJTJjekwVXVyvb3DcAZNIfoNfWZs6eo6Za3zdlTy1TM2dO1MO7nK1G1DrUffvg0cGVVfXSy41kTSWYn2ar9+3HAXsDPJjeqwVTVX1fVnKqaR/M6+W5VvW6SwxpIkk3bDwiRZFPgZcB6f2WADYQ5ewqaLnnbnD01TdWcPS0L46p6ABj5StQrgdOrasnkRjWYJKcCFwBPTbIiyZsmO6YB7QEcSvPu9tL2Z9/JDmpATwbOTnIZzT/0b1fVen3JnGniicB5SX4C/Aj4ZlX9v0mOSX0wZ09Z0yVvm7OnpimZs6fl5dokSZKkiZqWM8aSJEnSRFkYS5IkSVgYS5IkSYCFsSRJkgRYGEuSJEmAhbGmuCTvS7IkyWXtpYJ2T/LOJJv0cd++2kmShsOcrfWdl2vTlJXkhcBHgT2r6t4k2wIbAT8AFlTVjePcf1k/7SRJa86crenAGWNNZU8GbqyqewHaZPkqYDuai7WfDZDkE0kWt7MUx7bL3tGj3cuSXJDkkiRfTrJZu/z4JFe0Mxz/uO6HKUnTgjlb6z1njDVltUnwPGAT4H+AL1XVud2zCkm2rqqbk8wAvgO8o6ou62zXzlx8Fdinqu5M8l5gY+DjNN9S9bSqqiRbVdUt63qskrS+M2drOnDGWFNWVd0B/A5wOLAK+FKSw3o0fXWSS4AfA88A5vdo84J2+flJLgXeAOwI3AbcA3wqyf8B7hr2OCRpQ2DO1nQwc7IDkMZSVQ8C5wDnJLmcJjk+LMlOwLuB51fVb5KcBMzq0VWAb1fVIY9akewGvBQ4GDgC+INhjkGSNhTmbK3vnDHWlJXkqUl27li0K3ANcDuwebtsC+BO4NYkTwT26Wjf2e5CYI8kT2n73iTJLu2hvy2rahHwznYbkqQJMmdrOnDGWFPZZsC/JdkKeABYSnOI7hDgW0l+XVUvSfJjYAlwNXB+x/1P7Gp3GHBqko3b9e+nScRfTzKLZobiL9fFwCRpGjJna73nh+8kSZIkPJVCkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQIsjCVJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWNI0kuS1Sc6a7DgmKslhSc7ruH1Hkt8aUt9/k+RT7d/zklSSmUPqe4c21hnD6E+SJpuFsaR1JsmLk/wgya1Jbk5yfpLnt+tWKw776OtRRV5VfaGqXrY2Yl+Xqmqzqrp6rDZJ9kyyoo++PlRVbx5GXEmWJdmro+9ftbE+OIz+JWmyDWXWQJLGk2QL4BvAnwKnAxsBvwvcO5lxDUOSGVOxOEwys6oemOw4JGl94YyxpHVlF4CqOrWqHqyqu6vqrKq6LMnTgU8CL2wPzd8CkGS/JD9OcluS5UmO6ejve+3vW9r7vLDHKQkvSnJRO0N9UZIXdaw7J8nftbPWtyc5K8m2Heu/nOS69r7fS/KMjnUnJflEkkVJ7gTeleT6ztnrJAclubTXjkiyTZKF7bh+BPx21/pK8pRHk3+eAAAFRElEQVT2732TXNHGeG2SdyfZFPgWsF079juSbJfkmCRfSXJKktuAw9plp3SF8MYkK5P8OsmRXeP6+47bD89KJzkZ2AH473Z77+metW9jWNgeDVia5C0dfR2T5PQkn2/HsiTJgo71723Hd3uSq5K8tNe+k6S1ycJY0rryc+DBJJ9Lsk+Sx4+sqKorgbcBF7SH5rdqV90JvB7YCtgP+NMkr2jX/V77e6v2Phd0bizJ1sA3gY8B2wAfBb6ZZJuOZq8B/gR4As0M9rs71n0L2Llddwnwha7xvAb4ILA58G/ATcAfdqx/HXDyKPviBOAe4MnAG9uf0XwaeGtVbQ48E/huVd0J7AOsbMe+WVWtbNsfCHyFZp91xzziJe3YXgYc1Xl6xGiq6lDgV8DL2+19pEezU4EVwHbAq4APdRW4BwCntbEtBD4OkOSpwBHA89tx/hGwbLyYJGnYLIwlrRNVdRvwYqCA/wRWtbOLTxzjPudU1eVV9VBVXUZTeP1+n5vcD/hFVZ1cVQ9U1anAz4CXd7T5bFX9vKrupjm9Y9eObX+mqm6vqnuBY4DnJNmy475fr6rz29juAT5HUwyPFOV/BHyxO6j2g2oHAUdX1Z1V9dP2vqO5H5ifZIuq+k1VXTLOuC+oqq+1cd09Sptj221fDnwWOGScPseVZC7N4/veqrqnqi4FPgUc2tHsvKpa1J52cjLwnHb5g8DGNON8bFUtq6pfrmlMkjRRFsaS1pmqurKqDquqOTSzn9sB/zJa+yS7Jzk7yaokt9LMKm87Wvsu2wHXdC27Bti+4/Z1HX/fBWzWbndGkuOT/LI9JWFZ26Zz28u7+j4FeHmSzYBXA9+vql/3iGs2zec7Ou/fHWeng4B9gWuSnJvkhWO07RXXeG2uodlXa2o74Oaqur2r77H296z2POilwDtp3oDckOS0JMOISZImxMJY0qSoqp8BJ9EUyNDMJHf7Is0h97lVtSXNecgZo32nlcCOXct2AK7tI7zX0JySsBewJTCvXZ6ONqttv6quBS4AXkkzSzraaRSrgAeAuV1x9VRVF1XVgTSndHyNZmb7UdsfLa5RdG975DSMO4FNOtY9aQJ9rwS2TrJ5V9/97G+q6otV9WKax6yAf+jnfpI0TBbGktaJJE9LcmSSOe3tuTSH8C9sm1wPzEmyUcfdNqeZhbwnyW40BeuIVcBDwGjX+10E7JLkNUlmJvljYD7NlTHGsznN1TJuoikUP9TXIOHzwHuAZwFn9GrQnkbwVeCYJJskmQ+8oVfbJBuluTbzllV1P3AbzWkH0OyvbbpO7+jX37bbfgbNOdZfapdfCuybZOskT6KZxe10PaPs76paDvwA+HCSWUmeDbyJ0c9zfliSpyb5gyQb05x7fTePjFOS1hkLY0nryu3A7sAP2ys5XAj8FBi5KsJ3gSXAdUlubJf9GXBcktuBo3lktpSquovmw2/nJ7klyQs6N1ZVNwH7t/3fRFOw7l9VNzK+z9OcBnAtcAWPFO/jOYNmxvOM9gNyozmC5rSN62hmzT87RttDgWXtKR1voz2PuZ1xPxW4uh3/RE49OBdYCnwH+MeqGvlSlJOBn9CcOnIWjxTMIz4MvL/d3rt5tENoZtdX0uyLD1TVt/uIZ2PgeOBGmn3yBOBvJjAeSRqKVPVz1E2S1I8kv6S5isT/THYskqSJccZYkoYkyUE058d+d7JjkSRNnN98J0lDkOQcmnOYD62qhyY5HEnSADyVQpIkScJTKSRJkiTAwliSJEkCLIwlSZIkwMJYkiRJAiyMJUmSJMDCWJIkSQLg/wNE+Rt8mBxN7gAAAABJRU5ErkJggg==\n",
                        "text/plain": [
                            "<Figure size 864x288 with 2 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "fig, axes = plt.subplots(1, 2, figsize=(12, 4))\n",
                "titles = ['Stationary distribution for the recurrent class {0}'.format(recurrent_class)\n",
                "          for recurrent_class in mc1.recurrent_classes]\n",
                "\n",
                "for ax, title, dist in zip(axes, titles, mc1.stationary_distributions):\n",
                "    draw_histogram(dist, ax=ax, title=title, xlabel='States')\n",
                "\n",
                "fig.suptitle('Stationary distributions', y=-0.05, fontsize=12)\n",
                "plt.show()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Simulation"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Let us simulate our Markov chain `mc1`.\n",
                "The `simualte` method generates a sample path\n",
                "of length given by the first argument, `ts_length`,\n",
                "with an initial state as specified by an optional argument `init`;\n",
                "if not specified, the initial state is randomly drawn. "
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "A sample path from state `0`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
                            "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
                            "       0, 0, 0, 0, 0, 0])"
                        ]
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.simulate(50, init=0)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "As is clear from the transition matrix `P`,\n",
                "if it starts at state `0`, the chain stays there forever,\n",
                "i.e., `0` is an absorbing state, a state that constitutes a singleton recurrent class."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Start with state `1`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([1, 4, 4, 4, 4, 4, 4, 1, 4, 1, 4, 1, 4, 1, 4, 1, 4, 4, 4, 4, 4, 1,\n",
                            "       4, 1, 4, 1, 4, 4, 4, 4, 1, 4, 1, 4, 4, 1, 4, 1, 4, 1, 4, 4, 4, 1,\n",
                            "       4, 1, 4, 1, 4, 1])"
                        ]
                    },
                    "execution_count": 20,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.simulate(50, init=1)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "You can observe that the chain stays in the recurrent class $\\{1,4\\}$\n",
                "and visits states `1` and `4` with certain frequencies."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "If `init` is not specified, the initial state is randomly chosen:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
                            "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
                            "       0, 0, 0, 0, 0, 0])"
                        ]
                    },
                    "execution_count": 21,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.simulate(50)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "**Note on reproducibility**:\n",
                "The `simulate` method offers an option `random_state` to set a random seed\n",
                "to initialize the pseudo-random number generator.\n",
                "As you provide the same random seed value,\n",
                "`simulate` returns the same outcome."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "For example, the following will always give the same sequence:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([2, 4, 1, 4, 4, 4, 4, 1, 4, 1, 4, 4, 1, 4, 1, 4, 4, 4, 4, 4, 1, 4,\n",
                            "       4, 1, 4, 1, 4, 1, 4, 4, 1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 1, 4, 1, 4,\n",
                            "       1, 4, 4, 4, 4, 4])"
                        ]
                    },
                    "execution_count": 22,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc1.simulate(50, random_state=12345)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Time series averages"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Now, let us compute the frequency distribution along a sample path, given by\n",
                "$$\n",
                "\\frac{1}{t} \\sum_{\\tau=0}^{t-1} \\mathbf{1}\\{X_{\\tau} = s\\}\n",
                "\\quad (s \\in S).\n",
                "$$"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [],
            "source": [
                "def time_series_dist(mc, t, init=None, random_state=None):\n",
                "    \"\"\"\n",
                "    Return the distribution of visits by a sample path of length t\n",
                "    of mc with an initial state init.\n",
                "    \n",
                "    \"\"\"\n",
                "    t_max = np.max(t)\n",
                "    dim = 1\n",
                "    try:\n",
                "        ts_size = len(t)  # t is an array\n",
                "        ts_array = t\n",
                "        dim = 2\n",
                "    except:  # t is an int\n",
                "        ts_size = 1\n",
                "        ts_array = [t]\n",
                "        \n",
                "    X = mc.simulate(ts_length=t_max, init=init, random_state=random_state)\n",
                "    dists = np.empty((ts_size, mc.n))\n",
                "    bins = np.arange(mc.n+1)\n",
                "    for i, length in enumerate(ts_array):\n",
                "        hist, bin_edges = np.histogram(X[:length], bins=bins)\n",
                "        dists[i, :] = hist / length\n",
                "    if dim == 1:\n",
                "        return dists[0]\n",
                "    else:\n",
                "        return dists"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Here is a frequency distribution along a sample path,  of length 100,\n",
                "from initial state `1`, which is a recurrent state:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([0.  , 0.35, 0.  , 0.  , 0.65, 0.  ])"
                        ]
                    },
                    "execution_count": 24,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "time_series_dist(mc1, t=100, init=1)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Length 10,000:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 25,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([0.   , 0.332, 0.   , 0.   , 0.668, 0.   ])"
                        ]
                    },
                    "execution_count": 25,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "time_series_dist(mc1, t=10**4, init=1)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The distribution becomes close to the stationary distribution `(0, 1/3, 0, 0, 2/3, 0)`."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Plot the frequency distributions for a couple of different time lengths:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 26,
            "metadata": {},
            "outputs": [],
            "source": [
                "def plot_time_series_dists(mc, init, ts, seed=None, figsize=(12,4)):\n",
                "    dists = time_series_dist(mc, t=ts, init=init, random_state=seed)\n",
                "\n",
                "    fig, axes = plt.subplots(1, len(ts), figsize=figsize)\n",
                "    titles = ['t={0}'.format(t) for t in ts]\n",
                "\n",
                "    for ax, title, dist in zip(axes, titles, dists):\n",
                "        draw_histogram(dist, ax=ax, title=title, xlabel='States')\n",
                "\n",
                "    fig.suptitle('Time series distributions with init={0}'.format(init),\n",
                "                 y=-0.05, fontsize=12)\n",
                "    plt.show()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 27,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAE3CAYAAACpToeUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHtZJREFUeJzt3XmYZHV97/H3R9YoCFEmLgwwGHABr+uIeomKSwyogTyJUXjUiFFHn8hVo16DyqOIGpFo9F7FhbjgyuKSZNRR3BCi1wVUQAeCdy4OYQQF2REV0O/945zGoqieru4+1d2n5/16nnqmqs6p3/lV9YfiU6dOVaWqkCRJkrZ0d1jsCUiSJElLgcVYkiRJwmIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMV4USTYmecIcbldJfpnkhvb0/knMT5oyj6yekOTCJL9LcviI5X+f5GdJrk3ywSTbdTJhqTWJ59k03pLkyvZ0XJJ0O3NtaRbjeTbJqiSnJ7kxyX/OZfvLlcW4fx5YVTu0p+ct9mSkaZwL/B3w/eEFSf4MOBJ4PLAKuBfw+oWcnDSD6Z5n1wB/ATwQeADwFOAFizFBifk9z54E/AC4K/Aa4FNJVkx4vr1gMV5gST4K7A58tt0b8crFnpM0ynyyWlXHV9VXgV+PWPxs4ANVtb6qrgbeABzexZwlmOjz7LOBt1XVpqr6KfA2zK7mYTGeZ5PcG3gI8Lqq+lVVfRr4IfBX87s3y4PFeIFV1bOA/wL+vN0bcVySazZzOnJoiDPbt0Y+k2TVgt8BbTE6yOp09qXZ0zHlXOBuSe7a9X3QlmmCz7OjsrvvRO+MlrVFep7dF7ioqq4fWm6Wga0XewKCqtp5zFUfA3wbuCPwRuBzSR5UVbdMbHLSgFlkdXN2AK4duDx1fkfgyg7Gl26no+fZUdndIUmqqjqdsLZYC/A8O7xsavmuHWy399xj3CNVdWZV3VRV1wAvAfYE7rfI05Jm6wbgzgOXp85fP2JdaUHN8Dw7Krs3WIq1BG3ueXZ42dRyn4OxGC+W2zyJDnz6edTp1TOM4yeiNUldZXXQepoPL015IPDzqnJvsbo0iefZUdld3+20tQVa6OfZ9cC9kuw4tNws46EUi+XnNJ8QBaCqdpjpBkn2BbahOUD+D2je4vspcMGE5ijBHLIKkGRbmhfeAbZJsj1wU1X9DvgIcGKSjwOXAUcBJ3Y8b2kSz7MfAV6WZB1NmXk58M5up60t0II+z1bVj5OcA7wuyVHAQTTfsuKH73CP8WJ5M3BUeyD9K8a8zd2AU4DrgItovn7lKVV182SmKAFzyyrAl4BfAf8dOKE9/2iAqvoicBxwOnBxe3pdl5OWmMzz7PuAz9IU5x8Bn2+vk+ZjMZ5nDwVWA1cDxwJPraor5nk/loV4aJQkSZLkHmNJkiQJGKMYtz8jeHmSH02zPEn+d5INSc5L8pDupynNntlVX5ld9ZG51XIwzh7jE4EDN7P8IGDv9rQGeM/8pyV14kTMrvrpRMyu+udEzK16bsZiXFVnAldtZpVDgI9U49vAzknu0dUEpbkyu+ors6s+MrdaDro4xnhX4JKBy5vw11PUD2ZXfWV21UfmVkteF99jPOoHJkZ+1UWSNTRvn3CnO93pofe973072Ly2ZN/73vd+UVUr5nhzs6tFMc/cwpjZNbfqms+56qtxs9tFMd4E7DZweSVw6agVq+oEmu/aY/Xq1XX22Wd3sHltyZJcPI+bm10tinnmFsbMrrlV13zOVV+Nm90uDqVYC/xN+2nTRwDXVtVlHYwrTZrZVV+ZXfWRudWSN+Me4yQnAQcAuyTZRPPLKdsAVNV7gXXAk4ANwI3AcyY1WWk2zK76yuyqj8ytloMZi3FVHTbD8gJe1NmMpI6YXfWV2VUfmVstB/7ynSRJkoTFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRIwZjFOcmCSC5NsSHLkiOW7Jzk9yQ+SnJfkSd1PVZo9s6s+MrfqK7OrvpuxGCfZCjgeOAjYBzgsyT5Dqx0FnFpVDwYOBd7d9USl2TK76iNzq74yu1oOxtljvB+woaouqqqbgJOBQ4bWKeDO7fmdgEu7m6I0Z2ZXfWRu1VdmV703TjHeFbhk4PKm9rpBRwPPTLIJWAf8j1EDJVmT5OwkZ19xxRVzmK40K2ZXfWRu1VdmV703TjHOiOtq6PJhwIlVtRJ4EvDRJLcbu6pOqKrVVbV6xYoVs5+tNDtmV31kbtVXZle9N04x3gTsNnB5Jbd/6+O5wKkAVfUtYHtgly4mKM2D2VUfmVv1ldlV741TjM8C9k6yZ5JtaQ6WXzu0zn8BjwdIcj+aoPvehxab2VUfmVv1ldlV781YjKvqFuAI4DTgAppPk65PckySg9vVXg48P8m5wEnA4VU1/PaJtKDMrvrI3KqvzK6Wg63HWamq1tEcJD943WsHzp8P7N/t1KT5M7vqI3OrvjK76jt/+U6SJEnCYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAmDrxZ6AJElbmlVHfr6zsTYe++TOxpK2dO4xliRJkrAYS5IkSYDFWJIkSQI8xliSJEljWu7Hx7vHWJIkScJiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBIxZjJMcmOTCJBuSHDnNOk9Lcn6S9Uk+0e00pbkxu+ojc6u+Mrvquxl/+S7JVsDxwJ8Cm4CzkqytqvMH1tkbeBWwf1VdneSPJjVhaVxmV31kbtVXZlfLwTh7jPcDNlTVRVV1E3AycMjQOs8Hjq+qqwGq6vJupynNidlVH5lb9ZXZVe+NU4x3BS4ZuLypvW7QvYF7J/lmkm8nObCrCUrzYHbVR+ZWfWV21XszHkoBZMR1NWKcvYEDgJXAfyS5f1Vdc5uBkjXAGoDdd9991pPV5Kw68vOdjbXx2Cd3NtY8md1lrsvcwpLJrrlVX5ld9d44e4w3AbsNXF4JXDpinX+vqpur6ifAhTTBv42qOqGqVlfV6hUrVsx1ztK4zK76yNyqr8yuem+cYnwWsHeSPZNsCxwKrB1a59+AxwIk2YXmrZKLupyoNAdmV31kbtVXZle9N2MxrqpbgCOA04ALgFOran2SY5Ic3K52GnBlkvOB04H/WVVXTmrS0jjMrvrI3KqvzK6Wg3GOMaaq1gHrhq577cD5Al7WnqQlw+yqj8yt+srsqu/85TtJkiQJi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkYMxinOTAJBcm2ZDkyM2s99QklWR1d1OU5s7sqo/MrfrK7KrvZizGSbYCjgcOAvYBDkuyz4j1dgReDHyn60lKc2F21UfmVn1ldrUcjLPHeD9gQ1VdVFU3AScDh4xY7w3AccCvO5yfNB9mV31kbtVXZle9t/UY6+wKXDJweRPw8MEVkjwY2K2qPpfkFdMNlGQNsAZg9913n/1s52jVkZ/vbKyNxz65s7E0cb3Obpe5BbPbI73OrbZoZle9N84e44y4rm5dmNwBeDvw8pkGqqoTqmp1Va1esWLF+LOU5sbsqo/MrfrK7Kr3xinGm4DdBi6vBC4duLwjcH/g60k2Ao8A1npAvZYAs6s+MrfqK7Or3hunGJ8F7J1kzyTbAocCa6cWVtW1VbVLVa2qqlXAt4GDq+rsicxYGp/ZVR+ZW/WV2VXvzViMq+oW4AjgNOAC4NSqWp/kmCQHT3qC0lyZXfWRuVVfmV0tB+N8+I6qWgesG7rutdOse8D8pyV1w+yqj8yt+srsqu/85TtJkiQJi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQLG/B5jSZKk5W7VkZ/vbKyNxz65s7G0cNxjLEmSJGExliRJkgCLsSRJkgRYjCVJkiTAD99JkjrW5QeYwA8xSVo47jGWJEmSsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkoAxi3GSA5NcmGRDkiNHLH9ZkvOTnJfkq0n26H6q0uyZXfWRuVVfmV313YzFOMlWwPHAQcA+wGFJ9hla7QfA6qp6APAp4LiuJyrNltlVH5lb9ZXZ1XIwzh7j/YANVXVRVd0EnAwcMrhCVZ1eVTe2F78NrOx2mtKcmF31kblVX5ld9d44xXhX4JKBy5va66bzXOAL85mU1BGzqz4yt+ors6ve23qMdTLiuhq5YvJMYDXwmGmWrwHWAOy+++5jTlGaM7OrPjK36iuzq94bZ4/xJmC3gcsrgUuHV0ryBOA1wMFV9ZtRA1XVCVW1uqpWr1ixYi7zlWbD7KqPzK36yuyq98YpxmcBeyfZM8m2wKHA2sEVkjwYeB9NyC/vfprSnJhd9ZG5VV+ZXfXejMW4qm4BjgBOAy4ATq2q9UmOSXJwu9o/ATsAn0xyTpK10wwnLRizqz4yt+ors6vlYJxjjKmqdcC6oeteO3D+CR3PS+qE2VUfmVv1ldlV3/nLd5IkSRIWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEnAmMU4yYFJLkyyIcmRI5Zvl+SUdvl3kqzqeqLSXJhd9ZG5VV+ZXfXdjMU4yVbA8cBBwD7AYUn2GVrtucDVVbUX8HbgLV1PVJots6s+MrfqK7Or5WCcPcb7ARuq6qKqugk4GThkaJ1DgA+35z8FPD5JupumNCdmV31kbtVXZle9t/UY6+wKXDJweRPw8OnWqapbklwL3BX4xeBKSdYAa9qLNyS5cBZz3WV4vI6NNX7m99p2SdyHSY+/wI/RHptZZnYHzOPvMun5L8Q2ltpj1Ifcwpb3d1lq44+1DZ9zb2dL/Lv0cvwllN1bjVOMR72SqzmsQ1WdAJwwxjZvP4nk7KpaPZfbLoXxF2IbfR9/Atswuz0YfyG20bPxl0RuoXeP27IbfyG2sRyz699l+Y8/yW2McyjFJmC3gcsrgUunWyfJ1sBOwFVdTFCaB7OrPjK36iuzq94bpxifBeydZM8k2wKHAmuH1lkLPLs9/1Tga1V1u1eA0gIzu+ojc6u+MrvqvRkPpWiPAToCOA3YCvhgVa1PcgxwdlWtBT4AfDTJBppXfodOYK5zfjtwiYy/ENvo+/idbsPs9mb8hdhGb8ZfQrmFHj1uy3T8hdjGcsyuf5flP/7EthFfqEmSJEn+8p0kSZIEWIwlSZIkoCfFeKafmJzn2B9McnmSH3U57sD4uyU5PckFSdYneckEtrF9ku8mObfdxuu73ka7na2S/CDJ5yYw9sYkP0xyTpKzux5/MUwyt+34vc7ucshtO77Znf34Zne87ficO0v2hRm3YXZnUlVL+kRzAP//A+4FbAucC+zT4fiPBh4C/GhC878H8JD2/I7Aj7ucfztugB3a89sA3wEeMYH78jLgE8DnJjD2RmCXhc7XpE6Tzm27jV5ndznkth3f7M5+G2Z3vO34nDu7+2RfmHkbZneGUx/2GI/zE5NzVlVnMsHvUKyqy6rq++3564ELaH75p8ttVFXd0F7cpj11+qnKJCuBJwPv73LcZWyiuYX+Z9fcLllmd+bxze7SZF+YeRtmdwZ9KMajfmKy06AslCSrgAfTvELreuytkpwDXA58uaq63sY7gFcCv+t43CkFfCnJ99L8FGjfLZvcwuSyuwxyC2Z3STO701puuYVllF37wmZNNLt9KMZj/XzkUpdkB+DTwEur6rqux6+q31bVg2h+aWi/JPfvauwkTwEur6rvdTXmCPtX1UOAg4AXJXn0BLe1EJZFbmGy2V0GuQWzu2SZ3c1abrmFZZJd+8KMJprdPhTjcX5icklLsg1NyD9eVZ+Z5Laq6hrg68CBHQ67P3Bwko00b009LsnHOhyfqrq0/fdy4F9p3hLrs97nFhYuu33NLZjdpcrsbt4yzC0sg+zaF2Y26ez2oRiP8xOTS1aS0PzSzwVV9c8T2saKJDu35/8AeALwn12NX1WvqqqVVbWK5vH/WlU9s6vxk9wpyY5T54EnAhP51O8C6nVuYfLZ7XtuwewuVWZ385ZpbqHn2bUvzGwhsrvki3FV3QJM/cTkBcCpVbW+q/GTnAR8C7hPkk1JntvV2K39gWfRvGo6pz09qeNt3AM4Pcl5NE8MX66qiXw11YTcDfhGknOB7wKfr6ovLvKc5mXSuYVlkd2+5xbM7pyY3UW37HIL9oUxmd0Z+JPQkiRJEj3YYyxJkiQtBIuxJEmShMVYkiRJAizGkiRJEmAxliRJkgCL8YJI8pok65Oc1379ysOTvDTJHce47VjrSV0zt+ors6u+MruLz69rm7AkjwT+GTigqn6TZBdgW+D/AKur6hcz3H7jOOtJXTK36iuzq74yu0uDe4wn7x7AL6rqNwBtYJ8K3JPmS7ZPB0jyniRnt68UX99e9+IR6z0xybeSfD/JJ9P8pjpJjk1yfvsq860Lfze1zJhb9ZXZVV+Z3SXAPcYT1gbxG8Adga8Ap1TVGcOv7JLcpaquSrIV8FXgxVV13uB67avHzwAHVdUvk/wDsB3wLppf47lvVVWSndvfQJfmxNyqr8yu+srsLg3uMZ6wqroBeCiwBrgCOCXJ4SNWfVqS7wM/APYF9hmxziPa67+Z5Bzg2cAewHXAr4H3J/lL4Mau74e2LOZWfWV21Vdmd2nYerEnsCWoqt8CXwe+nuSHNAG9VZI9gVcAD6uqq5OcCGw/YqjQ/K75YbdbkOwHPB44lOa34h/X5X3Qlsfcqq/MrvrK7C4+9xhPWJL7JNl74KoHARcD1wM7ttfdGfglcG2SuwEHDaw/uN63gf2T7NWOfcck927fftmpqtYBL223Ic2ZuVVfmV31ldldGtxjPHk7AO9MsjNwC7CB5m2Sw4AvJLmsqh6b5AfAeuAi4JsDtz9haL3DgZOSbNcuP4rmP4Z/T7I9zavEv1+IO6Zlzdyqr8yu+srsLgF++E6SJEnCQykkSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWNriJHl1kvcv9jxmI8kzknxpwttYlaSSbN1e/kKSZ3c09qOSXDhweWOSJ3Qxdjve+iQHdDVeV2bKWpLDk3xjFuONfT+X6mMiaWlLVS32HCR1KMkNAxfvCPwG+G17+QVV9fGFn9XSl2QV8BNgm6q6ZRa3K2Dvqtowi9tsBJ5XVV+Z5TRJciKwqaqOmu1tF9OoxzfJ4TSPw59MeNtHA3tV1TPncNunAS8FHgR8t6oO6HZ2kpaSrRd7ApK6VVU7TJ2fTwFbKpJsPZuiutT0ff7iKuAdwH2Bxy3yXCRNmIdSSFuYJEcn+Vh7furwgeckuSTJ1UlemORhSc5Lck2Sdw3d/m+TXNCue1qSPabZzvZJPpbkynacs5LcrV22U5IPJLksyU+TvDHJVu2yw5N8M8nbk1wFHD38lnuS+yb5cpKrklzY7tWbWvakJOcnub4d+xXTzG+rJG9N8oskFwFPHlr+9STPa8/vleSMJNe265/SXn9mu/q5SW5I8vQkByTZlOQfkvwM+NDUdUNTeFg7z6uTfCjJ9gP3/zaHF7R/o72SrAGeAbyy3d5n2+W3HpqRZLsk70hyaXt6R5Lt2mVTc3t5ksvbx/85c3jsLk7y0Pb8M9v57dNefl6Sf2vP35o1YOqxuqad+yMHxntr+zj8JMlBo7Y54n4eneTUJB9p57s+yerhdZMcCLwaeHq73XOnG3+UqvpKVZ0KXDqb20nqJ4uxJICHA3sDT6fZO/Ya4AnAvsDTkjwGIMlf0JSMvwRWAP8BnDTNmM8GdgJ2A+4KvBD4Vbvsw8AtwF7Ag4EnAs8bms9FwB8BbxocNMmdgC8Dn2iXHwa8O8m+7SofoDlkZEfg/sDXppnf84GntNtfDTx1mvUA3gB8CfhDYCXwToCqenS7/IFVtUNVndJevjtwF2APYM00Yz4D+DPgj4F7AzMeGlFVJwAfB45rt/fnI1Z7DfAImrf+HwjsNzT23Wn+LrsCzwWOT/KH7bJxH7szgAPa84+m+Vs9ZuDyGSNuM/VY7dzO/Vvt5YcDFwK7AMcBH0iSabY77GDgZGBnYC3wruEVquqLwD8Cp7TbfSBAkne3L9hGnc4bc/uSlhmLsSSAN1TVr6vqS8AvgZOq6vKq+ilN+X1wu94LgDdX1QXt4QH/CDxomr3GN9MU4r2q6rdV9b2quq7da3wQ8NKq+mVVXQ68HTh04LaXVtU7q+qWqvrV0LhPATZW1Yfa5d8HPs3vi+3NwD5J7lxVV7fLR3ka8I6quqSqrgLevJnH52aaknvP9nGa6QNjvwNeV1W/GTH/Ke8a2PabaAp+F54BHNP+/a4AXg88a2D5ze3ym6tqHXADcJ+BZeM8dmfw+yL8KJrHburyYxhdjKdzcVX9S1X9luYF0z2Au415229U1br2th+leSEwlqr6u6raeZrTA2Yxf0nLiMVYEsDPB87/asTlqeOW9wD+19SeNZrjL0Oz93HYR4HTgJPbt/SPS7JNO8Y2wGUD47yPZu/vlEs2M9c9gIcP7uGjKYN3b5f/FfAk4OL28IdHTjPOPYe2c/FmtvnK9n5+t33L/m83sy7AFVX16xnWGd72PWdYf1z35Lb3ZXjsK4eOeb6R3/99x33szgAeleTuwFbAKcD+aT5gtxNwzizm+7OpM1V1Y3t2h2nWnfa2NPdj+7TfKiJJc+ETiKTZuAR40zjfbFFVN9PsrXx9W5jW0bxlvo7mmzJ22cyH0jb3dTmXAGdU1Z9Os92zgEPaEn4EcCrN4RzDLhu6fvfN3Jef0Rx6QZI/Ab6S5MzNfBPFOF/3M7ztqWNYf0nzbSK027s7tzXT2JfSvHhYP2LszRr3sauqDUluBF4MnFlV17fHU6+h2Yv7u1HDjzOHCbndtpO8F5juWyourqp9p1kmaRlzj7Gk2Xgv8Kqp43nTfIjur0etmOSxSf5bmg/VXUfzNv1vq+oymuN135bkzknukOSPp45jHsPngHsneVaSbdrTw5LcL8m2ab7zeKe2mF/H77+qbtipwIuTrGyPsT1yug0m+eskK9uLV9MUralxfw7ca8y5D3pRu+270By3PXV88rnAvkkelOYDeUcP3W6m7Z0EHJVkRZJdgNcCH9vM+gDM8rGDZq/xEfz+sImvD10edgXNISZzeazm6+fAqiS3/j+vql7YHnM86nRrKU7zIc3taXYk3SHNh0q3WYT7IGkBWIwlja2q/hV4C83hEdcBP6I5XniUuwOfoilYF9AUpqmC9jfAtsD5NEXzUzTHlo4zh+tpPqx3KM2e0J+1c9quXeVZwMZ2fi9k+r2C/0JzqMe5wPeBz2xmsw8DvpPmO6LXAi+pqp+0y44GPtwe1vG06QYY4RM0LxAuak9vbO/fj4FjgK8A/xcYPp75AzTHAV8z9e0PQ94InA2cB/ywvW9vHHNO4z520Pw9d+T33zYxfPk22sMk3gR8s537I8acUxc+2f57ZZLpjpuezrNoDid6D83x1L+iyY6kZcgf+JAkSZJwj7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEwP8HD4N43qJ7MjcAAAAASUVORK5CYII=\n",
                        "text/plain": [
                            "<Figure size 864x288 with 4 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "init = 1\n",
                "ts = [5, 10, 50, 100]\n",
                "plot_time_series_dists(mc1, init, ts)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Start with state `2`,\n",
                "which is a transient state:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 28,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAE3CAYAAACpToeUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHzlJREFUeJzt3XuUZGV97vHvE64qCIlMRBlgMOAFPF5HNIdEMWoCYiArMQhHjRjN6Eo4xmiOGZWliDckmpijJIZoglcuXpJMZCJqghCNGkYFdCCYOTiEERTkjqiA/s4fezcWRfV0dfeu7t49389ataaq9q53v1X9UDy1a1dVqgpJkiRpW/cziz0BSZIkaSmwGEuSJElYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGiyLJ5iRPn8PtKsn3k9zWnt47iflJU+aR1dOSXJ7kJ0mOG7H8j5J8J8nNSf42yU6dTFhqTeJ5No23Jbm+PZ2SJN3OXNuaxXieTbIqyXlJbk/yn3PZ/nJlMe6fR1fVLu3pxYs9GWkaFwO/D3x1eEGSXwPWAk8DVgEPAd6wkJOTZjDd8+wa4DeARwOPAp4FvGQxJigxv+fZM4CvAQ8AXgt8LMmKCc+3FyzGCyzJB4F9gH9q90a8arHnJI0yn6xW1alV9S/AD0csfgHwvqraWFU3Am8EjutizhJM9Hn2BcA7qmpLVX0beAdmV/OwGM+zSR4KPA54fVX9oKo+Dnwd+K353ZvlwWK8wKrq+cB/A7/e7o04JclNWzmtHRrigvatkU8kWbXgd0DbjA6yOp2DaPZ0TLkYeGCSB3R9H7RtmuDz7KjsHjTRO6NlbZGeZw8CrqiqW4eWm2Vg+8WegKCqdh9z1acAXwLuC7wJ+GSSx1TVXRObnDRgFlndml2AmwcuT53fFbi+g/Gle+noeXZUdndJkqqqTiesbdYCPM8OL5tavlcH2+099xj3SFVdUFV3VNVNwB8C+wGPWORpSbN1G3D/gctT528dsa60oGZ4nh2V3dssxVqCtvY8O7xsarnPwViMF8s9nkQHPv086vSaGcbxE9GapK6yOmgjzYeXpjwa+G5VubdYXZrE8+yo7G7sdtraBi308+xG4CFJdh1abpbxUIrF8l2aT4gCUFW7zHSDJAcBO9AcIH8fmrf4vg1cNqE5SjCHrAIk2ZHmhXeAHZLsDNxRVT8BPgCcnuTDwDXACcDpHc9bmsTz7AeAVyRZT1NmXgm8q9tpaxu0oM+zVfXNJBcBr09yAnA4zbes+OE73GO8WN4KnNAeSP/HY97mgcBZwC3AFTRfv/KsqrpzMlOUgLllFeDTwA+A/wmc1p5/MkBVfQo4BTgPuLI9vb7LSUtM5nn2r4F/oinO3wDOaa+T5mMxnmePAVYDNwInA8+uquvmeT+WhXholCRJkuQeY0mSJAkYoxi3PyN4bZJvTLM8Sf5vkk1JLknyuO6nKc2e2VVfmV31kbnVcjDOHuPTgcO2svxw4ID2tAb4q/lPS+rE6Zhd9dPpmF31z+mYW/XcjMW4qi4AbtjKKkcBH6jGl4DdkzyoqwlKc2V21VdmV31kbrUcdHGM8V7AVQOXt+Cvp6gfzK76yuyqj8ytlrwuvsd41A9MjPyqiyRraN4+4X73u9/jH/7wh3eweW3LvvKVr3yvqlbM8eZmV4tinrmFMbNrbtU1n3PVV+Nmt4tivAXYe+DySuDqUStW1Wk037XH6tWra8OGDR1sXtuyJFfO4+ZmV4tinrmFMbNrbtU1n3PVV+Nmt4tDKdYBv9N+2vRJwM1VdU0H40qTZnbVV2ZXfWRuteTNuMc4yRnAocAeSbbQ/HLKDgBV9R5gPfBMYBNwO/DCSU1Wmg2zq74yu+ojc6vlYMZiXFXHzrC8gD/obEZSR8yu+srsqo/MrZYDf/lOkiRJwmIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCRizGCc5LMnlSTYlWTti+T5JzkvytSSXJHlm91OVZs/sqo/MrfrK7KrvZizGSbYDTgUOBw4Ejk1y4NBqJwBnV9VjgWOAv+x6otJsmV31kblVX5ldLQfj7DE+GNhUVVdU1R3AmcBRQ+sUcP/2/G7A1d1NUZozs6s+MrfqK7Or3hunGO8FXDVweUt73aATgecl2QKsB/73qIGSrEmyIcmG6667bg7TlWbF7KqPzK36yuyq98YpxhlxXQ1dPhY4vapWAs8EPpjkXmNX1WlVtbqqVq9YsWL2s5Vmx+yqj8yt+srsqvfGKcZbgL0HLq/k3m99vAg4G6CqvgjsDOzRxQSleTC76iNzq74yu+q9cYrxhcABSfZLsiPNwfLrhtb5b+BpAEkeQRN03/vQYjO76iNzq74yu+q9GYtxVd0FHA+cC1xG82nSjUlOSnJku9orgd9LcjFwBnBcVQ2/fSItKLOrPjK36iuzq+Vg+3FWqqr1NAfJD173uoHzlwKHdDs1af7MrvrI3KqvzK76zl++kyRJkrAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkAbD9Yk9AkuZq1dpzOh1v88lHdDqeJKlf3GMsSZIkYTGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAH8SWpIkSWNatfaczsbafPIRnY3VFfcYS5IkSbjHWJKkBbfc97pJfeUeY0mSJAmLsSRJkgRYjCVJkiTAYixJkiQBYxbjJIcluTzJpiRrp1nn6CSXJtmY5CPdTlOaG7OrPjK36iuzq76b8VspkmwHnAo8A9gCXJhkXVVdOrDOAcCrgUOq6sYkPz+pCUvjMrvqI3OrvjK7Wg7G2WN8MLCpqq6oqjuAM4Gjhtb5PeDUqroRoKqu7Xaa0pyYXfWRuVVfmV313jjFeC/gqoHLW9rrBj0UeGiSLyT5UpLDupqgNA9mV31kbtVXZle9N84PfGTEdTVinAOAQ4GVwL8leWRV3XSPgZI1wBqAffbZZ9aT1eQs0y+bN7vqI3OrvjK76r1x9hhvAfYeuLwSuHrEOv9YVXdW1beAy2mCfw9VdVpVra6q1StWrJjrnKVxmV31kblVX5ld9d44xfhC4IAk+yXZETgGWDe0zj8ATwVIsgfNWyVXdDlRaQ7MrvrI3KqvzK56b8ZiXFV3AccD5wKXAWdX1cYkJyU5sl3tXOD6JJcC5wH/p6qun9SkpXGYXfWRuVVfmV0tB+McY0xVrQfWD133uoHzBbyiPUlLhtlVH5lb9ZXZVd/5y3eSJEkSFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJwJjFOMlhSS5PsinJ2q2s9+wklWR1d1OU5s7sqo/MrfrK7KrvZizGSbYDTgUOBw4Ejk1y4Ij1dgVeBny560lKc2F21UfmVn1ldrUcjLPH+GBgU1VdUVV3AGcCR41Y743AKcAPO5yfNB9mV31kbtVXZle9N04x3gu4auDylva6uyV5LLB3VX1yawMlWZNkQ5IN11133awnK82S2VUfmVv1ldlV741TjDPiurp7YfIzwJ8Dr5xpoKo6rapWV9XqFStWjD9LaW7MrvrI3KqvzK56b5xivAXYe+DySuDqgcu7Ao8EPpdkM/AkYJ0H1GsJMLvqI3OrvjK76r1xivGFwAFJ9kuyI3AMsG5qYVXdXFV7VNWqqloFfAk4sqo2TGTG0vjMrvrI3KqvzK56b8ZiXFV3AccD5wKXAWdX1cYkJyU5ctITlObK7KqPzK36yuxqOdh+nJWqaj2wfui6102z7qHzn5bUDbOrPjK36iuzq77zl+8kSZIkLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAkY83uMJUka16q153Q63uaTj+h0PGk6XWbX3PaTe4wlSZIkLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSMGYxTnJYksuTbEqydsTyVyS5NMklSf4lyb7dT1WaPbOrPjK36iuzq76bsRgn2Q44FTgcOBA4NsmBQ6t9DVhdVY8CPgac0vVEpdkyu+ojc6u+MrtaDsbZY3wwsKmqrqiqO4AzgaMGV6iq86rq9vbil4CV3U5TmhOzqz4yt+ors6veG6cY7wVcNXB5S3vddF4E/PN8JiV1xOyqj8yt+srsqve2H2OdjLiuRq6YPA9YDTxlmuVrgDUA++yzz5hTnL9Va8/pbKzNJx+x4ONrznqd3S5zBWarR3qdW23TzK56b5w9xluAvQcurwSuHl4pydOB1wJHVtWPRg1UVadV1eqqWr1ixYq5zFeaDbOrPjK36iuzq94bpxhfCByQZL8kOwLHAOsGV0jyWOCvaUJ+bffTlObE7KqPzK36yuyq92YsxlV1F3A8cC5wGXB2VW1MclKSI9vV/hTYBfhokouSrJtmOGnBmF31kblVX5ldLQfjHGNMVa0H1g9d97qB80/veF5SJ8yu+sjcqq/MrvrOX76TJEmSsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJgO0XewKSFs+qted0Ntbmk4/obCxJkhaDe4wlSZIkLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEkAbL/YE5AkaTZWrT2n0/E2n3xEp+NJ6q+x9hgnOSzJ5Uk2JVk7YvlOSc5ql385yaquJyrNhdlVH5lb9ZXZVd/NuMc4yXbAqcAzgC3AhUnWVdWlA6u9CLixqvZPcgzwNuA5k5iwNC6zqz4yt+orszuzLt/t8J2OyRhnj/HBwKaquqKq7gDOBI4aWuco4P3t+Y8BT0uS7qYpzYnZVR+ZW/WV2VXvjXOM8V7AVQOXtwBPnG6dqroryc3AA4DvDa6UZA2wpr14W5LLZzHXPYbH69hY4+dtk93GpMefpyXxGA3ZdyvLzO6AefxdluLffSLjT/oxGtCH3MI28ndZwL/7RLbhc+69bBN/l238OXfsbQzYWnbvNk4xHvVKruawDlV1GnDaGNu89ySSDVW1ei63XQrjL8Q2+j7+BLZhdnsw/kJso2fjL4ncQu8et2U3/kJsYzlm17/L8h9/ktsY51CKLcDeA5dXAldPt06S7YHdgBu6mKA0D2ZXfWRu1VdmV703TjG+EDggyX5JdgSOAdYNrbMOeEF7/tnAv1bVvV4BSgvM7KqPzK36yuyq92Y8lKI9Buh44FxgO+Bvq2pjkpOADVW1Dngf8MEkm2he+R0zgbnO+e3AJTL+Qmyj7+N3ug2z25vxF2IbvRl/CeUWevS4LdPxF2IbyzG7/l2W//gT20Z8oSZJkiT5k9CSJEkSYDGWJEmSgJ4U45l+YnKeY/9tkmuTfKPLcQfG3zvJeUkuS7IxyR9OYBs7J/mPJBe323hD19tot7Ndkq8l+eQExt6c5OtJLkqyoevxF8Mkc9uO3+vsLofctuOb3dmPb3bH247PubNkX5hxG2Z3JlW1pE80B/D/P+AhwI7AxcCBHY7/ZOBxwDcmNP8HAY9rz+8KfLPL+bfjBtilPb8D8GXgSRO4L68APgJ8cgJjbwb2WOh8Teo06dy22+h1dpdDbtvxze7st2F2x9uOz7mzu0/2hZm3YXZnOPVhj/E4PzE5Z1V1ARP8DsWquqaqvtqevxW4jOaXf7rcRlXVbe3FHdpTp5+qTLISOAJ4b5fjLmMTzS30P7vmdskyuzOPb3aXJvvCzNswuzPoQzEe9ROTnQZloSRZBTyW5hVa12Nvl+Qi4FrgM1XV9TbeCbwK+EnH404p4NNJvpLmp0D7btnkFiaX3WWQWzC7S5rZndZyyy0so+zaF7ZqotntQzEe6+cjl7okuwAfB15eVbd0PX5V/biqHkPzS0MHJ3lkV2MneRZwbVV9pasxRzikqh4HHA78QZInT3BbC2FZ5BYmm91lkFswu0uW2d2q5ZZbWCbZtS/MaKLZ7UMxHucnJpe0JDvQhPzDVfWJSW6rqm4CPgcc1uGwhwBHJtlM89bUryT5UIfjU1VXt/9eC/w9zVtifdb73MLCZbevuQWzu1SZ3a1bhrmFZZBd+8LMJp3dPhTjcX5icslKEppf+rmsqv5sQttYkWT39vx9gKcD/9nV+FX16qpaWVWraB7/f62q53U1fpL7Jdl16jzwq8BEPvW7gHqdW5h8dvueWzC7S5XZ3bplmlvoeXbtCzNbiOwu+WJcVXcBUz8xeRlwdlVt7Gr8JGcAXwQelmRLkhd1NXbrEOD5NK+aLmpPz+x4Gw8CzktyCc0Tw2eqaiJfTTUhDwQ+n+Ri4D+Ac6rqU4s8p3mZdG5hWWS377kFszsnZnfRLbvcgn1hTGZ3Bv4ktCRJkkQP9hhLkiRJC8FiLEmSJGExliRJkgCLsSRJkgRYjCVJkiTAYrwgkrw2ycYkl7Rfv/LEJC9Pct8xbjvWelLXzK36yuyqr8zu4vPr2iYsyS8CfwYcWlU/SrIHsCPw78DqqvreDLffPM56UpfMrfrK7KqvzO7S4B7jyXsQ8L2q+hFAG9hnAw+m+ZLt8wCS/FWSDe0rxTe0171sxHq/muSLSb6a5KNpflOdJCcnubR9lfn2hb+bWmbMrfrK7KqvzO4S4B7jCWuD+HngvsBngbOq6vzhV3ZJfq6qbkiyHfAvwMuq6pLB9dpXj58ADq+q7yf5E2An4N00v8bz8KqqJLu3v4EuzYm5VV+ZXfWV2V0a3GM8YVV1G/B4YA1wHXBWkuNGrHp0kq8CXwMOAg4csc6T2uu/kOQi4AXAvsAtwA+B9yb5TeD2ru+Hti3mVn1ldtVXZndp2H6xJ7AtqKofA58DPpfk6zQBvVuS/YA/Bp5QVTcmOR3YecRQofld82PvtSA5GHgacAzNb8X/Spf3Qdsec6u+MrvqK7O7+NxjPGFJHpbkgIGrHgNcCdwK7Nped3/g+8DNSR4IHD6w/uB6XwIOSbJ/O/Z9kzy0fftlt6paD7y83YY0Z+ZWfWV21Vdmd2lwj/Hk7QK8K8nuwF3AJpq3SY4F/jnJNVX11CRfAzYCVwBfGLj9aUPrHQeckWSndvkJNP8x/GOSnWleJf7RQtwxLWvmVn1ldtVXZncJ8MN3kiRJEh5KIUmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMbSNifJa5K8d7HnMRtJnpvk0xPexqoklWT79vI/J3lBR2P/cpLLBy5vTvL0LsZux9uY5NCuxuvKTFlLclySz89ivLHv51J9TCQtbamqxZ6DpA4luW3g4n2BHwE/bi+/pKo+vPCzWvqSrAK+BexQVXfN4nYFHFBVm2Zxm83Ai6vqs7OcJklOB7ZU1Qmzve1iGvX4JjmO5nH4pQlv+0Rg/6p63hxu+3bgKGBP4NvAW6rqA93OUNJSsf1iT0BSt6pql6nz8ylgS0WS7WdTVJeavs9ffB/4deCbwBOATyXZVFX/vrjTkjQJHkohbWOSnJjkQ+35qcMHXpjkqiQ3JnlpkickuSTJTUnePXT7301yWbvuuUn2nWY7Oyf5UJLr23EuTPLAdtluSd6X5Jok307ypiTbtcuOS/KFJH+e5AbgxOG33JM8PMlnktyQ5PIkRw8se2aSS5Pc2o79x9PMb7skb0/yvSRXAEcMLf9ckhe35/dPcn6Sm9v1z2qvv6Bd/eIktyV5TpJDk2xJ8idJvgP83dR1Q1N4QjvPG5P8XZKdB+7/PQ4vaP9G+ydZAzwXeFW7vX9ql999aEaSnZK8M8nV7emdSXZql03N7ZVJrm0f/xfO4bG7Msnj2/PPa+d3YHv5xUn+oT1/d9aAqcfqpnbuvzgw3tvbx+FbSQ4ftc0R9/PEJGcn+UA7341JVg+vm+Qw4DXAc9rtXjzd+KNU1eur6j+r6idV9WXg34BfnOl2kvrJYiwJ4InAAcBzgHcCrwWeDhwEHJ3kKQBJfoOmZPwmsIKmJJwxzZgvAHYD9gYeALwU+EG77P3AXcD+wGOBXwVePDSfK4CfB948OGiS+wGfAT7SLj8W+MskB7WrvI/mkJFdgUcC/zrN/H4PeFa7/dXAs6dZD+CNwKeBnwVWAu8CqKont8sfXVW7VNVZ7eU9gZ8D9gXWTDPmc4FfA34BeCgw46ERVXUa8GHglHZ7vz5itdcCTwIeAzwaOHho7D1p/i57AS8CTk3ys+2ycR+784FD2/NPpvlbPWXg8vkjbjP1WO3ezv2L7eUnApcDewCnAO9Lkmm2O+xI4Exgd2Ad8O7hFarqU8BbgLPa7T4aIMlfti/YRp0uGbWxJPeh2Wu8ccz5SeoZi7EkgDdW1Q+r6tM0bx2fUVXXVtW3acrvY9v1XgK8taouaw8PeAvwmGn2Gt9JU4j3r6ofV9VXquqWdq/x4cDLq+r7VXUt8OfAMQO3vbqq3lVVd1XVD4bGfRawuar+rl3+VeDj/LTY3gkcmOT+VXVju3yUo4F3VtVVVXUD8NatPD530pTcB7eP00wfGPsJ8Pqq+tGI+U9598C230xT8LvwXOCk9u93HfAG4PkDy+9sl99ZVeuB24CHDSwb57E7n58W4V+meeymLj+F0cV4OldW1d9U1Y9pXjA9CHjgmLf9fFWtb2/7QZoXAmOpqt+vqt2nOT1qmpu9B7gYOHfc7UjqF4uxJIDvDpz/wYjLU8ct7wv8xdSeNeAGIDR7H4d9kKZAnNm+pX9Kkh3aMXYArhkY569p9v5OuWorc90XeOLgHj6aMrhnu/y3gGcCV7aHP0z3tveDh7Zz5Va2+ar2fv5H+5b9725lXYDrquqHM6wzvO0Hz7D+uB7MPe/L8NjXDx3zfDs//fuO+9idD/xykj2B7YCzgEPSfMBuN+CiWcz3O1Nnqur29uwu06w77W1p7sfOab9VpGtJ/pRmL/rR5afWpWXLD99Jmo2rgDeP880WVXUnzd7KN7SFaT3NW+brab4pY4+tfChta8XjKuD8qnrGNNu9EDiqLeHHA2fTHM4x7Jqh6/fZyn35Ds2hFyT5JeCzSS7YyjdRjFOchrd9dXv++zTfJkK7vT25p5nGvprmxcPU2/2DY2/VuI9dVW1KcjvwMuCCqrq1PZ56Dc1e3J+MGn6cOUzIvbad5D3AdN9ScWVVHTSw7hto3uV4SlXdMpkpSloK3GMsaTbeA7x66njeNB+i++1RKyZ5apL/keZDdbfQvE3/46q6huZ43XckuX+Sn0nyC1PHMY/hk8BDkzw/yQ7t6QlJHpFkxzTfebxbW8xv4adfVTfsbOBlSVa2x9iunW6DSX47ycr24o00RWtq3O8CDxlz7oP+oN32z9Ectz11fPLFwEFJHpPmA3knDt1upu2dAZyQZEWSPYDXAR/ayvoAzPKxg2av8fH89LCJzw1dHnYdzSEmc3ms5uu7wKokd/8/r6pe2h5zPOo0WIpfDfwv4BlVdf0izF3SArIYSxpbVf098DaawyNuAb5BsydtlD2Bj9EUrMtoCtNUQfsdYEfgUpqi+TGaY0vHmcOtNB/WO4ZmT+h32jnt1K7yfGBzO7+XMv1ewb+hOdTjYuCrwCe2stknAF9O8x3R64A/rKpvtctOBN7fHtZx9HQDjPARmhcIV7SnN7X375vAScBngf8Cho9nfh/NccA3TX37w5A3ARuAS4Cvt/ftTWPOadzHDpq/56789Nsmhi/fQ3uYxJuBL7Rzf9KYc+rCR9t/r08y3XHT03kLzV73/2q/1eK2JK/pdnqSlgp/4EOSJEnCPcaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSAP8ftp/Kp18NqjUAAAAASUVORK5CYII=\n",
                        "text/plain": [
                            "<Figure size 864x288 with 4 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "init = 2\n",
                "ts = [5, 10, 50, 100]\n",
                "plot_time_series_dists(mc1, init, ts)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Run the above cell several times;\n",
                "you will observe that the limit distribution differs across sample paths.\n",
                "Sometimes the state is absorbed into the recurrent class $\\{0\\}$,\n",
                "while other times it is absorbed into the recurrent class $\\{1,4\\}$."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 29,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Some sample path with init=2\n"
                    ]
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAE3CAYAAACpToeUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAH0dJREFUeJzt3XuUZGV97vHvI8NFBSHKRJQBBgOo4PE6oh4SwXgJiEJWYhCOGjHq6IocNeoxo7IU8YZEozmKUSKKVy5ekowyijeEaLyACuiAmDk4hBEU5I6ogP7OH3u3Fk31dHXPru7ePd/PWrWoqr3r3W9VP1M8tWtXVaoKSZIkaXN3l/megCRJkrQQWIwlSZIkLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEW43mRZH2SJ8zidpXkF0lubk/vH8f8pAmbkNUTk1yS5LdJjhyy/O+S/DTJDUk+kGTrTiYstcbxPJvGW5Nc056OT5JuZ67NzXw8zyZZnuSsJLck+eFstr9YWYz75yFVtW17et58T0aawgXA3wLfnbwgyZ8Bq4DHA8uB+wGvn8vJSdOY6nl2JfDnwEOABwNPAV4wHxOU2LTn2VOA7wH3Al4DfDLJ0jHPtxcsxnMsyUeAXYHPtHsjXjnfc5KG2ZSsVtUJVfVl4FdDFj8bOKmq1lbVdcAbgCO7mLMEY32efTbw9qraUFU/Ad6O2dUmmI/n2SR7AQ8HXldVv6yqTwHfB/5y0+7N4mAxnmNV9Szgv4Gntnsjjk9y/UZOqyYNcU771sinkyyf8zugzUYHWZ3KPjR7OiZcANw7yb26vg/aPI3xeXZYdvcZ653RojZPz7P7AJdW1U2TlptlYMl8T0BQVTuMuOr+wDeBuwFvBD6b5KFVdfvYJicNmEFWN2Zb4IaByxPntwOu6WB86U46ep4dlt1tk6SqqtMJa7M1B8+zk5dNLN+5g+32nnuMe6SqzqmqW6vqeuAlwO7AA+d5WtJM3QzcY+DyxPmbhqwrzalpnmeHZfdmS7EWoI09z05eNrHc52AsxvPlDk+iA59+HnZ69TTj+IlojVNXWR20lubDSxMeAvysqtxbrC6N43l2WHbXdjttbYbm+nl2LXC/JNtNWm6W8VCK+fIzmk+IAlBV2053gyT7AFvSHCB/V5q3+H4CXDymOUowi6wCJNmK5oV3gC2TbAPcWlW/BT4MnJzkY8CVwNHAyR3PWxrH8+yHgZclWUNTZl4OvKvbaWszNKfPs1X1oyTnA69LcjRwEM23rPjhO9xjPF/eAhzdHkj/ihFvc2/gNOBG4FKar195SlXdNp4pSsDssgrwBeCXwP8ETmzPPxagqj4PHA+cBVzWnl7X5aQlxvM8+z7gMzTF+QfAGe110qaYj+fZw4EVwHXAccDTqurqTbwfi0I8NEqSJElyj7EkSZIEjFCM258RvCrJD6ZYniT/N8m6JBcmeXj305Rmzuyqr8yu+sjcajEYZY/xycCBG1l+ELBne1oJ/POmT0vqxMmYXfXTyZhd9c/JmFv13LTFuKrOAa7dyCqHAh+uxjeBHZLcp6sJSrNldtVXZld9ZG61GHRxjPHOwOUDlzfgr6eoH8yu+srsqo/MrRa8Lr7HeNgPTAz9qoskK2nePuHud7/7Ix7wgAd0sHltzr7zne/8vKqWzvLmZlfzYhNzCyNm19yqaz7nqq9GzW4XxXgDsMvA5WXAFcNWrKoTab5rjxUrVtR5553Xwea1OUty2Sbc3OxqXmxibmHE7Jpbdc3nXPXVqNnt4lCK1cBft582fTRwQ1Vd2cG40riZXfWV2VUfmVsteNPuMU5yCnAAsGOSDTS/nLIlQFW9F1gDPBlYB9wCPGdck5Vmwuyqr8yu+sjcajGYthhX1RHTLC/gRZ3NSOqI2VVfmV31kbnVYuAv30mSJElYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJArr55TtJkqSxW77qjM7GWn/cwZ2NpcXDPcaSJEkSFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJ8CehJWmz48/qStJw7jGWJEmSsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJGLEYJzkwySVJ1iVZNWT5rknOSvK9JBcmeXL3U5Vmzuyqj8yt+srsqu+mLcZJtgBOAA4C9gaOSLL3pNWOBk6vqocBhwPv6Xqi0kyZXfWRuVVfmV0tBqPsMd4XWFdVl1bVrcCpwKGT1ingHu357YErupuiNGtmV31kbtVXZle9N0ox3hm4fODyhva6QccAz0yyAVgD/O9hAyVZmeS8JOddffXVs5iuNCNmV31kbtVXZle9N0oxzpDratLlI4CTq2oZ8GTgI0nuNHZVnVhVK6pqxdKlS2c+W2lmzK76yNyqr8yuem+UYrwB2GXg8jLu/NbHc4HTAarqG8A2wI5dTFDaBGZXfWRu1VdmV703SjE+F9gzye5JtqI5WH71pHX+G3g8QJIH0gTd9z4038yu+sjcqq/Mrnpv2mJcVbcDRwFnAhfTfJp0bZJjkxzSrvZy4PlJLgBOAY6sqslvn0hzyuyqj8yt+srsajFYMspKVbWG5iD5weteO3D+ImC/bqcmbTqzqz4yt+ors6u+85fvJEmSJCzGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEjBiMU5yYJJLkqxLsmqKdQ5LclGStUk+3u00pdkxu+ojc6u+MrvquyXTrZBkC+AE4InABuDcJKur6qKBdfYEXgXsV1XXJfnDcU1YGpXZVR+ZW/WV2dViMMoe432BdVV1aVXdCpwKHDppnecDJ1TVdQBVdVW305Rmxeyqj8yt+srsqvdGKcY7A5cPXN7QXjdoL2CvJF9P8s0kB3Y1QWkTmF31kblVX5ld9d60h1IAGXJdDRlnT+AAYBnwH0keVFXX32GgZCWwEmDXXXed8WSlGTK76iNzq74yu+q9UfYYbwB2Gbi8DLhiyDr/XlW3VdWPgUtogn8HVXViVa2oqhVLly6d7ZylUZld9ZG5VV+ZXfXeKMX4XGDPJLsn2Qo4HFg9aZ1/Ax4HkGRHmrdKLu1yotIsmF31kblVX5ld9d60xbiqbgeOAs4ELgZOr6q1SY5Ncki72pnANUkuAs4C/k9VXTOuSUujMLvqI3OrvjK7WgxGOcaYqloDrJl03WsHzhfwsvYkLRhmV31kbtVXZld95y/fSZIkSViMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBMCS+Z6A1FfLV53R6Xjrjzu40/EkSdLMuMdYkiRJwmIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkYMRinOTAJJckWZdk1UbWe1qSSrKiuylKs2d21UfmVn1ldtV30xbjJFsAJwAHAXsDRyTZe8h62wEvBr7V9SSl2TC76iNzq74yu1oMRtljvC+wrqourapbgVOBQ4es9wbgeOBXHc5P2hRmV31kbtVXZle9t2SEdXYGLh+4vAF41OAKSR4G7FJVn03yiqkGSrISWAmw6667zny2m6nlq87odLz1xx3c6Xij6PI+zGD+Zld9ZG7VV2ZXvTfKHuMMua5+tzC5C/AO4OXTDVRVJ1bViqpasXTp0tFnKc2O2VUfmVv1ldlV741SjDcAuwxcXgZcMXB5O+BBwFeTrAceDaz2gHotAGZXfWRu1VdmV703SjE+F9gzye5JtgIOB1ZPLKyqG6pqx6paXlXLgW8Ch1TVeWOZsTQ6s6s+MrfqK7Or3pu2GFfV7cBRwJnAxcDpVbU2ybFJDhn3BKXZMrvqI3OrvjK7WgxG+fAdVbUGWDPputdOse4Bmz4tqRtmV31kbtVXZld95y/fSZIkSViMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJGDEYpzkwCSXJFmXZNWQ5S9LclGSC5N8Oclu3U9Vmjmzqz4yt+ors6u+m7YYJ9kCOAE4CNgbOCLJ3pNW+x6woqoeDHwSOL7riUozZXbVR+ZWfWV2tRiMssd4X2BdVV1aVbcCpwKHDq5QVWdV1S3txW8Cy7qdpjQrZld9ZG7VV2ZXvTdKMd4ZuHzg8ob2uqk8F/jcpkxK6ojZVR+ZW/WV2VXvLRlhnQy5roaumDwTWAHsP8XylcBKgF133XXEKS58y1ed0dlY6487uLOxZHbnW9//bXQ5fxj5Pphb9ZXZVe+Nssd4A7DLwOVlwBWTV0ryBOA1wCFV9ethA1XViVW1oqpWLF26dDbzlWbC7KqPzK36yuyq90YpxucCeybZPclWwOHA6sEVkjwMeB9NyK/qfprSrJhd9ZG5VV+ZXfXetMW4qm4HjgLOBC4GTq+qtUmOTXJIu9o/ANsCn0hyfpLVUwwnzRmzqz4yt+ors6vFYJRjjKmqNcCaSde9duD8Ezqel9QJs6s+MrfqK7OrvvOX7yRJkiQsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQJgyXxPQNL8Wb7qjM7GWn/cwZ2NJUnSfHCPsSRJkoTFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAWDLfE5AkaaFZvuqMzsZaf9zBnY01qi7nD/NzH6T54B5jSZIkCYuxJEmSBFiMJUmSJMBjjCVJPePxs9Jwi+Hfxnwf3z/SHuMkBya5JMm6JKuGLN86yWnt8m8lWT7jmUhjYHbVR+ZWfWV21XfT7jFOsgVwAvBEYANwbpLVVXXRwGrPBa6rqj2SHA68FXj6OCYsjcrsqo/MrfrK7E5vvveGanqj7DHeF1hXVZdW1a3AqcChk9Y5FPhQe/6TwOOTpLtpSrNidtVH5lZ9ZXbVe6McY7wzcPnA5Q3Ao6Zap6puT3IDcC/g54MrJVkJrGwv3pzkkhnMdcfJ43Vs3OOPtI28dbzjb+I2FuJjtNtGVu1Vdhfw32Xc8x9pG3Pxb2Pc4w/chz7kFja/v8tCG3/kbYx7/J5ld7P4u9gXOu0LvzNKMR72Sq5msQ5VdSJw4gjbvPMkkvOqasVsbrsQxp+LbfR9/DFsw+z2YPy52EbPxl8QuYXePW6Lbvy52MZizK5/l8U//ji3McqhFBuAXQYuLwOumGqdJEuA7YFru5igtAnMrvrI3KqvzK56b5RifC6wZ5Ldk2wFHA6snrTOauDZ7fmnAV+pqju9ApTmmNlVH5lb9ZXZVe9NeyhFewzQUcCZwBbAB6pqbZJjgfOqajVwEvCRJOtoXvkdPoa5zvrtwAUy/lxso+/jd7oNs9ub8ediG70ZfwHlFnr0uC3S8ediG4sxu/5dFv/4Y9tGfKEmSZIk+ZPQkiRJEmAxliRJkoCeFOPpfmJyE8f+QJKrkvygy3EHxt8lyVlJLk6yNslLxrCNbZJ8O8kF7TZe3/U22u1skeR7ST47hrHXJ/l+kvOTnNf1+PNhnLltx+91dhdDbtvxze7Mxze7o23H59wZsi9Muw2zO52qWtAnmgP4/x9wP2Ar4AJg7w7HfyzwcOAHY5r/fYCHt+e3A37U5fzbcQNs257fEvgW8Ogx3JeXAR8HPjuGsdcDO851vsZ1Gndu2230OruLIbft+GZ35tswu6Ntx+fcmd0n+8L02zC705z6sMd4lJ+YnLWqOocxfodiVV1ZVd9tz98EXEzzyz9dbqOq6ub24pbtqdNPVSZZBhwMvL/LcRexseYW+p9dc7tgmd3pxze7C5N9YfptmN1p9KEYD/uJyU6DMleSLAceRvMKreuxt0hyPnAV8MWq6nob7wReCfy243EnFPCFJN9J81Ogfbdocgvjy+4iyC2Y3QXN7E5pseUWFlF27QsbNdbs9qEYj/TzkQtdkm2BTwEvraobux6/qn5TVQ+l+aWhfZM8qKuxkzwFuKqqvtPVmEPsV1UPBw4CXpTksWPc1lxYFLmF8WZ3EeQWzO6CZXY3arHlFhZJdu0L0xprdvtQjEf5ickFLcmWNCH/WFV9epzbqqrrga8CB3Y47H7AIUnW07w19adJPtrh+FTVFe1/rwL+leYtsT7rfW5h7rLb19yC2V2ozO7GLcLcwiLIrn1heuPObh+K8Sg/MblgJQnNL/1cXFX/OKZtLE2yQ3v+rsATgB92NX5VvaqqllXVcprH/ytV9cyuxk9y9yTbTZwHngSM5VO/c6jXuYXxZ7fvuQWzu1CZ3Y1bpLmFnmfXvjC9ucjugi/GVXU7MPETkxcDp1fV2q7GT3IK8A3g/kk2JHluV2O39gOeRfOq6fz29OSOt3Ef4KwkF9I8MXyxqsby1VRjcm/ga0kuAL4NnFFVn5/nOW2ScecWFkV2+55bMLuzYnbn3aLLLdgXRmR2p+FPQkuSJEn0YI+xJEmSNBcsxpIkSRIWY0mSJAmwGEuSJEmAxViSJEkCLMZzIslrkqxNcmH79SuPSvLSJHcb4bYjrSd1zdyqr8yu+srszj+/rm3MkjwG+EfggKr6dZIdga2A/wRWVNXPp7n9+lHWk7pkbtVXZld9ZXYXBvcYj999gJ9X1a8B2sA+DbgvzZdsnwWQ5J+TnNe+Unx9e92Lh6z3pCTfSPLdJJ9I85vqJDkuyUXtq8y3zf3d1CJjbtVXZld9ZXYXAPcYj1kbxK8BdwO+BJxWVWdPfmWX5J5VdW2SLYAvAy+uqgsH12tfPX4aOKiqfpHk74GtgXfT/BrPA6qqkuzQ/ga6NCvmVn1ldtVXZndhcI/xmFXVzcAjgJXA1cBpSY4csuphSb4LfA/YB9h7yDqPbq//epLzgWcDuwE3Ar8C3p/kL4Bbur4f2ryYW/WV2VVfmd2FYcl8T2BzUFW/Ab4KfDXJ92kC+jtJdgdeATyyqq5LcjKwzZChQvO75kfcaUGyL/B44HCa34r/0y7vgzY/5lZ9ZXbVV2Z3/rnHeMyS3D/JngNXPRS4DLgJ2K697h7AL4AbktwbOGhg/cH1vgnsl2SPduy7Jdmrfftl+6paA7y03YY0a+ZWfWV21Vdmd2Fwj/H4bQu8K8kOwO3AOpq3SY4APpfkyqp6XJLvAWuBS4GvD9z+xEnrHQmckmTrdvnRNP8Y/j3JNjSvEv9uLu6YFjVzq74yu+ors7sA+OE7SZIkCQ+lkCRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNps5Pk1UneP9/zmIkkz0jyhTFvY3mSSrKkvfy5JM/uaOw/SXLJwOX1SZ7QxdjteGuTHNDVeF2ZLmtJjkzytRmMN/L9XKiPiaSFLVU133OQ1KEkNw9cvBvwa+A37eUXVNXH5n5WC1+S5cCPgS2r6vYZ3K6APatq3Qxusx54XlV9aYbTJMnJwIaqOnqmt51Pwx7fJEfSPA5/POZtHwPsUVXPnMVt3wYcCuwE/AR4c1V9uNsZSloolsz3BCR1q6q2nTi/KQVsoUiyZCZFdaHp+/zFL4CnAj8CHgl8Psm6qvrP+Z2WpHHwUAppM5PkmCQfbc9PHD7wnCSXJ7kuyQuTPDLJhUmuT/LuSbf/myQXt+uemWS3KbazTZKPJrmmHefcJPdul22f5KQkVyb5SZI3JtmiXXZkkq8neUeSa4FjJr/lnuQBSb6Y5NoklyQ5bGDZk5NclOSmduxXTDG/LZK8LcnPk1wKHDxp+VeTPK89v0eSs5Pc0K5/Wnv9Oe3qFyS5OcnTkxyQZEOSv0/yU+CDE9dNmsIj23lel+SDSbYZuP93OLyg/RvtkWQl8Azgle32PtMu/92hGUm2TvLOJFe0p3cm2bpdNjG3lye5qn38nzOLx+6yJI9ozz+znd/e7eXnJfm39vzvsgZMPFbXt3N/zMB4b2sfhx8nOWjYNofcz2OSnJ7kw+181yZZMXndJAcCrwae3m73gqnGH6aqXldVP6yq31bVt4D/AB4z3e0k9ZPFWBLAo4A9gacD7wReAzwB2Ac4LMn+AEn+nKZk/AWwlKYknDLFmM8Gtgd2Ae4FvBD4ZbvsQ8DtwB7Aw4AnAc+bNJ9LgT8E3jQ4aJK7A18EPt4uPwJ4T5J92lVOojlkZDvgQcBXppjf84GntNtfATxtivUA3gB8AfgDYBnwLoCqemy7/CFVtW1VndZe3gm4J7AbsHKKMZ8B/BnwR8BewLSHRlTVicDHgOPb7T11yGqvAR4NPBR4CLDvpLF3ovm77Aw8FzghyR+0y0Z97M4GDmjPP5bmb7X/wOWzh9xm4rHaoZ37N9rLjwIuAXYEjgdOSpIptjvZIcCpwA7AauDdk1eoqs8DbwZOa7f7EIAk72lfsA07XThsY0nuSrPXeO2I85PUMxZjSQBvqKpfVdUXaN46PqWqrqqqn9CU34e1670AeEtVXdweHvBm4KFT7DW+jaYQ71FVv6mq71TVje1e44OAl1bVL6rqKuAdwOEDt72iqt5VVbdX1S8njfsUYH1VfbBd/l3gU/y+2N4G7J3kHlV1Xbt8mMOAd1bV5VV1LfCWjTw+t9GU3Pu2j9N0Hxj7LfC6qvr1kPlPePfAtt9EU/C78Azg2PbvdzXweuBZA8tva5ffVlVrgJuB+w8sG+WxO5vfF+E/oXnsJi7vz/BiPJXLqupfquo3NC+Y7gPce8Tbfq2q1rS3/QjNC4GRVNXfVtUOU5wePMXN3gtcAJw56nYk9YvFWBLAzwbO/3LI5YnjlncD/mlizxpwLRCavY+TfYSmQJzavqV/fJIt2zG2BK4cGOd9NHt/J1y+kbnuBjxqcA8fTRncqV3+l8CTgcvawx+metv7vpO2c9lGtvnK9n5+u33L/m82si7A1VX1q2nWmbzt+06z/qjuyx3vy+Sxr5l0zPMt/P7vO+pjdzbwJ0l2ArYATgP2S/MBu+2B82cw359OnKmqW9qz206x7pS3pbkf26T9VpGuJfkHmr3oh5WfWpcWLT98J2kmLgfeNMo3W1TVbTR7K1/fFqY1NG+Zr6H5powdN/KhtI0Vj8uBs6vqiVNs91zg0LaEHwWcTnM4x2RXTrp+143cl5/SHHpBkj8GvpTknI18E8UoxWnytq9oz/+C5ttEaLe3E3c03dhX0Lx4mHi7f3DsjRr1sauqdUluAV4MnFNVN7XHU6+k2Yv722HDjzKHMbnTtpO8F5jqWyouq6p9BtZ9Pc27HPtX1Y3jmaKkhcA9xpJm4r3AqyaO503zIbq/GrZikscl+R9pPlR3I83b9L+pqitpjtd9e5J7JLlLkj+aOI55BJ8F9kryrCRbtqdHJnlgkq3SfOfx9m0xv5Hff1XdZKcDL06yrD3GdtVUG0zyV0mWtRevoylaE+P+DLjfiHMf9KJ22/ekOW574vjkC4B9kjw0zQfyjpl0u+m2dwpwdJKlSXYEXgt8dCPrAzDDxw6avcZH8fvDJr466fJkV9McYjKbx2pT/QxYnuR3/8+rqhe2xxwPOw2W4lcB/wt4YlVdMw9zlzSHLMaSRlZV/wq8lebwiBuBH9DsSRtmJ+CTNAXrYprCNFHQ/hrYCriIpmh+kubY0lHmcBPNh/UOp9kT+tN2Tlu3qzwLWN/O74VMvVfwX2gO9bgA+C7w6Y1s9pHAt9J8R/Rq4CVV9eN22THAh9rDOg6baoAhPk7zAuHS9vTG9v79CDgW+BLwX8Dk45lPojkO+PqJb3+Y5I3AecCFwPfb+/bGEec06mMHzd9zO37/bROTL99Be5jEm4Cvt3N/9Ihz6sIn2v9ek2Sq46an8maave7/1X6rxc1JXt3t9CQtFP7AhyRJkoR7jCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiQA/j+rjMkXSbzVmwAAAABJRU5ErkJggg==\n",
                        "text/plain": [
                            "<Figure size 864x288 with 4 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Another sample path with init=2\n"
                    ]
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAE3CAYAAACpToeUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHzdJREFUeJzt3XuUZGV97vHvE64qCIlMRBlgMOAFPPGSEc0hUYyagBjISgzCUSOJZuJKOMZojhmVpYg3JJqYoySGaIJXLl6SjDKKN4Ro1DAqoAPBzMEhjKAgd0QF9Hf+2LuxKKqnq7t3dffu+X7WqjVVtXe9+63qh+KpXbuqUlVIkiRJ27qfWewJSJIkSUuBxViSJEnCYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxXhRJNid56hxuV0m+n+S29vTOScxPmjKPrJ6W5PIkP0ly3Ijlf5bkO0luTvKPSXbqZMJSaxLPs2m8Kcn17emUJOl25trWLMbzbJJVSc5LcnuS/5zL9pcri3H/PKqqdmlPL1jsyUjTuBj4Y+CrwwuS/AawFngKsAp4CPCahZycNIPpnmfXAL8FPAr4ReAZwB8txgQl5vc8ewbwNeABwCuBDyVZMeH59oLFeIEleS+wD/DRdm/EyxZ7TtIo88lqVZ1aVZ8Bfjhi8fOAd1XVxqq6EXgtcFwXc5Zgos+zzwPeUlVbqurbwFswu5qHxXieTfJQ4LHAq6vqB1X1YeDrwO/M794sDxbjBVZVzwX+G/jNdm/EKUlu2spp7dAQF7RvjXwkyaoFvwPaZnSQ1ekcRLOnY8rFwAOTPKDr+6Bt0wSfZ0dl96CJ3hkta4v0PHsQcEVV3Tq03CwD2y/2BARVtfuYqz4J+BJwX+B1wMeSPLqq7prY5KQBs8jq1uwC3Dxweer8rsD1HYwv3UtHz7OjsrtLklRVdTphbbMW4Hl2eNnU8r062G7vuce4R6rqgqq6o6puAv4U2A94xCJPS5qt24D7D1yeOn/riHWlBTXD8+yo7N5mKdYStLXn2eFlU8t9DsZivFju8SQ68OnnUadXzDCOn4jWJHWV1UEbaT68NOVRwHeryr3F6tIknmdHZXdjt9PWNmihn2c3Ag9JsuvQcrOMh1Islu/SfEIUgKraZaYbJDkI2IHmAPn70LzF923gsgnNUYI5ZBUgyY40L7wD7JBkZ+COqvoJ8B7g9CTvB64BTgBO73je0iSeZ98DvCTJepoy81Lgbd1OW9ugBX2erapvJrkIeHWSE4DDab5lxQ/f4R7jxfJG4IT2QPo/H/M2DwTOAm4BrqD5+pVnVNWdk5miBMwtqwCfBH4A/E/gtPb8EwGq6hPAKcB5wJXt6dVdTlpiMs+zfw98lKY4fwM4p71Omo/FeJ49BlgN3AicDDyzqq6b5/1YFuKhUZIkSZJ7jCVJkiRgjGLc/ozgtUm+Mc3yJPm/STYluSTJY7ufpjR7Zld9ZXbVR+ZWy8E4e4xPBw7byvLDgQPa0xrg7+Y/LakTp2N21U+nY3bVP6djbtVzMxbjqroAuGErqxwFvKcaXwJ2T/KgriYozZXZVV+ZXfWRudVy0MUxxnsBVw1c3oK/nqJ+MLvqK7OrPjK3WvK6+B7jUT8wMfKrLpKsoXn7hPvd736/9PCHP7yDzWtb9pWvfOV7VbVijjc3u1oU88wtjJldc6uu+Zyrvho3u10U4y3A3gOXVwJXj1qxqk6j+a49Vq9eXRs2bOhg89qWJblyHjc3u1oU88wtjJldc6uu+Zyrvho3u10cSrEO+L3206ZPAG6uqms6GFeaNLOrvjK76iNzqyVvxj3GSc4ADgX2SLKF5pdTdgCoqncA64GnA5uA24Hfn9Rkpdkwu+ors6s+MrdaDmYsxlV17AzLC/iTzmYkdcTsqq/MrvrI3Go58JfvJEmSJCzGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkoAxi3GSw5JcnmRTkrUjlu+T5LwkX0tySZKndz9VafbMrvrI3KqvzK76bsZinGQ74FTgcOBA4NgkBw6tdgJwdlU9BjgG+NuuJyrNltlVH5lb9ZXZ1XIwzh7jg4FNVXVFVd0BnAkcNbROAfdvz+8GXN3dFKU5M7vqI3OrvjK76r1xivFewFUDl7e01w06EXhOki3AeuB/jxooyZokG5JsuO666+YwXWlWzK76yNyqr8yuem+cYpwR19XQ5WOB06tqJfB04L1J7jV2VZ1WVauravWKFStmP1tpdsyu+sjcqq/MrnpvnGK8Bdh74PJK7v3Wx/OBswGq6ovAzsAeXUxQmgezqz4yt+ors6veG6cYXwgckGS/JDvSHCy/bmid/waeApDkETRB970PLTazqz4yt+ors6vem7EYV9VdwPHAucBlNJ8m3ZjkpCRHtqu9FPjDJBcDZwDHVdXw2yfSgjK76iNzq74yu1oOth9npapaT3OQ/OB1rxo4fylwSLdTk+bP7KqPzK36yuyq7/zlO0mSJAmLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEgDbL/YEJGmuVq09p9PxNp98RKfjSZL6xT3GkiRJEhZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwB/4kCRpwXX54zT+MI3UHfcYS5IkSViMJUmSJMBiLEmSJAEeYyxJkqQxLffj491jLEmSJGExliRJkgCLsSRJkgRYjCVJkiRgzGKc5LAklyfZlGTtNOscneTSJBuTfKDbaUpzY3bVR+ZWfWV21XczfitFku2AU4GnAVuAC5Osq6pLB9Y5AHg5cEhV3Zjk5yc1YWlcZld9ZG7VV2ZXy8E4e4wPBjZV1RVVdQdwJnDU0Dp/CJxaVTcCVNW13U5TmhOzqz4yt+ors6veG6cY7wVcNXB5S3vdoIcCD03yhSRfSnJYVxOU5sHsqo/MrfrK7Kr3xvmBj4y4rkaMcwBwKLAS+Lckj6yqm+4xULIGWAOwzz77zHqympxl+oXdZld9ZG7VV2ZXvTfOHuMtwN4Dl1cCV49Y51+r6s6q+hZwOU3w76GqTquq1VW1esWKFXOdszQus6s+MrfqK7Or3hunGF8IHJBkvyQ7AscA64bW+RfgyQBJ9qB5q+SKLicqzYHZVR+ZW/WV2VXvzViMq+ou4HjgXOAy4Oyq2pjkpCRHtqudC1yf5FLgPOD/VNX1k5q0NA6zqz4yt+ors6vlYJxjjKmq9cD6oeteNXC+gJe0J2nJMLvqI3OrvjK76jt/+U6SJEnCYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJGLMYJzksyeVJNiVZu5X1npmkkqzuborS3Jld9ZG5VV+ZXfXdjMU4yXbAqcDhwIHAsUkOHLHersCLgC93PUlpLsyu+sjcqq/MrpaDcfYYHwxsqqorquoO4EzgqBHrvRY4Bfhhh/OT5sPsqo/MrfrK7Kr3xinGewFXDVze0l53tySPAfauqo9tbaAka5JsSLLhuuuum/VkpVkyu+ojc6u+MrvqvXGKcUZcV3cvTH4G+GvgpTMNVFWnVdXqqlq9YsWK8WcpzY3ZVR+ZW/WV2VXvjVOMtwB7D1xeCVw9cHlX4JHA55JsBp4ArPOAei0BZld9ZG7VV2ZXvTdOMb4QOCDJfkl2BI4B1k0trKqbq2qPqlpVVauALwFHVtWGicxYGp/ZVR+ZW/WV2VXvzViMq+ou4HjgXOAy4Oyq2pjkpCRHTnqC0lyZXfWRuVVfmV0tB9uPs1JVrQfWD133qmnWPXT+05K6YXbVR+ZWfWV21Xf+8p0kSZKExViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBIz5rRSSJEnL3aq153Q21uaTj+hsLC0c9xhLkiRJWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQL8ujZJUse6/Mor8GuvJC0c9xhLkiRJWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkScCYxTjJYUkuT7IpydoRy1+S5NIklyT5TJJ9u5+qNHtmV31kbtVXZld9N2MxTrIdcCpwOHAgcGySA4dW+xqwuqp+EfgQcErXE5Vmy+yqj8yt+srsajkYZ4/xwcCmqrqiqu4AzgSOGlyhqs6rqtvbi18CVnY7TWlOzK76yNyqr8yuem+cYrwXcNXA5S3tddN5PvDx+UxK6ojZVR+ZW/WV2VXvbT/GOhlxXY1cMXkOsBp40jTL1wBrAPbZZ58xpzh/q9ae09lYm08+YsHH15z1Ortd5grMVo/0Orfappld9d44e4y3AHsPXF4JXD28UpKnAq8EjqyqH40aqKpOq6rVVbV6xYoVc5mvNBtmV31kbtVXZle9N04xvhA4IMl+SXYEjgHWDa6Q5DHA39OE/NrupynNidlVH5lb9ZXZVe/NWIyr6i7geOBc4DLg7KramOSkJEe2q/0lsAvwwSQXJVk3zXDSgjG76iNzq74yu1oOxjnGmKpaD6wfuu5VA+ef2vG8pE6YXfWRuVVfmV31nb98J0mSJGExliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEgDbL/YEJC2eVWvP6WyszScf0dlYkiQtBvcYS5IkSViMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSANh+sScgSdJsrFp7TqfjbT75iE7Hk9RfY+0xTnJYksuTbEqydsTynZKc1S7/cpJVXU9Umguzqz4yt+ors6u+m3GPcZLtgFOBpwFbgAuTrKuqSwdWez5wY1Xtn+QY4E3AsyYxYWlcZld9ZG7VV2Z3Zl2+2+E7HZMxzh7jg4FNVXVFVd0BnAkcNbTOUcC72/MfAp6SJN1NU5oTs6s+MrfqK7Or3hvnGOO9gKsGLm8BHj/dOlV1V5KbgQcA3xtcKckaYE178bYkl89irnsMj9exscbPmya7jUmPP09L4jEasu9WlpndAfP4uyzFv/tExp/0YzSgD7mFbeTvsoB/94lsw+fce9km/i7b+HPu2NsYsLXs3m2cYjzqlVzNYR2q6jTgtDG2ee9JJBuqavVcbrsUxl+IbfR9/Alsw+z2YPyF2EbPxl8SuYXePW7LbvyF2MZyzK5/l+U//iS3Mc6hFFuAvQcurwSunm6dJNsDuwE3dDFBaR7MrvrI3KqvzK56b5xifCFwQJL9kuwIHAOsG1pnHfC89vwzgc9W1b1eAUoLzOyqj8yt+srsqvdmPJSiPQboeOBcYDvgH6tqY5KTgA1VtQ54F/DeJJtoXvkdM4G5zvntwCUy/kJso+/jd7oNs9ub8RdiG70ZfwnlFnr0uC3T8RdiG8sxu/5dlv/4E9tGfKEmSZIk+ZPQkiRJEmAxliRJkoCeFOOZfmJynmP/Y5Jrk3yjy3EHxt87yXlJLkuyMcmfTmAbOyf5jyQXt9t4TdfbaLezXZKvJfnYBMbenOTrSS5KsqHr8RfDJHPbjt/r7C6H3Lbjm93Zj292x9uOz7mzZF+YcRtmdyZVtaRPNAfw/z/gIcCOwMXAgR2O/0TgscA3JjT/BwGPbc/vCnyzy/m34wbYpT2/A/Bl4AkTuC8vAT4AfGwCY28G9ljofE3qNOncttvodXaXQ27b8c3u7Ldhdsfbjs+5s7tP9oWZt2F2Zzj1YY/xOD8xOWdVdQET/A7Fqrqmqr7anr8VuIzml3+63EZV1W3txR3aU6efqkyyEjgCeGeX4y5jE80t9D+75nbJMrszj292lyb7wszbMLsz6EMxHvUTk50GZaEkWQU8huYVWtdjb5fkIuBa4FNV1fU23gq8DPhJx+NOKeCTSb6S5qdA+27Z5BYml91lkFswu0ua2Z3WcsstLKPs2he2aqLZ7UMxHuvnI5e6JLsAHwZeXFW3dD1+Vf24qh5N80tDByd5ZFdjJ3kGcG1VfaWrMUc4pKoeCxwO/EmSJ05wWwthWeQWJpvdZZBbMLtLltndquWWW1gm2bUvzGii2e1DMR7nJyaXtCQ70IT8/VX1kUluq6puAj4HHNbhsIcARybZTPPW1K8leV+H41NVV7f/Xgv8M81bYn3W+9zCwmW3r7kFs7tUmd2tW4a5hWWQXfvCzCad3T4U43F+YnLJShKaX/q5rKr+akLbWJFk9/b8fYCnAv/Z1fhV9fKqWllVq2ge/89W1XO6Gj/J/ZLsOnUe+HVgIp/6XUC9zi1MPrt9zy2Y3aXK7G7dMs0t9Dy79oWZLUR2l3wxrqq7gKmfmLwMOLuqNnY1fpIzgC8CD0uyJcnzuxq7dQjwXJpXTRe1p6d3vI0HAecluYTmieFTVTWRr6aakAcCn09yMfAfwDlV9YlFntO8TDq3sCyy2/fcgtmdE7O76JZdbsG+MCazOwN/ElqSJEmiB3uMJUmSpIVgMZYkSZKwGEuSJEmAxViSJEkCLMaSJEkSYDFeEElemWRjkkvar195fJIXJ7nvGLcdaz2pa+ZWfWV21Vdmd/H5dW0TluSXgb8CDq2qHyXZA9gR+HdgdVV9b4bbbx5nPalL5lZ9ZXbVV2Z3aXCP8eQ9CPheVf0IoA3sM4EH03zJ9nkASf4uyYb2leJr2uteNGK9X0/yxSRfTfLBNL+pTpKTk1zavsp888LfTS0z5lZ9ZXbVV2Z3CXCP8YS1Qfw8cF/g08BZVXX+8Cu7JD9XVTck2Q74DPCiqrpkcL321eNHgMOr6vtJ/gLYCXg7za/xPLyqKsnu7W+gS3NibtVXZld9ZXaXBvcYT1hV3Qb8ErAGuA44K8lxI1Y9OslXga8BBwEHjljnCe31X0hyEfA8YF/gFuCHwDuT/DZwe9f3Q9sWc6u+MrvqK7O7NGy/2BPYFlTVj4HPAZ9L8nWagN4tyX7AnwOPq6obk5wO7DxiqND8rvmx91qQHAw8BTiG5rfif63L+6Btj7lVX5ld9ZXZXXzuMZ6wJA9LcsDAVY8GrgRuBXZtr7s/8H3g5iQPBA4fWH9wvS8BhyTZvx37vkke2r79sltVrQde3G5DmjNzq74yu+ors7s0uMd48nYB3pZkd+AuYBPN2yTHAh9Pck1VPTnJ14CNwBXAFwZuf9rQescBZyTZqV1+As1/DP+aZGeaV4l/thB3TMuauVVfmV31ldldAvzwnSRJkoSHUkiSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxtM1J8ook71zsecxGkmcn+eSEt7EqSSXZvr388STP62jsX01y+cDlzUme2sXY7Xgbkxza1XhdmSlrSY5L8vlZjDf2/Vyqj4mkpS1VtdhzkNShJLcNXLwv8CPgx+3lP6qq9y/8rJa+JKuAbwE7VNVds7hdAQdU1aZZ3GYz8IKq+vQsp0mS04EtVXXCbG+7mEY9vkmOo3kcfmXC2z4R2L+qnjOH274ZOArYE/g28Iaqek+3M5S0VGy/2BOQ1K2q2mXq/HwK2FKRZPvZFNWlpu/zF98HfhP4JvA44BNJNlXVvy/utCRNgodSSNuYJCcmeV97furwgd9PclWSG5O8MMnjklyS5KYkbx+6/R8kuaxd99wk+06znZ2TvC/J9e04FyZ5YLtstyTvSnJNkm8neV2S7dplxyX5QpK/TnIDcOLwW+5JHp7kU0luSHJ5kqMHlj09yaVJbm3H/vNp5rddkjcn+V6SK4AjhpZ/LskL2vP7Jzk/yc3t+me111/Qrn5xktuSPCvJoUm2JPmLJN8B/mnquqEpPK6d541J/inJzgP3/x6HF7R/o/2TrAGeDbys3d5H2+V3H5qRZKckb01ydXt6a5Kd2mVTc3tpkmvbx//35/DYXZnkl9rzz2nnd2B7+QVJ/qU9f3fWgKnH6qZ27r88MN6b28fhW0kOH7XNEffzxCRnJ3lPO9+NSVYPr5vkMOAVwLPa7V483fijVNWrq+o/q+onVfVl4N+AX57pdpL6yWIsCeDxwAHAs4C3Aq8EngocBByd5EkASX6LpmT8NrCCpiScMc2YzwN2A/YGHgC8EPhBu+zdwF3A/sBjgF8HXjA0nyuAnwdePzhokvsBnwI+0C4/FvjbJAe1q7yL5pCRXYFHAp+dZn5/CDyj3f5q4JnTrAfwWuCTwM8CK4G3AVTVE9vlj6qqXarqrPbynsDPAfsCa6YZ89nAbwC/ADwUmPHQiKo6DXg/cEq7vd8csdorgScAjwYeBRw8NPaeNH+XvYDnA6cm+dl22biP3fnAoe35J9L8rZ40cPn8EbeZeqx2b+f+xfby44HLgT2AU4B3Jck02x12JHAmsDuwDnj78ApV9QngDcBZ7XYfBZDkb9sXbKNOl4zaWJL70Ow13jjm/CT1jMVYEsBrq+qHVfVJmreOz6iqa6vq2zTl9zHten8EvLGqLmsPD3gD8Ohp9hrfSVOI96+qH1fVV6rqlnav8eHAi6vq+1V1LfDXwDEDt726qt5WVXdV1Q+Gxn0GsLmq/qld/lXgw/y02N4JHJjk/lV1Y7t8lKOBt1bVVVV1A/DGrTw+d9KU3Ae3j9NMHxj7CfDqqvrRiPlPefvAtl9PU/C78GzgpPbvdx3wGuC5A8vvbJffWVXrgduAhw0sG+exO5+fFuFfpXnspi4/idHFeDpXVtU/VNWPaV4wPQh44Ji3/XxVrW9v+16aFwJjqao/rqrdpzn94jQ3ewdwMXDuuNuR1C8WY0kA3x04/4MRl6eOW94X+JupPWvADUBo9j4Oey9NgTizfUv/lCQ7tGPsAFwzMM7f0+z9nXLVVua6L/D4wT18NGVwz3b57wBPB65sD3+Y7m3vBw9t58qtbPNl7f38j/Yt+z/YyroA11XVD2dYZ3jbD55h/XE9mHvel+Gxrx865vl2fvr3HfexOx/41SR7AtsBZwGHpPmA3W7ARbOY73emzlTV7e3ZXaZZd9rb0tyPndN+q0jXkvwlzV70o8tPrUvLlh++kzQbVwGvH+ebLarqTpq9la9pC9N6mrfM19N8U8YeW/lQ2taKx1XA+VX1tGm2eyFwVFvCjwfOpjmcY9g1Q9fvs5X78h2aQy9I8ivAp5NcsJVvohinOA1v++r2/Pdpvk2Ednt7ck8zjX01zYuHqbf7B8feqnEfu6ralOR24EXABVV1a3s89Rqavbg/GTX8OHOYkHttO8k7gOm+peLKqjpoYN3X0LzL8aSqumUyU5S0FLjHWNJsvAN4+dTxvGk+RPe7o1ZM8uQk/yPNh+puoXmb/sdVdQ3N8bpvSXL/JD+T5BemjmMew8eAhyZ5bpId2tPjkjwiyY5pvvN4t7aY38JPv6pu2NnAi5KsbI+xXTvdBpP8bpKV7cUbaYrW1LjfBR4y5twH/Um77Z+jOW576vjki4GDkjw6zQfyThy63UzbOwM4IcmKJHsArwLet5X1AZjlYwfNXuPj+elhE58bujzsOppDTObyWM3Xd4FVSe7+f15VvbA95njUabAUvxz4X8DTqur6RZi7pAVkMZY0tqr6Z+BNNIdH3AJ8g2ZP2ih7Ah+iKViX0RSmqYL2e8COwKU0RfNDNMeWjjOHW2k+rHcMzZ7Q77Rz2qld5bnA5nZ+L2T6vYL/QHOox8XAV4GPbGWzjwO+nOY7otcBf1pV32qXnQi8uz2s4+jpBhjhAzQvEK5oT69r7983gZOATwP/BQwfz/wumuOAb5r69ochrwM2AJcAX2/v2+vGnNO4jx00f89d+em3TQxfvof2MInXA19o5/6EMefUhQ+2/16fZLrjpqfzBpq97v/VfqvFbUle0e30JC0V/sCHJEmShHuMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJAD+P+aEyqmbkOj+AAAAAElFTkSuQmCC\n",
                        "text/plain": [
                            "<Figure size 864x288 with 4 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "init = 2\n",
                "ts = [5, 10, 50, 100]\n",
                "seeds = [222, 2222]\n",
                "descriptions = ['{0} sample path with init={1}'.format(adjective, init) for adjective\n",
                "                in ['Some'] + ['Another'] + ['Yet another']*(len(seeds)-1)]\n",
                "\n",
                "for seed, description in zip(seeds, descriptions):\n",
                "    print(description)\n",
                "    plot_time_series_dists(mc1, init, ts, seed=seed)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "In fact,\n",
                "for almost every sample path of a finite Markov chain $\\{X_t\\}$,\n",
                "for some recurrent class $C$ we have\n",
                "$$\n",
                "\\frac{1}{t} \\sum_{\\tau=0}^{t-1} \\mathbf{1}\\{X_{\\tau} = s\\} \\to \\psi^C[s]\n",
                "\\quad \\text{as $t \\to \\infty$}\n",
                "$$\n",
                "for all states $s$,\n",
                "where $\\psi^C$ is the stationary distribution associated with the recurrent class $C$.\n",
                "\n",
                "If the initial state $s_0$ is a recurrent state,\n",
                "then the recurrent class $C$ above is the one that contains $s_0$,\n",
                "while if it is a transient state,\n",
                "then the recurrent class to which the convergence occurs depends on the sample path."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Let us simulate with the remaining states, `3`, `4`, and `5`."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 30,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAD0CAYAAACYReA6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHslJREFUeJzt3Xu4XHV97/H3x0BECZdaUhXCxQqoaBXbCHo4rai0JYDg0+MFjlqpaOp5ylGPWkurtYharbWKF1qlpWK9gFRtRYkFrKBCxRJQqCGlzYNRIiDhEiDI3e/5Y60tk+3svdcOk9kz2e/X88yTmbV+s37fmZ3P3t9Za81MqgpJkiRpvnvYXBcgSZIkjQIbY0mSJAkbY0mSJAmwMZYkSZIAG2NJkiQJsDGWJEmSABvjOZPkK0leMc36jyb5047bujDJqwZXnaRe5lUaL2ZWm8vGeICSrE1ySJexVbWsqj7R3u/YJBdNWv+aqnrHlqiznXOvJJVkm1ncp/PjG6QkFyRZn+T2JFckOWrYNWjrY163rCTPbmt+51zVoK2Lmd0y2nnvSrKxvZw37BpGSecfmDSHXgdcVVX3JzkQ+GqSfavq+rkuTNLPS7It8EHg23Ndi6ROnl9VX53rIkaBe4y3kIlXqEnel+TWJN9Psqxn/YVJXpXkScBHgWe1r9Q2tOtPn9jTkuQXkny53Wt6a3t9Scc6Dkiyst3b+uMk729XfaP9d0M777OSPD7J15LcnOSmJJ9OsnO7nU8CewBfase/uV3+zCT/lmRDuzf34AE8fZuoqiur6v6Jm8C2wO6Dnkfzl3kduDcC5wH/uYW2r3nOzGpLsTHesg4ErgZ2Ad4LnJYkvQOqajXwGuBbVbWoqnbus52HAR8H9qQJzl3ARzrW8EHgg1W1I/B44Kx2+W+0/+7czvstIMC7gV2BJ9E0nye2db4c+CHNq8pFVfXeJLsB5wDvBB4FvAn4fJLF/Qppf9lsmOLy5ekeRHvfu2n2QF0IrOz4+KWuzGuPzc1rkj2BVwIndXzM0uYysz0eyt9Y4NPtC4Pzkjyt42PfKtkYb1k/qKq/raoHgE8AjwUePduNVNXNVfX5qvpJVd0BvAt4dse73wfsnWSXqtpYVZdMM8+aqjq/qu6pqvXA+2eY52XAiqpaUVU/rarzaRrWw6bY/hFVtfMUlyOmexDt+h3abZ9bVT+d/mFLs2ZeN93+5ub1Q8CfVtXGGR+t9NCY2U23v7mZfSmwF80LgwuAcyf2ZM9HNsZb1g0TV6rqJ+3VRbPdSJJHJvlYkh8kuZ3mEM3OSRZ0uPtxwL7Afya5NMmU4UjyS0nOTPKjdp5P0bwSn8qewIt6X5UC/5Pml9PAVdV9VfUV4LeTHLkl5tC8Zl4foiTPB3aoqs8OapvSNMzsAFTVxVV1V/vC4N3ABuDXBznHOPHNd6OhZlj/RuAJwIFVdUOS/YHv0ByWmX7DVf8NHJPkYcDvAJ9L8otTzPnudvlTq+rmJC9g08NJk+9zLfDJqnr1THVA8/E5TB22b1bVsinWTbYNzSEraS6Y16nz+jxgaZKJhmUn4IEkv1JVfpqM5oqZnd3f2KLDY99aucd4NPwYWJJk4RTrd6A552lDkkcBf9Z1w0lelmRxe+rBhnbxA8B64KfAL0+aZ2M7z27AH/aps3f8p4DnJ/ntJAuSbJfk4EzxpoVqPj5n0RSXvoFN8sQky5I8Ism2SV5Gc+7W17s+B9KAmdep/8D+Kc3es/3by9nA3wK/1/U5kLYAMzv139g9khyUZGG7/T+k2Yt9cdfnYGtjYzwavgasAm5IclOf9ScDjwBuAi4B/mUW2z4UWJVkI82bBI6uqrvbw07vAi5uD9E8E3g78KvAbTQn/H9h0rbeDby1Hf+mqroWOAr4E5pfAtfSBH2Q/69C8+aEG9s5Xge8pKouH+Ac0myY1ylU1R1VdcPEhabZuLOqbhnUHNJmMLNT2wH4G+BW4Eft41lWVTcPcI6xkqqZjjBIkiRJWz/3GEuSJEl0aIyT/H2SG5N8b4r1SfKhJGuSXJnkVwdfpqSuzKw0PsyrNFq67DE+neack6ksA/ZpL8tpzlWRNHdOx8xK4+J0zKs0MmZsjKvqG8B0b5w4CviHalxC89l/W+RzbCXNzMxK48O8SqNlEOcY70bzTskJ69plkkaTmZXGh3mVhmgQX/DR70Og+37URZLlNIeC2H777X/tiU984gCml7YOl1122U1VtXgIU3XKrHmVpjZqeQUzK02na2YH0RivA3bvub0EuK7fwKo6FTgVYOnSpbVy5coBTC9tHZL8YEhTdcqseZWmNmp5BTMrTadrZgfRGJ8NHJ/kTOBA4Laqun42G9jrhHMGUEY3a99z+NDmkkbUQ86spKExr9IQzdgYJzkDOBjYJck6mq9K3Bagqj4KrAAOA9YAP8Gv/pTmlJmVxscw8jqsnU/ueNLWYMbGuKqOmWF9AX8wsIokPSRmVhof5lUaLX7znSRJkoSNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBHRsjJMcmuTqJGuSnNBn/R5JLkjynSRXJjls8KVK6sK8SuPFzEqjY8bGOMkC4BRgGbAfcEyS/SYNeytwVlU9HTga+OtBFyppZuZVGi9mVhotXfYYHwCsqaprqupe4EzgqEljCtixvb4TcN3gSpQ0C+ZVGi9mVhoh23QYsxtwbc/tdcCBk8acCJyX5P8C2wOHDKQ6SbNlXqXxYmalEdJlj3H6LKtJt48BTq+qJcBhwCeT/Ny2kyxPsjLJyvXr18++WkkzMa/SeDGz0gjp0hivA3bvub2Enz+McxxwFkBVfQvYDthl8oaq6tSqWlpVSxcvXrx5FUuajnmVxouZlUZIl8b4UmCfJI9LspDmxP+zJ435IfA8gCRPogmtL1el4TOv0ngxs9IImbExrqr7geOBc4HVNO+MXZXkpCRHtsPeCLw6yRXAGcCxVTX5UJCkLcy8SuPFzEqjpcub76iqFcCKScve1nP9KuCgwZYmaXOYV2m8mFlpdPjNd5IkSRI2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAGwzVwXIEkaX3udcM5Q5ln7nsOHMo+k+c09xpIkSRI2xpIkSRJgYyxJkiQBHRvjJIcmuTrJmiQnTDHmxUmuSrIqyWcGW6akrsyrND7MqzRaZnzzXZIFwCnAbwLrgEuTnF1VV/WM2Qf4Y+Cgqro1yS9tqYIlTc28SuPDvEqjp8se4wOANVV1TVXdC5wJHDVpzKuBU6rqVoCqunGwZUrqyLxK48O8SiOmS2O8G3Btz+117bJe+wL7Jrk4ySVJDh1UgZJmxbxK48O8SiOmy+cYp8+y6rOdfYCDgSXAN5M8pao2bLKhZDmwHGCPPfaYdbGSZmRepfExsLyCmZUGocse43XA7j23lwDX9Rnzxaq6r6q+D1xNE+RNVNWpVbW0qpYuXrx4c2uWNDXzKo2PgeUVzKw0CF0a40uBfZI8LslC4Gjg7Elj/hl4DkCSXWgO/VwzyEIldWJepfFhXqURM2NjXFX3A8cD5wKrgbOqalWSk5Ic2Q47F7g5yVXABcAfVtXNW6poSf2ZV2l8mFdp9HQ5x5iqWgGsmLTsbT3XC3hDe5E0h8yrND7MqzRa/OY7SZIkCRtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJ6NgYJzk0ydVJ1iQ5YZpxL0xSSZYOrkRJs2FepfFiZqXRMWNjnGQBcAqwDNgPOCbJfn3G7QC8Fvj2oIuU1I15lcaLmZVGS5c9xgcAa6rqmqq6FzgTOKrPuHcA7wXuHmB9kmbHvErjxcxKI6RLY7wbcG3P7XXtsp9J8nRg96r68gBrkzR75lUaL2ZWGiFdGuP0WVY/W5k8DPgA8MYZN5QsT7Iyycr169d3r1JSV+ZVGi9mVhohXRrjdcDuPbeXANf13N4BeApwYZK1wDOBs/u9OaCqTq2qpVW1dPHixZtftaSpmFdpvJhZaYR0aYwvBfZJ8rgkC4GjgbMnVlbVbVW1S1XtVVV7AZcAR1bVyi1SsaTpmFdpvJhZaYTM2BhX1f3A8cC5wGrgrKpaleSkJEdu6QIldWdepfFiZqXRsk2XQVW1Algxadnbphh78EMvS9LmMq/SeDGz0ujwm+8kSZIkbIwlSZIkwMZYkiRJAjqeYyxJo2avE84Zyjxr33P4UOaRJM099xhLkiRJ2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJQMfGOMmhSa5OsibJCX3WvyHJVUmuTPKvSfYcfKmSujCv0vgwr9JombExTrIAOAVYBuwHHJNkv0nDvgMsraqnAp8D3jvoQiXNzLxK48O8SqOnyx7jA4A1VXVNVd0LnAkc1Tugqi6oqp+0Ny8Blgy2TEkdmVdpfJhXacR0aYx3A67tub2uXTaV44Cv9FuRZHmSlUlWrl+/vnuVkroyr9L4GFhewcxKg9ClMU6fZdV3YPIyYCnwl/3WV9WpVbW0qpYuXry4e5WSujKv0vgYWF7BzEqDsE2HMeuA3XtuLwGumzwoySHAW4BnV9U9gylP0iyZV2l8mFdpxHTZY3wpsE+SxyVZCBwNnN07IMnTgY8BR1bVjYMvU1JH5lUaH+ZVGjEzNsZVdT9wPHAusBo4q6pWJTkpyZHtsL8EFgH/mOS7Sc6eYnOStiDzKo0P8yqNni6nUlBVK4AVk5a9ref6IQOuS9JmMq/S+DCv0mjxm+8kSZIkbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQI6NsZJDk1ydZI1SU7os/7hST7brv92kr0GXaikbsyrNF7MrDQ6ZmyMkywATgGWAfsBxyTZb9Kw44Bbq2pv4APAXwy6UEkzM6/SeDGz0mjZpsOYA4A1VXUNQJIzgaOAq3rGHAWc2F7/HPCRJKmqGmCtGoK9TjhnKPOsfc/hQ5lnHjKv0ngxs/PEsP6+wmj/jR31PqPLqRS7Adf23F7XLus7pqruB24DfnGzKpL0UJhXabyYWWmEdNljnD7LJr9K7TKGJMuB5e3NjUmu7jD/VHYBbprtnTLYA1CbVcOAjWUNA/45bFYNW8BDrWHPAdQwqnmFMf0Z+TtjNGoYwd8Zg8grjG5mt4af0bytwd9bfZ+DTpnt0hivA3bvub0EuG6KMeuSbAPsBNwyeUNVdSpwapfCZpJkZVUtHcS2rMEatpYaGNG8wmg8P3Ndw1zPbw2jVUNrJDM7Cs+PNYxGDXM9/7Br6HIqxaXAPkkel2QhcDRw9qQxZwOvaK+/EPia5z5Jc8K8SuPFzEojZMY9xlV1f5LjgXOBBcDfV9WqJCcBK6vqbOA04JNJ1tC8ij16SxYtqT/zKo0XMyuNli6nUlBVK4AVk5a9ref63cCLBlvajAZ2iPchsIaGNTRGoYZRzSuMxvMz1zXM9fxgDRNGoQZgZDM7Cs+PNTTmuoa5nh+GWEM8GiNJkiT5ldCSJEkSMKaN8UxfnzmE+f8+yY1JvjfsuXtq2D3JBUlWJ1mV5HVzUMN2Sf49yRVtDW8fdg1tHQuSfCfJl+do/rVJ/iPJd5OsnIsaRtlc57WtYU4za143qWNO89rWYGanMdeZNa+jk9e2lnn1N3bsTqVovz7zv4DfpPkIm0uBY6rqqmnvONgafgPYCPxDVT1lWPNOquGxwGOr6vIkOwCXAS8Y8vMQYPuq2phkW+Ai4HVVdcmwamjreAOwFNixqo4Y5tzt/GuBpVU115/zOHJGIa9tHXOaWfO6SR1zmte2hrWY2b5GIbPmdXTy2tYyr/7GjuMe4599fWZV3QtMfH3m0FTVN+jzGZJDruH6qrq8vX4HsJqf/7akLV1DVdXG9ua27WWor7SSLAEOB/5umPOqsznPK8x9Zs1rw7yOhTnPrHkdjbzC/MzsODbGXb4+c15JshfwdODbczD3giTfBW4Ezq+qYddwMvBm4KdDnrdXAecluSzNN0/pQeZ1EvM653kFMzsdM9tjnucVRiOzQ83rODbGnb4ac75Isgj4PPD6qrp92PNX1QNVtT/NtzUdkGRoh72SHAHcWFWXDWvOKRxUVb8KLAP+oD0MqIZ57WFeRyKvYGanY2Zb8zmvMFKZHWpex7Ex7vL1mfNCe97R54FPV9UX5rKWqtoAXAgcOsRpDwKObM8/OhN4bpJPDXF+AKrquvbfG4F/ojkUqYZ5bZnX0cgrmNkZmFnMa2skMjvsvI5jY9zl6zO3eu2J+acBq6vq/XNUw+IkO7fXHwEcAvznsOavqj+uqiVVtRfN/4OvVdXLhjU/QJLt2zdnkGR74LeAOfu0khFkXjGvMBp5BTPbwbzPrHltjEJm5yKvY9cYV9X9wMTXZ64GzqqqVcOsIckZwLeAJyRZl+S4Yc7fOgh4Oc0ruO+2l8OGXMNjgQuSXEnzy/T8qpqzj2CaI48GLkpyBfDvwDlV9S9zXNPIGIW8wkhk1ryODjM7jVHIrHkFzOuEoed17D6uTZIkSdoSxm6PsSRJkrQl2BhLkiRJ2BhLkiRJgI2xJEmSBNgYS5IkSYCN8VYjyVuSrEpyZfvRMgcmeX2SR3a4b6dxkgbHzErjw7zOH35c21YgybOA9wMHV9U9SXYBFgL/BiytqptmuP/aLuMkDYaZlcaHeZ1f3GO8dXgscFNV3QPQhu+FwK40HxB+AUCSv0mysn3V+/Z22Wv7jPutJN9KcnmSf2y/L54k70lyVfuK+X3Df5jSVsPMSuPDvM4j7jHeCrShugh4JPBV4LNV9fXJr1KTPKqqbkmyAPhX4LVVdWXvuPaV8BeAZVV1Z5I/Ah4OfITmm4ieWFWVZOf2+9slzZKZlcaHeZ1f3GO8FaiqjcCvAcuB9cBnkxzbZ+iLk1wOfAd4MrBfnzHPbJdfnOS7wCuAPYHbgbuBv0vyO8BPBv04pPnCzErjw7zOL9vMdQEajKp6ALgQuDDJf9CE7WeSPA54E/CMqro1yenAdn02FZrvZD/m51YkBwDPA44GjgeeO8jHIM0nZlYaH+Z1/nCP8VYgyROS7NOzaH/gB8AdwA7tsh2BO4HbkjwaWNYzvnfcJcBBSfZut/3IJPu2h5J2qqoVwOvbOSRtBjMrjQ/zOr+4x3jrsAj4cJKdgfuBNTSHfI4BvpLk+qp6TpLvAKuAa4CLe+5/6qRxxwJnJHl4u/6tNMH+YpLtaF7x/r9hPDBpK2VmpfFhXucR33wnSZIk4akUkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xtK8l+RPkvzdXNcxG0lemuS8LTzHXkkqyTbt7a8kecWAtv3rSa7uub02ySGD2Ha7vVVJDh7U9jrO+c4kNyW5YZjzStIg+c130lYuycaem48E7gEeaG//flV9evhVjb4kewHfB7atqvtncb8C9qmqNbO4z1rgVVX11VmWSZLTgXVV9dbZ3ndQkuwO/BewZ1XdOKBtzvp5bO+3EPgMsBTYE3hOVV3Ysz7Ae4BXtYtOA/6o2j+GSfZvlz0JWA0cV1XffWiPRtK4cI+xtJWrqkUTF+CHwPN7lo1dUzyxB3dcjXv9U9gTuHlzmuIt9HxcBLwM6Lf3ejnwAuBpwFOBI4Dfb2tZCHwR+BTwC8AngC+2yyXNAzbG0jyX5MQkn2qvT5w+8HtJrk1ya5LXJHlGkiuTbEjykUn3f2WS1e3Yc5PsOcU82yX5VJKb2+1cmuTR7bqdkpyW5PokP2oPyy9o1x2b5OIkH0hyC3Biu+yinm0/Mcn5SW5JcnWSF/esOyzJVUnuaLf9pinqW5Dkfe3pANcAh09af2GSV7XX907y9SS3teM/2y7/Rjv8iiQbk7wkycFJ1iX5o/Y0g49PLJtUwjPaOm9N8vEk2/U8/ot6B7Y/o72TLAdeCry5ne9L7fqfnZqR5OFJTk5yXXs5OcnD23UTtb0xyY3t8/97s3nu2nnOB3Ztazi9XX5ke0rHhva5e1LPfda2z8eVwJ2Tm+N+z2O/n1k/VXVvVZ1cVRfx4JGRXq8A/qqq1lXVj4C/Ao5t1x0MbAOcXFX3VNWHgADP7Tq/pPFmYyypnwOBfYCXACcDbwEOAZ4MvDjJswGSvAD4E+B3gMXAN4EzptjmK4CdgN2BXwReA9zVrvsEcD+wN/B04Ld48FD3RD3XAL8EvKt3o0m2p2nMPtOuPwb46yRPboecRnPKyA7AU4CvTVHfq2n2Hj6d5jD8C6cYB/AO4DyavYpLgA8DVNVvtOuf1u6R/2x7+zHAo2j2rC6fYpsvBX4beDywLzDjqRFVdSrwaeC97XzP7zPsLcAzgf1p9pIeMGnbj6H5uewGHAeckuQX2nUzPnft6R/LgOvaGo5Nsi/N/4PX0/y/WAF8adKe12NoXnzsPPlUlX7PY5I92iZ7qsv/nun5aj0ZuKLn9hXtsol1V06cVtG6sme9pK2cjbGkft5RVXdX1XnAncAZVXVju4ftmzTNIzSHoN9dVavb5ubPgf2n2Gt8H01DvHdVPVBVl1XV7e1e42XA66vqzvZw/AeAo3vue11Vfbiq7q+quyZt9whgbVV9vF1/OfB5Hmxs7wP2S7JjVd3aru/nxTR7Cq+tqluAd0/z/NxH0+Tu2j5PF00zFuCnwJ+1eyEn1z/hIz1zv4umcRyElwIntT+/9cDbgZf3rL+vXX9fVa0ANgJP6FnX5bmb7CXAOVV1flXdB7wPeATwP3rGfKh9vFM9H5uoqh9W1c7TXD7TsbZFwG09t28DFiVJn3UT63fouG1JY87GWFI/P+65flef24va63sCH5zYawfcQnPoebc+2/wkcC5wZntI/71Jtm23sS1wfc92Pkaz93fCtdPUuidwYO/eQ5pm8DHt+v8FHAb8oD394VlTbGfXSfP8YJo539w+zn9vTxd45TRjAdZX1d0zjJk8964zjO9qVzZ9LJO3ffOkPbY/4cGfb9fnbto5q+qnNI+v9//FdD/TLWkjsGPP7R2Bje1e4snrJtbfMaTaJM0xG2NJD8W1NIfae/fcPaKq/m3ywHaP5Nuraj+aPYdHAL/bbuMeYJeebexYVb2Hr6f7+Jxrga9PqmFRVf2fdt5Lq+oomkb7n4GzptjO9TSneUzYY6oJq+qGqnp1Ve1Ks9f8r5PsPU2NXT7+Z/Lc17XX76T5NBEAkjyGTc207etoXjz02/a0ZvHcTTtnuzd2d+BHvZvvuK2JbezRnm881eWlHTe1iuaUkglPa5dNrHtqW++Ep/asl7SVszGW9FB8FPjjifN507yJ7kX9BiZ5TpJfSfOmuttpDtM/UFXX05yv+1dJdkzysCSPnziPuYMvA/smeXmSbdvLM5I8KcnCNJ95vFN7SP92+r8hC5qm77VJlrTn2J4w1YRJXpRkSXvzVpomb2K7PwZ+uWPtvf6gnftRNOdtT5yffAXw5CT7t2/IO3HS/Waa7wzgrUkWJ9kFeBvNpy5Ma5bP3WRnAYcneV57VOCNNC9+fu4F0zQ2eVztqRSLprn87BNW2jccbtfeXJjmjZ8Tze4/AG9IsluSXdvaTm/XXdg+xte22zi+XT7VeemStjI2xpI2W1X9E/AXNKdH3A58j+Z84X4eA3yOpsFaDXydBxu03wUWAlfRNJqfAx7bsYY7aN6sdzTNnsob2poe3g55ObC2re81NB/j1c/f0pzqcQVwOfCFaaZ9BvDtNJ8RfTbwuqr6frvuROAT7WkdL55qA318huYFwjXt5Z3t4/sv4CTgq8B/03wUWa/TaM4D3pDkn/ts953ASpo3kf1H+9je2bGmrs/dJqrq6nbsh4GbgOfTfEzgvR3nhc1/HgGupjnlZzean+ldPLgH+2PAl2iei+8B57TLaOt7Ac3/xw3AK4EXzLJuSWPML/iQJEmScI+xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTA/wfp3WJvojZoOQAAAABJRU5ErkJggg==\n",
                        "text/plain": [
                            "<Figure size 864x216 with 3 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "inits = [3, 4, 5]\n",
                "t = 100\n",
                "\n",
                "fig, axes = plt.subplots(1, 3, figsize=(12, 3))\n",
                "\n",
                "for init, ax in zip(inits, axes):\n",
                "    draw_histogram(time_series_dist(mc1, t=t, init=init), ax=ax,\n",
                "                   title='Initial state = {0}'.format(init),\n",
                "                   xlabel='States')\n",
                "\n",
                "fig.suptitle('Time series distributions for t={0}'.format(t),\n",
                "             y=-0.05, fontsize=12)\n",
                "plt.show()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "#### Cross sectional averages"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Next, let us repeat the simulation for many times (say, 10,000 times)\n",
                "and obtain the distribution of visits to each state at a given time period `T`.\n",
                "That is, we want to simulate the marginal distribution at time `T`."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 31,
            "metadata": {},
            "outputs": [],
            "source": [
                "def cross_sectional_dist(mc, T, init=None, num_reps=10**4, random_state=None):\n",
                "    \"\"\"\n",
                "    Return the distribution of visits at time T by num_reps times of simulation\n",
                "    of mc with an initial state init.\n",
                "    \n",
                "    \"\"\"\n",
                "    T_max = np.max(T)\n",
                "    dim = 1\n",
                "    try:\n",
                "        Ts_size = len(T)  # T is an array\n",
                "        Ts_array = T\n",
                "        dim = 2\n",
                "    except:  # T is an int\n",
                "        Ts_size = 1\n",
                "        Ts_array = [T]\n",
                "    \n",
                "    x = mc.simulate(ts_length=T_max+1, init=init, num_reps=num_reps,\n",
                "                    random_state=random_state)[:, Ts_array]\n",
                "    dists = np.empty((x.shape[-1], mc.n))\n",
                "    bins = np.arange(mc.n+1)\n",
                "    for i in range(x.shape[-1]):\n",
                "        hist, bin_edges = np.histogram(x[:, i], bins=bins)\n",
                "        dists[i, :] = hist / num_reps\n",
                "    if dim == 1:\n",
                "        return dists[0]\n",
                "    else:\n",
                "        return dists"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Start with state `1`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 32,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([0.    , 0.3352, 0.    , 0.    , 0.6648, 0.    ])"
                        ]
                    },
                    "execution_count": 32,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "init = 1\n",
                "T = 10\n",
                "cross_sectional_dist(mc1, init=init, T=T)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 33,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([0.    , 0.3312, 0.    , 0.    , 0.6688, 0.    ])"
                        ]
                    },
                    "execution_count": 33,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "T = 100\n",
                "cross_sectional_dist(mc1, init=init, T=T)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The distribution is close to the stationary distribution `(0, 1/3, 0, 0, 2/3, 0)`."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Plot the simulated marginal distribution at `T` for some values of `T`."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 34,
            "metadata": {},
            "outputs": [],
            "source": [
                "def plot_cross_sectional_dists(mc, init, Ts, num_reps=10**4, seed=None,\n",
                "                               figsize=(12,4)):\n",
                "    dists = cross_sectional_dist(mc, T=Ts, init=init, num_reps=num_reps,\n",
                "                                 random_state=seed)\n",
                "\n",
                "    fig, axes = plt.subplots(1, len(Ts), figsize=figsize)\n",
                "    titles = ['T={0}'.format(T) for T in Ts]\n",
                "\n",
                "    for ax, title, dist in zip(axes, titles, dists):\n",
                "        draw_histogram(dist, ax=ax, title=title, xlabel='States')\n",
                "\n",
                "    fig.suptitle('Cross sectional distributions with init={0}'.format(init),\n",
                "                 y=-0.05, fontsize=12)\n",
                "    plt.show()"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 35,
            "metadata": {
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAE3CAYAAACpToeUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAH7xJREFUeJzt3XuUpHV95/H3R4ZLuEcYjVzGQcXLyDFKRtTFC4rrMqDgrhwXRFcMStzIuizexigE8UZMjnGz4gWNwXgB0aCZyBA0CgiIwggIDmTMBAcZxggoF0Hl5nf/eJ7RoqZ7urqnqrufnvfrnDp0Vf3q9/yq6jPNp556qitVhSRJkrS5e9hML0CSJEmaDSzGkiRJEhZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi/G0SXJ3z+k3SX7Vc/6oSczz6iTfS3JXkrVJPpBk3ijXrs3bELN7RJJVSe5MckuSTyfZcZRr1+ZtiNk9OsmDffMdMMKlazMzxKzuk+T8JLcl2eCLKpI8PMmXk9yT5MYkrxjuPek+i/E0qart15+AHwMv6bnsc5OYalvgeGBX4BnAgcCbh79iqTHE7F4K7F9VOwGPAeYB7xnBkiVgqNkFuKx3vqq6cPgr1uZqiFm9HzgbOGac608D7gMeCRwFfDTJkzdl7XONexo7pqo+2nP25iSfA54/U+uRBlVVN/Vd9CDwuJlYiyTNRVW1CliVZIPfrUm2A14G7FNVdwOXJFkGvApYOr0rnb3cYzxLJHlFkjs2clowzk2fC6yczrVKvSaT3STPTnIn8AuaX9AfmrGFa7M3yd+7T2vfnv5hkhM9hE3TaRM6Qq/HAw9W1Q97Lvs+4B7jHv7DniWq6vPA5ydzmySvARYDrx3JoqQBTCa7VXUJsFOS3YHXAWtGuDRpoyaR3W8B+wA30pSILwAPAO8f3eqk35lKRxjD9sCdfZfdCeywifPOKe4x7qgkLwVOBZZU1W0zvR5pMqrqZuCfgbNmei3SRKrqhqr6UVX9pqquBU4BDp/pdUmTdDfQ/4HnHWnewVPLYjxLJDmq71Op/afet6MPAj5Bc3D+tTO3amly2e0zD3jsdK5V6rUJ2S0g07lWbd42Iau9fgjMS7J3z2V/iIdjPoSHUswS7adOJ/zkaZIXtOP+a1VdPvKFSROYRHaPAi4GbgIWAO8FvjHa1Unjm0R2lwBXVtVPkzwROBH44qjXJ603iawG2BrYqj2/TXPzureq7klyDnBKktcCTwUOA/7T6FbePe4x7p4TgZ2A5T2vFM+b6UVJA1gEfJvm7bxLgVU0xxlLs92BwDVJ7gGWA+cA75vZJUljejTwK363F/hXNL9r1/tT4PeAW4Azgf9ZVe4x7pGqDf7+syRJkrTZcY+xJEmSxADFOMmn0nx96w/GuT5J/ibJ6iTXJNl3+MuUJs/sqqvMrrrI3GouGGSP8RnAQRu5fgmwd3s6FvjoRsZK0+kMzK666QzMrrrnDMytOm7CYlxV3wJ+vpEhhwF/X43vADsnedSwFihNldlVV5lddZG51VwwjGOMd6f580vrrW0vk2Y7s6uuMrvqInOrWW8Yf8d4rD9yPuafukhyLM3bJ2y33XZ/9MQnPnEIm9fm7Hvf+95tVTV/ijc3u5oRm5hbGDC75lbD5u9cddWg2R1GMV4L7Nlzfg9g3VgDq+p04HSAxYsX14oVK4aweW3Okty4CTc3u5oRm5hbGDC75lbD5u9cddWg2R3GoRTLgP/Rftr0mcCdVfWTIcwrjZrZVVeZXXWRudWsN+Ee4yRnAgcAuyZZC/w5sCVAVX2M5luADgZWA78EXjOqxUqTYXbVVWZXXWRuNRdMWIyr6sgJri/gDUNbkTQkZlddZXbVReZWc4HffCdJkiRhMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEDFiMkxyUZFWS1UmWjnH9giQXJLkqyTVJDh7+UqXJM7vqInOrrjK76roJi3GSLYDTgCXAIuDIJIv6hr0TOLuqngYcAXxk2AuVJsvsqovMrbrK7GouGGSP8X7A6qq6oaruA84CDusbU8CO7c87AeuGt0RpysyuusjcqqvMrjpvkGK8O3BTz/m17WW9TgZemWQtsBz4X2NNlOTYJCuSrLj11lunsFxpUsyuusjcqqvMrjpvkGKcMS6rvvNHAmdU1R7AwcBnkmwwd1WdXlWLq2rx/PnzJ79aaXLMrrrI3KqrzK46b5BivBbYs+f8Hmz41scxwNkAVXUZsA2w6zAWKG0Cs6suMrfqKrOrzhukGF8B7J1kryRb0Rwsv6xvzI+BAwGSPIkm6L73oZlmdtVF5lZdZXbVeRMW46p6ADgOOB+4nubTpCuTnJLk0HbYm4DXJfk+cCZwdFX1v30iTSuzqy4yt+oqs6u5YN4gg6pqOc1B8r2XndTz83XA/sNdmrTpzK66yNyqq8yuus5vvpMkSZKwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJGDAv2MsSbPRwqXnDnW+NaceMtT5JEnd4h5jSZIkCYuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJADmzfQCJEna3Cxceu7Q5lpz6iFDm0va3LnHWJIkScJiLEmSJAEWY0mSJAnwGGNJkiQNaK4fH+8eY0mSJAmLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgQMWIyTHJRkVZLVSZaOM+blSa5LsjLJ54e7TGlqzK66yNyqq8yuum7eRAOSbAGcBvxnYC1wRZJlVXVdz5i9gbcD+1fV7UkeMaoFS4Myu+oic6uuMruaCwbZY7wfsLqqbqiq+4CzgMP6xrwOOK2qbgeoqluGu0xpSsyuusjcqqvMrjpvkGK8O3BTz/m17WW9Hg88PsmlSb6T5KBhLVDaBGZXXWRu1VVmV5034aEUQMa4rMaYZ2/gAGAP4OIk+1TVHQ+ZKDkWOBZgwYIFk16sNElmV11kbtVVZledN8ge47XAnj3n9wDWjTHmH6vq/qr6EbCKJvgPUVWnV9Xiqlo8f/78qa5ZGpTZVReZW3WV2VXnDVKMrwD2TrJXkq2AI4BlfWO+AjwfIMmuNG+V3DDMhUpTYHbVReZWXWV21XkTFuOqegA4DjgfuB44u6pWJjklyaHtsPOBnyW5DrgAeEtV/WxUi5YGYXbVReZWXWV2NRcMcowxVbUcWN532Uk9PxdwQnuSZg2zqy4yt+oqs6uu85vvJEmSJCzGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAmDeTC9gOixceu7Q5lpz6iFDm2s28TGSJEmbO/cYS5IkSViMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBAxYjJMclGRVktVJlm5k3OFJKsni4S1Rmjqzqy4yt+oqs6uum7AYJ9kCOA1YAiwCjkyyaIxxOwBvBL477EVKU2F21UXmVl1ldjUXDLLHeD9gdVXdUFX3AWcBh40x7t3AB4BfD3F90qYwu+oic6uuMrvqvEGK8e7ATT3n17aX/VaSpwF7VtVXNzZRkmOTrEiy4tZbb530YqVJMrvqInOrrjK76rxBinHGuKx+e2XyMOCvgTdNNFFVnV5Vi6tq8fz58wdfpTQ1ZlddZG7VVWZXnTdIMV4L7Nlzfg9gXc/5HYB9gAuTrAGeCSzzgHrNAmZXXWRu1VVmV503SDG+Atg7yV5JtgKOAJatv7Kq7qyqXatqYVUtBL4DHFpVK0ayYmlwZlddZG7VVWZXnTdhMa6qB4DjgPOB64Gzq2plklOSHDrqBUpTZXbVReZWXWV2NRfMG2RQVS0HlvdddtI4Yw/Y9GVJw2F21UXmVl1ldtV1fvOdJEmShMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAgb8SmhJkqS5buHSc4c215pTDxnaXJo+7jGWJEmSsBhLkiRJgIdSSJKGbJhvR4NvSUuaPu4xliRJkrAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRIA82Z6AZLGtnDpuUOdb82phwx1PkmS5hr3GEuSJElYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEjBgMU5yUJJVSVYnWTrG9SckuS7JNUm+keTRw1+qNHlmV11kbtVVZlddN2ExTrIFcBqwBFgEHJlkUd+wq4DFVfUU4EvAB4a9UGmyzK66yNyqq8yu5oJB9hjvB6yuqhuq6j7gLOCw3gFVdUFV/bI9+x1gj+EuU5oSs6suMrfqKrOrzhukGO8O3NRzfm172XiOAc7blEVJQ2J21UXmVl1ldtV58wYYkzEuqzEHJq8EFgPPG+f6Y4FjARYsWDDgEqUpM7vqInOrrjK76rxB9hivBfbsOb8HsK5/UJIXAu8ADq2qe8eaqKpOr6rFVbV4/vz5U1mvNBlmV11kbtVVZledN0gxvgLYO8leSbYCjgCW9Q5I8jTg4zQhv2X4y5SmxOyqi8ytusrsqvMmLMZV9QBwHHA+cD1wdlWtTHJKkkPbYX8JbA98McnVSZaNM500bcyuusjcqqvMruaCQY4xpqqWA8v7Ljup5+cXDnld0lCYXXWRuVVXmV11nd98J0mSJGExliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgQMWIyTHJRkVZLVSZaOcf3WSb7QXv/dJAuHvVBpKsyuusjcqqvMrrpuwmKcZAvgNGAJsAg4MsmivmHHALdX1eOAvwb+YtgLlSbL7KqLzK26yuxqLhhkj/F+wOqquqGq7gPOAg7rG3MY8On25y8BBybJ8JYpTYnZVReZW3WV2VXnzRtgzO7ATT3n1wLPGG9MVT2Q5E5gF+C23kFJjgWObc/enWTVJNa6a/98QzbQ/Nm017az4j6Mev5pfowevZHrzG6PTXheRr3+6djGbHuMupBb2Pyel9k2/0Db8HfuBjbH56WT88+i7P7WIMV4rFdyNYUxVNXpwOkDbHPDRSQrqmrxVG47G+afjm10ff4RbMPsdmD+6dhGx+afFbmFzj1uc27+6djGXMyuz8vcn3+U2xjkUIq1wJ495/cA1o03Jsk8YCfg58NYoLQJzK66yNyqq8yuOm+QYnwFsHeSvZJsBRwBLOsbswx4dfvz4cA3q2qDV4DSNDO76iJzq64yu+q8CQ+laI8BOg44H9gC+FRVrUxyCrCiqpYBfwt8Jslqmld+R4xgrVN+O3CWzD8d2+j6/EPdhtntzPzTsY3OzD+Lcgsdetzm6PzTsY25mF2fl7k//8i2EV+oSZIkSX7znSRJkgRYjCVJkiSgI8V4oq+Y3MS5P5XkliQ/GOa8PfPvmeSCJNcnWZnkf49gG9skuTzJ99ttvGvY22i3s0WSq5J8dQRzr0lybZKrk6wY9vwzYZS5befvdHbnQm7b+c3u5Oc3u4Ntx9+5k2RfmHAbZnciVTWrTzQH8P878BhgK+D7wKIhzv9cYF/gByNa/6OAfdufdwB+OMz1t/MG2L79eUvgu8AzR3BfTgA+D3x1BHOvAXad7nyN6jTq3Lbb6HR250Ju2/nN7uS3YXYH246/cyd3n+wLE2/D7E5w6sIe40G+YnLKqupbjPBvKFbVT6rqyvbnXwDX03zzzzC3UVV1d3t2y/Y01E9VJtkDOAT45DDnncNGmlvofnbN7axldiee3+zOTvaFibdhdifQhWI81ldMDjUo0yXJQuBpNK/Qhj33FkmuBm4Bvl5Vw97Gh4C3Ar8Z8rzrFfC1JN9L81WgXTdncgujy+4cyC2Y3VnN7I5rruUW5lB27QsbNdLsdqEYD/T1kbNdku2BfwCOr6q7hj1/VT1YVU+l+aah/ZLsM6y5k7wYuKWqvjesOcewf1XtCywB3pDkuSPc1nSYE7mF0WZ3DuQWzO6sZXY3aq7lFuZIdu0LExppdrtQjAf5islZLcmWNCH/XFWdM8ptVdUdwIXAQUOcdn/g0CRraN6aekGSzw5xfqpqXfvfW4Av07wl1mWdzy1MX3a7mlswu7OV2d24OZhbmAPZtS9MbNTZ7UIxHuQrJmetJKH5pp/rq+qDI9rG/CQ7tz//HvBC4F+HNX9Vvb2q9qiqhTSP/zer6pXDmj/Jdkl2WP8z8CJgJJ/6nUadzi2MPrtdzy2Y3dnK7G7cHM0tdDy79oWJTUd2Z30xrqoHgPVfMXk9cHZVrRzW/EnOBC4DnpBkbZJjhjV3a3/gVTSvmq5uTwcPeRuPAi5Icg3NL4avV9VI/jTViDwSuCTJ94HLgXOr6p9neE2bZNS5hTmR3a7nFszulJjdGTfncgv2hQGZ3Qn4ldCSJEkSHdhjLEmSJE0Hi7EkSZKExViSJEkCLMaSJEkSYDGWJEmSAIvxtEjyjiQrk1zT/vmVZyQ5Psm2A9x2oHHSsJlbdZXZVVeZ3Znnn2sbsSTPAj4IHFBV9ybZFdgK+DawuKpum+D2awYZJw2TuVVXmV11ldmdHdxjPHqPAm6rqnsB2sAeDuxG80e2LwBI8tEkK9pXiu9qL3vjGONelOSyJFcm+WKa71QnyalJrmtfZf7V9N9NzTHmVl1ldtVVZncWcI/xiLVBvATYFvgX4AtVdVH/K7skD6+qnyfZAvgG8MaquqZ3XPvq8RxgSVXdk+RtwNbAh2m+jeeJVVVJdm6/A12aEnOrrjK76iqzOzu4x3jEqupu4I+AY4FbgS8kOXqMoS9PciVwFfBkYNEYY57ZXn5pkquBVwOPBu4Cfg18Msl/A3457PuhzYu5VVeZXXWV2Z0d5s30AjYHVfUgcCFwYZJraQL6W0n2At4MPL2qbk9yBrDNGFOF5nvNj9zgimQ/4EDgCJrvin/BMO+DNj/mVl1ldtVVZnfmucd4xJI8IcnePRc9FbgR+AWwQ3vZjsA9wJ1JHgks6RnfO+47wP5JHtfOvW2Sx7dvv+xUVcuB49ttSFNmbtVVZlddZXZnB/cYj972wP9LsjPwALCa5m2SI4Hzkvykqp6f5CpgJXADcGnP7U/vG3c0cGaSrdvr30nzj+Efk2xD8yrx/0zHHdOcZm7VVWZXXWV2ZwE/fCdJkiThoRSSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLGmWS/KxJCdOw3bWJHnhgGOPTnJJz/m7kzxmSOv4sySfbH9emKSSzBvS3AvatW4xjPmGKcl5SV69kevPSPKeAeca+H7O5sdE0vSzGEsdkuQVSVa0/yP/SVsmnj3T6xqW/sIJUFWvr6p3z9SaBlFV21fVDRsbk+SAJGsHmOt9VfXaYayrv+xX1Y/btT44jPmHqaqWVNWnYewcTHKuge9n/9gkFyaZ0uOf5N1Jrk3yQJKTpzKHpJllMZY6IskJwIeA9wGPBBYAHwEOG2f8UPYyavr4nHXeauCtwLkzvRBJU2MxljogyU7AKcAbquqcqrqnqu6vqn+qqre0Y05O8qUkn01yF3B0kq2TfCjJuvb0oSRbt+N3TfLVJHck+XmSi5M8rL3ubUluTvKLJKuSHDjOug5Ocl077uYkb+657sVJrm7n/3aSp/Rct2eSc5LcmuRnST6c5EnAx4BntXvE72jHPuQt9CSvS7K6XfOyJLv1XFdJXp/k35LcnuS0JGmve2ySb7bbuy3J55LsPODjv0u7rbuSXA48tu/6SvK48R6TJNsB5wG7tfft7iS7jfOcnZzks31L+OP2+ftJkjf1bLf/sfntXukkn6F58fRP7fbemr5DM9o1LGsfy9VJXtcz18lJzk7y9+19WZlkcc/1E2YkyV7t878+V59MckvP9Z9Ncnz784VJXjteDlq/n+TcdpvfTfKQ56Fn3v77eWGavbmXtrf9WpJd+8cmeS/wHODD7bY/PNb846mqT1fVecAvJnM7SbOHxVjqhmcB2wBfnmDcYcCXgJ2BzwHvAJ4JPBX4Q2A/4J3t2DcBa4H5NHug/wyoJE8AjgOeXlU7AP8FWDPO9v4W+JN23D7ANwGS7At8CvgTYBfg48CyNEV9C+CrwI3AQmB34Kyquh54PXBZ+9b2BqU1yQuA9wMvBx7VznFW37AXA09v7+/L2/UDpL3tbsCTgD2Bk8e5X/1OA37dbvOP29N4NnhMquoeYAmwrr1v21fVunZ8/3M2lucDewMvApZmgGOhq+pVwI+Bl7Tb+8AYw86kycBuwOHA+/oK7qE0j+/OwDLgwwCDZqSqfgTcBTytveg5wN1t+QV4LnBR3202loMjgXcBv0+zd/a9Ez0OPV4BvAZ4BLAV8Ob+AVX1DuBi4Lh228e19/eatuCPdfrIJNYgaZazGEvdsAtwW1U9MMG4y6rqK1X1m6r6FXAUcEpV3VJVt9KUile1Y++nKXqPbvc+X1xVBTwIbA0sSrJlVa2pqn8fZ3v3t+N2rKrbq+rK9vLXAR+vqu9W1YPtsaP30pT0/WiK2FvaPd+/rqpBjyc9CvhUVV1ZVfcCb6fZs7iwZ8ypVXVHVf0YuIDmRQFVtbqqvl5V97aPxQeB5020wbbIvww4qV3vD4BPb+Qm4z0m4+l/zsbyrnbb1wJ/R1MQN0mSPYFnA29rn4OrgU/yu3wAXFJVy9vjbz9D82IDJpeRi4DnJfmD9vyX2vN7ATsC35/Ess+pqsvbfwefo31uB/R3VfXD9jE+ezK3raqnVNXO45z+dBJrkDTLWYylbvgZsGsmPgb1pr7zu9HsVV3vxvYygL+k2ev2tSQ3JFkKTYEEjqfZm3pLkrN6D1fo8zLgYODGJBcleVZ7+aOBN/XuWaPZQ7tb+98bByj5Y3nI/amqu2kem917xvxHz8+/BLYHSPKI9r7c3B628Flg1wG2OR+Yx0Mf2xvHGQvjPybj6X/OJhrT+xxuit2An1dV79v+N7Lxx3KbJPMmmZGLgANo9g5/C7iQ5gXJ84CLq+o3k1jzmM/tNNxW0mbCYix1w2U0b+W/dIJx1Xd+HU1JXW9BexlV9YuqelNVPQZ4CXDC+rfRq+rzVfXs9rYF/MWYG6u6oqoOo3l7+is0e+KgKXLv7duztm1Vndlet2Cckt+//n4PuT/tsbu7ADdPcDtoDqMo4ClVtSPwSprDKyZyK/AATaFfb8F4gzfymIx33ya6z4yx7fWHYdwDbNtz3R/wUBubex3w8CQ79M09yGM5cEZoivFzaMrxRcAlwP40xfiicW4zyGMyKhtsuz2++u5xTh+biUVKGg2LsdQBVXUncBJwWpKXJtk2yZZJliQZ69jR9c4E3plkfvtho5No9pSu/3Dc45KE5jjQB4EHkzwhyQvSfEjv18Cv2useIslWSY5KslNV3d8zB8AngNcneUYa2yU5pC1hlwM/AU5tL98myf7t7X4K7JFkq3Huz+eB1yR5aru+9wHfrao1AzyMOwB3A3ck2R14ywC3oT2M4Bzg5PZxXwSM+fd2J3hMfgrskuaDlJN1YrvtJ9McJ/uF9vKrgYOTPLw9VOH4vtv9FBjz7ytX1U3At4H3t8/BU4BjGP84598aNCPtdv6tvf6VwLeq6q52XS9j/GI8UQ5GaYPHrKqe3HNseP/p9evHtf8mt6H5f+u89nH17yNLHWIxljqiqj4InEDz4blbafa8HkezV3I87wFWANcA1wJXtpdB82Guf6Epi5cBH6mqC2mOHT0VuI3m7edH0HwwbyyvAta0hya8nqb8UFUraI4z/jBwO80hG0e31z1Is4f6cTQfDlsL/Pd2vm8CK4H/SHLbGI/BN4ATgX+gKdePBY7YyP3v9S5gX+BOmj+ndc6At4Pmcd6e5vE4g+Y43/GM95j8K80LlRvaw0smczjERTSP4TeAv6qqr7WXf4bmGN01wNf4XWFe7/00L4zuSM9fDOlxJM0HINfRfLDzz6vq6wOsZzIZWb/+n7XHfa8/H+CqccZvNAcj9n+Bw9P8VZO/meRtP0HzIuBImg++/oqHHrMtaZZL81kbSZIkafPmHmNJkiQJi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIA+P/PdBDD+E8d/QAAAABJRU5ErkJggg==\n",
                        "text/plain": [
                            "<Figure size 864x288 with 4 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "init = 1\n",
                "Ts = [2, 3, 5, 10]\n",
                "plot_cross_sectional_dists(mc1, init, Ts)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Starting with a transient state `2`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([0.502 , 0.1641, 0.    , 0.0007, 0.3318, 0.0014])"
                        ]
                    },
                    "execution_count": 36,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "init = 2\n",
                "T = 10\n",
                "cross_sectional_dist(mc1, init=init, T=T)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 37,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([0.4962, 0.1691, 0.    , 0.    , 0.3347, 0.    ])"
                        ]
                    },
                    "execution_count": 37,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "T = 100\n",
                "dist = cross_sectional_dist(mc1, init=init, T=T)\n",
                "dist"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 38,
            "metadata": {
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEWCAYAAACdaNcBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGQpJREFUeJzt3XvUXHV97/H3R8JFwk0lWiBAqMQLeqxoDuJBWyrYA17A1Xo84KXqUVmellqPaMV6qaJVqq16qnihalUQEO+pYtEqoCgoAQENFE9KA0TABAQU5O73/LF3cJg8T555kkkm/Hi/1npWZvb+zd7fvWfPZ37zmz07qSokSW15wKQLkCSNn+EuSQ0y3CWpQYa7JDXIcJekBhnuktQgw70BST6S5M0bYT3Lkxw4YtuXJDl74P7NSX53THX8dZKP9bcXJKkkc8a07N36Wjcbx/K0dqPs7/753XPE5d1zbIyz7X1Rc+Ge5PlJlvQHzDVJvp7kKZOua1yGQxOgql5ZVW+fVE2jqKptqurytbVJsn+SFSMs651V9fJx1DX8hlVVV/a13j2O5c+ijrcmOXEt828e+PtNklsH7r9gFut5bJLTk1yXZI0fuSR5cJIvJbklyRVJnj80//n99FuSfDnJg2e3pfc2vL+TnJlknZ/b2Rwbg23Xp5OQ5JlJzk5yY5Jrk/xTkm1nu5xxayrck7wGeD/wTuBhwG7Ah4BDp2k/lt6eNp7763PWB+A2VbUNcCXw7IFpn5nFou4ETgVeNs3844A76F4/LwA+nOQxAP2/HwVe1M//Nd3r6/5ue+AdwM7Ao4H5wHsmWhFAVTXxR7eDbwb+x1ravBX4PHAi8Evg5cCWdG8IV/d/7we27NvvCHwVuBH4BfBd4AH9vNcDPwN+BVwGHDDNOp8BXNK3+xnw2oF5zwIu7Jf/feBxA/N2Bb4IrAKuBz5Id+DcBtzdb+uNfdtPAu8YeOwrgGV9zYuBnQfmFfBK4P8BN9C9mNPPezjw7X591wGfAXYYeOxy4MBptvMh/bp+CfwQeDtw9tB695xunwBzgVuB3/TbdjPdi2Wq5+ytwIn9shb0yz6if/6uAY4aWO/wvtkfWNHfPqFf3639+v5qYHlz+jY799v1i36fvmLoeDoV+HS/LUuBRWs5/v4vcFW/HecDT+2nH0QXqHf2dVw0w7E+7fMwi9fLnkANTZvb1/GIgWknAMf2t98JnDQw7+F9+22nWP7bgA/0tzcHbgHe3d9/IN1x/KDB/Q38Ld2xfVu/Hz440zE7zWt8+Nh4Md0b4nXAG6dpe2XfdvWx9+T12Ld/DPx4XNm2znVMuoCxbUj3Arlr9YtyLU/8ncBz6D61PBA4BjgXeCgwjy5k3963fxfwkf7g3Bx4KhDgkf2LdOeBg+jh06zzmoEX8YOAJ/S3nwCsBJ4EbNYfgMvp3mw2Ay4C3te/4LYCntI/7iUMhGY/7ZP0AQY8rT+In9Av6wPAdwbaFt0b1g50n2xWAQf18/YEnt4/bh7wHeD9A49dzvThfgpd0M0FHksX2tOF+3T7ZH/64J3hORt8US7ol31yv+7/0m/TgcP7Zqp1DG8Ta4b7WXS9062Ax/fLPmCgttvo3qw2oztezl3L8fdCujfBOcBRwLXAVgPLOnHEY32N5wF4Pl0nYbq/3YbaTxXuewO3Dk17LfAv/e2vAK8fmn8z8MQpanwafcAB/w34D+AHA/MummZ/nwm8fGhZ0x6z07zGh4+Nf+qPm98DbgcevZa2cwaW9ZQZ9ulTpqnh/cApGyLnZvPX0rDMQ4DrququGdqdU1VfrqrfVNWtdB89j6mqlVW1iq7H8aK+7Z3ATsDuVXVnVX23umfvbroA3CvJ5lW1vKr+Y5r13dm3266qbqiqC/rprwA+WlU/qKq7q+pTdAfevsA+dD3G11XVLVV1W1WdPfXi1/AC4BNVdUFV3Q68AXhykgUDbY6tqhur6krgDLrQoqqWVdU3q+r2fl+8F/iDmVbYfxn2J8Bb+np/AnxqLQ+Zbp9MZ/g5m8rb+nX/GPhn4PCZ6p5Jkl3pXuCv75+DC4GP8dvjA7o3sNOqGzM+gS5AplRVJ1bV9VV1V1X9A90x9Mj1rbNf9klVtcNa/q4cYTHbADcNTbsJ2HbE+YPOARYmeQjw+8DHgV2SbEN3TJ014qatNuUxO6K3VdWtVXURXadp2udoUFWdPcM+XeM1meTpdB21t8yivg2ipXC/HthxhDHZq4bu7wxcMXD/in4adONmy4BvJLk8ydHQhSDwarp3/pVJTkmyM1P7E7qe3RVJzkry5H767sBR/ZcwNya5kW4oZuf+3ytGeKOayr22p6pupts3uwy0uXbg9q/pXrQkeWi/LT9L8ku6oZAdR1jnPLre6OC+vWKatjD9PpnO8HM2U5vB53B97Az8oqp+NbTste3LraY7BpMcleTSJDf1z/f2jLZ/N5abge2Gpm1HN+Q0yvx79G/CS+iC/Pfpwvz7wH6sW7hPecxuhMeOLMm+wEnAc6vqpxtiHbPRUrifQ/cR+TkztBs+Q+BquqBdbbd+GlX1q6o6qqp+F3g28JokB/TzTqqqp/SPLeDvplxZ1XlVdSjdsM+X6YYuoAujvx3qCWxdVSf383abJiRmuoznvbYnyVy6TzU/m+Fx0A0rFN3Y/3Z0wwgZ4XGr6IbEdh2Yttt0jdeyT6bbtlEuXTq87qv727cAWw/M+51ZLPtq4MFDZz7sxmj78l6SPJXue5rnAQ+qqh3oer2r9+96XZ41yQuGzqgZ/pv2+RjwU2BOkoUD036P7rsE+n/v6fX2p7Zu2T9uKmfRDcHsDZzX3//vdJ9MvzPNYyZ1mdqpzhx66gz79KkDbfem+27mf1XVtzZm4dNpJtyr6ia6j0LHJXlOkq2TbJ7k4CTvXstDTwbelGRekh37ZZwIkORZSfZMErovwe4G7k7yyCRPS7Il3RvKrf28e0myRf+i276q7hxYBnTjgK9M8qR05vanVG1L94XkNcCx/fStkuzXP+7nwPwkW0yzPScBL03y+L6+d9KNdS4fYTduS/9FbZJdgNeN8Bj6IYkvAm/t9/tedB9N1zDDPvk58JAk24+y3iFv7tf9GOClwGf76RcCz0h3it/v0H3iGvRzYMrz76vqKrre5rv65+BxdGeZzObslNW2pXsDXEUXoG/h3r3gnwMLkqzTa7KqPlMDZ9RM8XclQH+sbQVs0d/fqj9OqKpb6J7HY/rjbj+6M81O6FfzGeDZfejNpfu+6otDn2wGnQX8KXBJVd1BP54O/Gc/7DeVaZ+PDWwV3Zfr96y7H4Zd2z79LnSnlwL/CvxFVf3LBGqfUjPhDlBV7wVeA7yJ7sm6CjiSrnc4nXfQfXy8GPgxcEE/DWAh8G90gXcO8KGqOpOut3Is3ReX19L1QP96muW/CFjeD3O8kq43TFUtoRt3/yDdGQDL6L4sXR2Wz6b70utKYAXwP/vlfZuuB3Vtkuum2AffAt4MfIHuDeLhwGFr2f5Bb6P7IvYm4Gt0L/RRHUn3cfdaui8x/3ktbafbJ/9O92Z7eT9UNZuhlbPo9uG3gL+vqm/000+gG2ddDnyD34b+au+ie3O/Mclrp1ju4XRftl0NfAn4m6r65izqWu104Ot0vdwr6DoFg0NJn+v/vT7JTN9BrI/d6Tojq3vjt9Kd7bXan9F9+biS7rn431W1FKD/95V0Ib+S7g3rz9ayru/3y1rdS7+Ebrun67VDd0bRc5PckOQfR9+s9VNVv6Y7W+d7/bGw7ywefhTd0OTHB3r1S2d60Ia2+hQ4SVJDmuq5S5I6M4Z7kk8kWZnkJ9PMT5J/TLIsycVJnjD+MiVJszFKz/2TdD8Qms7BdGPTC+l+Jfjh9S9LkrQ+Zgz3qvoO3U+vp3Mo8OnqnAvskGSncRUoSZq9cVyEaRfu/a3/in7aNcMNkxxB17tn7ty5T3zUox41htVL0v3H+eeff11VzZup3TjCfaofuUx5Ck5VHQ8cD7Bo0aJasmTJGFYvSfcfSdb26+97jONsmRXc+9eB8/ntrwMlSRMwjnBfDPxpf9bMvsBNVbXGkIwkaeOZcVgmycl0l0ndMd3/kvM3dJe/pao+ApxGdxGoZXQX5XnphipWkjSaGcO9qtZ66dTqfuL652OrSJK03vyFqiQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1KCRwj3JQUkuS7IsydFTzN8tyRlJfpTk4iTPGH+pkqRRzRjuSTYDjgMOBvYCDk+y11CzNwGnVtXewGHAh8ZdqCRpdKP03PcBllXV5VV1B3AKcOhQmwK2629vD1w9vhIlSbM1SrjvAlw1cH9FP23QW4EXJlkBnAb8xVQLSnJEkiVJlqxatWodypUkjWKUcM8U02ro/uHAJ6tqPvAM4IQkayy7qo6vqkVVtWjevHmzr1aSNJJRwn0FsOvA/fmsOezyMuBUgKo6B9gK2HEcBUqSZm+UcD8PWJhkjyRb0H1huniozZXAAQBJHk0X7o67SNKEzBjuVXUXcCRwOnAp3VkxS5Mck+SQvtlRwCuSXAScDLykqoaHbiRJG8mcURpV1Wl0X5QOTnvLwO1LgP3GW5okaV35C1VJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDRop3JMclOSyJMuSHD1Nm+cluSTJ0iQnjbdMSdJszJmpQZLNgOOApwMrgPOSLK6qSwbaLATeAOxXVTckeeiGKliSNLNReu77AMuq6vKqugM4BTh0qM0rgOOq6gaAqlo53jIlSbMxSrjvAlw1cH9FP23QI4BHJPleknOTHDTVgpIckWRJkiWrVq1at4olSTMaJdwzxbQauj8HWAjsDxwOfCzJDms8qOr4qlpUVYvmzZs321olSSMaJdxXALsO3J8PXD1Fm69U1Z1V9Z/AZXRhL0magFHC/TxgYZI9kmwBHAYsHmrzZeAPAZLsSDdMc/k4C5UkjW7GcK+qu4AjgdOBS4FTq2ppkmOSHNI3Ox24PsklwBnA66rq+g1VtCRp7VI1PHy+cSxatKiWLFkykXVL0n1VkvOratFM7fyFqiQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDVozqQLGLbg6K9NuoRZWX7sMyddgiStwZ67JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNWikcE9yUJLLkixLcvRa2j03SSVZNL4SJUmzNWO4J9kMOA44GNgLODzJXlO02xZ4FfCDcRcpSZqdUXru+wDLquryqroDOAU4dIp2bwfeDdw2xvokSetglHDfBbhq4P6Kfto9kuwN7FpVX13bgpIckWRJkiWrVq2adbGSpNGMEu6ZYlrdMzN5APA+4KiZFlRVx1fVoqpaNG/evNGrlCTNyijhvgLYdeD+fODqgfvbAo8FzkyyHNgXWOyXqpI0OaOE+3nAwiR7JNkCOAxYvHpmVd1UVTtW1YKqWgCcCxxSVUs2SMWSpBnNGO5VdRdwJHA6cClwalUtTXJMkkM2dIGSpNmbM0qjqjoNOG1o2lumabv/+pclSVof/kJVkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1KCRfsQkqX0Ljv7apEuYleXHPnPSJWzS7LlLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoNGCvckByW5LMmyJEdPMf81SS5JcnGSbyXZffylSpJGNWO4J9kMOA44GNgLODzJXkPNfgQsqqrHAZ8H3j3uQiVJoxul574PsKyqLq+qO4BTgEMHG1TVGVX16/7uucD88ZYpSZqNUcJ9F+Cqgfsr+mnTeRnw9almJDkiyZIkS1atWjV6lZKkWRkl3DPFtJqyYfJCYBHwnqnmV9XxVbWoqhbNmzdv9ColSbMyZ4Q2K4BdB+7PB64ebpTkQOCNwB9U1e3jKU+StC5G6bmfByxMskeSLYDDgMWDDZLsDXwUOKSqVo6/TEnSbMwY7lV1F3AkcDpwKXBqVS1NckySQ/pm7wG2AT6X5MIki6dZnCRpIxhlWIaqOg04bWjaWwZuHzjmuiRJ68FfqEpSg0bquWs8Fhz9tUmXMGvLj33mpEuQtA7suUtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoNGCvckByW5LMmyJEdPMX/LJJ/t5/8gyYJxFypJGt2M4Z5kM+A44GBgL+DwJHsNNXsZcENV7Qm8D/i7cRcqSRrdKD33fYBlVXV5Vd0BnAIcOtTmUOBT/e3PAwckyfjKlCTNxpwR2uwCXDVwfwXwpOnaVNVdSW4CHgJcN9goyRHAEf3dm5Ncti5Fr6Mdh+sZh2w6n1HcvvuulrcNPDbHbfdRGo0S7lP1wGsd2lBVxwPHj7DOsUuypKoWTWLdG4Pbd9/V8raB2zcpowzLrAB2Hbg/H7h6ujZJ5gDbA78YR4GSpNkbJdzPAxYm2SPJFsBhwOKhNouBF/e3nwt8u6rW6LlLkjaOGYdl+jH0I4HTgc2AT1TV0iTHAEuqajHwceCEJMvoeuyHbcii19FEhoM2IrfvvqvlbQO3byJiB1uS2uMvVCWpQYa7JDWo+XCf6dIJ93VJPpFkZZKfTLqWcUuya5IzklyaZGmSv5x0TeOUZKskP0xyUb99b5t0TeOWZLMkP0ry1UnXsiEkWZ7kx0kuTLJk0vUManrMvb90wk+Bp9OdrnkecHhVXTLRwsYoye8DNwOfrqrHTrqecUqyE7BTVV2QZFvgfOA5rTx//a+451bVzUk2B84G/rKqzp1waWOT5DXAImC7qnrWpOsZtyTLgUVVtcn9CK31nvsol064T6uq79Dobwqq6pqquqC//SvgUrpfQzehOjf3dzfv/5rpbSWZDzwT+Nika7k/aj3cp7p0QjPhcH/SX2l0b+AHk61kvPphiwuBlcA3q6ql7Xs/8FfAbyZdyAZUwDeSnN9fXmWT0Xq4j3RZBG3akmwDfAF4dVX9ctL1jFNV3V1Vj6f75fc+SZoYWkvyLGBlVZ0/6Vo2sP2q6gl0V839836YdJPQeriPcukEbcL6segvAJ+pqi9Oup4NpapuBM4EDppwKeOyH3BIPyZ9CvC0JCdOtqTxq6qr+39XAl+iGwreJLQe7qNcOkGbqP4Lx48Dl1bVeyddz7glmZdkh/72A4EDgX+fbFXjUVVvqKr5VbWA7nX37ap64YTLGqskc/sv+kkyF/gjYJM5a63pcK+qu4DVl064FDi1qpZOtqrxSnIycA7wyCQrkrxs0jWN0X7Ai+h6fRf2f8+YdFFjtBNwRpKL6Toi36yqJk8ZbNTDgLOTXAT8EPhaVf3rhGu6R9OnQkrS/VXTPXdJur8y3CWpQYa7JDXIcJekBhnuktQgw13NSvLG/mqLF/enUT4pyauTbD3CY0dqJ22qPBVSTUryZOC9wP5VdXuSHYEtgO8zwlX8NuWr/UmjsOeuVu0EXFdVtwP0If1cYGe6Hw6dAZDkw0mWDF5PPcmrpmj3R0nOSXJBks/117shybFJLuk/Hfz9xt9MaWr23NWkPnzPBrYG/g34bFWdNdwjT/LgqvpFf+3/bwGvqqqLB9v1vf4vAgdX1S1JXg9sCXyQ7tfBj6qqSrJDf40YaeLsuatJ/XXSnwgcAawCPpvkJVM0fV6SC4AfAY8B9pqizb799O/1l+d9MbA78EvgNuBjSf4Y+PW4t0NaV3MmXYC0oVTV3XRXWjwzyY/pQvkeSfYAXgv816q6Icknga2mWFTorvty+Bozkn2AA+gujnUk8LRxboO0ruy5q0lJHplk4cCkxwNXAL8Ctu2nbQfcAtyU5GF01+RebbDducB+Sfbsl711kkf0Qz/bV9VpwKv7dUibBHvuatU2wAf6S+reBSyjG6I5HPh6kmuq6g+T/AhYClwOfG/g8ccPtXsJcHKSLfv5b6J7A/hKkq3oevf/Z2NsmDQKv1CVpAY5LCNJDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoP+P6lkSlwgMPDGAAAAAElFTkSuQmCC\n",
                        "text/plain": [
                            "<Figure size 432x288 with 1 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "draw_histogram(dist,\n",
                "               title='Cross sectional distribution at T={T} with init={init}'\n",
                "                     .format(T=T, init=init),\n",
                "               xlabel='States')"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Observe that the distribution is close to a convex combination of\n",
                "the stationary distributions `(1, 0, 0, 0, 0, 0)` and `(0, 1/3, 0, 0, 2/3, 0)`,\n",
                "which is a stationary distribution itself."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "How the simulated marginal distribution evolves:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 39,
            "metadata": {
                "scrolled": true
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAE3CAYAAACpToeUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuUpHV95/H3JwyXcI8yGrmMoKJm5BgvE9TgBS/rMmggu3JcEI0YlbiRzbJeEhIvURKVmBzjZsULMQbjBSSGJBPBoFFAUVQGRHQgmAkZZBwioNxVFPzuH88zWhTd09U9Vd3963m/zulDV9Wvfs+vqj7TfOqpp6pSVUiSJEnbup9b6AVIkiRJi4HFWJIkScJiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGeN0nuGPj5SZIfDJw+dhbzvDjJpUluS7IxyduTLJvk2rVtG2N2j05ydZJbk9yQ5INJdp/k2rVtG2N2j0tyz9B8h05w6drGjDGrByU5L8lNSe7zRRVJ7pfk75PcmeTaJC8Y7y1pn8V4nlTVrpt/gG8BvzZw3kdmMdXOwInAXsATgGcCrxn/iqXOGLP7BeCQqtoDeAiwDPjjCSxZAsaaXYCLB+erqgvGv2Jtq8aY1R8DZwEvnebyU4EfAQ8EjgXek+RRW7P2pcY9jY2pqvcMnPx2ko8AT1+o9Uijqqrrhs66B3jYQqxFkpaiqroauDrJff62JtkFeB5wUFXdAVyUZA3wIuCk+V3p4uUe40UiyQuS3LKFnxXTXPWpwLr5XKs0aDbZTfLkJLcCt9P9gX7ngi1c27xZ/t19bP/y9DeTvMFD2DSftqIjDHo4cE9VfXPgvK8B7jEe4D/sRaKqPgp8dDbXSfISYBXwsoksShrBbLJbVRcBeyTZB3g5sGGCS5O2aBbZ/RxwEHAtXYn4GHA38LbJrU76mbl0hCnsCtw6dN6twG5bOe+S4h7jRiX5deAUYHVV3bTQ65Fmo6q+DfwzcOZCr0WaSVVdU1X/UVU/qaqvAycDRy30uqRZugMYfsPz7nSv4KlnMV4kkhw79K7U4Z/Bl6MPA/6S7uD8ry/cqqXZZXfIMuCh87lWadBWZLeAzOdatW3biqwO+iawLMmBA+f9Mh6OeS8eSrFI9O86nfGdp0me0Y/7b1X1lYkvTJrBLLJ7LPB54DpgBfAW4DOTXZ00vVlkdzVwWVV9J8kjgTcAfzvp9UmbzSKrAXYEduhP79Rdve6qqjuTnA2cnORlwGOAI4FfndzK2+Me4/a8AdgDOHfgmeInF3pR0ghWAl+keznvC8DVdMcZS4vdM4ErktwJnAucDbx1YZckTenBwA/42V7gH9D9rd3st4GfB24AzgD+Z1W5x3hAqu7z+c+SJEnSNsc9xpIkSRIjFOMkH0j39a3fmObyJPmLJOuTXJHkceNfpjR7ZletMrtqkbnVUjDKHuPTgcO2cPlq4MD+53jgPVsYK82n0zG7atPpmF2153TMrRo3YzGuqs8B39vCkCOBv6nOl4A9kzxoXAuU5srsqlVmVy0yt1oKxnGM8T50H7+02cb+PGmxM7tqldlVi8ytFr1xfI7xVB9yPuVHXSQ5nu7lE3bZZZfHP/KRjxzD5rUtu/TSS2+qquVzvLrZ1YLYytzCiNk1txo3/+aqVaNmdxzFeCOw38DpfYFNUw2sqtOA0wBWrVpVa9euHcPmtS1Lcu1WXN3sakFsZW5hxOyaW42bf3PVqlGzO45DKdYAv9G/2/SJwK1Vdf0Y5pUmzeyqVWZXLTK3WvRm3GOc5AzgUGCvJBuBPwS2B6iq99J9C9DhwHrg+8BLJrVYaTbMrlpldtUic6ulYMZiXFXHzHB5Aa8c24qkMTG7apXZVYvMrZYCv/lOkiRJwmIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCRixGCc5LMnVSdYnOWmKy1ckOT/JV5NckeTw8S9Vmj2zqxaZW7XK7Kp1MxbjJNsBpwKrgZXAMUlWDg17PXBWVT0WOBp497gXKs2W2VWLzK1aZXa1FIyyx/hgYH1VXVNVPwLOBI4cGlPA7v3vewCbxrdEac7MrlpkbtUqs6vmjVKM9wGuGzi9sT9v0JuAFybZCJwL/K+pJkpyfJK1SdbeeOONc1iuNCtmVy0yt2qV2VXzRinGmeK8Gjp9DHB6Ve0LHA58KMl95q6q06pqVVWtWr58+exXK82O2VWLzK1aZXbVvFGK8UZgv4HT+3Lflz5eCpwFUFUXAzsBe41jgdJWMLtqkblVq8yumjdKMb4EODDJAUl2oDtYfs3QmG8BzwRI8kt0Qfe1Dy00s6sWmVu1yuyqeTMW46q6GzgBOA+4iu7dpOuSnJzkiH7Yq4GXJ/kacAZwXFUNv3wizSuzqxaZW7XK7GopWDbKoKo6l+4g+cHz3jjw+5XAIeNdmrT1zK5aZG7VKrOr1vnNd5IkSRIWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAkYsRgnOSzJ1UnWJzlpmjHPT3JlknVJPjreZUpzY3bVInOrVpldtW7ZTAOSbAecCvwXYCNwSZI1VXXlwJgDgd8HDqmqm5M8YFILlkZldtUic6tWmV0tBaPsMT4YWF9V11TVj4AzgSOHxrwcOLWqbgaoqhvGu0xpTsyuWmRu1Sqzq+aNUoz3Aa4bOL2xP2/Qw4GHJ/lCki8lOWxcC5S2gtlVi8ytWmV21bwZD6UAMsV5NcU8BwKHAvsCn09yUFXdcq+JkuOB4wFWrFgx68VKs2R21SJzq1aZXTVvlD3GG4H9Bk7vC2yaYsw/VtWPq+o/gKvpgn8vVXVaVa2qqlXLly+f65qlUZldtcjcqlVmV80bpRhfAhyY5IAkOwBHA2uGxvwD8HSAJHvRvVRyzTgXKs2B2VWLzK1aZXbVvBmLcVXdDZwAnAdcBZxVVeuSnJzkiH7YecB3k1wJnA+8tqq+O6lFS6Mwu2qRuVWrzK6WglQNH/4zP1atWlVr165dkG1r6UhyaVWtms9tml1tLXOrVpldtWrU7PrNd5IkSRIWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEmAxViSJEkCLMaSJEkSYDGWJEmSAIuxJEmSBFiMJUmSJMBiLEmSJAEWY0mSJAmwGEuSJEkALFvoBUiSJI1i/5POGdtcG055ztjm0tLhHmNJkiQJi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIE+JXQkrTN8Wt1JWlq7jGWJEmSsBhLkiRJgMVYkiRJAizGkiRJEjBiMU5yWJKrk6xPctIWxh2VpJKsGt8Spbkzu2qRuVWrzK5aN2MxTrIdcCqwGlgJHJNk5RTjdgN+B/jyuBcpzYXZVYvMrVpldrUUjLLH+GBgfVVdU1U/As4Ejpxi3B8Bbwd+OMb1SVvD7KpF5latMrtq3ijFeB/guoHTG/vzfirJY4H9quoTW5ooyfFJ1iZZe+ONN856sdIsmV21yNyqVWZXzRulGGeK8+qnFyY/B/w58OqZJqqq06pqVVWtWr58+eirlObG7KpF5latMrtq3ijFeCOw38DpfYFNA6d3Aw4CLkiyAXgisMYD6rUImF21yNyqVWZXzRulGF8CHJjkgCQ7AEcDazZfWFW3VtVeVbV/Ve0PfAk4oqrWTmTF0ujMrlpkbtUqs6vmzViMq+pu4ATgPOAq4KyqWpfk5CRHTHqB0lyZXbXI3KpVZldLwbJRBlXVucC5Q+e9cZqxh279sqTxMLtqkblVq8yuWuc330mSJElYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEjDi5xhL0mK0/0nnjHW+Dac8Z6zzSZLaYjGWJGmejfNJnU/oNJ+WenY9lEKSJEnCYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgA/x1iaM79cQpKkpcU9xpIkSRIWY0mSJAnwUAr1lvpXPEqSJM3EPcaSJEkSFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBFmNJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkoARi3GSw5JcnWR9kpOmuPxVSa5MckWSzyR58PiXKs2e2VWLzK1aZXbVuhmLcZLtgFOB1cBK4JgkK4eGfRVYVVWPBj4OvH3cC5Vmy+yqReZWrTK7WgpG2WN8MLC+qq6pqh8BZwJHDg6oqvOr6vv9yS8B+453mdKcmF21yNyqVWZXzRulGO8DXDdwemN/3nReCnxyaxYljYnZVYvMrVpldtW8ZSOMyRTn1ZQDkxcCq4CnTXP58cDxACtWrBhxidKcmV21yNyqVWZXzRtlj/FGYL+B0/sCm4YHJXkW8DrgiKq6a6qJquq0qlpVVauWL18+l/VKs2F21SJzq1aZXTVvlGJ8CXBgkgOS7AAcDawZHJDkscD76EJ+w/iXKc2J2VWLzK1aZXbVvBmLcVXdDZwAnAdcBZxVVeuSnJzkiH7YnwK7An+b5PIka6aZTpo3ZlctMrdqldnVUjDKMcZU1bnAuUPnvXHg92eNeV3SWJhdtcjcqlVmV63zm+8kSZIkLMaSJEkSMOKhFJIkjWr/k84Z63wbTnnOWOeTpjPO7JrbNi2KYjzpIBr0hedjIEmSFjsPpZAkSZKwGEuSJEmAxViSJEkCFskxxpLuyzcwSZI0v9xjLEmSJGExliRJkgCLsSRJkgRYjCVJkiTAYixJkiQBfiqFpAnyGw8lSS2xGI9BS19pPd02JEmStnUeSiFJkiRhMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEwLKFXoAkLVb7n3TOWOfbcMpzxjqfJGm83GMsSZIkYTGWJEmSAIuxJEmSBHiMsbRNG+cxtB4/K0lqncVYkqQhPmmUtk0eSiFJkiRhMZYkSZIAD6WQJGnJ8TO4pbkZaY9xksOSXJ1kfZKTprh8xyQf6y//cpL9x71QaS7MrlpkbtUqs6vWzViMk2wHnAqsBlYCxyRZOTTspcDNVfUw4M+BPxn3QqXZMrtqkblVq8yuloJR9hgfDKyvqmuq6kfAmcCRQ2OOBD7Y//5x4JlJMr5lSnNidtUic6tWmV01L1W15QHJUcBhVfWy/vSLgCdU1QkDY77Rj9nYn/73fsxNQ3MdDxzfn3wEcPUs1roXcNOMo+Zu0vPPxzZan38u23hwVS2f6gKz28z887GNxTZ/C7mFxXe/bWvzz8c2lmJ2t8XHZVubfy7bmDa7g0Z5891Uz+SG2/QoY6iq04DTRtjmfReRrK2qVXO57mKYfz620fr8E9iG2W1g/vnYRmPzL4rcQnP325Kbfz62sRSz6+Oy9Oef5DZGOZRiI7DfwOl9gU3TjUmyDNgD+N44FihtBbOrFplbtcrsqnmjFONLgAOTHJBkB+BoYM3QmDXAi/vfjwI+WzMdoyFNntlVi8ytWmV21bwZD6WoqruTnACcB2wHfKCq1iU5GVhbVWuAvwI+lGQ93TO/oyew1jm/HLhI5p+PbbQ+/1i3YXabmX8+ttHM/Isot9DQ/bZE55+PbSzF7Pq4LP35J7aNGd98J0mSJG0L/EpoSZIkCYuxJEmSBDRSjGf6ismtnPsDSW7oP1tx7JLsl+T8JFclWZfkf09gGzsl+UqSr/XbePO4t9FvZ7skX03yiQnMvSHJ15NcnmTtuOdfCJPMbT9/09ldCrnt5ze7s5/f7I62Hf/mzpJ9YcZtmN2ZVNWi/qE7gP/fgYcAOwBfA1aOcf6nAo8DvjGh9T8IeFz/+27AN8e5/n7eALv2v28PfBl44gRuy6uAjwKfmMDcG4C95jtfk/qZdG77bTSd3aWQ235+szv7bZjd0bbj39zZ3Sb7wszbMLsz/LSwx3iUr5ics6r6HBP8DMWqur6qLut/vx24CthnzNuoqrqjP7l9/zPWd1Um2Rd4DvD+cc67hE00t9B+ds3tomV2Z57f7C5O9oWZt2F2Z9BCMd4HuG7g9EbGHJT5kmR/4LF0z9DGPfd2SS4HbgA+XVXj3sY7gd8FfjLmeTcr4FNJLk33VaCtWzK5hclldwnkFszuomZ2p7XUcgtLKLv2hS2aaHZbKMYjfX3kYpdkV+DvgBOr6rZxz19V91TVY+i+aejgJAeNa+4kzwVuqKpLxzXnFA6pqscBq4FXJnnqBLc1H5ZEbmGy2V0CuQWzu2iZ3S1aarmFJZJd+8KMJprdForxKF8xuagl2Z4u5B+pqrMnua2qugW4ADhsjNMeAhyRZAPdS1PPSPLhMc5PVW3q/3sD8Pd0L4m1rPncwvxlt9XcgtldrMzuli3B3MISyK59YWaTzm4LxXiUr5hctJKE7pt+rqqqd0xoG8uT7Nn//vPAs4B/Hdf8VfX7VbVvVe1Pd/9/tqpeOK75k+ySZLfNvwPPBibyrt951HRuYfLZbT23YHYXK7O7ZUs0t9B4du0LM5uP7C76YlxVdwObv2LyKuCsqlo3rvmTnAFcDDwiycYkLx3X3L1DgBfRPWu6vP85fMzbeBBwfpIr6P4wfLqqJvLRVBPyQOCiJF8DvgKcU1X/vMBr2iqTzi0siey2nlswu3Nidhfcksst2BdGZHZn4FdCS5IkSTSwx1iSJEmaDxZjSZIkCYuxJEmSBFiMJUmSJMBiLEmSJAEW43mR5HVJ1iW5ov/4lSckOTHJziNcd6Rx0riZW7XK7KpVZnfh+XFtE5bkScA7gEOr6q4kewE7AF8EVlXVTTNcf8Mo46RxMrdqldlVq8zu4uAe48l7EHBTVd0F0Af2KGBvug/ZPh8gyXuSrO2fKb65P+93phj37CQXJ7ksyd+m+051kpyS5Mr+Weafzf/N1BJjbtUqs6tWmd1FwD3GE9YH8SJgZ+BfgI9V1YXDz+yS3K+qvpdkO+AzwO9U1RWD4/pnj2cDq6vqziS/B+wIvIvu23geWVWVZM/+O9ClOTG3apXZVavM7uLgHuMJq6o7gMcDxwM3Ah9LctwUQ5+f5DLgq8CjgJVTjHlif/4XklwOvBh4MHAb8EPg/Un+O/D9cd8ObVvMrVpldtUqs7s4LFvoBWwLquoe4ALggiRfpwvoTyU5AHgN8CtVdXOS04GdppgqdN9rfsx9LkgOBp4JHE33XfHPGOdt0LbH3KpVZletMrsLzz3GE5bkEUkOHDjrMcC1wO3Abv15uwN3ArcmeSCwemD84LgvAYckeVg/985JHt6//LJHVZ0LnNhvQ5ozc6tWmV21yuwuDu4xnrxdgf+XZE/gbmA93cskxwCfTHJ9VT09yVeBdcA1wBcGrn/a0LjjgDOS7Nhf/nq6fwz/mGQnumeJ/2c+bpiWNHOrVpldtcrsLgK++U6SJEnCQykkSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWJIkSQIsxpIkSRJgMZYkSZIAi7EkSZIEWIwlSZIkwGIsSZIkARZjSZIkCbAYS5IkSYDFWNIil+S9Sd4wD9vZkORZI449LslFA6fvSPKQMa3jD5K8v/99/ySVZNmY5l7Rr3W7ccw3Tkk+meTFW7j89CR/POJcI9/OxXyfSJp/FmOpIUlekGRt/z/y6/sy8eSFXte4DBdOgKp6RVX90UKtaRRVtWtVXbOlMUkOTbJxhLneWlUvG8e6hst+VX2rX+s945h/nKpqdVV9EKbOwSznGvl2Do9NckGSWd//SR6Q5Iwkm5LcmuQLSZ4wl/VLWjgWY6kRSV4FvBN4K/BAYAXwbuDIacaPZS+j5o+PWdN2BS4BHg/cD/ggcE6SXRd0VZJmxWIsNSDJHsDJwCur6uyqurOqflxV/1RVr+3HvCnJx5N8OMltwHFJdkzyzn4v1qb+9x378Xsl+USSW5J8L8nnk/xcf9nvJfl2ktuTXJ3kmdOs6/AkV/bjvp3kNQOXPTfJ5f38X0zy6IHL9ktydpIbk3w3ybuS/BLwXuBJ/R7xW/qx93oJPcnLk6zv17wmyd4Dl1WSVyT5tyQ3Jzk1SfrLHprks/32bkrykSR7jnj/37/f1m1JvgI8dOjySvKw6e6TJLsAnwT27m/bHUn2nuYxe1OSDw8t4Tf7x+/6JK8e2O7wffPTvdJJPkT35Omf+u39boYOzejXsKa/L9cnefnAXG9KclaSv+lvy7okqwYunzEjSQ7oH//NuXp/khsGLv9wkhP73y9I8rLpctD7hSTn9Nv8cpJ7PQ4D8w7fzguS/FG6vbi3J/lUkr2GxyZ5C/AU4F39tt811fxTqaprquodVXV9Vd1TVacBOwCPGHUOSQvPYiy14UnATsDfzzDuSODjwJ7AR4DXAU8EHgP8MnAw8Pp+7KuBjcByuj3QfwBUkkcAJwC/UlW7Af8V2DDN9v4K+K1+3EHAZwGSPA74APBbwP2B9wFr0hX17YBPANcC+wP7AGdW1VXAK4CL+5e271NakzwDeBvwfOBB/RxnDg17LvAr/e19fr9+gPTX3Rv4JWA/4E3T3K5hpwI/7Lf5m/3PdO5zn1TVncBqYFN/23atqk39+OHHbCpPBw4Eng2clBGOha6qFwHfAn6t397bpxh2Bl0G9gaOAt46VHCPoLt/9wTWAO8CGDUjVfUfwG3AY/uzngLc0ZdfgKcCFw5dZ0s5OAZ4M/ALwHrgLTPdDwNeALwEeABdYX3N8ICqeh3weeCEftsn9Lf3ir7gT/Xz7qk2luQx/XbWz2KNkhaYxVhqw/2Bm6rq7hnGXVxV/1BVP6mqHwDHAidX1Q1VdSNdqXhRP/bHdEXvwf3e589XVQH3ADsCK5NsX1Ubqurfp9nej/txu1fVzVV1WX/+y4H3VdWX+71nHwTuoivpB9MVsdf2e75/WFWjHk96LPCBqrqsqu4Cfp9uz+L+A2NOqapbqupbwPl0TwqoqvVV9emququ/L94BPG2mDfZF/nnAG/v1foPuZfLpTHefTGf4MZvKm/ttfx34a7qCuFWS7Ac8Gfi9/jG4HHg/P8sHwEVVdW5//O2H6J5swOwyciHwtCS/2J/+eH/6AGB34GuzWPbZVfWV/t/BR+gf2xH9dVV9s7+Pz5rNdavq0VW15zQ/vz08PsnudPfXm6vq1lmsUdICsxhLbfgusFdmPgb1uqHTe9PtVd3s2v48gD+l25v1qSTXJDkJugIJnEi3N/WGJGcOHq4w5HnA4cC1SS5M8qT+/AcDrx7cs0a3h3bv/r/XjlDyp3Kv21NVd9DdN/sMjPnPgd+/T3fs5+Y3R53Zv/x/G/BhYK8RtrkcWMa979trpxkL098n0xl+zGYaM/gYbo29ge9V1e1Dc2/pvtwpybJZZuRC4FC6vcOfAy6ge0LyNODzVfWTWax5ysd2Hq47siQ/D/wT8KWqetsktiFpcizGUhsupnsp/9dnGFdDpzfRldTNVvTnUVW3V9Wrq+ohwK8Br9r8MnpVfbSqntxft4A/mXJjVZdU1ZF0L0//A92eOOiK3FuG9qztXFVn9JetmKbkD69/2L1uT3/s7v2Bb89wPegOoyjg0VW1O/BCusMrZnIjcDddod9sxXSDt3CfTHfbZrrNTLHtzYdh3AnsPHDZL3JvW5p7E3C/JLsNzT3KfTlyRuiK8VPoyvGFwEXAIXTF+MJprjPKfTIp99l2f3z1HdP8vHdg3I50j/m36Q4jktQYi7HUgP7l2DcCpyb59SQ7J9k+yeokUx07utkZwOuTLO/fbPRGuj2lm98c97AkoTsO9B7gniSPSPKM/n/yPwR+0F92L0l2SHJskj2q6scDcwD8JfCKJE9IZ5ckz+lL2FeA64FT+vN3SnJIf73vAPsm2WGa2/NR4CVJHtOv763Al6tqwwh3427AHcAtSfYBXjvCdegPIzgbeFN/v68Epvy83Rnuk+8A90/3RsrZekO/7UfRHSf7sf78y4HDk9yvP1ThxKHrfQeY8vOVq+o64IvA2/rH4NHAS5n+OOefGjUj/Xb+rb/8hcDnquq2fl3PY/piPFMOJuk+91lVPWrg2PDhn1cAJNme7jCRHwC/Mcs94ZIWCYux1IiqegfwKro3z91It+f1BLo9VNP5Y2AtcAXwdeCy/jzo3sz1L3Rl8WLg3VV1Ad2xo6cAN9G9/PwAujfmTeVFwIb+0IRX0JUfqmot3XHG7wJupjtk47j+snvo9lA/jO7NYRuB/9HP91lgHfCfSW6a4j74DPAG4O/oyvVDgaO3cPsHvRl4HHArcA5d2R3VCXQvvf8ncDrdcb7Tme4++Ve6JyrX9IeXzOZwiAvp7sPPAH9WVZ/qz/8Q3TG6G4BP8bPCvNnb6J4Y3ZKBTwwZcAzdGyA30b2x8w+r6tMjrGc2Gdm8/u/2x31vPh3gq9OM32IOJuz/Akel+1STv5jF9X6V7o2fz6Z78rV5j/JTJrJKSROR7r02kiRJ0rbNPcaSJEkSFmOL1AWEAAAAOElEQVRJkiQJsBhLkiRJgMVYkiRJAizGkiRJEmAxliRJkgCLsSRJkgRYjCVJkiTAYixJkiQB8P8BM7Nvt3j1Sv0AAAAASUVORK5CYII=\n",
                        "text/plain": [
                            "<Figure size 864x288 with 4 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "init = 2\n",
                "Ts = [2, 3, 5, 10]\n",
                "plot_cross_sectional_dists(mc1, init, Ts)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Since our Markov chain is aperiodic (i.e., every recurrent class is aperiodic),\n",
                "the marginal disribution at time $T$ converges as $T \\to \\infty$\n",
                "to some stationary distribution,\n",
                "and the limit distribution depends on the initial state,\n",
                "according to the probabilities that the state is absorbed into the recurrent classes."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "For initial states `3`, `4`, and `5`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 40,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAD0CAYAAACYReA6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHydJREFUeJzt3X+cXHV97/HX2wAivytEKwSIVfyB1l+NqA9spWpbggo+Wmuh6pXWSrlXbvWKVmqFIrVCbWu1V1qlarGAICraqFFQARUFJSBiA2JzaYAQkIAECAIS/Nw/zolOlp3ds2F2d2b39Xw85pGdc75zzmfO7Dv7me+cmUlVIUmSJM13D5vtAiRJkqRhYGMsSZIkYWMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYzxrknwxyWsnWP/BJMd23NaFSf5kcNVJ6mVepdFiZrWlbIwHKMnqJC/uMraqllbVx9rbHZ7kojHrj6yqv56OOtt9Lk5SSbaawm06379BSnJBknVJ7kzyvSSHzHQNmnvM6/RK8oK25nfNVg2aW8zs9Gj3e0+SDe3lvJmuYZh0fsCkWfRG4Kqq2pjkOcBXkjyhqm6a7cIkPViSrYH3A9+e7VokdfKyqvrKbBcxDJwxniabnqEm+fsktyf57yRLe9ZfmORPkjwZ+CDwvPaZ2vp2/ambZlqS/FKSz7ezpre3Py/qWMd+SVa0s60/SvLedtXX23/Xt/t9XpLHJTk/yW1Jbk1yRpJd2u2cBuwFfK4d/+ft8ucm+VaS9e1s7gEDOHybqaorq2rjpqvA1sCeg96P5i/zOnBHA+cBP5im7WueM7OaLjbG0+s5wDXAbsB7gI8kSe+AqroaOBK4uKp2qKpdxtnOw4B/A/amCc49wAc61vB+4P1VtRPwOODsdvlvtP/u0u73YiDAicDuwJNpms/j2zpfA1xP86xyh6p6T5I9gC8A7wIeCbwF+HSSheMV0v5ns77P5fMT3Yn2tvfSzEBdCKzoeP+lrsxrjy3Na5K9gT8GTuh4n6UtZWZ7PJS/scAZ7ROD85I8veN9n5NsjKfXdVX1r1X1APAx4DHAo6e6kaq6rao+XVU/qaq7gL8BXtDx5vcDj0+yW1VtqKpLJtjPqqr6clXdV1XrgPdOsp9XA8uranlV/ayqvkzTsB7UZ/svrapd+lxeOtGdaNfv2G773Kr62cR3W5oy87r59rc0r/8EHFtVGya9t9JDY2Y33/6WZvZVwGKaJwYXAOdumsmej2yMp9fNm36oqp+0P+4w1Y0k2S7Jh5Jcl+ROmpdodkmyoMPNXwc8AfhBkkuT9A1HkkclOSvJje1+Tqd5Jt7P3sDv9z4rBZ5P85/TwFXV/VX1ReB3khw8HfvQvGZeH6IkLwN2rKpPDGqb0gTM7ABU1Ter6p72icGJwHrg1we5j1Him++GQ02y/mjgicBzqurmJM8AvkvzsszEG676L+CwJA8Dfhf4VJJd++zzxHb506rqtiQvZ/OXk8be5gbgtKp6/WR1QPPxOfQP2zeqammfdWNtRfOSlTQbzGv/vL4IWJJkU8OyM/BAkl+tKj9NRrPFzE7tb2zR4b7PVc4YD4cfAYuSbNNn/Y405zytT/JI4K+6bjjJq5MsbE89WN8ufgBYB/wM+JUx+9nQ7mcP4K3j1Nk7/nTgZUl+J8mCJNsmOSB93rRQzcfn7NDnMm5gkzwpydIkj0iydZJX05y79bWux0AaMPPa/w/ssTSzZ89oL8uAfwX+qOsxkKaBme3/N3avJPsn2abd/ltpZrG/2fUYzDU2xsPhfGAlcHOSW8dZ/z7gEcCtwCXAl6aw7QOBlUk20LxJ4NCqurd92elvgG+2L9E8F3gn8CzgDpoT/s8Zs60TgXe0499SVTcAhwBvp/lP4AaaoA/y9yo0b064pd3HG4E/qKrLB7gPaSrMax9VdVdV3bzpQtNs3F1VPx7UPqQtYGb72xH4F+B24Mb2/iytqtsGuI+RkqrJXmGQJEmS5j5njCVJkiQ6NMZJPprkliT/2Wd9kvxTklVJrkzyrMGXKakrMyuNDvMqDZcuM8an0pxz0s9SYJ/2cgTNuSqSZs+pmFlpVJyKeZWGxqSNcVV9HZjojROHAP9ejUtoPvtvWj7HVtLkzKw0OsyrNFwGcY7xHjTvlNxkTbtM0nAys9LoMK/SDBrEF3yM9yHQ437URZIjaF4KYvvtt/+1Jz3pSQB8/8Y7BlBGN7+6x84zti9pKi677LJbq2rhDOyqU2b75VXS8OUVZv9vrH9fNcy6ZnYQjfEaYM+e64uAteMNrKpTgFMAlixZUitWrABg8TFfGEAZ3aw46SUzti9pKpJcN0O76pTZfnmVNHx5hdn/G+vfVw2zrpkdxKkUy4D/0b5z9rnAHVV10wC2K2l6mFlpdJhXaQZNOmOc5EzgAGC3JGtovipxa4Cq+iCwHDgIWAX8BL/6U5pVZlYaHeZVGi6TNsZVddgk6wt4w8AqkvSQmFlpdJhXabj4zXeSJEkSNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRLQsTFOcmCSa5KsSnLMOOv3SnJBku8muTLJQYMvVVIX5lUaLWZWGh6TNsZJFgAnA0uBfYHDkuw7Ztg7gLOr6pnAocA/D7pQSZMzr9JoMbPScOkyY7wfsKqqrq2qnwJnAYeMGVPATu3POwNrB1eipCkwr9JoMbPSENmqw5g9gBt6rq8BnjNmzPHAeUn+N7A98OKBVCdpqsyrNFrMrDREuswYZ5xlNeb6YcCpVbUIOAg4LcmDtp3kiCQrkqxYt27d1KuVNBnzKo0WMysNkS6N8Rpgz57ri3jwyzivA84GqKqLgW2B3cZuqKpOqaolVbVk4cKFW1axpImYV2m0mFlpiHRpjC8F9kny2CTb0Jz4v2zMmOuBFwEkeTJNaH26Ks088yqNFjMrDZFJG+Oq2ggcBZwLXE3zztiVSU5IcnA77Gjg9Um+B5wJHF5VY18KkjTNzKs0WsysNFy6vPmOqloOLB+z7Lien68C9h9saZK2hHmVRouZlYaH33wnSZIkYWMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRIAW812AZKk0bX4mC/MyH5Wn/SSGdmPpPnNGWNJkiQJG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkoCOjXGSA5Nck2RVkmP6jHllkquSrEzy8cGWKakr8yqNDvMqDZdJP64tyQLgZOC3gDXApUmWVdVVPWP2Af4C2L+qbk/yqOkqWFJ/5lUaHeZVGj5dZoz3A1ZV1bVV9VPgLOCQMWNeD5xcVbcDVNUtgy1TUkfmVRod5lUaMl0a4z2AG3qur2mX9XoC8IQk30xySZIDB1WgpCkxr9LoMK/SkOnyzXcZZ1mNs519gAOARcA3kjy1qtZvtqHkCOAIgL322mvKxUqalHmVRsfA8gpmVhqELjPGa4A9e64vAtaOM+Y/qur+qvpv4BqaIG+mqk6pqiVVtWThwoVbWrOk/syrNDoGllcws9IgdGmMLwX2SfLYJNsAhwLLxoz5LPCbAEl2o3np59pBFiqpE/MqjQ7zKg2ZSRvjqtoIHAWcC1wNnF1VK5OckOTgdti5wG1JrgIuAN5aVbdNV9GSxmdepdFhXqXh0+UcY6pqObB8zLLjen4u4M3tRdIsMq/S6DCv0nDxm+8kSZIkbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiSgY2Oc5MAk1yRZleSYCca9IkklWTK4EiVNhXmVRouZlYbHpI1xkgXAycBSYF/gsCT7jjNuR+DPgG8PukhJ3ZhXabSYWWm4dJkx3g9YVVXXVtVPgbOAQ8YZ99fAe4B7B1ifpKkxr9JoMbPSEOnSGO8B3NBzfU277OeSPBPYs6o+P8DaJE2deZVGi5mVhkiXxjjjLKufr0weBvwjcPSkG0qOSLIiyYp169Z1r1JSV+ZVGi1mVhoiXRrjNcCePdcXAWt7ru8IPBW4MMlq4LnAsvHeHFBVp1TVkqpasnDhwi2vWlI/5lUaLWZWGiJdGuNLgX2SPDbJNsChwLJNK6vqjqraraoWV9Vi4BLg4KpaMS0VS5qIeZVGi5mVhsikjXFVbQSOAs4FrgbOrqqVSU5IcvB0FyipO/MqjRYzKw2XrboMqqrlwPIxy47rM/aAh16WpC1lXqXRYmal4eE330mSJEnYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBHT8VApJGjaLj/nCjOxn9UkvmZH9SJJmnzPGkiRJEjbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkS0LExTnJgkmuSrEpyzDjr35zkqiRXJvlqkr0HX6qkLsyrNDrMqzRcJm2MkywATgaWAvsChyXZd8yw7wJLquppwKeA9wy6UEmTM6/S6DCv0vDpMmO8H7Cqqq6tqp8CZwGH9A6oqguq6ift1UuARYMtU1JH5lUaHeZVGjJdGuM9gBt6rq9pl/XzOuCL461IckSSFUlWrFu3rnuVkroyr9LoGFhewcxKg9ClMc44y2rcgcmrgSXA3423vqpOqaolVbVk4cKF3auU1JV5lUbHwPIKZlYahK06jFkD7NlzfRGwduygJC8G/hJ4QVXdN5jyJE2ReZVGh3mVhkyXGeNLgX2SPDbJNsChwLLeAUmeCXwIOLiqbhl8mZI6Mq/S6DCv0pCZtDGuqo3AUcC5wNXA2VW1MskJSQ5uh/0dsAPwySRXJFnWZ3OSppF5lUaHeZWGT5dTKaiq5cDyMcuO6/n5xQOuS9IWMq/S6DCv0nDxm+8kSZIkbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQI6NsZJDkxyTZJVSY4ZZ/3Dk3yiXf/tJIsHXaikbsyrNFrMrDQ8Jm2MkywATgaWAvsChyXZd8yw1wG3V9XjgX8E/nbQhUqanHmVRouZlYZLlxnj/YBVVXVtVf0UOAs4ZMyYQ4CPtT9/CnhRkgyuTEkdmVdptJhZaYh0aYz3AG7oub6mXTbumKraCNwB7DqIAiVNiXmVRouZlYbIVh3GjPestLZgDEmOAI5or25Ick2H/fezG3DrVG+Uwb4AtUU1DJg1zJ0a9h5ADcOaVxjRx8j/M4ajhgE/DltUwxiDyCsMb2bnwmNkDXNj/4OqoVNmuzTGa4A9e64vAtb2GbMmyVbAzsCPx26oqk4BTulS2GSSrKiqJYPYljVYw1ypgSHNKwzH8ZntGmZ7/9YwXDW0hjKzw3B8rGE4apjt/c90DV1OpbgU2CfJY5NsAxwKLBszZhnw2vbnVwDnV9WDns1KmnbmVRotZlYaIpPOGFfVxiRHAecCC4CPVtXKJCcAK6pqGfAR4LQkq2iexR46nUVLGp95lUaLmZWGS5dTKaiq5cDyMcuO6/n5XuD3B1vapAb2Eu9DYA0Na2gMQw3DmlcYjuMz2zXM9v7BGjYZhhqAoc3sMBwfa2jMdg2zvX+YwRriqzGSJEmSXwktSZIkASPaGE/29ZkzsP+PJrklyX/O9L57atgzyQVJrk6yMskbZ6GGbZN8J8n32hreOdM1tHUsSPLdJJ+fpf2vTvL9JFckWTEbNQyz2c5rW8OsZta8blbHrOa1rcHMTmC2M2tehyevbS3z6m/syJ1K0X595g+B36L5CJtLgcOq6qoZrOE3gA3Av1fVU2dqv2NqeAzwmKq6PMmOwGXAy2f4OATYvqo2JNkauAh4Y1VdMlM1tHW8GVgC7FRVL53Jfbf7Xw0sqarZ/pzHoTMMeW3rmNXMmtfN6pjVvLY1rMbMjmsYMmtehyevbS3z6m/sKM4Yd/n6zGlVVV9nnM+QnOEabqqqy9uf7wKu5sHfljTdNVRVbWivbt1eZvSZVpJFwEuAD8/kftXZrOcVZj+z5rVhXkfCrGfWvA5HXmF+ZnYUG+MuX585ryRZDDwT+PYs7HtBkiuAW4AvV9VM1/A+4M+Bn83wfnsVcF6Sy9J885R+wbyOYV5nPa9gZidiZnvM87zCcGR2RvM6io1xp6/GnC+S7AB8GnhTVd050/uvqgeq6hk039a0X5IZe9kryUuBW6rqspnaZx/7V9WzgKXAG9qXAdUwrz3M61DkFczsRMxsaz7nFYYqszOa11FsjLt8fea80J539GngjKo6ZzZrqar1wIXAgTO42/2Bg9vzj84CXpjk9BncPwBVtbb99xbgMzQvRaphXlvmdTjyCmZ2EmYW89oaiszOdF5HsTHu8vWZc157Yv5HgKur6r2zVMPCJLu0Pz8CeDHwg5naf1X9RVUtqqrFNL8H51fVq2dq/wBJtm/fnEGS7YHfBmbt00qGkHnFvMJw5BXMbAfzPrPmtTEMmZ2NvI5cY1xVG4FNX595NXB2Va2cyRqSnAlcDDwxyZokr5vJ/bf2B15D8wzuivZy0AzX8BjggiRX0vxn+uWqmrWPYJoljwYuSvI94DvAF6rqS7Nc09AYhrzCUGTWvA4PMzuBYciseQXM6yYznteR+7g2SZIkaTqM3IyxJEmSNB1sjCVJkiRsjCVJkiTAxliSJEkCbIwlSZIkwMZ4zkjyl0lWJrmy/WiZ5yR5U5LtOty20zhJg2NmpdFhXucPP65tDkjyPOC9wAFVdV+S3YBtgG8BS6rq1kluv7rLOEmDYWal0WFe5xdnjOeGxwC3VtV9AG34XgHsTvMB4RcAJPmXJCvaZ73vbJf92TjjfjvJxUkuT/LJ9vviSXJSkqvaZ8x/P/N3U5ozzKw0OszrPOKM8RzQhuoiYDvgK8AnquprY5+lJnlkVf04yQLgq8CfVdWVvePaZ8LnAEur6u4kbwMeDnyA5puInlRVlWSX9vvbJU2RmZVGh3mdX5wxngOqagPwa8ARwDrgE0kOH2foK5NcDnwXeAqw7zhjntsu/2aSK4DXAnsDdwL3Ah9O8rvATwZ9P6T5wsxKo8O8zi9bzXYBGoyqegC4ELgwyfdpwvZzSR4LvAV4dlXdnuRUYNtxNhWa72Q/7EErkv2AFwGHAkcBLxzkfZDmEzMrjQ7zOn84YzwHJHlikn16Fj0DuA64C9ixXbYTcDdwR5JHA0t7xveOuwTYP8nj221vl+QJ7UtJO1fVcuBN7T4kbQEzK40O8zq/OGM8N+wA/N8kuwAbgVU0L/kcBnwxyU1V9ZtJvgusBK4Fvtlz+1PGjDscODPJw9v176AJ9n8k2ZbmGe//mYk7Js1RZlYaHeZ1HvHNd5IkSRKeSiFJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsaY5J8sEkx87AflYneXHHsYcnuajn+oYkvzKgOt6e5MPtz4uTVJKBfKtpkr3aWhcMYnuSNOxsjKU5LMkfJlnRNjc3JflikufPdl2DMrbhBKiqI6vqr2erpi6qaoequnaiMUkOSLKmw7beXVV/Moi6xjb7VXV9W+sDg9j+FOo4PsnpE6zf0HP5WZJ7eq6/agr7eWqSc5PcmuRBXwOb5JFJPpPk7iTXJfnDLb1PkkaDjbE0RyV5M/A+4N3Ao4G9gH8GDukzfiCzjJo58/Uxa5v1HapqB+B64GU9y86YwqbuB84GXtdn/cnAT2ny8yrgX5I85aHULmm42RhLc1CSnYETgDdU1TlVdXdV3V9Vn6uqt7Zjjk/yqSSnJ7kTODzJw5O8L8na9vK+JA9vx++W5PNJ1if5cZJvJHlYu+5tSW5McleSa5K8qE9dByW5qh13Y5K39Kx7aZIr2u1/K8nTetbtmeScJOuS3JbkA0meDHwQeF47U7i+HXtqknf13Pb1SVa1NS9LsnvPukpyZJL/SnJ7kpOTpF33uCTnt/u7NckZSXbpePx3bfd1Z5LvAI8bs76SPL7fMUmyPfBFYPeemdDd+zxm482u/nH7+N2U5Oie/Y49Nj+flU5yGs2Tp8+1+/vzjDk1o61hWXssVyV5fc+2jk9ydpJ/b+/LyiRLJjhG709yQ3uMLkvy6+3yA4G3A3/Q1vG9Lsd8S1TVNVX1EWDlOPVtD/wecGxVbaiqi4BlwGumqx5Js8/GWJqbngdsC3xmknGHAJ8CdgHOAP4SeC7wDODpwH7AO9qxRwNrgIU0M2hvByrJE4GjgGdX1Y7A7wCr++zvI8CftuOeCpwPkORZwEeBPwV2BT4ELGsb9QXA54HrgMXAHsBZVXU1cCRwcTtT+KCmNckLgROBVwKPabdx1phhLwWe3d7fV7b1A6S97e7Ak4E9geP73K+xTgbubff5x+2lnwcdk6q6G1gKrO2ZCV3bjh/7mI3nN4F9gN8GjkmHc6Gr6jVsPvv6nnGGnUnzO7A78Arg3WOeBB1Mc3x3oWkiPzDBLi+l+T17JPBx4JNJtq2qL9G8yvGJto6nT1b7WGlOIVo/wWWvDpt5AvBAVf2wZ9n3AGeMpTnMxliam3YFbq2qjZOMu7iqPltVP6uqe2heLj6hqm6pqnXAO/nFDNn9NI3e3u3s8zeqqoAHgIcD+ybZuqpWV9X/67O/+9txO1XV7VV1ebv89cCHqurbVfVAVX0MuI+mSd+PphF7azvzfW87e9fFq4CPVtXlVXUf8Bc0M8yLe8acVFXrq+p64AKaZo2qWlVVX66q+9pj8V7gBZPtsG3kfw84rq33P4GPTXCTfsekn7GP2Xje2e77+8C/AYdNVvdkkuwJPB94W/sYXAF8mM1nUC+qquXtOcmn0TzZGFdVnV5Vt1XVxqr6B5rfoSc+1DrbbX+8qnaZ4HJ9h83sANwxZtkdwI6DqFHScLIxluam24DdMvk5qDeMub47zazqJte1ywD+DlgFnJfk2iTHQNNAAm+imU29JclZvacrjPF7wEHAdUm+luR57fK9gaN7Z/VoZmh3b/+9rkOTP57N7k9VbaA5Nnv0jLm55+ef0DREJHlUe19ubE9bOB3YrcM+FwJbsfmxva7PWOh/TPoZ+5hNNqb3MXwodgd+XFV3jdn2RMdy236/g0mOTnJ1kjvax3tnuh3fmbIB2GnMsp2Au8YZK2mOsDGW5qaLaV7Kf/kk48a+E38tTZO6yV7tMqrqrqo6uqp+BXgZ8OZNL6O3M3TPb29bwN+Ou7OqS6vqEOBRwGdp3vgETSP3N2Nm9barqjPbdXv1abAe9EkCE92f9rzRXYEbJ7kdNKdRFPC0qtoJeDXN6RWTWQdspGnoN+n70v0Ex6TffZvsPjPOvjedhnE3sF3Pul+ewrbXAo9M0jtjuhfdjuVm2vOJ30Zz6sovtafB3MEvjm+X+zjR9l+VzT+5Yuyly6kUPwS2SrJPz7KnM875yJLmDhtjaQ6qqjuA44CTk7w8yXZJtk6yNMl4545ucibwjiQLk+zWbuN0+Pmb4x6fJMCdNKdQPJDkiUlemOZNevcC97TrNpNkm7Zh2bmq7u/ZBsC/AkcmeU4a2yd5SduEfQe4CTipXb5tkv3b2/0IWJRkmz735+PAHyV5Rlvfu4FvV9XqDodxR5pZw/VJ9gDe2uE2tKcRnAMc3x73fYHXjjd2kmPyI2DXNG+knKpj230/Bfgj4BPt8iuAg9J8DNkv08z09/oRMO7nK1fVDcC3gBPbx+BpNJ/mMJVPgdhkR5onD+toms/j2Hx29kfA4rRv7pyqqjqj95MrxrlcD9D+rm0LbNNe37b9PaE9z/sc4IT2925/mvO7T9uSmiSNBhtjaY6qqvcCb6Z589w6mpnXo2hmJft5F7ACuBL4PnB5uwyaN3N9haZZvBj456q6kObc0JOAW2leSn8UzRvzxvMaYHV7asKRNLOwVNUKmvOMPwDcTnPKxuHtugdoZqgfT/PmsDXAH7TbO59mBu/mJLeOcwy+ChwLfJqmuX4ccOgE97/XO4Fn0cxkfoGmSerqKJpTMm4GTqU5z7effsfkBzRPVK5tTy+ZyukQX6M5hl8F/r6qzmuXn0bzBrLVwHn8omHe5ESaJ0br0/OJIT0Oo3kD5FqaN3b+VVV9eQp1bXIuzadu/JDmdIx72fz0j0+2/96WZLJzrh+KvWmeyG2aBb4HuKZn/f8CHgHcQvNY/M+qcsZYmsPSvHdGkiRJmt+cMZYkSZKwMZYkSZIAG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJgP8PR/6CoLQqJpEAAAAASUVORK5CYII=\n",
                        "text/plain": [
                            "<Figure size 864x216 with 3 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "inits = [3, 4, 5]\n",
                "T = 10\n",
                "\n",
                "fig, axes = plt.subplots(1, 3, figsize=(12, 3))\n",
                "\n",
                "for init, ax in zip(inits, axes):\n",
                "    draw_histogram(cross_sectional_dist(mc1, T=T, init=init), ax=ax,\n",
                "                   title='Initial state = {0}'.format(init),\n",
                "                   xlabel='States')\n",
                "\n",
                "fig.suptitle('Cross sectional distribution at T={0}'.format(T),\n",
                "             y=-0.05, fontsize=12)\n",
                "plt.show()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Powers of $P$"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "collapsed": true
            },
            "source": [
                "The marginal distributions at time $T$ are obtained by $P^T$."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 41,
            "metadata": {},
            "outputs": [],
            "source": [
                "np.set_printoptions(suppress=True)  # Suppress printing with floating point notation"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 42,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "P^10 =\n",
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33398438 0.         0.         0.66601562 0.        ]\n",
                        " [0.49807228 0.16679179 0.00001694 0.0007677  0.33319974 0.00115155]\n",
                        " [0.99902344 0.         0.         0.00097656 0.         0.        ]\n",
                        " [0.         0.33300781 0.         0.         0.66699219 0.        ]\n",
                        " [0.99902344 0.         0.         0.         0.         0.00097656]]\n",
                        "P^20 =\n",
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33333397 0.         0.         0.66666603 0.        ]\n",
                        " [0.49999809 0.16666679 0.         0.00000076 0.33333321 0.00000114]\n",
                        " [0.99999905 0.         0.         0.00000095 0.         0.        ]\n",
                        " [0.         0.33333302 0.         0.         0.66666698 0.        ]\n",
                        " [0.99999905 0.         0.         0.         0.         0.00000095]]\n",
                        "P^30 =\n",
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33333333 0.         0.         0.66666667 0.        ]\n",
                        " [0.5        0.16666667 0.         0.         0.33333333 0.        ]\n",
                        " [1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33333333 0.         0.         0.66666667 0.        ]\n",
                        " [1.         0.         0.         0.         0.         0.        ]]\n"
                    ]
                }
            ],
            "source": [
                "Ts = [10, 20, 30]\n",
                "for T in Ts:\n",
                "    print('P^{T} =\\n{P_T}'.format(T=T, P_T=np.linalg.matrix_power(mc1.P, T)))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "In the canonical form:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 43,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Q =\n",
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.         1.         0.         0.         0.        ]\n",
                        " [0.         0.5        0.5        0.         0.         0.        ]\n",
                        " [0.         0.         0.33333333 0.33333333 0.33333333 0.        ]\n",
                        " [0.5        0.         0.         0.         0.         0.5       ]\n",
                        " [0.5        0.         0.         0.         0.5        0.        ]]\n",
                        "Q^10 =\n",
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33398438 0.66601562 0.         0.         0.        ]\n",
                        " [0.         0.33300781 0.66699219 0.         0.         0.        ]\n",
                        " [0.49807228 0.16679179 0.33319974 0.00001694 0.0007677  0.00115155]\n",
                        " [0.99902344 0.         0.         0.         0.00097656 0.        ]\n",
                        " [0.99902344 0.         0.         0.         0.         0.00097656]]\n",
                        "Q^20 =\n",
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33333397 0.66666603 0.         0.         0.        ]\n",
                        " [0.         0.33333302 0.66666698 0.         0.         0.        ]\n",
                        " [0.49999809 0.16666679 0.33333321 0.         0.00000076 0.00000114]\n",
                        " [0.99999905 0.         0.         0.         0.00000095 0.        ]\n",
                        " [0.99999905 0.         0.         0.         0.         0.00000095]]\n",
                        "Q^30 =\n",
                        "[[1.         0.         0.         0.         0.         0.        ]\n",
                        " [0.         0.33333333 0.66666667 0.         0.         0.        ]\n",
                        " [0.         0.33333333 0.66666667 0.         0.         0.        ]\n",
                        " [0.5        0.16666667 0.33333333 0.         0.         0.        ]\n",
                        " [1.         0.         0.         0.         0.         0.        ]\n",
                        " [1.         0.         0.         0.         0.         0.        ]]\n"
                    ]
                }
            ],
            "source": [
                "Q = mc1.P[np.ix_(permutation, permutation)]\n",
                "print('Q =\\n{Q}'.format(Q=Q))\n",
                "for T in Ts:\n",
                "    print('Q^{T} =\\n{Q_T}'.format(T=T, Q_T=np.linalg.matrix_power(Q, T)))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Observe that the first three rows, which correspond to the recurrent states,\n",
                "are close to the stationary distributions associated with the corresponding recurrent classes."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Example 2: Periodic chain"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Consider the Markov chain given by the following stochastic matrix,\n",
                "taken from Exercise 9 (see also Exercise 11) in Jarvis and Shier\n",
                "(where the actual values of non-zero probabilities are not important):"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 44,
            "metadata": {},
            "outputs": [],
            "source": [
                "P = np.zeros((10, 10))\n",
                "P[0, 3] = 1\n",
                "P[1, [0, 4]] = 1/2\n",
                "P[2, 6] = 1\n",
                "P[3, [1, 2, 7]] = 1/3\n",
                "P[4, 3] = 1\n",
                "P[5, 4] = 1\n",
                "P[6, 3] = 1\n",
                "P[7, [6, 8]] = 1/2\n",
                "P[8, 9] = 1\n",
                "P[9, 5] = 1"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 45,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[[0.    0.    0.    1.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.5   0.    0.    0.    0.5   0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    1.    0.    0.    0.   ]\n",
                        " [0.    0.333 0.333 0.    0.    0.    0.    0.333 0.    0.   ]\n",
                        " [0.    0.    0.    1.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    1.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.5   0.    0.5   0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.    0.    0.    1.   ]\n",
                        " [0.    0.    0.    0.    0.    1.    0.    0.    0.    0.   ]]\n"
                    ]
                }
            ],
            "source": [
                "np.set_printoptions(precision=3)  # Reduce the number of digits printed\n",
                "print(P)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 46,
            "metadata": {},
            "outputs": [],
            "source": [
                "mc2 = MarkovChain(P)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "This Markov chain is irreducible:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 47,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "True"
                        ]
                    },
                    "execution_count": 47,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc2.is_irreducible"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "This Markov chain is periodic:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 48,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "False"
                        ]
                    },
                    "execution_count": 48,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc2.is_aperiodic"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Its period, which we denote by $d$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 49,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "3\n"
                    ]
                }
            ],
            "source": [
                "d = mc2.period\n",
                "print(d)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Identify the cyclic classes:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 50,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "[array([0, 4, 6, 8]), array([3, 9]), array([1, 2, 5, 7])]"
                        ]
                    },
                    "execution_count": 50,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc2.cyclic_classes"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Cyclic normal form"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "If a Markov chain is periodic with period $d \\geq 2$,\n",
                "then its transition probability matrix is written in the form (\"cyclic normal form\")\n",
                "$$\n",
                "\\begin{pmatrix}\n",
                "0       & P_0    & 0      & 0      & \\cdots & 0 \\\\\n",
                "0       & 0      & P_1    & 0      & \\cdots & 0 \\\\\n",
                "0       & 0      & 0      & P_2    & \\cdots & 0 \\\\\n",
                "\\vdots  & \\vdots & \\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
                "0       & 0      & 0      & 0      & \\cdots & P_{d-2} \\\\\n",
                "P_{d-1} & 0      & 0      & 0      & \\cdots & 0\n",
                "\\end{pmatrix}.\n",
                "$$"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Represent our Markov chain in cyclic normal form:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 51,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[[0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    1.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.    0.    1.    0.   ]\n",
                        " [0.5   0.5   0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    1.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    1.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.5   0.5   0.    0.    0.    0.    0.    0.   ]]\n"
                    ]
                }
            ],
            "source": [
                "permutation = np.concatenate(mc2.cyclic_classes)\n",
                "Q = mc2.P[np.ix_(permutation, permutation)]\n",
                "print(Q)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Re-define the Markov chain with the above matrix `Q`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 52,
            "metadata": {},
            "outputs": [],
            "source": [
                "mc2 = MarkovChain(Q)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Obtain the block components $P_0, \\cdots, P_{d-1}$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 53,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "P_0 =\n",
                        "[[1. 0.]\n",
                        " [1. 0.]\n",
                        " [1. 0.]\n",
                        " [0. 1.]]\n",
                        "P_1 =\n",
                        "[[0.333 0.333 0.    0.333]\n",
                        " [0.    0.    1.    0.   ]]\n",
                        "P_2 =\n",
                        "[[0.5 0.5 0.  0. ]\n",
                        " [0.  0.  1.  0. ]\n",
                        " [0.  1.  0.  0. ]\n",
                        " [0.  0.  0.5 0.5]]\n"
                    ]
                }
            ],
            "source": [
                "P_blocks = []\n",
                "\n",
                "for i in range(d):\n",
                "    P_blocks.append(mc2.P[np.ix_(mc2.cyclic_classes[i%d], mc2.cyclic_classes[(i+1)%d])])\n",
                "    print('P_{i} =\\n{P_block}'.format(i=i, P_block=P_blocks[i]))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "$P^d$ is block diagonal:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 54,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[[0.167 0.167 0.5   0.167 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.167 0.167 0.5   0.167 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.167 0.167 0.5   0.167 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    1.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.833 0.167 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.167 0.167 0.5   0.167]]\n"
                    ]
                }
            ],
            "source": [
                "P_power_d = np.linalg.matrix_power(mc2.P, d)\n",
                "print(P_power_d)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 55,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "0th diagonal block of P^d =\n",
                        "[[0.167 0.167 0.5   0.167]\n",
                        " [0.167 0.167 0.5   0.167]\n",
                        " [0.167 0.167 0.5   0.167]\n",
                        " [0.    1.    0.    0.   ]]\n",
                        "1st diagonal block of P^d =\n",
                        "[[0.833 0.167]\n",
                        " [1.    0.   ]]\n",
                        "2nd diagonal block of P^d =\n",
                        "[[0.333 0.333 0.    0.333]\n",
                        " [0.333 0.333 0.    0.333]\n",
                        " [0.333 0.333 0.    0.333]\n",
                        " [0.167 0.167 0.5   0.167]]\n"
                    ]
                }
            ],
            "source": [
                "P_power_d_blocks = []\n",
                "ordinals = ['0th', '1st', '2nd']\n",
                "\n",
                "for i in range(d):\n",
                "    P_power_d_blocks.append(P_power_d[np.ix_(mc2.cyclic_classes[i], mc2.cyclic_classes[i])])\n",
                "    print('{ordinal} diagonal block of P^d =\\n{P_power_d_block}'\n",
                "          .format(ordinal=ordinals[i], P_power_d_block=P_power_d_blocks[i]))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The $i$th diagonal block of $P^d$ equals $P_i P_{i+1} \\cdots P_{d-1} P_0 \\cdots P_{i-1}$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 56,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "P_0 P_1 P_2 =\n",
                        "[[0.167 0.167 0.5   0.167]\n",
                        " [0.167 0.167 0.5   0.167]\n",
                        " [0.167 0.167 0.5   0.167]\n",
                        " [0.    1.    0.    0.   ]]\n",
                        "P_1 P_2 P_0 =\n",
                        "[[0.833 0.167]\n",
                        " [1.    0.   ]]\n",
                        "P_2 P_0 P_1 =\n",
                        "[[0.333 0.333 0.    0.333]\n",
                        " [0.333 0.333 0.    0.333]\n",
                        " [0.333 0.333 0.    0.333]\n",
                        " [0.167 0.167 0.5   0.167]]\n"
                    ]
                }
            ],
            "source": [
                "products = []\n",
                "\n",
                "for i in range(d):\n",
                "    R = np.eye(P_blocks[i].shape[0])\n",
                "    string = ''\n",
                "    for j in range(d):\n",
                "        R = R.dot(P_blocks[(i+j)%d])\n",
                "        string += 'P_{0} '.format((i+j)%d)\n",
                "    products.append(R)\n",
                "    print(string + '=\\n{R}'.format(R=R))"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 57,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "True\n",
                        "True\n",
                        "True\n"
                    ]
                }
            ],
            "source": [
                "for matrix0, matrix1 in zip(P_power_d_blocks, products):\n",
                "    print(np.array_equal(matrix0, matrix1))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Stationary distributions"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The Markov chain `mc2` has a unique stationary distribution,\n",
                "which we denote by $\\pi$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 58,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "1"
                        ]
                    },
                    "execution_count": 58,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "len(mc2.stationary_distributions)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 59,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[0.048 0.095 0.143 0.048 0.286 0.048 0.095 0.095 0.048 0.095]\n"
                    ]
                }
            ],
            "source": [
                "pi = mc2.stationary_distributions[0]\n",
                "print(pi)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 60,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEWCAYAAACKSkfIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGsxJREFUeJzt3X20HXV97/H3p4kBeZDH1ApJINT4ELQFPQaVFq0gBlFiK9bA1Qut3hRLbmupS2P1Ao33Wmpd3q7bpmquRBGFiPjQXAkFFNCqoDlABAJGQgzkEJBAUJDnwOf+MRMZNifZc5KTvQ/5fV5rnXXm4Tfz+87OyefM/u2ZObJNRESU4bf6XUBERPROQj8ioiAJ/YiIgiT0IyIKktCPiChIQj8ioiAJ/eg7Sf9F0qX9rmOkJJ0s6fuN+V9LOmiU9v13kj5XTx8oyZLGj9K+p9S1jhuN/cWzS0I/upL0B5J+KOlXkjZI+oGkV9XrnhZ8Lfb1jACz/WXbR2+P2nvJ9m62V2+pjaTXSxpqsa+P237vaNQlaY2koxr7vr2u9YnR2H88u4zKmUPsuCQ9D/gW8D7gAmAC8IfAo/2sazRIGjcWg0/SeNsb+11H7Jhyph/dvAjA9vm2n7D9sO1LbV8v6aXAZ4DX1MMFvwSQdKyk6yTdL2mtpDMb+/te/f2X9TavGWaY5LWSltXvLJZJem1j3ZWSPla/23hA0qWS9m2s/6qku+ptvyfp4Ma6L0j6tKSlkh4ETpP0i+a7Dklvl7R8uBdC0j6SltTH9WPgdzvWW9IL6+k3S7qprvEOSR+QtCtwMbBffey/lrSfpDMlXSjpS5LuB06ul32po4Q/l7RO0p2S/rbjuP5nY/437yYknQtMAf5f3d8HO99t1TUsqd/FrZL03xr7OlPSBZK+WB/LCkkDw70+8eyQ0I9ufgY8IekcScdI2mvTCts3A6cAV9XDBXvWqx4E/iuwJ3As8D5Jb6vXHVF/37Pe5qpmZ5L2Bi4C/g+wD/Ap4CJJ+zSanQj8GfDbVO88PtBYdzEwrV53LfDljuM5EfhfwO7AvwD3Am9srH8XcO5mXosFwCPAC4A/r78252zgL2zvDrwMuNz2g8AxwLr62Hezva5uPwu4kOo166x5kz+qj+1oYF5zyGZzbL8buB14a93fJ4Zpdj4wBOwHHA98XNKRjfXHAYvr2pYA/9qt3xi7EvqxRbbvB/4AMPB/gfX1WeHzt7DNlbZvsP2k7eupQuV1Lbs8FrjF9rm2N9o+H/gp8NZGm8/b/pnth6mGnA5p9L3I9gO2HwXOBH5f0h6Nbf/d9g/q2h4BzqEK+k2/cN4EnNdZVP2h59uB020/aPvGetvNeRyYLul5tu+zfW2X477K9jfruh7eTJu/r/u+Afg8cEKXfXYlaTLVv++HbD9ieznwOeDdjWbft720Hgo7F/j9be03+iehH13Zvtn2ybYnUZ217gf88+baSzpM0hWS1kv6FdW7gX03177DfsBtHctuA/ZvzN/VmH4I2K3ud5yksyTdWg+TrKnbNPte27HvLwFvlbQb8KfAf9q+c5i6JlJ9BtbcvrPOprcDbwZuk/RdSa/ZQtvh6urW5jaq12pb7QdssP1Ax7639HrvrFG6kih6L6EfI2L7p8AXqMIfqncAnc6jGgaYbHsPqnF/baF90zrggI5lU4A7WpR3ItUwyVHAHsCB9XI12jytf9t3AFcBf0x1dru5oZ31wEZgckddw7K9zPYsqmGmb1K9I3lG/5urazM6+940NPQgsEtj3e+MYN/rgL0l7d6x7zavdzwLJfRjiyS9RNLfSppUz0+mGla4um7yC2CSpAmNzXanOnt8RNIMqjDeZD3wJLC569mXAi+SdKKk8ZLeCUynuoKom92priq6lyoEP97qIOGLwAeBlwPfGK5BPbTxdeBMSbtImg6cNFxbSRNU3Xuwh+3HgfuBTVcJ/QLYp2PIqa3/Ufd9MNVnGl+ply8H3ixpb0m/A7y/Y7tfsJnX2/Za4IfAP0jaWdLvAe9h858rxLNcQj+6eQA4DPhRfcXL1cCNwKarRy4HVgB3SbqnXvaXwHxJDwCn89RZLrYfovog9QeSfinp1c3ObN8LvKXe/71UYfwW2/fQ3RephibuAG7iqV9M3XyD6t3FN+oPWzdnLtVQ0l1U73Y+v4W27wbW1MNMp1B/blC/UzofWF0f/0iGaL4LrAK+A3zS9qYb2s4FfkI1nHUpT/0y2OQfgI/W/X2AZzqB6l3ROqrX4gzbl42grngWUf6ISgRIupXqaptv97uWiO0pZ/pRPElvpxr3vrzftURsb61CX9JMSSvrGzfmDbP+FEk3SFou6fv1eOemW+4frpcvl/SZ0T6AiG0h6Urg08Cptp/sczkR213X4Z36+uSfUd3AMgQsA06wfVOjzfPq67mRdBzwl7ZnSjoQ+Jbtlz1jxxER0XNtzvRnAKtsr7b9GNWdebOaDTYFfm1X2l1+FhERPdbmBov9efpNIUNUV3M8jaRTgdOobot/Q2PVVEnXUV229lHb/znMtnOAOQC77rrrK1/ykpe0PoCIiIBrrrnmHtsTu7VrE/oaZtkzzuRtLwAWSDoR+CjVNcx3AlNs3yvplcA3JR3c8c4A2wuBhQADAwMeHBxsUVZERGwiaUt3iP9Gm+GdIZ5+J+AknroTcDiLgbcB2H60vu4a29cAt1I/tTEiInqvTegvA6ZJmlrfdTmb6hb735A0rTF7LHBLvXxi/UEwqv6i0DRgi39kIiIitp+uwzu2N0qaC1wCjAMW2V4haT4waHsJMLd+zOvjwH08dXv6EVR3Zm6kug39FNsbtseBREREd2PujtyM6UdEjJyka2x3/QM3uSM3IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIK0Cn1JMyWtlLRK0rxh1p8i6QZJyyV9X9L0xroP19utlPSm0Sw+IiJGpmvoSxoHLACOAaYDJzRDvXae7ZfbPgT4BPCpetvpwGzgYGAm8G/1/iIiog/anOnPAFbZXm37MWAxMKvZwPb9jdldAdfTs4DFth+1/XNgVb2/iIjog/Et2uwPrG3MDwGHdTaSdCpwGjABeENj26s7tt1/mG3nAHMApkyZ0qbuiIjYCm3O9DXMMj9jgb3A9u8CHwI+OsJtF9oesD0wceLEFiVFRMTWaBP6Q8DkxvwkYN0W2i8G3raV20ZExHbUJvSXAdMkTZU0geqD2SXNBpKmNWaPBW6pp5cAsyXtJGkqMA348baXHRERW6PrmL7tjZLmApcA44BFtldImg8M2l4CzJV0FPA4cB9wUr3tCkkXADcBG4FTbT+xnY4lIiK6kP2MIfa+GhgY8ODgYL/LiIh4VpF0je2Bbu1yR25EREES+hERBUnoR0QUpM3NWRFjzoHzLupZX2vOOrZnfUVsbznTj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgrUJf0kxJKyWtkjRvmPWnSbpJ0vWSviPpgMa6JyQtr7+WjGbxERExMl3/Rq6kccAC4I3AELBM0hLbNzWaXQcM2H5I0vuATwDvrNc9bPuQUa47IiK2Qpsz/RnAKturbT8GLAZmNRvYvsL2Q/Xs1cCk0S0zIiJGQ5vQ3x9Y25gfqpdtznuAixvzO0salHS1pLcNt4GkOXWbwfXr17coKSIitkbX4R1AwyzzsA2ldwEDwOsai6fYXifpIOBySTfYvvVpO7MXAgsBBgYGht13RERsuzZn+kPA5Mb8JGBdZyNJRwEfAY6z/eim5bbX1d9XA1cCh25DvRERsQ3ahP4yYJqkqZImALOBp12FI+lQ4LNUgX93Y/leknaqp/cFDgeaHwBHREQPdR3esb1R0lzgEmAcsMj2CknzgUHbS4B/AnYDvioJ4HbbxwEvBT4r6UmqXzBndVz1ExERPdRmTB/bS4GlHctOb0wftZntfgi8fFsKjIiI0ZM7ciMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSKvQlzRT0kpJqyTNG2b9aZJuknS9pO9IOqCx7iRJt9RfJ41m8RERMTJdQ1/SOGABcAwwHThB0vSOZtcBA7Z/D7gQ+ES97d7AGcBhwAzgDEl7jV75ERExEm3O9GcAq2yvtv0YsBiY1Wxg+wrbD9WzVwOT6uk3AZfZ3mD7PuAyYObolB4RESPVJvT3B9Y25ofqZZvzHuDirdw2IiK2o/Et2miYZR62ofQuYAB43Ui2lTQHmAMwZcqUFiVFRMTWaHOmPwRMbsxPAtZ1NpJ0FPAR4Djbj45kW9sLbQ/YHpg4cWLb2iMiYoTahP4yYJqkqZImALOBJc0Gkg4FPksV+Hc3Vl0CHC1pr/oD3KPrZRER0Qddh3dsb5Q0lyqsxwGLbK+QNB8YtL0E+CdgN+CrkgBut32c7Q2SPkb1iwNgvu0N2+VIIiKiqzZj+theCiztWHZ6Y/qoLWy7CFi0tQVGRMToyR25EREFSehHRBQkoR8RUZCEfkREQRL6EREFSehHRBQkoR8RUZCEfkREQRL6EREFSehHRBQkoR8RUZCEfkREQRL6EREFSehHRBQkoR8RUZCEfkREQRL6EREFSehHRBQkoR8RUZCEfkREQRL6EREFSehHRBSkVehLmilppaRVkuYNs/4ISddK2ijp+I51T0haXn8tGa3CIyJi5MZ3ayBpHLAAeCMwBCyTtMT2TY1mtwMnAx8YZhcP2z5kFGqNiIht1DX0gRnAKturASQtBmYBvwl922vqdU9uhxojImKUtBne2R9Y25gfqpe1tbOkQUlXS3rbiKqLiIhR1eZMX8Ms8wj6mGJ7naSDgMsl3WD71qd1IM0B5gBMmTJlBLuOiIiRaHOmPwRMbsxPAta17cD2uvr7auBK4NBh2iy0PWB7YOLEiW13HRERI9Qm9JcB0yRNlTQBmA20ugpH0l6Sdqqn9wUOp/FZQERE9FbX0Le9EZgLXALcDFxge4Wk+ZKOA5D0KklDwDuAz0paUW/+UmBQ0k+AK4CzOq76iYiIHmozpo/tpcDSjmWnN6aXUQ37dG73Q+Dl21hj1A6cd1HP+lpz1rE96ysieid35EZEFCShHxFRkIR+RERBEvoREQVJ6EdEFCShHxFRkIR+RERBEvoREQVJ6EdEFCShHxFRkIR+RERBEvoREQVJ6EdEFCShHxFRkIR+RERBEvoREQVJ6EdEFCShHxFRkIR+RERBEvoREQVJ6EdEFCShHxFRkIR+RERBWoW+pJmSVkpaJWneMOuPkHStpI2Sju9Yd5KkW+qvk0ar8IiIGLmuoS9pHLAAOAaYDpwgaXpHs9uBk4HzOrbdGzgDOAyYAZwhaa9tLzsiIrZGmzP9GcAq26ttPwYsBmY1G9heY/t64MmObd8EXGZ7g+37gMuAmaNQd0REbIU2ob8/sLYxP1Qva6PVtpLmSBqUNLh+/fqWu46IiJFqE/oaZplb7r/VtrYX2h6wPTBx4sSWu46IiJFqE/pDwOTG/CRgXcv9b8u2ERExysa3aLMMmCZpKnAHMBs4seX+LwE+3vjw9mjgwyOuss8OnHdRz/pac9axPesrdgy9+vnc0s9m/o88Zay/Fl3P9G1vBOZSBfjNwAW2V0iaL+k4AEmvkjQEvAP4rKQV9bYbgI9R/eJYBsyvl0VERB+0OdPH9lJgacey0xvTy6iGbobbdhGwaBtqjIiIUZI7ciMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCpLQj4goSKvQlzRT0kpJqyTNG2b9TpK+Uq//kaQD6+UHSnpY0vL66zOjW35ERIzE+G4NJI0DFgBvBIaAZZKW2L6p0ew9wH22XyhpNvCPwDvrdbfaPmSU646IiK3Q5kx/BrDK9mrbjwGLgVkdbWYB59TTFwJHStLolRkREaOhTejvD6xtzA/Vy4ZtY3sj8Ctgn3rdVEnXSfqupD8crgNJcyQNShpcv379iA4gIiLa6zq8Awx3xu6Wbe4Epti+V9IrgW9KOtj2/U9raC8EFgIMDAw8bd8HzruoRYmjY81Zx/asr2er/Hs8Ja/F2NOrf5Nn879HmzP9IWByY34SsG5zbSSNB/YANth+1Pa9ALavAW4FXrStRUdExNZpE/rLgGmSpkqaAMwGlnS0WQKcVE8fD1xu25Im1h8EI+kgYBqwenRKj4iIkeo6vGN7o6S5wCXAOGCR7RWS5gODtpcAZwPnSloFbKD6xQBwBDBf0kbgCeAU2xu2x4FERER3bcb0sb0UWNqx7PTG9CPAO4bZ7mvA17axxoiIGCW5IzcioiAJ/YiIgiT0IyIKktCPiChIQj8ioiAJ/YiIgiT0IyIKktCPiChIQj8ioiAJ/YiIgiT0IyIKktCPiChIQj8ioiAJ/YiIgiT0IyIKktCPiChIQj8ioiAJ/YiIgiT0IyIKktCPiChIQj8ioiAJ/YiIgrQKfUkzJa2UtErSvGHW7yTpK/X6H0k6sLHuw/XylZLeNHqlR0TESHUNfUnjgAXAMcB04ARJ0zuavQe4z/YLgf8N/GO97XRgNnAwMBP4t3p/ERHRB23O9GcAq2yvtv0YsBiY1dFmFnBOPX0hcKQk1csX237U9s+BVfX+IiKiD2R7yw2k44GZtt9bz78bOMz23EabG+s2Q/X8rcBhwJnA1ba/VC8/G7jY9oUdfcwB5tSzLwZWbuNx7Qvcs437GA1joY6xUAOMjTrGQg0wNuoYCzXA2KhjLNQA217HAbYndms0vsWONMyyzt8Um2vTZltsLwQWtqilFUmDtgdGa3/P5jrGQg1jpY6xUMNYqWMs1DBW6hgLNfSyjjbDO0PA5Mb8JGDd5tpIGg/sAWxouW1ERPRIm9BfBkyTNFXSBKoPZpd0tFkCnFRPHw9c7mrcaAkwu766ZyowDfjx6JQeEREj1XV4x/ZGSXOBS4BxwCLbKyTNBwZtLwHOBs6VtIrqDH92ve0KSRcANwEbgVNtP7GdjqVp1IaKttFYqGMs1ABjo46xUAOMjTrGQg0wNuoYCzVAj+ro+kFuRETsOHJHbkREQRL6EREF2eFCv9sjI3pUwyJJd9f3L/SFpMmSrpB0s6QVkv66DzXsLOnHkn5S1/D3va6hUcs4SddJ+lYfa1gj6QZJyyUN9rGOPSVdKOmn9c/Ha3rc/4vr12DT1/2S3t/LGhq1/E39s3mjpPMl7dyHGv667n9FT14H2zvMF9UHzbcCBwETgJ8A0/tQxxHAK4Ab+/havAB4RT29O/CzXr8WVPdp7FZPPwf4EfDqPr0epwHnAd/q47/JGmDffvXfqOMc4L319ARgzz7WMg64i+rGol73vT/wc+C59fwFwMk9ruFlwI3ALlQX1nwbmLY9+9zRzvTbPDJiu7P9PaqrmPrG9p22r62nHwBupvoh72UNtv3revY59VfPrxyQNAk4Fvhcr/seayQ9j+qk5GwA24/Z/mUfSzoSuNX2bX3qfzzw3Pr+ol3o/X1EL6V6asFDtjcC3wX+eHt2uKOF/v7A2sb8ED0OurGofurpoVRn2r3ue5yk5cDdwGW2e14D8M/AB4En+9B3k4FLJV1TP3qkHw4C1gOfr4e7Pidp1z7VAtXl3ef3o2PbdwCfBG4H7gR+ZfvSHpdxI3CEpH0k7QK8maff0DrqdrTQb/XYh5JI2g34GvB+2/f3un/bT9g+hOpu7BmSXtbL/iW9Bbjb9jW97HczDrf9Cqon1p4q6Yg+1DCeaujx07YPBR4E+vXZ1wTgOOCrfep/L6qRgKnAfsCukt7Vyxps30z1VOLLgP+gGpLeuD373NFCP499aJD0HKrA/7Ltr/ezlnoI4UqqR2z30uHAcZLWUA33vUHSl3pcAwC219Xf7wa+QX+eODsEDDXecV1I9UugH44BrrX9iz71fxTwc9vrbT8OfB14ba+LsH227VfYPoJqWPiW7dnfjhb6bR4ZUYT60dZnAzfb/lSfapgoac96+rlU/8l+2ssabH/Y9iTbB1L9PFxuu6dncwCSdpW0+6Zp4Giqt/Y9ZfsuYK2kF9eLjqS6Y74fTqBPQzu124FXS9ql/v9yJNVnXz0l6bfr71OAP2E7vyZtnrL5rOHNPDKi13VIOh94PbCvpCHgDNtn97iMw4F3AzfUY+oAf2d7aQ9reAFwTv2Hc34LuMB23y6Z7LPnA9+osoXxwHm2/6NPtfx34Mv1idFq4M96XUA9fv1G4C963fcmtn8k6ULgWqohlevozyMZviZpH+BxqkfV3Lc9O8tjGCIiCrKjDe9ERMQWJPQjIgqS0I+IKEhCPyKiIAn9iIiCJPSjSJI+Uj/V8Pr6SY+HSXp/fSlht21btYsYi3LJZhSnfpTwp4DX235U0r5UT5v8ITBg+54u269p0y5iLMqZfpToBcA9th8FqMP7eKrnr1wh6QoASZ+WNNj8WwCS/mqYdkdLukrStZK+Wj/vCElnSbqpfjfxyd4fZsQz5Uw/ilOH8vepHqX7beArtr/beQYvaW/bG+o7ir8D/JXt65vt6ncJXweOsf2gpA8BOwH/ClwFvMS2Je3Z50cYRwA5048C1c/4fyUwh+oxw1+RdPIwTf9U0rVUt+cfDEwfps2r6+U/qB93cRJwAHA/8AjwOUl/Ajw02scRsTV2qGfvRLRl+wmqp35eKekGqrD+DUlTgQ8Ar7J9n6QvAMP9KT1R/Z2AE56xQppB9RCv2cBc4A2jeQwRWyNn+lGc+m+0TmssOgS4DXiA6k9LAjyP6lnzv5L0fKrHAG/SbHc1cLikF9b73kXSi+ohpD3qB9y9v+4jou9yph8l2g34l/qxzxuBVVRDPScAF0u60/YfSboOWEH1JMofNLZf2NHuZOB8STvV6z9K9Yvh3+s/tC3gb3pxYBHd5IPciIiCZHgnIqIgCf2IiIIk9CMiCpLQj4goSEI/IqIgCf2IiIIk9CMiCvL/AcYAQxsncBxkAAAAAElFTkSuQmCC\n",
                        "text/plain": [
                            "<Figure size 432x288 with 1 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "draw_histogram(pi,\n",
                "               title='Stationary distribution', xlabel='States', ylim=(0, 0.35))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Obtain the stationary distributions $\\pi^0, \\ldots, \\pi^{d-1}$\n",
                "each associated with the diagonal blocks of $P^d$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 61,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "pi^0 =\n",
                        "[0.143 0.286 0.429 0.143]\n",
                        "pi^1 =\n",
                        "[0.857 0.143]\n",
                        "pi^2 =\n",
                        "[0.286 0.286 0.143 0.286]\n"
                    ]
                }
            ],
            "source": [
                "pi_s = []\n",
                "\n",
                "for i in range(d):\n",
                "    pi_s.append(MarkovChain(P_power_d_blocks[i]).stationary_distributions[0])\n",
                "    print('pi^{i} =\\n{pi_i}'.format(i=i, pi_i=pi_s[i]))"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 62,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAD3CAYAAAAe0ZKUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm4JHV97/H3R4Z9VZgYYVBQEZ2gUTMCihoiRFkiJNFrIIkGY0Jyr0SNGoPGS4BsJPGaPDch5iIiigIi0QR1DLiARhRlQGQVM8Agw4AM+yYC+r1/VA02Z87S55w6c6pn3q/n6Wd6+fWvvl3dnzPfrqruTlUhSZIkbeieMN8FSJIkSX1gYyxJkiRhYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjrFaSv0ryX0nOTrLFfNcjaWJJtk3yrST3J9ljvuuRNLEkL07yjSRfSXJGko3nuyZNzMZYtP+xPqOqXgZ8EfjdeS5J0uQeBA4Gzp7vQiRN6UbgFVX1i8D1wKHzXI8mYWMsgJcBn2/Pfx546TzWImkKVfVIVa2e7zokTa2qVlXVD9uLjwI/mc96NDkbYwE8EbinPX8P8KR5rEWSpPVOkl2BA4HPznctmpiNsQDuArZtz28L3DmPtUiStF5Jsg3wEeD1VfXwfNejidkYbyCSbJrk3iQ/bj+wc397/iFgAfCqduirgAvnr1JJMHlmk+w/3/VJ+qkh8noGcGxVXTvftWpyNsYbiKr6EfBW4EtVtVVVbQVcB7yyqv4JuDHJf9E0xqfMY6mSmDKzX0yyFHgl8MEkR8xjqdIGb7K8Ak8B9gKOSXJBkt+Yx1I1hQXzXYDWqecCVwAk2RJ4BnAlQFW9ex7rkjS+yTJ70DzWJWlt4+a1qr4KnDafhWl4bjHesDwW2vb8LVXl8cRSf5lZaXSY1/WAjfGGZTC0Pz9wXlI/mVlpdJjX9YCN8QYiyQ7AQuDq9qpnA9+bv4okTcbMSqPDvK4/bIw3HM8Drhv4kvGbgdcl2Wsea5I0MTMrjQ7zup5IVc13DZIkSdK8c4uxJEmShI2xJEmSBNgYS5IkScAQjXGSU5LcluTKCW5Pkv+bZHmSy5O8sPsyJQ3LzEqjw7xK/TLMFuNTgQMmuf1AYLf2dCTwgdmXJWkWTsXMSqPiVMyr1BtTNsbtTxlO9ssthwIfrcZFwHZJntJVgZKmx8xKo8O8Sv2yoIM5dgJuGri8sr3ulrEDkxxJ846XLbfc8hee/exnd7B4af1wySWX3F5VC9fBoobKrHmVJta3vIKZlSYzbGa7aIwzznXjfjlyVZ0EnASwZMmSWrZsWQeLl9YPSW5cV4sa57q1MmtepYn1La9gZqXJDJvZLr6VYiWw88DlRcCqDuaVNDfMrDQ6zKu0DnXRGJ8DvKH95OzewD1VtdYuHkm9YWal0WFepXVoykMpkpwB7AvskGQl8OfAxgBV9a/AUuAgYDnwIPDGuSpW0tTMrDQ6zKvUL1M2xlV1+BS3F/DmziqSNCtmVhod5lXqF3/5TpIkScLGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkABbMdwGStCHY5ejPzer+K044uKNKJEkTcYuxJEmShI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEnAkI1xkgOSXJtkeZKjx7n9qUnOT/LtJJcnOaj7UiUNw7xKo8XMSv0xZWOcZCPgROBAYDFweJLFY4a9Fzirql4AHAb8S9eFSpqaeZVGi5mV+mWYLcZ7Asur6vqqehg4Ezh0zJgCtmnPbwus6q5ESdNgXqXRYmalHhnmBz52Am4auLwS2GvMmGOB85L8EbAlsH8n1UmaLvMqjRYzK/XIMFuMM851Neby4cCpVbUIOAg4Lclacyc5MsmyJMtWr149/WolTcW8SqPFzEo9MkxjvBLYeeDyItbejfMm4CyAqvoGsBmww9iJquqkqlpSVUsWLlw4s4olTca8SqPFzEo9MkxjfDGwW5Jdk2xCc+D/OWPGfB/YDyDJc2hC69tVad0zr9JoMbNSj0zZGFfVo8BRwLnANTSfjL0qyfFJDmmHvQP4/STfAc4AjqiqsbuCJM0x8yqNFjMr9cswH76jqpYCS8dcd8zA+auBfbotTdJMmFdptJhZqT/85TtJkiQJG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAkYsjFOckCSa5MsT3L0BGNel+TqJFclOb3bMiUNy7xKo8O8Sv2yYKoBSTYCTgR+GVgJXJzknKq6emDMbsC7gX2q6q4kPzNXBUuamHmVRod5lfpnmC3GewLLq+r6qnoYOBM4dMyY3wdOrKq7AKrqtm7LlDQk8yqNDvMq9cwwjfFOwE0Dl1e21w16FvCsJBcmuSjJAeNNlOTIJMuSLFu9evXMKpY0GfMqjY7O8gpmVurCMI1xxrmuxlxeAOwG7AscDpycZLu17lR1UlUtqaolCxcunG6tkqZmXqXR0VlewcxKXRimMV4J7DxweRGwapwx/1FVj1TVDcC1NEGWtG6ZV2l0mFepZ4ZpjC8Gdkuya5JNgMOAc8aM+XfglwCS7ECz6+f6LguVNBTzKo0O8yr1zJSNcVU9ChwFnAtcA5xVVVclOT7JIe2wc4E7klwNnA/8SVXdMVdFSxqfeZVGh3mV+mfKr2sDqKqlwNIx1x0zcL6At7cnSfPIvEqjw7xK/eIv30mSJEnYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTYGEuSJEmAjbEkSZIE2BhLkiRJACyY7wL6bJejPzer+6844eCOKpEkSdJcc4uxJEmSxJCNcZIDklybZHmSoycZ99oklWRJdyVKmg7zKo0WMyv1x5SNcZKNgBOBA4HFwOFJFo8zbmvgLcA3uy5S0nDMqzRazKzUL8NsMd4TWF5V11fVw8CZwKHjjPsL4O+AhzqsT9L0mFdptJhZqUeGaYx3Am4auLyyve4xSV4A7FxVn51soiRHJlmWZNnq1aunXaykKZlXabSYWalHhmmMM8519diNyROAfwDeMdVEVXVSVS2pqiULFy4cvkpJwzKv0mgxs1KPDNMYrwR2Hri8CFg1cHlrYA/ggiQrgL2Bc/xwgDQvzKs0Wsys1CPDNMYXA7sl2TXJJsBhwDlrbqyqe6pqh6rapap2AS4CDqmqZXNSsaTJmFdptJhZqUembIyr6lHgKOBc4BrgrKq6KsnxSQ6Z6wIlDc+8SqPFzEr9MtQv31XVUmDpmOuOmWDsvrMvS9JMmVdptJhZqT/85TtJkiQJG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmwMZYkSZIAG2NJkiQJsDGWJEmSABtjSZIkCbAxliRJkgAbY0mSJAmABfNdQNd2OfpzM77vihMO7rASSZKk7s2m14G1+50+905dP9apuMVYkiRJwsZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQKGbIyTHJDk2iTLkxw9zu1vT3J1ksuTfCnJ07ovVdIwzKs0Osyr1C9TNsZJNgJOBA4EFgOHJ1k8Zti3gSVV9TzgbODvui5U0tTMqzQ6zKvUP8NsMd4TWF5V11fVw8CZwKGDA6rq/Kp6sL14EbCo2zIlDcm8SqPDvEo9M0xjvBNw08Dlle11E3kT8PnxbkhyZJJlSZatXr16+ColDcu8SqOjs7yCmZW6MExjnHGuq3EHJr8NLAH+frzbq+qkqlpSVUsWLlw4fJWShmVepdHRWV7BzEpdWDDEmJXAzgOXFwGrxg5Ksj/wZ8AvVtWPuilP0jSZV2l0mFepZ4bZYnwxsFuSXZNsAhwGnDM4IMkLgP8HHFJVt3VfpqQhmVdpdJhXqWembIyr6lHgKOBc4BrgrKq6KsnxSQ5ph/09sBXwySSXJTlngukkzSHzKo0O8yr1zzCHUlBVS4GlY647ZuD8/h3XJWmGzKs0Osyr1C/+8p0kSZKEjbEkSZIE2BhLkiRJgI2xJEmSBAz54bu5tMvRn5vV/VeccHBHlcy9DemxSppb/j2R1Eej/rfJLcaSJEkSNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBNsaSJEkSYGMsSZIkATbGkiRJEmBjLEmSJAE2xpIkSRJgYyxJkiQBQzbGSQ5Icm2S5UmOHuf2TZN8or39m0l26bpQScMxr9JoMbNSf0zZGCfZCDgROBBYDByeZPGYYW8C7qqqZwL/APxt14VKmpp5lUaLmZX6ZZgtxnsCy6vq+qp6GDgTOHTMmEOBj7Tnzwb2S5LuypQ0JPMqjRYzK/XIgiHG7ATcNHB5JbDXRGOq6tEk9wDbA7cPDkpyJHBke/H+JNcOsfwdxs7zuDmn/755wvm6nGsU5pvHufo+33zV9rQOltXrvM5An+dbp7XN89+TPj8PXc+3LvMK61dm+/y8dj1fn2ubcr4u89/3XmdgvqEyO0xjPN670prBGKrqJOCkIZb504mTZVW1ZDr3WVfz9bm2rufrc21dz9fn2oZZ3DjXmdc5mK/PtXU9X59r63q+dZxXWI8y2+fntev5+lxb1/P1uba5mG+YQylWAjsPXF4ErJpoTJIFwLbAnV0UKGlazKs0Wsys1CPDNMYXA7sl2TXJJsBhwDljxpwD/E57/rXAl6tqrXezkuaceZVGi5mVemTKQyna45mOAs4FNgJOqaqrkhwPLKuqc4APAaclWU7zLvawDmuc1m6hdTxfn2vrer4+19b1fH2ubVLmdZ3O1+faup6vz7V1Pd86yyusd5nt8/Pa9Xx9rq3r+fpcW+fzxTedkiRJkr98J0mSJAE2xpIkSRLQ48Z4qp/InMF8pyS5LcmVHcy1c5Lzk1yT5Kokb53lfJsl+VaS77TzHddBjRsl+XaSz3Yw14okVyS5LMmyDubbLsnZSb7brsMXz3Ce3dua1pzuTfK2Wdb2x+1zcGWSM5JsNsv53trOddVsa+u7LjNrXmc1Vy/z2s7VaWbN68z1Na/tfJ1ldi7y2s7by8z2Oa/tnJ1lds7yWlW9O9F8AOE64OnAJsB3gMWznPPlwAuBKzuo7ynAC9vzWwPfm019NN9RuVV7fmPgm8Des6zx7cDpwGc7eLwrgB06fH4/Avxee34TYLuOXjO3Ak+bxRw7ATcAm7eXzwKOmMV8ewBXAlvQfND1i8BuXa3HPp26zqx5ndVcvc/rwGtmxpk1r7Ne973MaztfZ5mdi7y2c/Uys33NaztHZ5mdy7z2dYvxMD+ROS1V9VU6+t7Hqrqlqi5tz98HXEPzhM90vqqq+9uLG7enGX8qMski4GDg5JnOMVeSbEPzR/RDAFX1cFXd3cHU+wHXVdWNs5xnAbB5mu8K3YK1v090Op4DXFRVD1bVo8BXgF+bZX191WlmzWs/zGFeoZvMmteZ6W1e2/k6y2zXeYX+ZnYE8grdZXbO8trXxni8n8ic8X9kcynJLsALaN6FzmaejZJcBtwGfKGqZjPfPwLvAn4ym5oGFHBekkvS/OTobDwdWA18uN0NdXKSLWdfIocBZ8xmgqq6GXgf8H3gFuCeqjpvFlNeCbw8yfZJtgAO4vFf5L8+GYnMmtdpm6u8wiwza15nZSTyCt1ktuO8Qn8z29u8QueZnbO89rUxHurnL+dbkq2AfwPeVlX3zmauqvpxVT2f5leP9kyyxwxr+hXgtqq6ZDb1jLFPVb0QOBB4c5KXz2KuBTS73D5QVS8AHgBme3zbJsAhwCdnOc8Tabaa7ArsCGyZ5LdnOl9VXQP8LfAF4D9pdlc+Opsae6z3mTWvM9J5XqGbzJrXWel9XqG7zHaV17amPme2t3lt5+kss3OZ1742xsP8ROa8SrIxTWA/XlWf6mredrfHBcABM5xiH+CQJCtodo+9IsnHZlnTqvbf24BP0+yGm6mVwMqBd+xn0wR5Ng4ELq2qH8xynv2BG6pqdVU9AnwKeMlsJqyqD1XVC6vq5TS7Gv97ljX2Va8za15nbC7yCt1k1rzOXK/zCnOT2Q7yCv3ObJ/zCh1ndq7y2tfGeJifyJw3SUJzDM81VfX+DuZbmGS79vzmNC+e785krqp6d1UtqqpdaNbbl6tqxltRkmyZZOs154FX0uzCmJGquhW4Kcnu7VX7AVfPdL7W4cxyF0/r+8DeSbZon+P9aI5tm7EkP9P++1Tg1+mmzj7qbWbNa+/yCt1k1rzOXG/zCt1mtsu8Qr8z2/O8QseZnau8TvmT0POhJviJzNnMmeQMYF9ghyQrgT+vqg/NcLp9gNcDV7THLQG8p6qWznC+pwAfSbIRzZuVs6pq1l8B05EnA59uXsMsAE6vqv+c5Zx/BHy8/YN8PfDGmU7UHlv0y8AfzLImquqbSc4GLqXZJfNtZv9Tk/+WZHvgEeDNVXXXLOfrpa4za15nrNd5he4ya15nrud5hW4z2+e8QveZ7WVeYU4yOyd59SehJUmSJPp7KIUkSZK0TtkYS5IkSdgYS5IkSYCNsSRJkgTYGEuSJEmAjfF6I8mfJbkqyeVJLkuyV5K3tV+1MtV9hxonqTtmVhod5nXD4de1rQeSvBh4P7BvVf0oyQ7AJsDXgSVVdfsU918xzDhJ3TCz0ugwrxsWtxivH54C3F5VPwJow/damt8iPz/J+QBJPpBkWfuu97j2ureMM+6VSb6R5NIkn0zze/UkOSHJ1e075vet+4cprTfMrDQ6zOsGxC3G64E2VF8DtgC+CHyiqr4y9l1qkidV1Z3tLwB9CXhLVV0+OK59J/wp4MCqeiDJnwKbAv8MfAN4dlVVku3a352XNE1mVhod5nXD4hbj9UBV3Q/8AnAksBr4RJIjxhn6uiSX0vwM488Bi8cZs3d7/YXtT3H+DvA04F7gIeDkJL8OPNj145A2FGZWGh3mdcOyYL4LUDeq6sfABcAFSa6gCdtjkuwKvBN4UVXdleRUYLNxpgrwhao6fK0bkj2B/YDDgKOAV3T5GKQNiZmVRod53XC4xXg9kGT3JLsNXPV84EbgPmDr9rptgAeAe5I8GThwYPzguIuAfZI8s517iyTPanclbVtVS4G3tcuQNANmVhod5nXD4hbj9cNWwD8l2Q54FFhOs8vncODzSW6pql9K8m3gKuB64MKB+580ZtwRwBlJNm1vfy9NsP8jyWY073j/eF08MGk9ZWal0WFeNyB++E6SJEnCQykkSZIkwMZYkiRJAmyMJUmSJMDGWJIkSQJsjCVJkiTAxliSJEkCbIwlSZIkwMZYkiRJAmyMpZGU5LeSnDffdUxXkiOSfG3g8v1Jnt7R3O9JcnJ7fpcklaSTX/dM8tS21o26mG/IZW6e5DNJ7knyyTlaxrFJPjZHc8/Zcz0XpnrNJFmRZP8Jbts3ycq5XL6kdcPGWJoDSV6a5OttU3NnkguTvKi97XENwxBzrfUfZlV9vKpeORe1r0tVtVVVXT/ZmGGbjqr666r6vS7qGtsEVdX321p/3MX8Q3ot8GRg+6r6H7OdrIvmbTaGea4lab75zlTqWJJtgM8C/xM4C9gEeBnwo/msqwtJNlrHzeFQkiyoqkfnu46OPQ343kwe13q6PiRpzrnFWOreswCq6oyq+nFV/bCqzquqy5M8B/hX4MXtruW7AZIcnOTbSe5NclOSYwfm+2r7793tfV48zm7qlyS5uN1CfXGSlwzcdkGSv2i3Wt+X5LwkOwzc/skkt7b3/WqSnxu47dQkH0iyNMkDwNuT/GBw63WS1yS5bLwVkWT7JOe0j+tbwDPG3F5JntmePyjJ1W2NNyd5Z5Itgc8DO7aP/f4kO7aHAJyd5GNJ7gWOmOCwgN9NsirJLUneMeZx/eXA5ce2piY5DXgq8Jl2ee8au9W+reGcdm/A8iS/PzDXsUnOSvLR9rFclWTJwO1/2j6++5Jcm2S/cdbbccAxwG+0NbwpyROSvDfJjUlua+ffth2/pr43Jfk+8OUx8427HtubN5mk1h2T/FuS1UluSPKW8Z7nGTzXk73eSfKG9nHekeR/Z2ALfpJNk/xj+7yuas9vOvg8JnlHu45uSfLGgXknXe4QXtS+Ru9K8uEkm02wLp6TJnd3t+v0kIHbNk/yf9rHd0+SryXZfJw5XtM+7j2SbNa+1u9o57w4yZOnWbukYVSVJ0+eOjwB2wB3AB8BDgSeOOb2I4CvjbluX+C5NG9Wnwf8APjV9rZdgAIWjDcH8CTgLuD1NHuBDm8vb9/efgFwHU3Dvnl7+YSBuX4X2BrYFPhH4LKB204F7gH2aWvbDLgaOHBgzKeBd0ywLs6k2Wq+JbAHcPPgY28f1zPb87cAL2vPPxF44cC6WTlm3mOBR4BfbevavL3uY2PW2Rntsp8LrAb2H3hcfzlm/a8cuLxizdjxngPgK8C/tOvj+e3c+w3U9hBwELAR8DfARe1tuwM3ATsOzPuMCdbdY49n4HlaDjwd2Ar4FHDamPo+2j7ezceZb6L1OFGtTwAuoWnQN2mXez3wqg6e632Z+PW+GLgfeGm73Pe1z/Wa5+544CLgZ4CFwNeBvxiY99F2zMbt43qQNoNTLPdxz/E4j28FcCWwM03mLqR9DQ2u23a5y4H3tPW/ArgP2L29/USaDO7UrvOX0GTvseUDb2znWLO+/gD4DLBFe59fALaZ7791njytjye3GEsdq6p7af5TL+CDwOp2S9qEW3iq6oKquqKqflJVl9M0dL845CIPBv67qk6rqker6gzgu8CrB8Z8uKq+V1U/pGlenj+w7FOq6r6q+hFNo/Tza7ZEtv6jqi5sa3uIpuH/bYAkTwJeBZw+tqg0H1R7DXBMVT1QVVe2953II8DiJNtU1V1VdekUj/sbVfXvbV0/nGDMce2yrwA+TPOmYVaS7Ezz/P5pVT1UVZcBJ9O8MVnja1W1tJrDTk4Dfr69/sc0TdDiJBtX1Yqqum7IRf8W8P6qur6q7gfeDRyWx39Y69j28U60PsYzUa0vAhZW1fFV9XA1xwd/EDhs7ATTfa6neL2/FvhMVX2tqh6macxrzHo4vqpuq6rVwHE8ft0/0t7+SFUtpWmydx9iucP456q6qaruBP6K8V9Pe9O8cTmhXW9fpjm06vAkT6B5g/PWqrq5mj1KX2+zt8bbgD8B9q2q5QOPaXuaRvnHVXVJ+3dGUsdsjKU5UFXXVNURVbWIZuvZjjRbY8eVZK8k57e7rO8B/hDYYaLxY+wI3DjmuhtptkitcevA+Qdp/uMmyUZJTkhyXZpDEla0YwaXfdOYuT8GvDrJVsDrgP+qqlvGqWshzdavwfuPrXPQa2i28N2Y5CtJXjzJ2PHqmmrMjTTrarZ2BO6sqvvGzD3Z+t4szXG/y2kan2OB25KcOXBIwzDLHVx/N9Ks38E3XMOsk7HGrZXmGOcd2133d6c57Oc9Y5a3xrSe6yle7zsOzlNVD9LsgWHg9rHrYXAd3lGPP7568PU+m5zBcK+nHYGbquonY8bu1C5rM5o9OBP5E+DEqhr8oORpwLnAme3hI3+XZONp1C1pSDbG0hyrqu/S7LrfY81V4ww7HTgH2LmqtqU5DjmTjB+0iqaJGfRUml3ZU/lN4FBgf2Bbmt25DCx7reVX1c3AN4Bfo9lSd9oEc6+m2a2985i6xlVVF1fVoTS7yP+dZsv2WsufqK4JjF32qvb8AzS7pdf42WnMvQp4UpKtx8w9zPqmqk6vqpfSPGcF/O0w92Pt5/mpNOv3B4PTT7boIZezxk3ADVW13cBp66o6aJyx03qumfz1fguwaM3A9vjb7QfuO956WMVwJlvuMCZ6PQ1aBezcbh0eHHszcDvNoSvPGOd+a7wSeG+S16y5ot36fVxVLaY59OJXgDdMo25JQ7IxljqW5Nnth38WtZd3ptnlelE75AfAoiSbDNxta5qtkA8l2ZOmYV1jNfATmmM8x7MUeFaS30yyIMlv0Byn+dkhyt2a5tsy7qBpFP96qAfZHMv6LprjNT893oB21/yngGOTbJFkMfA7441Nskma72betqoeAe6lOewAmvW1/ZjDO4b1v9tl/xzNcZufaK+/DDgoyZOS/CzNVtxBP2CC9V1VN9Ec1/o37Yeinge8Cfj4VMUk2T3JK9oPiz0E/JCfPs6pnAH8cZJd2631fw18oob/9onprsdvAfem+bDg5u3ehT3Sfu3goOk8163JXu9n0+yReEmbkeN4fPN6Bk3juDDNh0iPodmLMYzJljuMNydZ1B5C9B5++noa9E2aN17vSrJxkn1pDms6s92KfArw/jQfbNwozYdpNx24/1XAAcCJaz60l+SXkjy3PWTlXppDK3r37TDS+sDGWOrefcBewDfTfJPDRTQf2lnzrQhfpvnP79Ykt7fX/S/g+CT30fxHv2Zr6ZpdyX8FXNju0t57cGFVdQfNFqR30DS47wJ+papuZ2ofpdnNezPNh+oumnz4Yz5Ns9Xu01X1wCTjjqLZjX0rzVbzD08y9vXAivaQjj+kPY653eJ+BnB9+/inczjEV2g+xPQl4H1VteZHUU4DvkNz6Mh5rN3g/A1N83V3kneOM+/hNFvXV9Gsiz+vqi8MUc+mwAk0Ww5vpdk6/p4hH8spbd1fBW6gaaz/aMj7Tns9ts3uq2mOR7+hrflkmj0L45nOcz3Z6/0qmsd1Js3W4/uA2/jp1x3+JbAMuBy4Ari0vW5ruhAkAAAA2UlEQVQYEy53SKfTvF6ub09rLbc9LvoQmg/e3k7zIc03tOsf4J1t3RcDd9LsMXjCmDm+Q5PpDyY5kGaPxtk0TfE1NK/rOflhFmlDl6rp7l2TJEhyHfAHVfXF+a5F66926/jdwG5VdcN81yNp/eYWY0nT1h7/WIz5vlypC0le3R6SsSXN17VdwU8/GCpJc8ZfvpM0LUkuoDmG+fVjPnkvdeVQmsNGQnPYxGHl7k1J64CHUkiSJEl4KIUkSZIE2BhLkiRJgI2xJEmSBNgYS5IkSYCNsSRJkgTA/we42/7qLI/7wQAAAABJRU5ErkJggg==\n",
                        "text/plain": [
                            "<Figure size 864x216 with 3 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "fig, axes = plt.subplots(1, d, figsize=(12, 3))\n",
                "\n",
                "for i, ax in enumerate(axes):\n",
                "    pi_i_full_dim = np.zeros(mc2.n)\n",
                "    pi_i_full_dim[mc2.cyclic_classes[i]] = pi_s[i]\n",
                "    draw_histogram(pi_i_full_dim,\n",
                "                   ax=ax, title='$\\pi^{i}$'.format(i=i), xlabel='States')\n",
                "\n",
                "fig.suptitle('Stationary distributions for the diagonal blocks',\n",
                "             y=-0.05, fontsize=12)\n",
                "plt.show()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Verify that $\\pi^{i+1} = \\pi^i P_i$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 63,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "pi^0 P_0 =\n",
                        "[0.857 0.143]\n",
                        "pi^1 P_1 =\n",
                        "[0.286 0.286 0.143 0.286]\n",
                        "pi^2 P_2 =\n",
                        "[0.143 0.286 0.429 0.143]\n"
                    ]
                }
            ],
            "source": [
                "for i in range(d):\n",
                "    print('pi^{i} P_{i} =\\n{dot}'.format(i=i, dot=np.dot(pi_s[i], P_blocks[i])))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Verify that $\\pi = (\\pi^0 + \\cdots + \\pi^{d-1})/d$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 64,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[0.048 0.095 0.143 0.048 0.286 0.048 0.095 0.095 0.048 0.095]\n"
                    ]
                }
            ],
            "source": [
                "# Right hand side of the above identity\n",
                "rhs = np.zeros(mc2.n)\n",
                "\n",
                "for i in range(d):\n",
                "    rhs[mc2.cyclic_classes[i]] = pi_s[i]\n",
                "rhs /= d\n",
                "print(rhs)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 65,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "True"
                        ]
                    },
                    "execution_count": 65,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "np.allclose(pi, rhs)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "### Powers of $P$"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Since the Markov chain in consideration is periodic,\n",
                "the marginal distribution does not converge, but changes periodically."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Let us compute the powers of the transition probability matrix (in cyclic normal form):"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 66,
            "metadata": {},
            "outputs": [],
            "source": [
                "np.set_printoptions(suppress=True)  # Suppress printing with floating point notation"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Print $P^1, P^2, \\ldots, P^d$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 67,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "P^1 =\n",
                        "[[0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    1.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.    0.    1.    0.   ]\n",
                        " [0.5   0.5   0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    1.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    1.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.5   0.5   0.    0.    0.    0.    0.    0.   ]]\n",
                        "P^2 =\n",
                        "[[0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.    0.    1.    0.   ]\n",
                        " [0.167 0.167 0.5   0.167 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    1.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.5   0.5   0.    0.    0.    0.   ]]\n",
                        "P^3 =\n",
                        "[[0.167 0.167 0.5   0.167 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.167 0.167 0.5   0.167 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.167 0.167 0.5   0.167 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    1.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.833 0.167 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    1.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.333 0.333 0.    0.333]\n",
                        " [0.    0.    0.    0.    0.    0.    0.167 0.167 0.5   0.167]]\n"
                    ]
                }
            ],
            "source": [
                "for i in range(1, d+1):\n",
                "    print('P^{i} =\\n{P_i}'.format(i=i, P_i=np.linalg.matrix_power(mc2.P, i)))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Print $P^{2d}$, $P^{4d}$, and $P^{6d}$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 68,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "P^6 =\n",
                        "[[0.139 0.306 0.417 0.139 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.139 0.306 0.417 0.139 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.139 0.306 0.417 0.139 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.167 0.167 0.5   0.167 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.861 0.139 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.833 0.167 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.278 0.278 0.167 0.278]\n",
                        " [0.    0.    0.    0.    0.    0.    0.278 0.278 0.167 0.278]\n",
                        " [0.    0.    0.    0.    0.    0.    0.278 0.278 0.167 0.278]\n",
                        " [0.    0.    0.    0.    0.    0.    0.306 0.306 0.083 0.306]]\n",
                        "P^12 =\n",
                        "[[0.143 0.286 0.428 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.428 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.428 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.144 0.282 0.431 0.144 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.856 0.144 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.285 0.285 0.144 0.285]\n",
                        " [0.    0.    0.    0.    0.    0.    0.285 0.285 0.144 0.285]\n",
                        " [0.    0.    0.    0.    0.    0.    0.285 0.285 0.144 0.285]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.141 0.286]]\n",
                        "P^18 =\n",
                        "[[0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]]\n"
                    ]
                }
            ],
            "source": [
                "for i in [k*d for k in [2, 4, 6]]:\n",
                "    print('P^{i} =\\n{P_i}'.format(i=i, P_i=np.linalg.matrix_power(mc2.P, i)))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "$P^{kd}$ converges as $k \\to \\infty$ to a matrix that contains $\\pi^0, \\ldots, \\pi^{d-1}$."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Print $P^{kd+1}, \\ldots, P^{kd+d}$ with $k = 10$ for example:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 69,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "P^31 =\n",
                        "[[0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]]\n",
                        "P^32 =\n",
                        "[[0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]]\n",
                        "P^33 =\n",
                        "[[0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.143 0.286 0.429 0.143 0.    0.    0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.857 0.143 0.    0.    0.    0.   ]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]\n",
                        " [0.    0.    0.    0.    0.    0.    0.286 0.286 0.143 0.286]]\n"
                    ]
                }
            ],
            "source": [
                "for i in range(10*d+1, 10*d+1+d):\n",
                "    print('P^{i} =\\n{P_i}'.format(i=i, P_i=np.linalg.matrix_power(mc2.P, i)))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "But $P^i$ itself does not converge."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Simulation"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Plot the frequency distribution of visits to the states\n",
                "along a sample path starting at state `0`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 70,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[0.051 0.094 0.14  0.048 0.285 0.048 0.097 0.092 0.048 0.097]\n"
                    ]
                }
            ],
            "source": [
                "init = 0\n",
                "dist = time_series_dist(mc2, init=init, t=10**4)\n",
                "print(dist)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 71,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEWCAYAAACKSkfIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHENJREFUeJzt3XuUXGWd7vHvY0LC/SYZkYSQIFEIqCBNcIY1kVEuQRzC8XAJChMUJzKHHGU4Lg2XBRgdRcZRzmgcyTFhWCKEm5dWgwHk4iACaW6BhIk0IZA2IIGEOwY6/s4f+23cKarTu5NKVSfv81mrVmrv/e79/qq689Sud19aEYGZmeXhba0uwMzMmsehb2aWEYe+mVlGHPpmZhlx6JuZZcShb2aWEYf+JkLSOZJ+0Oo6+kPSJyXduJH7GCUpJA1O0zdImtygbf+tpMWl6aWSDmvEttP2Fko6tFHba3T/km6T9JmK21rrvWpUW2s8+Tz9gUHSy6XJrYHVwJo0/dmI+FHzqxr4JI0CHge2iIjufqwXwJiI6OzHOkuBz0TEzf0sE0n/CXRFxHn9XbcZJF0I7BURJ5fm3QZcEREbdWdjA9/X/YFZwD7AI8BpEfFAYyvcvHhPf4CIiG17HsCTwN+X5m1ygd+z572p2tTrz4GkIcDPgCuAnYDLgZ+l+dYLh/4mQtKFkq5Iz3uGND4laZmkVZJOl3SQpAWSnpf03Zr1Py3pkdR2nqQ9eulnS0lXSHoubWe+pHekZTtImiXpKUl/kPRVSYPSslMl/VbStyWtBC5M8+4obXtvSTdJWilpsaQTSss+KmmRpJfStr/QS32DJH1T0rOSlgBH1yx/c0hC0l6Sbpf0Qmp/dZr/m9T8QUkvSzpR0qGSuiR9SdLTwGU982pKOCjVuUrSZZK2LL3+O8oN089oL0lTgE8CX0z9/Twtf3O4SNJQSZdIWp4el0gampb11PZ/JD2T3v9P9fL+/J2kh0rTN0u6pzR9h6Rjy/1LmgCcA5yY6nuwtMk90s/1JUk3Stqll37Xeq/Str+Qfh9fkHR16b16s62kHwIjgZ+nvr9Yb/u9OBQYDFwSEasj4t8BAR/uxzay49DftB0MjAFOBC4BzgUOA/YFTpD0IYD0n/wc4OPAMOC/gKt62eZkYAdgd+DtwOnAa2nZ5UA3sBdwAHAEUB7zPRhYAvwV8C/ljUraBrgJuDItPwn4nqR9U5NZFMNY2wH7Abf0Ut8/Ah9L/bcBx/XSDuArwI0Ue4EjgO8ARMT4tPz96ZvU1Wl6V2BnYA9gSi/b/CRwJPAu4N1An8M1ETET+BFwcerv7+s0Oxf4ILA/8H5gXM22d6X4uQwHTgNmSNqpznZ+B+wlaRcV31b2A0ZI2k7SVsCBFD//cn2/Ar4GXJ3qe39p8SeAT1H8zIYAdT+Me3ECMAEYDbwPOLW2QUScwtrfbC8GSDscvT2mpdX3BRbE2mPUC9J864VDf9P2lYj4U0TcCLwCXBURz0TEHyj+Yx+Q2n0W+HpEPJLGvb8G7N/L3v4bFGG/V0SsiYh7I+LFtLd/FHBmRLwSEc8A3wYmldZdHhHfiYjuiHitZrsfA5ZGxGVp+X3A9fwltN8AxkraPiJWpeX1nECxZ7csIlYCX1/H+/MGRYDvlt6nO9bRFuDPwAVpr7G2/h7fLfX9LxQfXo3wSWB6+vmtAL4MnFJa/kZa/kZEzAVeBt5Tu5GI+BPQAYyn+FBcANwBHELxofJoRDzXj7oui4jfp/fjGooPpar+PSKWp/fq5/1ZNyJ2XMfjotRsW+CFmlVfALbrR43Zcehv2v5Yev5anelt0/M9gP/bs6cErKT4Gjy8zjZ/CMwD5qRhhoslbZG2sQXwVGk7l1LsAfZYto5a9wAOLu+xUQTdrmn5/wQ+CjyRhmT+upft7FbTzxPr6POL6XXeo+JMlU+voy3AihSa61Lb9259tK9qN9Z+LbXbfq7mQPWr/OXnW+t2iqGP8en5bcCH0uP2ftb1dMU+G71uFS8D29fM2x54qcH9bFYc+nlYRjF0Ut5b2ioi7qxtmPYkvxwRY4G/odhD/4e0jdXALqVtbB8R5a/S6zoVbBlwe00N20bEP6V+50fERIoPkZ9S7FXW8xTF0FOPkb11GBFPR8Q/RsRuFN92vidpr3XUWOVUttq+l6fnr1CcdQWApF1ZW1/bXk7xwVhv2/1VG/q303fot/I0vrf0ncb3e3uck5otBN4nSaVV35fmWy8c+nn4PnB2z/i5igOyx9drmA4EvlfFAdoXKYYV1kTEUxTj4/8maXtJb5P0rp7jBhX8Ani3pFMkbZEeB0naR9IQFef07xARb6R+1/SynWuAz0kakca0p/XSDknHSxqRJldRhEvPdv8I7Fmx9rIzUt87Uxwn6Tke8CCwr6T90wHLC2vW66u/q4DzJA1LB0vPpzgrZX3cSTH0Mw64JyIWkr5pAb/pZZ0/AqMktSIT3vLelM9mq/P4Wmp2G8XP83PpQPjUNL+340GGQz8LEfET4BsUQzYvAg9TjM/XsytwHUXwPkKxZ9gTPv9AcTBvEUWIXge8s2INL1Ec+J1EsQf7dKppaGpyCrA01Xc6cHK97QD/j2L46UHgPuDH6+j2IOBuFddAtAOfj4jH07ILgcvTUNMJvW2gjispPvyWpMdX0+v7PTAduBl4lGIcvWwWxTGL5yX9tM52v0oxFr8AeCi9tq/2o643RcQraf2FEfF6mv074Il0LKaea9O/z0nq7XjKxvJ1ig+859XLWVv1pNd2LMXv5fPAp4FjS6/Z6vDFWWZmGfGevplZRiqFvqQJKi6m6SydI1tefrqkhyQ9kC7+GJvmj5L0Wpr/gKTvN/oFmJlZdX0O76QDer8HDge6gPnASRGxqNRm+4h4MT0/BvhfETFBxX1RfhER+22c8s3MrD+q7OmPAzojYkk6QDIHmFhu0BP4yTa09vQvMzPrRZWbSg1n7QtSuihO/VqLpDOAsyjO7ijf+2K0pPspzgY5LyL+q866U0iXvW+zzTYH7r333pVfgJmZwb333vtsRAzrq12V0FedeW/Zk4+IGRT3A/kExT1DJlNcSDMyIp6TdCDwU0n71nwz6Lk3yUyAtra26OjoqFCWmZn1kLSuq9PfVGV4p4u1r0IcwbqvFJxDce4s6R4mz6Xn9wKPUdykyszMWqBK6M8HxkgareI+1ZMoLnR5k6QxpcmjKS5OIV1d2HPr3T0p7gi5pBGFm5lZ//U5vBMR3eny5nnAIGB2RCyUNB3oiIh2YKqK+4K/QXGlZs+fqxsPTJfUTXG59OnpjntmZtYCA+6KXI/pm5n1n6R7I6Ktr3a+ItfMLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjFQKfUkTJC2W1ClpWp3lp0t6SNIDku6QNLa07Oy03mJJRzayeDMz658+Q1/SIGAGcBQwFjipHOrJlRHx3ojYH7gY+FZadywwCdgXmAB8L23PzMxaoMqe/jigMyKWRMTrwBxgYrlBRLxYmtwGiPR8IjAnIlZHxONAZ9qemZm1wOAKbYYDy0rTXcDBtY0knQGcBQwBPlxa966adYfXWXcKMAVg5MiRVeo2M7P1UGVPX3XmxVtmRMyIiHcBXwLO6+e6MyOiLSLahg0bVqEkMzNbH1VCvwvYvTQ9Ali+jvZzgGPXc10zM9uIqoT+fGCMpNGShlAcmG0vN5A0pjR5NPBoet4OTJI0VNJoYAxwz4aXbWZm66PPMf2I6JY0FZgHDAJmR8RCSdOBjohoB6ZKOgx4A1gFTE7rLpR0DbAI6AbOiIg1G+m1mJlZHxTxliH2lmpra4uOjo5Wl2FmtkmRdG9EtPXVzlfkmpllxKFvZpYRh76ZWUYc+mZmGalyRa7ZgDNq2i+b1tfSi45uWl9mG5v39M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMlIp9CVNkLRYUqekaXWWnyVpkaQFkn4taY/SsjWSHkiP9kYWb2Zm/dPnn0uUNAiYARwOdAHzJbVHxKJSs/uBtoh4VdI/ARcDJ6Zlr0XE/g2u28zM1kOVPf1xQGdELImI14E5wMRyg4i4NSJeTZN3ASMaW6aZmTVCldAfDiwrTXeleb05DbihNL2lpA5Jd0k6tt4KkqakNh0rVqyoUJKZma2PPod3ANWZF3UbSicDbcCHSrNHRsRySXsCt0h6KCIeW2tjETOBmQBtbW11t21mZhuuyp5+F7B7aXoEsLy2kaTDgHOBYyJidc/8iFie/l0C3AYcsAH1mpnZBqgS+vOBMZJGSxoCTALWOgtH0gHApRSB/0xp/k6ShqbnuwCHAOUDwGZm1kR9Du9ERLekqcA8YBAwOyIWSpoOdEREO/CvwLbAtZIAnoyIY4B9gEsl/ZniA+aimrN+zMysiaqM6RMRc4G5NfPOLz0/rJf17gTeuyEFmplZ4/iKXDOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwy4tA3M8uIQ9/MLCMOfTOzjDj0zcwyUin0JU2QtFhSp6RpdZafJWmRpAWSfi1pj9KyyZIeTY/JjSzezMz6p8/QlzQImAEcBYwFTpI0tqbZ/UBbRLwPuA64OK27M3ABcDAwDrhA0k6NK9/MzPqjyp7+OKAzIpZExOvAHGBiuUFE3BoRr6bJu4AR6fmRwE0RsTIiVgE3ARMaU7qZmfVXldAfDiwrTXeleb05DbhhPdc1M7ONaHCFNqozL+o2lE4G2oAP9WddSVOAKQAjR46sUJKZma2PKnv6XcDupekRwPLaRpIOA84FjomI1f1ZNyJmRkRbRLQNGzasau1mZtZPVUJ/PjBG0mhJQ4BJQHu5gaQDgEspAv+Z0qJ5wBGSdkoHcI9I88zMrAX6HN6JiG5JUynCehAwOyIWSpoOdEREO/CvwLbAtZIAnoyIYyJipaSvUHxwAEyPiJUb5ZWYmVmfqozpExFzgbk1884vPT9sHevOBmavb4FmZtY4viLXzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4xUCn1JEyQtltQpaVqd5eMl3SepW9JxNcvWSHogPdobVbiZmfXf4L4aSBoEzAAOB7qA+ZLaI2JRqdmTwKnAF+ps4rWI2L8BtZqZ2QbqM/SBcUBnRCwBkDQHmAi8GfoRsTQt+/NGqNHMzBqkyvDOcGBZaborzatqS0kdku6SdGy/qjMzs4aqsqevOvOiH32MjIjlkvYEbpH0UEQ8tlYH0hRgCsDIkSP7sWkzM+uPKnv6XcDupekRwPKqHUTE8vTvEuA24IA6bWZGRFtEtA0bNqzqps3MrJ+qhP58YIyk0ZKGAJOASmfhSNpJ0tD0fBfgEErHAszMrLn6DP2I6AamAvOAR4BrImKhpOmSjgGQdJCkLuB44FJJC9Pq+wAdkh4EbgUuqjnrx8zMmqjKmD4RMReYWzPv/NLz+RTDPrXr3Qm8dwNrNDOzBvEVuWZmGam0p28Dw6hpv2xaX0svOrppfZlZ83hP38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy4hD38wsI5VCX9IESYsldUqaVmf5eEn3SeqWdFzNssmSHk2PyY0q3MzM+q/P0Jc0CJgBHAWMBU6SNLam2ZPAqcCVNevuDFwAHAyMAy6QtNOGl21mZuujyp7+OKAzIpZExOvAHGBiuUFELI2IBcCfa9Y9ErgpIlZGxCrgJmBCA+o2M7P1UCX0hwPLStNdaV4VldaVNEVSh6SOFStWVNy0mZn1V5XQV515UXH7ldaNiJkR0RYRbcOGDau4aTMz66/BFdp0AbuXpkcAyytuvws4tGbd2yqua2YVjJr2y6b0s/Sio5vSz6auWT8PWL+fSZXQnw+MkTQa+AMwCfhExe3PA75WOnh7BHB2v6tssYH+QzQzq6rP4Z2I6AamUgT4I8A1EbFQ0nRJxwBIOkhSF3A8cKmkhWndlcBXKD445gPT0zwzM2uBKnv6RMRcYG7NvPNLz+dTDN3UW3c2MHsDajSzAc7fhjcdviLXzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4w49M3MMuLQNzPLiEPfzCwjDn0zs4xUCn1JEyQtltQpaVqd5UMlXZ2W3y1pVJo/StJrkh5Ij+83tnwzM+uPwX01kDQImAEcDnQB8yW1R8SiUrPTgFURsZekScA3gBPTssciYv8G121mZuuhz9AHxgGdEbEEQNIcYCJQDv2JwIXp+XXAdyWpEQWOmvbLRmymkqUXHd20vszMWqHK8M5wYFlpuivNq9smIrqBF4C3p2WjJd0v6XZJf1uvA0lTJHVI6lixYkW/XoCZmVVXZU+/3h57VGzzFDAyIp6TdCDwU0n7RsSLazWMmAnMBGhra6vdtg0g/ub1F34vBp5m/Uw25Z9HlT39LmD30vQIYHlvbSQNBnYAVkbE6oh4DiAi7gUeA969oUWbmdn6qRL684ExkkZLGgJMAtpr2rQDk9Pz44BbIiIkDUsHgpG0JzAGWNKY0s3MrL/6HN6JiG5JU4F5wCBgdkQslDQd6IiIdmAW8ENJncBKig8GgPHAdEndwBrg9IhYuTFeiJmZ9a3KmD4RMReYWzPv/NLzPwHH11nveuD6DazRzMwaxFfkmpllxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpYRh76ZWUYc+mZmGXHom5llxKFvZpaRSqEvaYKkxZI6JU2rs3yopKvT8rsljSotOzvNXyzpyMaVbmZm/dVn6EsaBMwAjgLGAidJGlvT7DRgVUTsBXwb+EZadywwCdgXmAB8L23PzMxaoMqe/jigMyKWRMTrwBxgYk2bicDl6fl1wEckKc2fExGrI+JxoDNtz8zMWkARse4G0nHAhIj4TJo+BTg4IqaW2jyc2nSl6ceAg4ELgbsi4oo0fxZwQ0RcV9PHFGBKmnwPsHgDX9cuwLMbuI1GGAh1DIQaYGDUMRBqgIFRx0CoAQZGHQOhBtjwOvaIiGF9NRpcYUOqM6/2k6K3NlXWJSJmAjMr1FKJpI6IaGvU9jblOgZCDQOljoFQw0CpYyDUMFDqGAg1NLOOKsM7XcDupekRwPLe2kgaDOwArKy4rpmZNUmV0J8PjJE0WtIQigOz7TVt2oHJ6flxwC1RjBu1A5PS2T2jgTHAPY0p3czM+qvP4Z2I6JY0FZgHDAJmR8RCSdOBjohoB2YBP5TUSbGHPymtu1DSNcAioBs4IyLWbKTXUtawoaINNBDqGAg1wMCoYyDUAAOjjoFQAwyMOgZCDdCkOvo8kGtmZpsPX5FrZpYRh76ZWUY2u9Dv65YRTaphtqRn0vULLSFpd0m3SnpE0kJJn29BDVtKukfSg6mGLze7hlItgyTdL+kXLaxhqaSHJD0gqaOFdewo6TpJ/51+P/66yf2/J70HPY8XJZ3ZzBpKtfxz+t18WNJVkrZsQQ2fT/0vbMr7EBGbzYPiQPNjwJ7AEOBBYGwL6hgPfAB4uIXvxTuBD6Tn2wG/b/Z7QXGdxrbp+RbA3cAHW/R+nAVcCfyihT+TpcAureq/VMflwGfS8yHAji2sZRDwNMWFRc3uezjwOLBVmr4GOLXJNewHPAxsTXFizc3AmI3Z5+a2p1/llhEbXUT8huIsppaJiKci4r70/CXgEYpf8mbWEBHxcprcIj2afuaApBHA0cAPmt33QCNpe4qdklkAEfF6RDzfwpI+AjwWEU+0qP/BwFbp+qKtaf51RPtQ3LXg1YjoBm4H/sfG7HBzC/3hwLLSdBdNDrqBKN319ACKPe1m9z1I0gPAM8BNEdH0GoBLgC8Cf25B32UB3Cjp3nTrkVbYE1gBXJaGu34gaZsW1QLF6d1XtaLjiPgD8E3gSeAp4IWIuLHJZTwMjJf0dklbAx9l7QtaG25zC/1Kt33IiaRtgeuBMyPixWb3HxFrImJ/iquxx0nar5n9S/oY8ExE3NvMfntxSER8gOKOtWdIGt+CGgZTDD3+R0QcALwCtOrY1xDgGODaFvW/E8VIwGhgN2AbSSc3s4aIeITirsQ3Ab+iGJLu3ph9bm6h79s+lEjagiLwfxQRP25lLWkI4TaKW2w30yHAMZKWUgz3fVjSFU2uAYCIWJ7+fQb4Ca2542wX0FX6xnUdxYdAKxwF3BcRf2xR/4cBj0fEioh4A/gx8DfNLiIiZkXEByJiPMWw8KMbs7/NLfSr3DIiC+nW1rOARyLiWy2qYZikHdPzrSj+k/13M2uIiLMjYkREjKL4fbglIpq6NwcgaRtJ2/U8B46g+GrfVBHxNLBM0nvSrI9QXDHfCifRoqGd5Engg5K2Tv9fPkJx7KupJP1V+nck8HE28ntS5S6bm4zo5ZYRza5D0lXAocAukrqACyJiVpPLOAQ4BXgojakDnBMRc5tYwzuBy9MfznkbcE1EtOyUyRZ7B/CTIlsYDFwZEb9qUS3/G/hR2jFaAnyq2QWk8evDgc82u+8eEXG3pOuA+yiGVO6nNbdkuF7S24E3KG5Vs2pjdubbMJiZZWRzG94xM7N1cOibmWXEoW9mlhGHvplZRhz6ZmYZcehbliSdm+5quCDd6fFgSWemUwn7WrdSO7OByKdsWnbSrYS/BRwaEasl7UJxt8k7gbaIeLaP9ZdWaWc2EHlP33L0TuDZiFgNkML7OIr7r9wq6VYASf8hqaP8twAkfa5OuyMk/U7SfZKuTfc7QtJFkhalbxPfbP7LNHsr7+lbdlIo30FxK92bgasj4vbaPXhJO0fEynRF8a+Bz0XEgnK79C3hx8BREfGKpC8BQ4HvAr8D9o6IkLRji29hbAZ4T98ylO7xfyAwheI2w1dLOrVO0xMk3Udxef6+wNg6bT6Y5v823e5iMrAH8CLwJ+AHkj4OvNro12G2Pjare++YVRURayju+nmbpIcowvpNkkYDXwAOiohVkv4TqPen9ETxdwJOessCaRzFTbwmAVOBDzfyNZitD+/pW3bS32gdU5q1P/AE8BLFn5YE2J7iXvMvSHoHxW2Ae5Tb3QUcImmvtO2tJb07DSHtkG5wd2bqw6zlvKdvOdoW+E667XM30Ekx1HMScIOkpyLi7yTdDyykuBPlb0vrz6xpdypwlaShafl5FB8MP0t/aFvAPzfjhZn1xQdyzcwy4uEdM7OMOPTNzDLi0Dczy4hD38wsIw59M7OMOPTNzDLi0Dczy8j/B2M2HGdyo1r8AAAAAElFTkSuQmCC\n",
                        "text/plain": [
                            "<Figure size 432x288 with 1 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "draw_histogram(dist,\n",
                "               title='Time series distribution with init={0}'.format(init),\n",
                "               xlabel='States', ylim=(0, 0.35))\n",
                "plt.show()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Observe that the distribution is close to the (unique) stationary distribution $\\pi$."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 72,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "[0.048 0.095 0.143 0.048 0.286 0.048 0.095 0.095 0.048 0.095]\n"
                    ]
                }
            ],
            "source": [
                "print(pi)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Next, plot the simulated marginal distributions\n",
                "at $T = 10d+1, \\ldots, 11d, 11d+1, \\ldots, 12d$ with initial state `0`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 73,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAGRCAYAAABmC3VnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xu0ZAV55/3vz25a5CYG2ozQYGNsL+hrhOkQDAmSoHkBDYyJy4FEJ17RFXF0cExaTVBJNGgy0cmImn7BYLxACGrS0U7AKF7iCNKAIhcxLXI5AtIiF1ERwef9Y++W4nBOnzqndp3e1f39rFWLuux66qmq86Of2rVr71QVkiRJ0vbuIVu7AUmSJKkPHIwlSZIkHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwXjiJLlr4PTTJD8auPx7HT7OO5LckOTOJNcleeO029cmubrt4YVdPa60relDZpM8Lsk/JdmU5HtJzk3y+K4eW9pW9CSveyb5YpJbk9ye5EtJDunqsbVlDsYTpqp22XwCrgd+a+C6D3f4UKcDT6iq3YBfAX43yW8P3P5V4A+ASzp8TGmb05PM7g6sAx4P/DzwZeCfOnxsaZvQk7zeBbwYWA48Ang78M9Jlnb4+JqFL7JmVFVXT7vqp8BjB24/FSDJ3YvZl6SZbSmzVfVlmmEYgCTvBP44yR5VdevidSkJ5szr3cDVAEkeAtxHMyD/HHDLIra5XXKN8TYuyZr2q5gZT0Pc9y5gCtgZ+MiiNC1txxYps4cCNzsUS6MZZ16TXAbcTfNtz2lV5VC8CByMt3FVdUpV7T7baa77ArsCBwIfBO5YjJ6l7dm4M5tkBXAqcOI4+pe2J+PMa1U9BdgN+F3g38f0FDSNg7G2qBqXAj8C3rK1+5G0ZVvKbJLlwHnAe6rqzK3Rn6T7zfVvbFXd3WZ1TZJfXPQGt0MOxtu4JG+Y9ivbB5zmUWop8Avj6lNSY1yZTfIImqF4XVW9teu+pe3RIv4buwPwmNG61TAcjLdxVfW2wV/ZTj/NdJ8kD0ny8iSPSOMg4JXApweWWZZkRyDADkl2bH8kIGkE48hskt2Ac4EvVtWaxXs20rZtTHk9OMmvtv/OPizJH9HsTebCxXtm2y8HGc3mOcA3ge8DHwL+T3va7Dyar35+BVjbnj90kXuUdL8tZfY5wC8BL5q2RmvfrdOqtN3bUl4fSvM7gFuBbwNHAc+qqhu3Qp/bnVTV1u5BkiRJ2upcYyxJkiQxxGCc5P1Jbkly+Sy3J8lfJ9mY5LIkB3bfpqRhmVlpcphXqV+GWWN8BnDEFm4/EljVno4H3jt6W5JGcAZmVpoUZ2Bepd6YczCuqs8D39vCIscAf9fui+8CYPckj+qqQUnzY2alyWFepX5Z2kGNvYEbBi5PtdfdNH3BJMfTfOJl5513/s9PeMITOnh4adtw8cUXf7eqli/CQw2VWfMqza5veQUzK23JsJntYjDODNfNuKuLqlpLs2svVq9eXRs2bOjg4aVtQ5LrFuuhZrjuQZk1r9Ls+pZXMLPSlgyb2S72SjEF7DNweQXgvvak/jKz0uQwr9Ii6mIwXgf8t/aXswcDd1TVg77ikdQbZlaaHOZVWkRzbkqR5EzgMGDPJFPAm2iO2U1VvQ9YT3NUlo3AD4EXjatZSXMzs9LkMK9Sv8w5GFfVcXPcXjTH+JbUA2ZWmhzmVeoXj3wnSZIk4WAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRLgYCxJkiQBDsaSJEkS4GAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRLgYCxJkiQBDsaSJEkS4GAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRLgYCxJkiQBDsaSJEkSAEu3dgNauJVrPjnS/a895VkddSJJkjT5XGMsSZIkMeRgnOSIJFcn2ZhkzQy375vk/CSXJrksyVHdtyppGOZVmixmVuqPOQfjJEuAU4Ejgf2B45LsP22xPwbOrqoDgGOB93TdqKS5mVdpsphZqV+GWWN8ELCxqq6pqnuAs4Bjpi1TwG7t+YcDN3bXoqR5MK/SZDGzUo8MMxjvDdwwcHmqvW7Qm4HnJ5kC1gOvmqlQkuOTbEiyYdOmTQtoV9IczKs0Wcys1CPDDMaZ4bqadvk44IyqWgEcBXwwyYNqV9XaqlpdVauXL18+/24lzcW8SpPFzEo9MsxgPAXsM3B5BQ/+GuclwNkAVfUlYEdgzy4alDQv5lWaLGZW6pFhBuOLgFVJ9kuyjGbD/3XTlrkeOBwgyRNpQuv3ONLiM6/SZDGzUo/MORhX1b3ACcC5wFU0v4y9IsnJSY5uF3st8LIkXwXOBF5YVdO/CpI0ZuZVmixmVuqXoY58V1XraTb4H7zupIHzVwKHdNuapIUwr9JkMbNSf3jkO0mSJAkHY0mSJAlwMJYkSZIAB2NJkiQJcDCWJEmSAAdjSZIkCXAwliRJkgAHY0mSJAlwMJYkSZIAB2NJkiQJcDCWJEmSAAdjSZIkCXAwliRJkgAHY0mSJAlwMJYkSZIAB2NJkiQJcDCWJEmSAAdjSZIkCXAwliRJkgAHY0mSJAlwMJYkSZKAIQfjJEckuTrJxiRrZlnmeUmuTHJFko9026akYZlXaXKYV6lfls61QJIlwKnAM4Ep4KIk66rqyoFlVgGvBw6pqtuSPHJcDUuanXmVJod5lfpnmDXGBwEbq+qaqroHOAs4ZtoyLwNOrarbAKrqlm7blDQk8ypNDvMq9cwwg/HewA0Dl6fa6wY9Dnhcki8muSDJETMVSnJ8kg1JNmzatGlhHUvaEvMqTY7O8gpmVurCMINxZriupl1eCqwCDgOOA05LsvuD7lS1tqpWV9Xq5cuXz7dXSXMzr9Lk6CyvYGalLgwzGE8B+wxcXgHcOMMy/1RVP6mqbwFX0wRZ0uIyr9LkMK9SzwwzGF8ErEqyX5JlwLHAumnL/CPw6wBJ9qT56ueaLhuVNBTzKk0O8yr1zJyDcVXdC5wAnAtcBZxdVVckOTnJ0e1i5wK3JrkSOB94XVXdOq6mJc3MvEqTw7xK/TPn7toAqmo9sH7adScNnC/gxPYkaSsyr9LkMK9Sv3jkO0mSJAkHY0mSJAlwMJYkSZIAB2NJkiQJcDCWJEmSAAdjSZIkCXAwliRJkgAHY0mSJAlwMJYkSZIAB2NJkiQJcDCWJEmSAAdjSZIkCXAwliRJkgAHY0mSJAlwMJYkSZIAB2NJkiQJcDCWJEmSAAdjSZIkCXAwliRJkgAHY0mSJAkYcjBOckSSq5NsTLJmC8s9N0klWd1di5Lmw7xKk8XMSv0x52CcZAlwKnAksD9wXJL9Z1huV+C/Axd23aSk4ZhXabKYWalfhlljfBCwsaquqap7gLOAY2ZY7k+BdwB3d9ifpPkxr9JkMbNSjwwzGO8N3DBweaq97meSHADsU1Wf2FKhJMcn2ZBkw6ZNm+bdrKQ5mVdpsphZqUeGGYwzw3X1sxuThwDvBF47V6GqWltVq6tq9fLly4fvUtKwzKs0Wcys1CPDDMZTwD4Dl1cANw5c3hV4MvDZJNcCBwPr/HGAtFWYV2mymFmpR4YZjC8CViXZL8ky4Fhg3eYbq+qOqtqzqlZW1UrgAuDoqtowlo4lbYl5lSaLmZV6ZM7BuKruBU4AzgWuAs6uqiuSnJzk6HE3KGl45lWaLGZW6pelwyxUVeuB9dOuO2mWZQ8bvS1JC2VepcliZqX+GGowliRJ6tLKNZ8c6f7XnvKsjjqR7uchoSVJkiRcYyxJkobgGl5tD1xjLEmSJOFgLEmSJAFuSiFJ2s6MskmAmwNI2zbXGEuSJEk4GEuSJEmAg7EkSZIEOBhLkiRJgIOxJEmSBDgYS5IkSYC7a5Mk4S7MJAlcYyxJkiQBDsaSJEkS4GAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRLgYCxJkiQBDsaSJEkSMOQBPpIcAfxvYAlwWlWdMu32E4GXAvcCm4AXV9V1HfcqaQjmVVvbKAcLge3rgCHmVeqXOdcYJ1kCnAocCewPHJdk/2mLXQqsrqqnAOcA7+i6UUlzM6/S5DCvUv8MsynFQcDGqrqmqu4BzgKOGVygqs6vqh+2Fy8AVnTbpqQhmVdpcphXqWeGGYz3Bm4YuDzVXjeblwD/MtMNSY5PsiHJhk2bNg3fpaRhmVdpcnSWVzCzUheG2cY4M1xXMy6YPB9YDTx9pturai2wFmD16tUz1pA0EvMqTY7O8gpm1m3b1YVhBuMpYJ+ByyuAG6cvlOQZwBuBp1fVj7tpT9I8mVdpcphXqWeG2ZTiImBVkv2SLAOOBdYNLpDkAOBvgKOr6pbu25Q0JPMqTQ7zKvXMnGuMq+reJCcA59LsTub9VXVFkpOBDVW1DvgLYBfgH5IAXF9VR4+xb43BKF9D+RVUP5hXbYu21a/IzavUP0Ptx7iq1gPrp1130sD5Z3Tcl6QFMq/S5DCvUr945DtJkiQJB2NJkiQJcDCWJEmSAAdjSZIkCXAwliRJkgAHY0mSJAlwMJYkSZIAB2NJkiQJcDCWJEmSgCGPfCdJkibPKIfT7uuhtKVxco2xJEmShIOxJEmSBDgYS5IkSYCDsSRJkgQ4GEuSJEmAg7EkSZIEOBhLkiRJgIOxJEmSBDgYS5IkSYCDsSRJkgQMORgnOSLJ1Uk2Jlkzw+0PTfL37e0XJlnZdaOShmNepcliZqX+mHMwTrIEOBU4EtgfOC7J/tMWewlwW1U9Fngn8PauG5U0N/MqTRYzK/XLMGuMDwI2VtU1VXUPcBZwzLRljgE+0J4/Bzg8SbprU9KQzKs0Wcys1CNLh1hmb+CGgctTwC/PtkxV3ZvkDmAP4LuDCyU5Hji+vXhXkquHePw9p9cZUZf1+tzbnPUyv3UOXdaas94CbAvv66M7eCzzunj1+txb1/UWNf9bud6wtbrIK/Q4s9vY+9p1vT7ntet6fe5tPvWGyuwwg/FMn0prActQVWuBtUM85v2Fkw1VtXo+91msen3uret6fe6t63p97m2Yh5vhOvM6hnp97q3ren3uret6i5xX2IYy2+f3tet6fe6t63p97m0c9YbZlGIK2Gfg8grgxtmWSbIUeDjwvS4alDQv5lWaLGZW6pFhBuOLgFVJ9kuyDDgWWDdtmXXA77fnnwt8pqoe9GlW0tiZV2mymFmpR+bclKLdnukE4FxgCfD+qroiycnAhqpaB5wOfDDJRppPscd22OO8vhZa5Hp97q3ren3uret6fe5ti8zrotbrc29d1+tzb13XW7S8wjaX2T6/r13X63NvXdfrc2+d14sfOiVJkiSPfCdJkiQBDsaSJEkS0OPBeK5DZC6g3vuT3JLk8g5q7ZPk/CRXJbkiyatHrLdjki8n+Wpb7y0d9LgkyaVJPtFBrWuTfC3JV5Js6KDe7knOSfL19jV82gLrPL7tafPpziSvGbG3/9G+B5cnOTPJjiPWe3Vb64pRe+u7LjNrXkeq1cu8trU6zax5Xbi+5rWt11lmx5HXtm4vM9vnvLY1O8vs2PJaVb070fwA4ZvAY4BlwFeB/UeseShwIHB5B/09CjiwPb8r8I1R+qPZR+Uu7fkdgAuBg0fs8UTgI8AnOni+1wJ7dvj+fgB4aXt+GbB7R38zNwOPHqHG3sC3gIe1l88GXjhCvScDlwM70fzQ9d+AVV29jn06dZ1Z8zpSrd7ndeBvZsGZNa8jv/a9zGtbr7PMjiOvba1eZraveW1rdJbZcea1r2uMhzlE5rxU1efpaL+PVXVTVV3Snv8+cBXNG77QelVVd7UXd2hPC/5VZJIVwLOA0xZaY1yS7EbzP9HTAarqnqq6vYPShwPfrKrrRqyzFHhYmn2F7sSD9yc6H08ELqiqH1bVvcDngOeM2F9fdZpZ89oPY8wrdJNZ87owvc1rW6+zzHadV+hvZicgr9BdZseW174OxjMdInPB/5CNU5KVwAE0n0JHqbMkyVeAW4BPVdUo9d4F/CHw01F6GlDAeUkuTnPI0VE8BtgE/G37NdRpSXYevUWOBc4cpUBVfRv4S+B64Cbgjqo6b4SSlwOHJtkjyU7AUTxwR/7bkonIrHmdt3HlFUbMrHkdyUTkFbrJbMd5hf5mtrd5hc4zO7a89nUwHurwl1tbkl2AjwKvqao7R6lVVfdV1VNpjnp0UJInL7CnZwO3VNXFo/QzzSFVdSBwJPDKJIeOUGspzVdu762qA4AfAKNu37YMOBr4hxHrPIJmrcl+wF7Azkmev9B6VXUV8HbgU8C/0nxdee8oPfZY7zNrXhek87xCN5k1ryPpfV6hu8x2lde2pz5ntrd5bet0ltlx5rWvg/Ewh8jcqpLsQBPYD1fVx7qq237t8VngiAWWOAQ4Osm1NF+P/UaSD43Y043tf28BPk7zNdxCTQFTA5/Yz6EJ8iiOBC6pqu+MWOcZwLeqalNV/QT4GPAroxSsqtOr6sCqOpTmq8b/GLHHvup1Zs3rgo0jr9BNZs3rwvU6rzCezHaQV+h3ZvucV+g4s+PKa18H42EOkbnVJAnNNjxXVdVfdVBveZLd2/MPo/nj+fpCalXV66tqRVWtpHndPlNVC16LkmTnJLtuPg/8Js1XGAtSVTcDNyR5fHvV4cCVC63XOo4Rv+JpXQ8cnGSn9j0+nGbbtgVL8sj2v/sCv003ffZRbzNrXnuXV+gms+Z14XqbV+g2s13mFfqd2Z7nFTrO7LjyOuchobeGmuUQmaPUTHImcBiwZ5Ip4E1VdfoCyx0CvAD4WrvdEsAbqmr9Aus9CvhAkiU0H1bOrqqRdwHTkZ8HPt78DbMU+EhV/euINV8FfLj9H/I1wIsWWqjdtuiZwMtH7ImqujDJOcAlNF/JXMroh5r8aJI9gJ8Ar6yq20as10tdZ9a8Lliv8wrdZda8LlzP8wrdZrbPeYXuM9vLvMJYMjuWvHpI6AmT5K6BizsBPwbuay+/vKo+3NHjvIPmU+LDgduAtVX11hmW+33gDOBlVdWrX+hKfdCXzCYp4Ifcvy3pWVX10i4eW9pW9CivS4C3AC+m2WXdRuDXO9zLhGbhYDzB2m2cXlpV/zaG2o+n2VbpB0n2Bs4D/mRwW692Q/ov0Xzye5eDsbRlWzOz7WC8qqo2dv3Y0rZoK+f1z2i2v30RzSYIT6LZxd7dXfeiB+rlphTa+qrq6mlX/RR47LTr/hz4a+B5i9KUpFkNmVlJPbClvLYrnV4D/GLdv9/gTo4qqLn19cd36kiSNUlun+00xH3vovml6840R/nZfNtBwGrgfWN9AtJ2ZlyZbX0+yc1JPpZm/7CSRjCmvP4/NN/EPrfN6zeSvHK8z0SbORhv46rqlKrafbbTXPel2bbpQOCDwB3ws22f3gO8qqq62sG5JMaT2dbTgZXAE2h2zfWJNEefkrRAY8rrCpptjx9Hs8/f5wJvTvLM8T0TbeZgrC2qxqXAj2h+CADwB8BlVfWlrdeZpJnMklmq6vN1/yFiX03zD+4Tt1Kbkpg1rz9q/3tyVf2oqi6j2WfyUVujx+2Ng/E2Lskbktw122kepZYCv9CePxx4TvsVz800PxD4X0ne3XX/0vZmTJmdSTHzEdAkDWlMeb2s/a97R9gKHIy3cVX1tqraZbbTTPdJ8pAkL0/yiDQOAl4JfLpd5IU0a5qe2p420HzSfeP4n5G0bRtHZpM8KclTkyxJc5jd/wV8mxEPiCFt78aR16r6JvAF4I1JHprkicB/Bfq0/+VtloOxZvMc4JvA94EPAf+nPVFVt1fVzZtPwD3AnVV1x6zVJI3brJmlOYjA3wN30uz0fyXw7GoOyypp8W0pr9Ds4/jRwK3AJ2l25fbp6UXUPfdjLEmSJOEaY0mSJAkYYjBO8v4ktySZcefS7fYxf51kY5LLkhzYfZuShmVmpclhXqV+GWaN8RnAEVu4/UhgVXs6Hnjv6G1JGsEZmFlpUpyBeZV6Y87BuKo+D3xvC4scA/xduy++C4DdkzyqqwYlzY+ZlSaHeZX6pYujHu0N3DBweaq97qbpCyY5nuYTLzvvvPN/fsITntDBw0vbhosvvvi7VbV8ER5qqMyaV2l2fcsrmFlpS4bNbBeD8Uw7iJ9xVxdVtRZYC7B69erasGFDBw8vbRuSXLdYDzXDdQ/KrHmVZte3vIKZlbZk2Mx2sVeKKWCfgcsrgBs7qCtpPMysNDnMq7SIuhiM1wH/rf3l7MHAHVX1oK94JPWGmZUmh3mVFtGcm1IkORM4DNgzyRTwJmAHgKp6H7AeOArYCPwQeNG4mpU0NzMrTQ7zKvXLnINxVR03x+1Fc4xvST1gZqXJYV6lfvHId5IkSRIOxpIkSRLgYCxJkiQBDsaSJEkS4GAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRLgYCxJkiQBDsaSJEkS4GAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRLgYCxJkiQBDsaSJEkS4GAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRLgYCxJkiQBQw7GSY5IcnWSjUnWzHD7vknOT3JpksuSHNV9q5KGYV6lyWJmpf5YOtcCSZYApwLPBKaAi5Ksq6orBxb7Y+Dsqnpvkv2B9cDKMfSrASvXfHKk+197yrM66kR9YV6lyWJmpX4ZZo3xQcDGqrqmqu4BzgKOmbZMAbu15x8O3Nhdi5LmwbxKk8XMSj0yzGC8N3DDwOWp9rpBbwaen2SK5pPsq2YqlOT4JBuSbNi0adMC2pU0B/MqTRYzK/XIMINxZriupl0+DjijqlYARwEfTPKg2lW1tqpWV9Xq5cuXz79bSXMxr9JkMbNSjwwzGE8B+wxcXsGDv8Z5CXA2QFV9CdgR2LOLBiXNi3mVJouZlXpkmMH4ImBVkv2SLAOOBdZNW+Z64HCAJE+kCa3f40iLz7xKk8XMSj0y52BcVfcCJwDnAlfR/DL2iiQnJzm6Xey1wMuSfBU4E3hhVU3/KkjSmJlXabKYWalf5txdG0BVrafZ4H/wupMGzl8JHNJta5IWwrxKk8XMSv3hke8kSZIkHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwViSJEkCHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwViSJEkCHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwViSJEkCHIwlSZIkYMjBOMkRSa5OsjHJmlmWeV6SK5NckeQj3bYpaVjmVZoc5lXql6VzLZBkCXAq8ExgCrgoybqqunJgmVXA64FDquq2JI8cV8OSZmdepclhXqX+GWaN8UHAxqq6pqruAc4Cjpm2zMuAU6vqNoCquqXbNiUNybxKk8O8Sj0zzGC8N3DDwOWp9rpBjwMel+SLSS5IcsRMhZIcn2RDkg2bNm1aWMeStsS8SpOjs7yCmZW6MMxgnBmuq2mXlwKrgMOA44DTkuz+oDtVra2q1VW1evny5fPtVdLczKs0OTrLK5hZqQvDDMZTwD4Dl1cAN86wzD9V1U+q6lvA1TRBlrS4zKs0Ocyr1DPDDMYXAauS7JdkGXAssG7aMv8I/DpAkj1pvvq5pstGJQ3FvEqTw7xKPTPnYFxV9wInAOcCVwFnV9UVSU5OcnS72LnArUmuBM4HXldVt46raUkzM6/S5DCvUv/Mubs2gKpaD6yfdt1JA+cLOLE9SdqKzKs0Ocyr1C8e+U6SJEnCwViSJEkCHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwViSJEkCHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwViSJEkCHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwViSJEkCYOnWbkCSJG1/Vq755Ej3v/aUZ3XUiXS/odYYJzkiydVJNiZZs4XlnpukkqzurkVJ82FepcliZqX+mHMwTrIEOBU4EtgfOC7J/jMstyvw34ELu25S0nDMqzRZzKzUL8OsMT4I2FhV11TVPcBZwDEzLPenwDuAuzvsT9L8mFdpsphZqUeGGYz3Bm4YuDzVXvczSQ4A9qmqT2ypUJLjk2xIsmHTpk3zblbSnMyrNFnMrNQjwwzGmeG6+tmNyUOAdwKvnatQVa2tqtVVtXr58uXDdylpWOZVmixmVuqRYQbjKWCfgcsrgBsHLu8KPBn4bJJrgYOBdf44QNoqzKs0Wcys1CPDDMYXAauS7JdkGXAssG7zjVV1R1XtWVUrq2olcAFwdFVtGEvHkrbEvEqTxcxKPTLnYFxV9wInAOcCVwFnV9UVSU5OcvS4G5Q0PPMqTRYzK/XLUAf4qKr1wPpp1500y7KHjd6WpIUyr9JkMbNSf3hIaEmSJAkHY0mSJAlwMJYkSZIAB2NJkiQJcDCWJEmSAAdjSZIkCRhyd22SJGn7tnLNJ0e6/7WnPKujTqTxcY2xJEmShIOxJEmSBLgphSSpY37lLmlSucZYkiRJwsFYkiRJAhyMJUmSJMBtjCVJjLZdsNsES9pWuMZYkiRJwsFYkiRJAhyMJUmSJMDBWJIkSQL88Z0kaTvjDw0lzcY1xpIkSRIOxpIkSRIw5GCc5IgkVyfZmGTNDLefmOTKJJcl+XSSR3ffqqRhmFdpcphXqV/mHIyTLAFOBY4E9geOS7L/tMUuBVZX1VOAc4B3dN2opLmZV2lymFepf4ZZY3wQsLGqrqmqe4CzgGMGF6iq86vqh+3FC4AV3bYpaUjmVZoc5lXqmWEG472BGwYuT7XXzeYlwL/MdEOS45NsSLJh06ZNw3cpaVjmVZocneUVzKzUhWF215YZrqsZF0yeD6wGnj7T7VW1FlgLsHr16hlraOtxF0bbBPOqbc4o/2+CXv//qbO8gpmVujDMYDwF7DNweQVw4/SFkjwDeCPw9Kr6cTftSZon8ypNDvMq9cwwm1JcBKxKsl+SZcCxwLrBBZIcAPwNcHRV3dJ9m5KGZF6lyWFepZ6ZczCuqnuBE4BzgauAs6vqiiQnJzm6XewvgF2Af0jylSTrZiknaYzMqzQ5zKvUP0MdErqq1gPrp1130sD5Z3Tcl6QFMq/S5DCvUr945DtJkiQJB2NJkiQJcDCWJEmSgCGTZIGHAAAQcElEQVS3MZYkSZPH/dNL8+MaY0mSJAkHY0mSJAlwMJYkSZIAtzGWJEnbgFG2pwa3qVbDNcaSJEkSDsaSJEkS4GAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRLgYCxJkiQBDsaSJEkS4GAsSZIkAQ7GkiRJEuBgLEmSJAEOxpIkSRIw5GCc5IgkVyfZmGTNDLc/NMnft7dfmGRl141KGo55lSaLmZX6Y87BOMkS4FTgSGB/4Lgk+09b7CXAbVX1WOCdwNu7blTS3MyrNFnMrNQvw6wxPgjYWFXXVNU9wFnAMdOWOQb4QHv+HODwJOmuTUlDMq/SZDGzUo8sHWKZvYEbBi5PAb882zJVdW+SO4A9gO8OLpTkeOD49uJdSa4e4vH3nF5nRF3W63Nvc9bL/NY5dFlrznoLsC28r4/u4LHM6+LV63NvXddb1Pxv5XrD1uoir9DjzG5j72vX9fqc167r9bm3+dQbKrPDDMYzfSqtBSxDVa0F1g7xmPcXTjZU1er53Gex6vW5t67r9bm3ruv1ubdhHm6G68zrGOr1ubeu6/W5t67rLXJeYRvKbJ/f167r9bm3ruv1ubdx1BtmU4opYJ+ByyuAG2dbJslS4OHA97poUNK8mFdpsphZqUeGGYwvAlYl2S/JMuBYYN20ZdYBv9+efy7wmap60KdZSWNnXqXJYmalHplzU4p2e6YTgHOBJcD7q+qKJCcDG6pqHXA68MEkG2k+xR7bYY/z+lpokev1ubeu6/W5t67r9bm3LTKvi1qvz711Xa/PvXVdb9HyCttcZvv8vnZdr8+9dV2vz711Xi9+6JQkSZI88p0kSZIEOBhLkiRJQI8H47kOkbmAeu9PckuSyzuotU+S85NcleSKJK8esd6OSb6c5Kttvbd00OOSJJcm+UQHta5N8rUkX0myoYN6uyc5J8nX29fwaQus8/i2p82nO5O8ZsTe/kf7Hlye5MwkO45Y79VtrStG7a3vusyseR2pVi/z2tbqNLPmdeH6mte2XmeZHUde27q9zGyf89rW7CyzY8trVfXuRPMDhG8CjwGWAV8F9h+x5qHAgcDlHfT3KODA9vyuwDdG6Y9mH5W7tOd3AC4EDh6xxxOBjwCf6OD5Xgvs2eH7+wHgpe35ZcDuHf3N3Aw8eoQaewPfAh7WXj4beOEI9Z4MXA7sRPND138DVnX1Ovbp1HVmzetItXqf14G/mQVn1ryO/Nr3Mq9tvc4yO468trV6mdm+5rWt0Vlmx5nXvq4xHuYQmfNSVZ+no/0+VtVNVXVJe/77wFU0b/hC61VV3dVe3KE9LfhXkUlWAM8CTltojXFJshvN/0RPB6iqe6rq9g5KHw58s6quG7HOUuBhafYVuhMP3p/ofDwRuKCqflhV9wKfA54zYn991WlmzWs/jDGv0E1mzevC9Davbb3OMtt1XqG/mZ2AvEJ3mR1bXvs6GM90iMwF/0M2TklWAgfQfAodpc6SJF8BbgE+VVWj1HsX8IfAT0fpaUAB5yW5OM0hR0fxGGAT8Lft11CnJdl59BY5FjhzlAJV9W3gL4HrgZuAO6rqvBFKXg4cmmSPJDsBR/HAHflvSyYis+Z13saVVxgxs+Z1JBORV+gmsx3nFfqb2d7mFTrP7Njy2tfBeKjDX25tSXYBPgq8pqruHKVWVd1XVU+lOerRQUmevMCeng3cUlUXj9LPNIdU1YHAkcArkxw6Qq2lNF+5vbeqDgB+AIy6fdsy4GjgH0as8wiatSb7AXsBOyd5/kLrVdVVwNuBTwH/SvN15b2j9Nhjvc+seV2QzvMK3WTWvI6k93mF7jLbVV7bnvqc2d7mta3TWWbHmde+DsbDHCJzq0qyA01gP1xVH+uqbvu1x2eBIxZY4hDg6CTX0nw99htJPjRiTze2/70F+DjN13ALNQVMDXxiP4cmyKM4Erikqr4zYp1nAN+qqk1V9RPgY8CvjFKwqk6vqgOr6lCarxr/Y8Qe+6rXmTWvCzaOvEI3mTWvC9frvMJ4MttBXqHfme1zXqHjzI4rr30djIc5ROZWkyQ02/BcVVV/1UG95Ul2b88/jOaP5+sLqVVVr6+qFVW1kuZ1+0xVLXgtSpKdk+y6+TzwmzRfYSxIVd0M3JDk8e1VhwNXLrRe6zhG/IqndT1wcJKd2vf4cJpt2xYsySPb/+4L/Dbd9NlHvc2see1dXqGbzJrXhettXqHbzHaZV+h3ZnueV+g4s+PK65yHhN4aapZDZI5SM8mZwGHAnkmmgDdV1ekLLHcI8ALga+12SwBvqKr1C6z3KOADSZbQfFg5u6pG3gVMR34e+HjzN8xS4CNV9a8j1nwV8OH2f8jXAC9aaKF226JnAi8fsSeq6sIk5wCX0HwlcymjH2ryo0n2AH4CvLKqbhuxXi91nVnzumC9zit0l1nzunA9zyt0m9k+5xW6z2wv8wpjyexY8uohoSVJkiT6uymFJEmStKgcjCVJkiQcjCVJkiTAwViSJEkCHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwViSJEkCHIwlSZIkwMFYkiRJAhyMJUmSJMDBWJIkSQIcjCVJkiTAwViSJEkCHIwlSZIkwMFYUs8leV+SP1mEx7k2yTOGXPaFSf594PJdSR7TUR9vSHJae35lkkqytKPa+7a9LumiXpeS/EuS39/C7Wck+bMhaw39PPv8mkhafA7G0gRJ8rtJNrT/kN/UDhO/urX76sr0gROgql5RVX+6tXoaRlXtUlXXbGmZJIclmRqi1tuq6qVd9DV92K+q69te7+uifpeq6siq+gDM/Hcwz1pDP8/pyyb5bJIFvf7tB5nzk/wwydeH/aAlqT8cjKUJkeRE4F3A24CfB/YF3gMcM8vynaxl1OLxPZt4ZwKXAnsAbwTOSbJ867YkaT4cjKUJkOThwMnAK6vqY1X1g6r6SVX9c1W9rl3mzUnOSfKhJHcCL0zy0CTvSnJje3pXkoe2y++Z5BNJbk/yvSRfSPKQ9rY/SvLtJN9PcnWSw2fp66gkV7bLfTvJ/xy47dlJvtLW/79JnjJw2z5JPpZkU5Jbk7w7yROB9wFPa9eI394u+4Cv0JO8LMnGtud1SfYauK2SvCLJfyS5LcmpSdLe9gtJPtM+3neTfDjJ7kO+/nu0j3Vnki8DvzDt9kry2NlekyQ7A/8C7NU+t7uS7DXLe/bmJB+a1sKL2/fvpiSvHXjc6a/Nz9ZKJ/kgzYenf24f7w8zbdOMtod17Wu5McnLBmq9OcnZSf6ufS5XJFk9cPucfyNJ9mvf/81/V6cluWXg9g8leU17/rNJXjrb30HrEUk+2T7mhUke8D4M1J3+PD+b5E+TfLG973lJ9py+bJK3Ar8GvLt97HfPVH+Wx3wccCDwpqr6UVV9FPga8DvD1pC09TkYS5PhacCOwMfnWO4Y4Bxgd+DDNGutDgaeCvwicBDwx+2yrwWmgOU0a6DfAFSSxwMnAL9UVbsC/y9w7SyPdzrw8na5JwOfAUhyIPB+4OU0a8/+BliXZlBfAnwCuA5YCewNnFVVVwGvAL7UfrX9oKE1yW8Afw48D3hUW+OsaYs9G/il9vk+r+0fIO199wKeCOwDvHmW5zXdqcDd7WO+uD3N5kGvSVX9ADgSuLF9brtU1Y3t8tPfs5n8OrAK+E1gTYb4ir6qXgBcD/xW+3jvmGGxM2n+BvYCngu8bdqAezTN67s7sA54N8CwfyNV9S3gTuCA9qpfA+5qh1+AQ4HPTbvPlv4OjgPeAjwC2Ai8da7XYcDvAi8CHgksA/7n9AWq6o3AF4AT2sc+oX2+l7UD/kyn97R3fxJwTVV9f6DkV9vrJU0IB2NpMuwBfLeq7p1juS9V1T9W1U+r6kfA7wEnV9UtVbWJZqh4QbvsT2gGvUe3a5+/UFUF3Ac8FNg/yQ5VdW1VfXOWx/tJu9xuVXVbVV3SXv8y4G+q6sKquq/ddvTHNEP6QTSD2OvaNd93V9Ww25P+HvD+qrqkqn4MvJ5mzeLKgWVOqarbq+p64HyaDwVU1caq+lRV/bh9Lf4KePpcD9gO8r8DnNT2eznwgS3cZbbXZDbT37OZvKV97K8Bf0szII4kyT7ArwJ/1L4HXwFO4/6/D4B/r6r17fa3H6T5sAHz+xv5HPD0JP+pvXxOe3k/YDea4XFYH6uqL7c5+DDtezukv62qb7Sv8dnzuW9VPaWqdp/l9AftYrsAd0y76x3ArvPoUdJW5mAsTYZbgT0z9zaoN0y7vBfNWtXNrmuvA/gLmrVu5yW5JskaaAZI4DU0a1NvSXLW4OYK0/wOcBRwXZLPJXlae/2jgdcOrlmjWUO7V/vf64YY8mfygOdTVXfRvDZ7Dyxz88D5H9IMLCR5ZPtcvt1utvAhYM8hHnM5sJQHvrbXzbIszP6azGb6ezbXMoPv4Sj2Ar43bQ3ndWz5tdwxydJ5/o18DjiMZu3w54HP0nwgeTrwhar66Tx6nvG9XYT7DuMumkF/0G7A92dYVlJPORhLk+FLNF/l/5c5lqtpl2+kGVI327e9jqr6flW9tqoeA/wWcOLmr9Gr6iNV9avtfQt4+4wPVnVRVR1D8/X0P9KsiYNmkHvrtDVrO1XVme1t+84y5E/vf7oHPJ922909gG/PcT9oNqMo4ClVtRvwfJrNK+ayCbiXZqDfbN/ZFt7CazLbc5vrOTPDY2/eDOMHwE4Dt/0nHmhLtW8Efi7J4BrNfRnutRz6b4RmMP41muH4c8C/A4fQDMafm+U+w7wm4/Kgx263r75rltP72sWuAB4z7fX8xfZ6SRPCwViaAFV1B3AScGqS/5JkpyQ7JDkyyUzbjm52JvDHSZa3PzY6iWZN6eYfxz02SWi2A70PuC/J45P8Rpof6d0N/Ki97QGSLEvye0keXlU/GagB8P8Br0jyy2nsnORZ7dDwZeAm4JT2+h2THNLe7zvAiiTLZnk+HwFelOSpbX9vAy6sqmuHeBl3pVmrd3uSvYHXDXEf2s0IPga8uX3d9wdm3N/uHK/Jd4A90vyQcr7+pH3sJ9FsJ/v37fVfAY5K8nPtpgqvmXa/7wAz7l+5qm4A/i/w5+178BTgJcy+nfPPDPs30j7Of7S3Px/4fFXd2fb1O8w+GM/1dzBOD3rNqupJA9uGTz+9ol3mGzTvx5va1/M5wFOAjy76M5C0YA7G0oSoqr8CTqT58dwmmjWvJ9CslZzNnwEbgMtofiF/SXsdND/m+jeaYfFLwHuq6rM0246eAnyX5uvnR9L8MG8mLwCubTdNeAXN8ENVbaDZzvjdwG00m2y8sL3tPpo11I+l+XHYFPBf23qfoVnDdnOS787wGnwa+BOaYeMmmr1DHLuF5z/oLTR7DbgD+CTNsDusE2i+er8ZOINmO9/ZzPaafJ3mg8o17eYl89kc4nM0r+Gngb+sqvPa6z9Is43utcB53D8wb/bnNB+Mbs/AHkMGHEfzA8gbaX7Y+aaq+tQQ/cznb2Rz/7e2231vvhyaXZvNZIt/B2P2v4HnptmryV/P877HAqtp/uZPAZ7bbs8uaUKk+a2NJEmStH1zjbEkSZKEg7EkSZIEOBhLkiRJgIOxJEmSBDgYS5IkSYCDsSRJkgQ4GEuSJEmAg7EkSZIEwP8PvLeBk5vnac8AAAAASUVORK5CYII=\n",
                        "text/plain": [
                            "<Figure size 864x432 with 6 Axes>"
                        ]
                    },
                    "metadata": {},
                    "output_type": "display_data"
                }
            ],
            "source": [
                "init = 0\n",
                "k = 10\n",
                "Ts = [k*d + 1 + i for i in range(2*d)]\n",
                "num_reps = 10**2\n",
                "dists = cross_sectional_dist(mc2, T=Ts, init=init, num_reps=num_reps)\n",
                "\n",
                "fig, axes = plt.subplots(2, d, figsize=(12, 6))\n",
                "for dist, T, ax in zip(dists, Ts, axes.flatten()):\n",
                "    draw_histogram(dist, ax=ax,\n",
                "                   title='T = {T}'.format(T=T))\n",
                "fig.suptitle('Cross sectional distributions with init={init}'.format(init=init),\n",
                "             y=0.05, fontsize=12)\n",
                "plt.show()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Compare these with the rows of $P^{10d+1}, \\ldots, P^{10d+d}$."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Example 3: Nearly completely decomposable chain"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Consider the Markov chain given by the following stochastic matrix $P^{\\varepsilon}$,\n",
                "parameterized by $\\varepsilon$:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 74,
            "metadata": {},
            "outputs": [],
            "source": [
                "def P_epsilon(eps, p=0.5):\n",
                "    P = np.array([[1-(p+eps), p,         eps],\n",
                "                  [p,         1-(p+eps), eps],\n",
                "                  [eps,       eps,       1-2*eps]])\n",
                "    return P"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "If $\\varepsilon = 0$,\n",
                "then the Markovh chain is reducible into two recurrent classes, `[0, 1]` and `[2]`:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 75,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([[0.5, 0.5, 0. ],\n",
                            "       [0.5, 0.5, 0. ],\n",
                            "       [0. , 0. , 1. ]])"
                        ]
                    },
                    "execution_count": 75,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "P_epsilon(0)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 76,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "[array([0, 1]), array([2])]"
                        ]
                    },
                    "execution_count": 76,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "MarkovChain(P_epsilon(0)).recurrent_classes"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "If $\\varepsilon > 0$ but small, the chain is irreducible,\n",
                "but transition within each of the subsets `[0, 1]` and `[2]` is much more likely\n",
                "than that between these sets."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 77,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "array([[0.499, 0.5  , 0.001],\n",
                            "       [0.5  , 0.499, 0.001],\n",
                            "       [0.001, 0.001, 0.998]])"
                        ]
                    },
                    "execution_count": 77,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "P_epsilon(0.001)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 78,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "[array([0, 1, 2])]"
                        ]
                    },
                    "execution_count": 78,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "MarkovChain(P_epsilon(0.001)).recurrent_classes"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Analytically, the unique stationary distribution of the chain with $\\varepsilon > 0$\n",
                "is `(1/3, 1/3, 1/3)`, independent of the value of $\\varepsilon$."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "However,\n",
                "for such matrices with small values of $\\varepsilon > 0$,\n",
                "general purpose eigenvalue solvers are numerically unstable.\n",
                "\n",
                "For example, if we use [`numpy.linalg.eig`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eig.html)\n",
                "to compute the eigenvector that corresponds\n",
                "to the dominant (i.e., largest in magnitude) eigenvalue:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 79,
            "metadata": {},
            "outputs": [],
            "source": [
                "np.set_printoptions(precision=15)  # Increase the number of digits printed"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 80,
            "metadata": {
                "scrolled": true
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "epsilon = 1e-12\n",
                        "[0.33333662324477  0.33333662324477  0.333326753510459]\n",
                        "epsilon = 1e-13\n",
                        "[0.333344849397111 0.333344849397111 0.333310301205779]\n",
                        "epsilon = 1e-14\n",
                        "[0.333000773366967 0.333000773366967 0.333998453266065]\n",
                        "epsilon = 1e-15\n",
                        "[0.329141015582661 0.329141015582661 0.341717968834677]\n",
                        "epsilon = 1e-16\n",
                        "[0.5 0.5 0. ]\n",
                        "epsilon = 1e-17\n",
                        "[0.5 0.5 0. ]\n"
                    ]
                }
            ],
            "source": [
                "epsilons = [float('1e-{i}'.format(i=i)) for i in range(12, 18)]\n",
                "for eps in epsilons:\n",
                "    print('epsilon = {eps}'.format(eps=eps))\n",
                "    w, v = np.linalg.eig(P_epsilon(eps).T)\n",
                "    i = w.argmax()\n",
                "    print(v[:, i]/v[:, i].sum())"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The output becomes farther from the actual stationary distribution `(1/3, 1/3, 1/3)`\n",
                "as $\\varepsilon$ becomes smaller."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "The same applies to [`scipy.linalg.eig`](http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eig.htm):"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 81,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "epsilon = 1e-12\n",
                        "[0.33333662324477  0.33333662324477  0.333326753510459]\n",
                        "epsilon = 1e-13\n",
                        "[0.333344849397131 0.333344849397131 0.333310301205738]\n",
                        "epsilon = 1e-14\n",
                        "[0.333883550460886 0.333883550460886 0.332232899078228]\n",
                        "epsilon = 1e-15\n",
                        "[0.329141015582661 0.329141015582661 0.341717968834677]\n",
                        "epsilon = 1e-16\n",
                        "[0.344742003948057 0.344742003948057 0.310515992103886]\n",
                        "epsilon = 1e-17\n",
                        "[ 0.523579944426164  0.523579944426163 -0.047159888852327]\n"
                    ]
                }
            ],
            "source": [
                "import scipy.linalg\n",
                "\n",
                "for eps in epsilons:\n",
                "    print('epsilon = {eps}'.format(eps=eps))\n",
                "    w, v = scipy.linalg.eig(P_epsilon(eps), left=True, right=False)\n",
                "    i = w.argmax()\n",
                "    print(v[:, i]/v[:, i].sum())"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "`MarkovChain` in `quantecon` employs\n",
                "the algorithm called the \"[GTH algorithm](http://quanteconpy.readthedocs.org/en/stable/markov/gth_solve.html)\",\n",
                "which is a numerically stable variant of Gaussian elimination,\n",
                "specialized for Markov chains."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 82,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "epsilon = 1e-12\n",
                        "[0.333333333333333 0.333333333333333 0.333333333333333]\n",
                        "epsilon = 1e-13\n",
                        "[0.333333333333333 0.333333333333333 0.333333333333333]\n",
                        "epsilon = 1e-14\n",
                        "[0.333333333333333 0.333333333333333 0.333333333333333]\n",
                        "epsilon = 1e-15\n",
                        "[0.333333333333333 0.333333333333333 0.333333333333333]\n",
                        "epsilon = 1e-16\n",
                        "[0.333333333333333 0.333333333333333 0.333333333333333]\n",
                        "epsilon = 1e-17\n",
                        "[0.333333333333333 0.333333333333333 0.333333333333333]\n",
                        "epsilon = 1e-100\n",
                        "[0.333333333333333 0.333333333333333 0.333333333333333]\n"
                    ]
                }
            ],
            "source": [
                "for eps in epsilons + [1e-100]:\n",
                "    print('epsilon = {eps}'.format(eps=eps))\n",
                "    print(MarkovChain(P_epsilon(eps)).stationary_distributions[0])"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "collapsed": true
            },
            "source": [
                "It succeeds in obtaining the correct stationary distribution for any value of $\\varepsilon$."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": []
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.6.5"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 1
}