ipynb/chris_conlon-nevo.ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Random Coefficients Logit Tutorial with the Fake Cereal Data"
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "'0.7.0'"
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "import pyblp\n",
                "import numpy as np\n",
                "import pandas as pd\n",
                "\n",
                "pyblp.options.digits = 2\n",
                "pyblp.options.verbose = False\n",
                "pyblp.__version__"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "In this tutorial, we'll use data from [Nevo (2000)](https://pyblp.readthedocs.io/en/stable/references.html#nevo-2000) to solve the paper's fake cereal problem. Locations of CSV files that contain the data are in the [`data`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.data.html#module-pyblp.data) module.\n\n## Theory of Random Coefficients Logit\n\nThe random coefficients model extends the plain logit model by allowing for correlated tastes for different product characteristics.\nIn this  model (indirect) utility is given by\n\n$$u_{jti} = \\alpha_i p_{jt} + x_{jt} \\beta_i^x + \\xi_{jt} + \\epsilon_{jti}$$\n\nThe main addition is that $\\beta_i = (\\alpha_i, \\beta_i^x)$ have individual specific subscripts $i$.\n\nConditional on $\\beta_i$, the individual marketshares follow the same logit form as before. But now we must integrate over heterogeneous individuals to get the aggregate marketshares:\n\n$$s_{jt}(\\alpha, \\beta, \\theta) = \\int \\frac{\\exp(\\alpha_i p_{jt} + x_{jt} \\beta_i^x + \\xi_{jt})}{1 + \\sum_k \\exp(\\alpha_i p_{jt} + x_{kt} \\beta_i^x + \\xi_{kt})} f(\\alpha_i, \\beta_i \\mid \\theta).$$\n\nIn general, this integral needs to be calculated numerically. This also means that we can't directly linearize the model. It is common to re-parametrize the model to separate the aspects of mean utility that all individuals agree on, $\\delta_{jt} = \\alpha p_{jt} + x_{jt} \\beta^x + \\xi_{jt}$, from the individual specific heterogeneity, $\\mu_{jti}(\\theta)$. This gives us\n\n$$s_{jt}(\\delta_{jt}, \\theta) = \\int \\frac{\\exp(\\delta_{jt} + \\mu_{jti})}{1 + \\sum_k \\exp(\\delta_{kt} + \\mu_{kti})} f(\\mu_{ti} | \\theta).$$\n\nGiven a guess of $\\theta$ we can solve the system of nonlinear equations for the vector $\\delta$ which equates observed and predicted marketshares $s_{jt} = s_{jt}(\\delta, \\theta)$. Now we can perform a linear IV GMM regression of the form\n\n$$\\delta_{jt}(\\theta) = \\alpha p_{jt} + x_{jt} \\beta^x + \\xi_{jt}.$$\n\nThe moments are constructed by interacting the predicted residuals $\\hat{\\xi}_{jt}(\\theta)$ with instruments $z_{jt}$ to form\n\n$$\\bar{g}(\\theta) =\\frac{1}{N} \\sum_{j,t} z_{jt}' \\hat{\\xi}_{jt}(\\theta).$$"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "## Application of Random Coefficients Logit with the Fake Cereal Data\n\nTo include random coefficients we need to add a [`Formulation`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Formulation.html#pyblp.Formulation) for the nonlinear characteristics $X_2$.\n\nJust like in the logit case we have the same reserved field names in `product_data`:\n\n- `market_ids` are the unique market identifiers which we subscript $t$.\n- `shares` specifies the marketshares which need to be between zero and one, and within a market ID, $\\sum_{j} s_{jt} < 1$.\n- `prices` are prices $p_{jt}$. These have some special properties and are _always_ treated as endogenous.\n- `demand_instruments0`, `demand_instruments1`, and so on are numbered demand instruments. These represent only the _excluded_ instruments. The exogenous regressors in $X_1$ (of which $X_2$ is typically a subset) will be automatically added to the set of instruments.\n\nWe proceed with the following steps:\n\n1. Load the `product data` which at a minimum consists of `market_ids`, `shares`, `prices`, and at least a single column of demand instruments, `demand_instruments0`.\n2. Define a [`Formulation`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Formulation.html#pyblp.Formulation) for the $X_1$ (linear) demand model.\n\n    - This and all other formulas are similar to R and [patsy](https://patsy.readthedocs.io/en/stable/) formulas.\n    - It includes a constant by default. To exclude the constant, specify either a `0` or a `-1`.\n    - To efficiently include fixed effects, use the `absorb` option and specify which categorical variables you would like to absorb.\n    - Some model reduction may happen automatically. The constant will be excluded if you include fixed effects and some precautions are taken against collinearity. However, you will have to make sure that differently-named variables are not collinear.\n\n3. Define a [`Formulation`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Formulation.html#pyblp.Formulation) for the $X_2$ (nonlinear) demand model.\n\n    - Include only the variables over which we want random coefficients.\n    - Do not absorb or include fixed effects.\n    - It will include a random coefficient on the constant (to capture inside good vs. outside good preference) unless you specify not to with a `0` or a `-1`.\n\n4. Define an [`Integration`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Integration.html#pyblp.Integration) configuration to solve the market share integral from several available options:\n\n    - Monte Carlo integration (pseudo-random draws).\n    - Product rule quadrature.\n    - Sparse grid quadrature.\n\n3. Combine [`Formulation`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Formulation.html#pyblp.Formulation) classes, `product_data`, and the [`Integration`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Integration.html#pyblp.Integration) configuration to construct a [`Problem`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.html#pyblp.Problem).\n4. Use the [`Problem.solve`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.solve.html#pyblp.Problem.solve) method to estimate paramters.\n\n    - It is required to specify an initial guess of the nonlinear parameters. This serves two primary purposes: speeding up estimation and indicating to the solver through initial values of zero which parameters are restricted to be always zero."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Specification of Random Taste Parameters\n\nIt is common to assume that $f(\\beta_i \\mid \\theta)$ follows a multivariate normal distribution, and to break it up into three parts:\n\n1. A mean $K_1 \\times 1$ taste which all individuals agree on, $\\beta$.\n2. A $K_2 \\times K_2$ covariance matrix, $\\Sigma$ .\n3. Any $K_2 \\times D$ interactions, $\\Pi$, with observed $D \\times 1$ demographic data, $d_i$.\n\nTogether this gives us that\n\n$$\\beta_i \\sim N(\\beta + \\Pi d_i, \\Sigma).$$\n\n[`Problem.solve`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.solve.html#pyblp.Problem.solve) takes an initial guess $\\Sigma_0$ of $\\Sigma$. It guarantees that $\\hat{\\Sigma}$ (the estimated parameters) will have the same sparsity structure as $\\Sigma_0$. So any zero element of $\\Sigma$ is restricted to be zero in the solution $\\hat{\\Sigma}$. For example, a popular restriction is that $\\Sigma$ is diagonal, this can be achieved by passing a diagonal matrix as $\\Sigma_0$.\n\nAs is common with multivariate normal distributions, $\\Sigma$ is not estimated directly. Rather, its matrix square (Cholesky) root $L$ is estimated where $LL' = \\Sigma$."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Loading the Data\n\nThe `product_data` argument of [`Problem`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.html#pyblp.Problem) should be a structured array-like object with fields that store data. Product data can be a structured [NumPy](https://www.numpy.org/) array, a [pandas](https://pandas.pydata.org/) DataFrame, or other similar objects."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>market_ids</th>\n",
                            "      <th>city_ids</th>\n",
                            "      <th>quarter</th>\n",
                            "      <th>product_ids</th>\n",
                            "      <th>firm_ids</th>\n",
                            "      <th>brand_ids</th>\n",
                            "      <th>shares</th>\n",
                            "      <th>prices</th>\n",
                            "      <th>sugar</th>\n",
                            "      <th>mushy</th>\n",
                            "      <th>...</th>\n",
                            "      <th>demand_instruments10</th>\n",
                            "      <th>demand_instruments11</th>\n",
                            "      <th>demand_instruments12</th>\n",
                            "      <th>demand_instruments13</th>\n",
                            "      <th>demand_instruments14</th>\n",
                            "      <th>demand_instruments15</th>\n",
                            "      <th>demand_instruments16</th>\n",
                            "      <th>demand_instruments17</th>\n",
                            "      <th>demand_instruments18</th>\n",
                            "      <th>demand_instruments19</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>F1B04</td>\n",
                            "      <td>1</td>\n",
                            "      <td>4</td>\n",
                            "      <td>0.012417</td>\n",
                            "      <td>0.072088</td>\n",
                            "      <td>2</td>\n",
                            "      <td>1</td>\n",
                            "      <td>...</td>\n",
                            "      <td>2.116358</td>\n",
                            "      <td>-0.154708</td>\n",
                            "      <td>-0.005796</td>\n",
                            "      <td>0.014538</td>\n",
                            "      <td>0.126244</td>\n",
                            "      <td>0.067345</td>\n",
                            "      <td>0.068423</td>\n",
                            "      <td>0.034800</td>\n",
                            "      <td>0.126346</td>\n",
                            "      <td>0.035484</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>F1B06</td>\n",
                            "      <td>1</td>\n",
                            "      <td>6</td>\n",
                            "      <td>0.007809</td>\n",
                            "      <td>0.114178</td>\n",
                            "      <td>18</td>\n",
                            "      <td>1</td>\n",
                            "      <td>...</td>\n",
                            "      <td>-7.374091</td>\n",
                            "      <td>-0.576412</td>\n",
                            "      <td>0.012991</td>\n",
                            "      <td>0.076143</td>\n",
                            "      <td>0.029736</td>\n",
                            "      <td>0.087867</td>\n",
                            "      <td>0.110501</td>\n",
                            "      <td>0.087784</td>\n",
                            "      <td>0.049872</td>\n",
                            "      <td>0.072579</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>F1B07</td>\n",
                            "      <td>1</td>\n",
                            "      <td>7</td>\n",
                            "      <td>0.012995</td>\n",
                            "      <td>0.132391</td>\n",
                            "      <td>4</td>\n",
                            "      <td>1</td>\n",
                            "      <td>...</td>\n",
                            "      <td>2.187872</td>\n",
                            "      <td>-0.207346</td>\n",
                            "      <td>0.003509</td>\n",
                            "      <td>0.091781</td>\n",
                            "      <td>0.163773</td>\n",
                            "      <td>0.111881</td>\n",
                            "      <td>0.108226</td>\n",
                            "      <td>0.086439</td>\n",
                            "      <td>0.122347</td>\n",
                            "      <td>0.101842</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>F1B09</td>\n",
                            "      <td>1</td>\n",
                            "      <td>9</td>\n",
                            "      <td>0.005770</td>\n",
                            "      <td>0.130344</td>\n",
                            "      <td>3</td>\n",
                            "      <td>0</td>\n",
                            "      <td>...</td>\n",
                            "      <td>2.704576</td>\n",
                            "      <td>0.040748</td>\n",
                            "      <td>-0.003724</td>\n",
                            "      <td>0.094732</td>\n",
                            "      <td>0.135274</td>\n",
                            "      <td>0.088090</td>\n",
                            "      <td>0.101767</td>\n",
                            "      <td>0.101777</td>\n",
                            "      <td>0.110741</td>\n",
                            "      <td>0.104332</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>F1B11</td>\n",
                            "      <td>1</td>\n",
                            "      <td>11</td>\n",
                            "      <td>0.017934</td>\n",
                            "      <td>0.154823</td>\n",
                            "      <td>12</td>\n",
                            "      <td>0</td>\n",
                            "      <td>...</td>\n",
                            "      <td>1.261242</td>\n",
                            "      <td>0.034836</td>\n",
                            "      <td>-0.000568</td>\n",
                            "      <td>0.102451</td>\n",
                            "      <td>0.130640</td>\n",
                            "      <td>0.084818</td>\n",
                            "      <td>0.101075</td>\n",
                            "      <td>0.125169</td>\n",
                            "      <td>0.133464</td>\n",
                            "      <td>0.121111</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "<p>5 rows × 30 columns</p>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "  market_ids  city_ids  quarter product_ids  firm_ids  brand_ids    shares  \\\n",
                            "0      C01Q1         1        1       F1B04         1          4  0.012417   \n",
                            "1      C01Q1         1        1       F1B06         1          6  0.007809   \n",
                            "2      C01Q1         1        1       F1B07         1          7  0.012995   \n",
                            "3      C01Q1         1        1       F1B09         1          9  0.005770   \n",
                            "4      C01Q1         1        1       F1B11         1         11  0.017934   \n",
                            "\n",
                            "     prices  sugar  mushy          ...           demand_instruments10  \\\n",
                            "0  0.072088      2      1          ...                       2.116358   \n",
                            "1  0.114178     18      1          ...                      -7.374091   \n",
                            "2  0.132391      4      1          ...                       2.187872   \n",
                            "3  0.130344      3      0          ...                       2.704576   \n",
                            "4  0.154823     12      0          ...                       1.261242   \n",
                            "\n",
                            "   demand_instruments11  demand_instruments12  demand_instruments13  \\\n",
                            "0             -0.154708             -0.005796              0.014538   \n",
                            "1             -0.576412              0.012991              0.076143   \n",
                            "2             -0.207346              0.003509              0.091781   \n",
                            "3              0.040748             -0.003724              0.094732   \n",
                            "4              0.034836             -0.000568              0.102451   \n",
                            "\n",
                            "   demand_instruments14  demand_instruments15  demand_instruments16  \\\n",
                            "0              0.126244              0.067345              0.068423   \n",
                            "1              0.029736              0.087867              0.110501   \n",
                            "2              0.163773              0.111881              0.108226   \n",
                            "3              0.135274              0.088090              0.101767   \n",
                            "4              0.130640              0.084818              0.101075   \n",
                            "\n",
                            "   demand_instruments17  demand_instruments18  demand_instruments19  \n",
                            "0              0.034800              0.126346              0.035484  \n",
                            "1              0.087784              0.049872              0.072579  \n",
                            "2              0.086439              0.122347              0.101842  \n",
                            "3              0.101777              0.110741              0.104332  \n",
                            "4              0.125169              0.133464              0.121111  \n",
                            "\n",
                            "[5 rows x 30 columns]"
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "product_data = pd.read_csv(pyblp.data.NEVO_PRODUCTS_LOCATION)\n",
                "product_data.head()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "The product data contains `market_ids`, `product_ids`, `firm_ids`, `shares`, `prices`, a number of other firm IDs and product characteristics, and some pre-computed excluded `demand_instruments0`, `demand_instruments1`, and so on. The `product_ids` will be incorporated as fixed effects. \n\nFor more information about the instruments and the example data as a whole, refer to the [`data`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.data.html#module-pyblp.data) module."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Setting Up and Solving the Problem Without Demographics\n\nFormulations, product data, and an integration configuration are collectively used to initialize a [`Problem`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.html#pyblp.Problem). Once initialized, [`Problem.solve`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.solve.html#pyblp.Problem.solve) runs the estimation routine. The arguments to [`Problem.solve`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.solve.html#pyblp.Problem.solve) configure how estimation is performed. For example, `optimization` and `iteration` arguments configure the optimization and iteration routines that are used by the outer and inner loops of estimation.\n\nWe'll specify [`Formulation`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Formulation.html#pyblp.Formulation) configurations for $X_1$, the linear characteristics, and $X_2$, the nonlinear characteristics.\n\n- The formulation for $X_1$ consists of `prices` and fixed effects constructed from `product_ids`, which we will absorb using `absorb` argument of [`Formulation`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Formulation.html#pyblp.Formulation).\n- If we were interested in reporting estimates for each fixed effect, we could replace the formulation for $X_1$ with `Formulation('prices + C(product_ids)')`.\n- Because `sugar`, `mushy`, and the constant are collinear with `product_ids`, we can include them in $X_2$ but not in $X_1$."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "(prices + Absorb[C(product_ids)], 1 + prices + sugar + mushy)"
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "X1_formulation = pyblp.Formulation('0 + prices', absorb='C(product_ids)')\n",
                "X2_formulation = pyblp.Formulation('1 + prices + sugar + mushy')\n",
                "product_formulations = (X1_formulation, X2_formulation)\n",
                "product_formulations"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We also need to specify an [`Integration`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Integration.html#pyblp.Integration) configuration. We consider two alternatives:\n\n1. Monte Carlo draws: we simulate 50 individuals from a random normal distribution.\n2. Product rules: we construct nodes and weights according to a product rule that exactly integrates polynomials of degree $5 \\times 2 - 1 = 9$ or less."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Configured to construct nodes and weights with Monte Carlo simulation."
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc_integration = pyblp.Integration('monte_carlo', size=50, seed=0)\n",
                "mc_integration"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Configured to construct nodes and weights according to the level-5 Gauss-Hermite product rule."
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pr_integration = pyblp.Integration('product', size=5)\n",
                "pr_integration"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Dimensions:\n",
                            "============================================\n",
                            " T    N     F    I     K1    K2    MD    ED \n",
                            "---  ----  ---  ----  ----  ----  ----  ----\n",
                            "94   2256   5   4700   1     4     20    1  \n",
                            "============================================\n",
                            "\n",
                            "Formulations:\n",
                            "===========================================================\n",
                            "       Column Indices:           0       1       2      3  \n",
                            "-----------------------------  ------  ------  -----  -----\n",
                            " X1: Linear Characteristics    prices                      \n",
                            "X2: Nonlinear Characteristics    1     prices  sugar  mushy\n",
                            "==========================================================="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "mc_problem = pyblp.Problem(product_formulations, product_data, integration=mc_integration)\n",
                "mc_problem"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Dimensions:\n",
                            "=============================================\n",
                            " T    N     F     I     K1    K2    MD    ED \n",
                            "---  ----  ---  -----  ----  ----  ----  ----\n",
                            "94   2256   5   58750   1     4     20    1  \n",
                            "=============================================\n",
                            "\n",
                            "Formulations:\n",
                            "===========================================================\n",
                            "       Column Indices:           0       1       2      3  \n",
                            "-----------------------------  ------  ------  -----  -----\n",
                            " X1: Linear Characteristics    prices                      \n",
                            "X2: Nonlinear Characteristics    1     prices  sugar  mushy\n",
                            "==========================================================="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "pr_problem = pyblp.Problem(product_formulations, product_data, integration=pr_integration)\n",
                "pr_problem"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "As an illustration of how to configure the optimization routine, we'll use a simpler, non-default [`Optimization`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Optimization.html#pyblp.Optimization) configuration that doesn't support parameter bounds."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Configured to optimize using the BFGS algorithm implemented in SciPy with analytic gradients and options {}."
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "bfgs = pyblp.Optimization('bfgs')\n",
                "bfgs"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We estimate three versions of the model:\n\n1. An unrestricted covariance matrix for random tastes using Monte Carlo integration.\n2. An unrestricted covariance matrix for random tastes using the product rule.\n3. A restricted diagonal matrix for random tastes using Monte Carlo Integration.\n\nNotice that the only thing that changes when we estimate the restricted covariance is our initial guess of $\\Sigma_0$. The lower diagonal in this initial guess is ignored because we are optimizing over the upper triangular (Cholesky) root of $\\Sigma$."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Problem Results Summary:\n",
                            "========================================================================================================================\n",
                            "                                                                                                   Smallest    Largest  \n",
                            "Computation  GMM   Optimization   Objective   Fixed Point  Contraction  Objective    Gradient      Hessian     Hessian  \n",
                            "   Time      Step   Iterations   Evaluations  Iterations   Evaluations    Value    Infinity Norm  Eigenvalue  Eigenvalue\n",
                            "-----------  ----  ------------  -----------  -----------  -----------  ---------  -------------  ----------  ----------\n",
                            " 00:05:19     2         62           171        175814       541988     +3.0E+05     +8.0E-03      +1.5E+02    +1.2E+07 \n",
                            "========================================================================================================================\n",
                            "\n",
                            "Nonlinear Coefficient Estimates (Robust SEs in Parentheses):\n",
                            "======================================================\n",
                            "Sigma:      1         prices      sugar       mushy   \n",
                            "------  ----------  ----------  ----------  ----------\n",
                            "  1      +9.1E-02    +1.3E-01    +1.1E-01    -5.7E-02 \n",
                            "        (+1.8E+00)  (+8.2E+00)  (+9.5E+00)  (+4.3E+00)\n",
                            "                                                      \n",
                            "prices               -1.4E+00    +3.5E+00    +1.3E+01 \n",
                            "                    (+6.6E+01)  (+7.4E+01)  (+1.9E+01)\n",
                            "                                                      \n",
                            "sugar                            -1.1E-02    -1.4E-01 \n",
                            "                                (+1.7E-01)  (+1.8E-01)\n",
                            "                                                      \n",
                            "mushy                                        -2.6E-02 \n",
                            "                                            (+7.5E-01)\n",
                            "======================================================\n",
                            "\n",
                            "Beta Estimates (Robust SEs in Parentheses):\n",
                            "==========\n",
                            "  prices  \n",
                            "----------\n",
                            " -3.4E+01 \n",
                            "(+1.5E+01)\n",
                            "=========="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "results1 = mc_problem.solve(sigma=np.ones((4, 4)), optimization=bfgs)\n",
                "results1"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Problem Results Summary:\n",
                            "========================================================================================================================\n",
                            "                                                                                                   Smallest    Largest  \n",
                            "Computation  GMM   Optimization   Objective   Fixed Point  Contraction  Objective    Gradient      Hessian     Hessian  \n",
                            "   Time      Step   Iterations   Evaluations  Iterations   Evaluations    Value    Infinity Norm  Eigenvalue  Eigenvalue\n",
                            "-----------  ----  ------------  -----------  -----------  -----------  ---------  -------------  ----------  ----------\n",
                            " 00:10:49     2         69           153        100768       312910     +3.6E+05     +1.3E-02      +3.3E+00    +1.2E+07 \n",
                            "========================================================================================================================\n",
                            "\n",
                            "Nonlinear Coefficient Estimates (Robust SEs in Parentheses):\n",
                            "======================================================\n",
                            "Sigma:      1         prices      sugar       mushy   \n",
                            "------  ----------  ----------  ----------  ----------\n",
                            "  1      -9.8E-10    -1.0E-09    +3.2E-01    +6.7E-01 \n",
                            "        (+2.2E+09)  (+5.1E+09)  (+9.9E+01)  (+5.2E+01)\n",
                            "                                                      \n",
                            "prices               +1.4E-08    -6.2E+00    -1.1E+01 \n",
                            "                    (+2.8E+10)  (+1.6E+03)  (+9.3E+02)\n",
                            "                                                      \n",
                            "sugar                            +5.7E-02    +9.7E-02 \n",
                            "                                (+1.4E+01)  (+7.9E+00)\n",
                            "                                                      \n",
                            "mushy                                        -1.6E-01 \n",
                            "                                            (+1.5E+01)\n",
                            "======================================================\n",
                            "\n",
                            "Beta Estimates (Robust SEs in Parentheses):\n",
                            "==========\n",
                            "  prices  \n",
                            "----------\n",
                            " -3.1E+01 \n",
                            "(+3.1E+01)\n",
                            "=========="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "results2 = pr_problem.solve(sigma=np.ones((4, 4)), optimization=bfgs)\n",
                "results2"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Problem Results Summary:\n",
                            "========================================================================================================================\n",
                            "                                                                                                   Smallest    Largest  \n",
                            "Computation  GMM   Optimization   Objective   Fixed Point  Contraction  Objective    Gradient      Hessian     Hessian  \n",
                            "   Time      Step   Iterations   Evaluations  Iterations   Evaluations    Value    Infinity Norm  Eigenvalue  Eigenvalue\n",
                            "-----------  ----  ------------  -----------  -----------  -----------  ---------  -------------  ----------  ----------\n",
                            " 00:01:19     2         16           77          43049       134868     +4.0E+05     +3.9E-04      +2.4E+03    +1.4E+07 \n",
                            "========================================================================================================================\n",
                            "\n",
                            "Nonlinear Coefficient Estimates (Robust SEs in Parentheses):\n",
                            "======================================================\n",
                            "Sigma:      1         prices      sugar       mushy   \n",
                            "------  ----------  ----------  ----------  ----------\n",
                            "  1      +5.2E-02    +0.0E+00    +0.0E+00    +0.0E+00 \n",
                            "        (+1.1E+00)                                    \n",
                            "                                                      \n",
                            "prices               -4.3E-01    +0.0E+00    +0.0E+00 \n",
                            "                    (+8.0E+00)                        \n",
                            "                                                      \n",
                            "sugar                            +3.6E-02    +0.0E+00 \n",
                            "                                (+5.8E-02)            \n",
                            "                                                      \n",
                            "mushy                                        +5.0E-01 \n",
                            "                                            (+1.7E+00)\n",
                            "======================================================\n",
                            "\n",
                            "Beta Estimates (Robust SEs in Parentheses):\n",
                            "==========\n",
                            "  prices  \n",
                            "----------\n",
                            " -3.0E+01 \n",
                            "(+1.4E+00)\n",
                            "=========="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "results3 = mc_problem.solve(sigma=np.eye(4), optimization=bfgs)\n",
                "results3"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "We see that all three models give similar estimates of the price coefficient $\\hat{\\alpha} \\approx -30$. Note a few of the estimated terms on the diagonal of $\\Sigma$ are negative. Since the diagonal consists of standard deviations, negative values are unrealistic. When using another optimization routine that supports bounds (like the default L-BFGS-B routine), these diagonal elements are by default bounded from below by zero."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Adding Demographics to the Problem\n\nTo add demographic data we need to make two changes:\n\n1. We need to load `agent_data`, which for this cereal problem contains pre-computed Monte Carlo draws and demographics.\n2. We need to add an `agent_formulation` to the model.\n\nThe `agent data` has several reserved column names.\n\n- `market_ids` are the index that link the `agent data` to the `market_ids` in `product data`.\n- `weights` are the weights $w_{ti}$ attached to each agent. In each market, these should sum to one so that $\\sum_i w_{ti} = 1$. It is often the case the $w_{ti} = 1 / I_t$ where $I_t$ is the number of agents in market $t$, so that each agent gets equal weight. Other possibilities include quadrature nodes and weights.\n- `nodes0`, `nodes1`, and so on are the nodes at which the unobserved agent tastes $\\mu_{jti}$ are evaluated. The nodes should be labeled from $0, \\ldots, K_2 - 1$ where $K_2$ is the number of random coefficients.\n- Other fields are the realizations of the demographics $d$ themselves."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": [
                            "<div>\n",
                            "<style scoped>\n",
                            "    .dataframe tbody tr th:only-of-type {\n",
                            "        vertical-align: middle;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe tbody tr th {\n",
                            "        vertical-align: top;\n",
                            "    }\n",
                            "\n",
                            "    .dataframe thead th {\n",
                            "        text-align: right;\n",
                            "    }\n",
                            "</style>\n",
                            "<table border=\"1\" class=\"dataframe\">\n",
                            "  <thead>\n",
                            "    <tr style=\"text-align: right;\">\n",
                            "      <th></th>\n",
                            "      <th>market_ids</th>\n",
                            "      <th>city_ids</th>\n",
                            "      <th>quarter</th>\n",
                            "      <th>weights</th>\n",
                            "      <th>nodes0</th>\n",
                            "      <th>nodes1</th>\n",
                            "      <th>nodes2</th>\n",
                            "      <th>nodes3</th>\n",
                            "      <th>income</th>\n",
                            "      <th>income_squared</th>\n",
                            "      <th>age</th>\n",
                            "      <th>child</th>\n",
                            "    </tr>\n",
                            "  </thead>\n",
                            "  <tbody>\n",
                            "    <tr>\n",
                            "      <th>0</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>0.05</td>\n",
                            "      <td>0.434101</td>\n",
                            "      <td>-1.500838</td>\n",
                            "      <td>-1.151079</td>\n",
                            "      <td>0.161017</td>\n",
                            "      <td>0.495123</td>\n",
                            "      <td>8.331304</td>\n",
                            "      <td>-0.230109</td>\n",
                            "      <td>-0.230851</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>1</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>0.05</td>\n",
                            "      <td>-0.726649</td>\n",
                            "      <td>0.133182</td>\n",
                            "      <td>-0.500750</td>\n",
                            "      <td>0.129732</td>\n",
                            "      <td>0.378762</td>\n",
                            "      <td>6.121865</td>\n",
                            "      <td>-2.532694</td>\n",
                            "      <td>0.769149</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>2</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>0.05</td>\n",
                            "      <td>-0.623061</td>\n",
                            "      <td>-0.138241</td>\n",
                            "      <td>0.797441</td>\n",
                            "      <td>-0.795549</td>\n",
                            "      <td>0.105015</td>\n",
                            "      <td>1.030803</td>\n",
                            "      <td>-0.006965</td>\n",
                            "      <td>-0.230851</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>3</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>0.05</td>\n",
                            "      <td>-0.041317</td>\n",
                            "      <td>1.257136</td>\n",
                            "      <td>-0.683054</td>\n",
                            "      <td>0.259044</td>\n",
                            "      <td>-1.485481</td>\n",
                            "      <td>-25.583605</td>\n",
                            "      <td>-0.827946</td>\n",
                            "      <td>0.769149</td>\n",
                            "    </tr>\n",
                            "    <tr>\n",
                            "      <th>4</th>\n",
                            "      <td>C01Q1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>1</td>\n",
                            "      <td>0.05</td>\n",
                            "      <td>-0.466691</td>\n",
                            "      <td>0.226968</td>\n",
                            "      <td>1.044424</td>\n",
                            "      <td>0.092019</td>\n",
                            "      <td>-0.316597</td>\n",
                            "      <td>-6.517009</td>\n",
                            "      <td>-0.230109</td>\n",
                            "      <td>-0.230851</td>\n",
                            "    </tr>\n",
                            "  </tbody>\n",
                            "</table>\n",
                            "</div>"
                        ],
                        "text/plain": [
                            "  market_ids  city_ids  quarter  weights    nodes0    nodes1    nodes2  \\\n",
                            "0      C01Q1         1        1     0.05  0.434101 -1.500838 -1.151079   \n",
                            "1      C01Q1         1        1     0.05 -0.726649  0.133182 -0.500750   \n",
                            "2      C01Q1         1        1     0.05 -0.623061 -0.138241  0.797441   \n",
                            "3      C01Q1         1        1     0.05 -0.041317  1.257136 -0.683054   \n",
                            "4      C01Q1         1        1     0.05 -0.466691  0.226968  1.044424   \n",
                            "\n",
                            "     nodes3    income  income_squared       age     child  \n",
                            "0  0.161017  0.495123        8.331304 -0.230109 -0.230851  \n",
                            "1  0.129732  0.378762        6.121865 -2.532694  0.769149  \n",
                            "2 -0.795549  0.105015        1.030803 -0.006965 -0.230851  \n",
                            "3  0.259044 -1.485481      -25.583605 -0.827946  0.769149  \n",
                            "4  0.092019 -0.316597       -6.517009 -0.230109 -0.230851  "
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "agent_data = pd.read_csv(pyblp.data.NEVO_AGENTS_LOCATION)\n",
                "agent_data.head()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "The `agent formulation` tells us which columns of demographic information to interact with $X_2$."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "income + income_squared + age + child"
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "agent_formulation = pyblp.Formulation('0 + income + income_squared + age + child')\n",
                "agent_formulation"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "This tells us to include demographic realizations for `income`, `income_squared`, `age`, and the presence of children, `child`, but to ignore other possible demographics when interacting demographics with $X_2$. We should also generally exclude the constant from the demographic formula.\n\nNow we configure the [`Problem`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.html#pyblp.Problem) to include the `agent_formulation` and `agent_data`, which follow the `product_formulations` and `product_data`.\n\nWhen we display the class, it lists the demographic interactions in the table of formulations and reports $D = 4$, the dimension of the demographic draws."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Dimensions:\n",
                            "=================================================\n",
                            " T    N     F    I     K1    K2    D    MD    ED \n",
                            "---  ----  ---  ----  ----  ----  ---  ----  ----\n",
                            "94   2256   5   1880   1     4     4    20    1  \n",
                            "=================================================\n",
                            "\n",
                            "Formulations:\n",
                            "===================================================================\n",
                            "       Column Indices:           0           1           2      3  \n",
                            "-----------------------------  ------  --------------  -----  -----\n",
                            " X1: Linear Characteristics    prices                              \n",
                            "X2: Nonlinear Characteristics    1         prices      sugar  mushy\n",
                            "       d: Demographics         income  income_squared   age   child\n",
                            "==================================================================="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "nevo_problem = pyblp.Problem(\n",
                "    product_formulations,\n",
                "    product_data,\n",
                "    agent_formulation,\n",
                "    agent_data\n",
                ")\n",
                "nevo_problem"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "The initialized problem can be solved with [`Problem.solve`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.solve.html#pyblp.Problem.solve). We'll use the same starting values as [Nevo (2000)](https://pyblp.readthedocs.io/en/stable/references.html#nevo-2000). By passing a diagonal matrix as starting values for $\\Sigma$, we're choosing to ignore covariance terms. Similarly, zeros in the starting values for $\\Pi$ mean that those parameters will be fixed at zero.\n\nTo replicate common estimates, we'll use the non-default BFGS optimization routine, and we'll configure [`Problem.solve`](https://pyblp.readthedocs.io/en/stable/_api/pyblp.Problem.solve.html#pyblp.Problem.solve) to use 1-step GMM instead of 2-step GMM."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Problem Results Summary:\n",
                            "========================================================================================================================\n",
                            "                                                                                                   Smallest    Largest  \n",
                            "Computation  GMM   Optimization   Objective   Fixed Point  Contraction  Objective    Gradient      Hessian     Hessian  \n",
                            "   Time      Step   Iterations   Evaluations  Iterations   Evaluations    Value    Infinity Norm  Eigenvalue  Eigenvalue\n",
                            "-----------  ----  ------------  -----------  -----------  -----------  ---------  -------------  ----------  ----------\n",
                            " 00:01:18     1         51           58          46236       143503     +4.6E+00     +6.9E-06      +2.8E-05    +1.6E+04 \n",
                            "========================================================================================================================\n",
                            "\n",
                            "Nonlinear Coefficient Estimates (Robust SEs in Parentheses):\n",
                            "=====================================================================================================================\n",
                            "Sigma:      1         prices      sugar       mushy     |   Pi:      income    income_squared     age        child   \n",
                            "------  ----------  ----------  ----------  ----------  |  ------  ----------  --------------  ----------  ----------\n",
                            "  1      +5.6E-01    +0.0E+00    +0.0E+00    +0.0E+00   |    1      +2.3E+00      +0.0E+00      +1.3E+00    +0.0E+00 \n",
                            "        (+1.6E-01)                                      |          (+1.2E+00)                  (+6.3E-01)            \n",
                            "                                                        |                                                            \n",
                            "prices               +3.3E+00    +0.0E+00    +0.0E+00   |  prices   +5.9E+02      -3.0E+01      +0.0E+00    +1.1E+01 \n",
                            "                    (+1.3E+00)                          |          (+2.7E+02)    (+1.4E+01)                (+4.1E+00)\n",
                            "                                                        |                                                            \n",
                            "sugar                            -5.8E-03    +0.0E+00   |  sugar    -3.8E-01      +0.0E+00      +5.2E-02    +0.0E+00 \n",
                            "                                (+1.4E-02)              |          (+1.2E-01)                  (+2.6E-02)            \n",
                            "                                                        |                                                            \n",
                            "mushy                                        +9.3E-02   |  mushy    +7.5E-01      +0.0E+00      -1.4E+00    +0.0E+00 \n",
                            "                                            (+1.9E-01)  |          (+8.0E-01)                  (+6.7E-01)            \n",
                            "=====================================================================================================================\n",
                            "\n",
                            "Beta Estimates (Robust SEs in Parentheses):\n",
                            "==========\n",
                            "  prices  \n",
                            "----------\n",
                            " -6.3E+01 \n",
                            "(+1.5E+01)\n",
                            "=========="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "initial_sigma = np.diag([0.3302, 2.4526, 0.0163, 0.2441])\n",
                "initial_pi = np.array([\n",
                "  [ 5.4819,  0,      0.2037,  0     ],\n",
                "  [15.8935, -1.2000, 0,       2.6342],\n",
                "  [-0.2506,  0,      0.0511,  0     ],\n",
                "  [ 1.2650,  0,     -0.8091,  0     ]\n",
                "])\n",
                "nevo_results = nevo_problem.solve(\n",
                "    initial_sigma,\n",
                "    initial_pi,\n",
                "    optimization=bfgs,\n",
                "    method='1s'\n",
                ")\n",
                "nevo_results"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Results are similar to those in the original paper with a GMM objective value of $q(\\hat{\\theta}) = 4.56$ and a price coefficient of $\\hat{\\alpha} = -62.7$. "
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "### Restricting Parameters\n\nBecause the interactions between `price`, `income`, and `income_squared` are potentially collinear, we might worry that $\\hat{\\Pi}_{21} = 588$ and  $\\hat{\\Pi}_{22} = -30.2$ are pushing the price coefficient in opposite directions. Both are large in magnitude but statistically insignficant. One way of dealing with this is to restrict $\\Pi_{22} = 0$.\n\nThere are two ways we can do this:\n\n1. Change the initial $\\Pi_0$ values to make this term zero.\n2. Change the agent formula to drop `income_squared`.\n\nFirst, we'll change the initial $\\Pi_0$ values."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Problem Results Summary:\n",
                            "========================================================================================================================\n",
                            "                                                                                                   Smallest    Largest  \n",
                            "Computation  GMM   Optimization   Objective   Fixed Point  Contraction  Objective    Gradient      Hessian     Hessian  \n",
                            "   Time      Step   Iterations   Evaluations  Iterations   Evaluations    Value    Infinity Norm  Eigenvalue  Eigenvalue\n",
                            "-----------  ----  ------------  -----------  -----------  -----------  ---------  -------------  ----------  ----------\n",
                            " 00:00:57     1         34           41          34457       106706     +1.5E+01     +5.2E-06      +4.7E-02    +1.7E+04 \n",
                            "========================================================================================================================\n",
                            "\n",
                            "Nonlinear Coefficient Estimates (Robust SEs in Parentheses):\n",
                            "=====================================================================================================================\n",
                            "Sigma:      1         prices      sugar       mushy     |   Pi:      income    income_squared     age        child   \n",
                            "------  ----------  ----------  ----------  ----------  |  ------  ----------  --------------  ----------  ----------\n",
                            "  1      +3.7E-01    +0.0E+00    +0.0E+00    +0.0E+00   |    1      +3.1E+00      +0.0E+00      +1.2E+00    +0.0E+00 \n",
                            "        (+1.2E-01)                                      |          (+1.1E+00)                  (+1.0E+00)            \n",
                            "                                                        |                                                            \n",
                            "prices               +1.8E+00    +0.0E+00    +0.0E+00   |  prices   +4.2E+00      +0.0E+00      +0.0E+00    +1.2E+01 \n",
                            "                    (+9.2E-01)                          |          (+4.6E+00)                              (+5.2E+00)\n",
                            "                                                        |                                                            \n",
                            "sugar                            -4.4E-03    +0.0E+00   |  sugar    -1.9E-01      +0.0E+00      +2.8E-02    +0.0E+00 \n",
                            "                                (+1.2E-02)              |          (+3.5E-02)                  (+3.2E-02)            \n",
                            "                                                        |                                                            \n",
                            "mushy                                        +8.6E-02   |  mushy    +1.5E+00      +0.0E+00      -1.5E+00    +0.0E+00 \n",
                            "                                            (+1.9E-01)  |          (+6.5E-01)                  (+1.1E+00)            \n",
                            "=====================================================================================================================\n",
                            "\n",
                            "Beta Estimates (Robust SEs in Parentheses):\n",
                            "==========\n",
                            "  prices  \n",
                            "----------\n",
                            " -3.2E+01 \n",
                            "(+2.3E+00)\n",
                            "=========="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "restricted_pi = initial_pi.copy()\n",
                "restricted_pi[1, 1] = 0\n",
                "nevo_problem.solve(\n",
                "    initial_sigma,\n",
                "    restricted_pi,\n",
                "    optimization=bfgs,\n",
                "    method='1s'\n",
                ")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Now we'll drop both `income_squared` and the corresponding column in $\\Pi_0$."
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": [
                            "Problem Results Summary:\n",
                            "========================================================================================================================\n",
                            "                                                                                                   Smallest    Largest  \n",
                            "Computation  GMM   Optimization   Objective   Fixed Point  Contraction  Objective    Gradient      Hessian     Hessian  \n",
                            "   Time      Step   Iterations   Evaluations  Iterations   Evaluations    Value    Infinity Norm  Eigenvalue  Eigenvalue\n",
                            "-----------  ----  ------------  -----------  -----------  -----------  ---------  -------------  ----------  ----------\n",
                            " 00:00:54     1         34           41          34472       106726     +1.5E+01     +5.2E-06      +4.7E-02    +1.7E+04 \n",
                            "========================================================================================================================\n",
                            "\n",
                            "Nonlinear Coefficient Estimates (Robust SEs in Parentheses):\n",
                            "=====================================================================================================\n",
                            "Sigma:      1         prices      sugar       mushy     |   Pi:      income       age        child   \n",
                            "------  ----------  ----------  ----------  ----------  |  ------  ----------  ----------  ----------\n",
                            "  1      +3.7E-01    +0.0E+00    +0.0E+00    +0.0E+00   |    1      +3.1E+00    +1.2E+00    +0.0E+00 \n",
                            "        (+1.2E-01)                                      |          (+1.1E+00)  (+1.0E+00)            \n",
                            "                                                        |                                            \n",
                            "prices               +1.8E+00    +0.0E+00    +0.0E+00   |  prices   +4.2E+00    +0.0E+00    +1.2E+01 \n",
                            "                    (+9.2E-01)                          |          (+4.6E+00)              (+5.2E+00)\n",
                            "                                                        |                                            \n",
                            "sugar                            -4.4E-03    +0.0E+00   |  sugar    -1.9E-01    +2.8E-02    +0.0E+00 \n",
                            "                                (+1.2E-02)              |          (+3.5E-02)  (+3.2E-02)            \n",
                            "                                                        |                                            \n",
                            "mushy                                        +8.6E-02   |  mushy    +1.5E+00    -1.5E+00    +0.0E+00 \n",
                            "                                            (+1.9E-01)  |          (+6.5E-01)  (+1.1E+00)            \n",
                            "=====================================================================================================\n",
                            "\n",
                            "Beta Estimates (Robust SEs in Parentheses):\n",
                            "==========\n",
                            "  prices  \n",
                            "----------\n",
                            " -3.2E+01 \n",
                            "(+2.3E+00)\n",
                            "=========="
                        ]
                    },
                    "execution_count": 1,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "restricted_formulation = pyblp.Formulation('0 + income  + age + child')\n",
                "deleted_pi = np.delete(initial_pi, 1, axis=1)\n",
                "restricted_problem = pyblp.Problem(\n",
                "    product_formulations,\n",
                "    product_data,\n",
                "    restricted_formulation,\n",
                "    agent_data\n",
                ")\n",
                "restricted_problem.solve(\n",
                "    initial_sigma,\n",
                "    deleted_pi,\n",
                "    optimization=bfgs,\n",
                "    method='1s'\n",
                ")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "The parameter estimates and standard errors are identical for both approaches. Based on the number of fixed point iterations, there is some evidence that the solver took a slightly different path for each problem, but both restricted problems arrived at identical answers. The $\\hat{\\Pi}_{12}$ interaction term is still insignificant."
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.6.6"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 2
}