A COMPANION TO Theoretical Econometrics

Experimental Design and Results

This section details the design and results of a Monte Carlo experiment that we conducted to shed light on the performance of the Gibbs sampling algorithm discussed in Section 2. We generated data according to equations (22.6)-(22.12) using the true parameter values that are listed in column two of Table 22.3. The table does not list the discount rate and the intercepts in the school and home payoff functions, which were set to 0.95, 11,000, and 17,000 respectively, since these are not identified. In all of our experiments we set the number of people, N, to 2,000.

Data from this model were generated using two different assumptions about the way people formed expectations. First, we assumed that people had rational expectations. This required us to solve the resulting dynamic optimization prob­lem once to generate the optimal decision rules. Since the choice set includes only four discrete alternatives it is feasible to do this. Then, to simulate choice and wage paths requires only that we generate realizations of the appropriate stochastic variables. It is important to note that the polynomial future component used in the estimation procedure does not provide a perfect fit to the rational expecta­tions future component. Hence, analysis of this data sheds light on the effect that misspecification of the future component may have on inference.

Next, we assumed that agents used a future component that was actually a polynomial in the state variables to form decisions. Analysis of this data set sheds light on how the algorithm performs when the model is correctly specified. To ensure close comparability with the rational expectations case, we constructed this polynomial by regressing the rational expectations future components on a fourth-order polynomial in the state variables, constructed as described in the discussion preceding (22.13). We used the point estimates from this regression as the coefficients of our polynomial future component (see Appendix A for the specific form of the polynomial).

We found that a fourth-order polynomial provided a good approximation to the future component in the sense that if agents used the approximate instead of optimal decision rule they suffered rather small lifetime earnings losses. Evid­ence of this is given in Table 22.1, where we report the results of simulations

under optimal and suboptimal decision rules. The simulations were conducted as follows. First, for N = 2,000 people we drew five sets of lifetime (T = 40) realizations of the model's stochastic components {ei1t, ei2t, (%)у=1,4}. In Table 22.1 these are referred to as error sets one to five. For each of the five error sets we simulated lifetime choice histories for each of the 2,000 people under the optimal and approximate decision rules. We refer to the 10 data sets constructed in this way as 1-EMAX through 5-EMAX and 1-POLY through 5-POLY, respectively. We then calculated the mean of the present value of lifetime payoffs (pecuniary plus nonpecuniary) for each of the 2,000 people under the optimal and approx­imate decision rules, respectively, for each of the five error sets. These are re­ported in the second and third rows of Table 22.1. Holding the error set fixed, the source of any difference in the mean present value of lifetime payoffs lies in the use of different decision rules. The mean present values of dollar equivalent losses from using the suboptimal polynomial rules are small, ranging from 287 to 491. The percentage loss ranges from 8 hundredths of 1 percent to 14 hundredths of 1 percent. These findings are similar to those reported by Geweke and Keane (1999a) and Krusell and Smith (1995).

Table 22.2 reports the mean accepted wages and choice frequencies for the data generated from error-set two. The first set of columns report statistics for data generated according to the polynomial approximation (data set 2-POLY) while the second set of columns report results from the optimal decision rule (data set 2-EMAX). Under our parameterization, occupation one can be thought of as "unskilled" labor, while occupation two can be understood as "skilled" labor. The reason is the mean of the wage offer distribution is lower in occupa­tion two early in life, but it rises more quickly with experience. The choice pat­terns and mean accepted wages are similar under the two decision rules. School is chosen somewhat more often under the optimal decision rule, which helps to generate slightly higher lifetime earnings. Finally, note that selection effects leave the mean accepted wage in occupation two higher than that in occupation one throughout the lifecycle under both decision rules.

Next, we ran the Gibbs algorithm described in section two for 40,000 cycles on each data set. We achieved about three cycles per minute on a Sun ultra-2 workstation.5 Thus, while time requirements were substantial, they were minor compared to what estimation of such a model using a full solution of the dynamic programming problem would entail. Visual inspection of graphs of the draw sequences, as well as application of the split sequence diagnostic suggested by Gelman (1996) - which compares variability of the draws across subsequences - suggests that the algorithm converged for all 10 artificial data sets. In all cases, the final 15,000 draws from each run were used to simulate the parameters' marginal posterior distributions.

Table 22.3 reports the results of the Gibbs sampling algorithm when applied to the data generated with a polynomial future component. In this case, the econo­metric model is correctly specified. The first column of Table 22.3 is the para­meter label, the second column is the true value, and the remaining columns report the structural parameters' posterior means and standard deviations for each of the five data sets.6 The results are extremely encouraging. Across all runs, there was only one instance in which the posterior mean of a parameter for the first wage equation was more than two posterior standard deviations away from its true value: the intercept in data set one. In data sets four and five, all of the structural parameters' posterior means are within two posterior standard devia­tions of their true values. In the second data set, only the second wage equation's own experience term is slightly more than two posterior standard deviations from its true value. In the third data set the mean of the wage equation's error correlation is slightly more than two posterior standard deviations from the true value, as are a few of the second wage equation's parameters.

Careful examination of Table 22.3 reveals that the standard deviation of the nonpecuniary payoff was the most difficult parameter to pin down. In particular, the first two moments of the marginal posteriors of these parameters vary con­siderably across experiments, in relation to the variability of the other structural parameters' marginal posteriors. This result reflects earlier findings reported by Geweke and Keane (1999a). In the earlier work they found that relatively large changes in the value of the nonpecuniary component's standard deviation had only a small effect on choices. It appears that this is the case in the current experiment as well.

It is interesting to note that an OLS regression of accepted (observed) log - wages on the log-wage equation's regressors yields point estimates that differ sharply from the results of the Gibbs sampling algorithm. Table 22.4 contains point estimates and standard errors from such an accepted wage regression. Selection bias is apparent in the estimates of the log-wage equation's parameters in all data sets. This highlights the fact that the Bayesian simulation algorithm is doing an impressive job of implementing the appropriate dynamic selection correction.

Perhaps more interesting is the performance of the algorithm when taken to data that were generated using optimal decision rules. Table 22.5 reports the results of this analysis on data sets 1-EMAX to 5-EMAX. Again, the first column labels the parameter, and the second contains its data generating value. The performance of the algorithm is quite impressive. In almost all cases, the poste­rior means of the wage function parameters deviate only slightly from the true values in percentage terms. Also, the posterior standard deviations are in most cases quite small, suggesting that the data contain a great deal of information about these structural parameters - even without imposing the assumption that agents form the future component "optimally." Finally, despite the fact that the posterior standard deviations are quite small, the posterior means are rarely more than two posterior deviations away from the true values.7 As with the polynomial data, the standard deviation of the nonpecuniary component seems difficult to pin down. Unlike the polynomial data, the school payoff parameters are not pinned down as well as the wage equation parameters. This is perhaps not surprising since school payoffs are never observed.

Figure 22.1 contains the simulated posterior densities for a subset of the struc­tural parameters based on data set 3-EMAX. Each figure includes three triangles on its horizontal axis. The middle triangle defines the posterior mean, and the two flanking triangles mark the points two posterior standard deviations above and below the mean. A vertical line is positioned at the parameters' data genera­ting (true) value. These distributions emphasize the quality of the algorithm's performance in that the true parameter values are typically close to the posterior means. The figures also make clear that not all the parameters have approxi­mately normal distributions. For instance, the posterior density of the wage equations' error correlation is multi-modal.

The results of Table 22.5 indicate that in a case where agents form the future component optimally, we can still obtain reliable and precise inferences about structural parameters of the current payoff functions using a simplified and misspecified model that says the future component is a simple fourth-order poly­nomial in the state variables. But we are also interested in how well our method approximates the decision rule used by the agents. In Table 22.6 we consider an experiment in which we use the posterior means for the parameters n that

Education
Intercept
600 400 200 0
400
True = 0.05
True = 9
200
0
9.002 9.014 9.026
Own experience
600 400 t 200 0
True = 0.055 Jr 500 C Cd Z3 a
0
.0524
.0538	.0553
Experience in occupation two True = 0.0 і
400
500 tz Ы Z3 cr
0
0
-.00175	.00331
.00078
Errors' correlation Figure 22.1 Marginal posterior densities of first log-wage equation’s parameters from data set 3-EMAX* * On each graph, the vertical line indicates the data generating parameter value. The middle triangle indicates the empirical mean, and the two flanking triangles are located two standard deviations from the mean

characterize how agents form expectations to form an estimate of agents' decision rules. We then simulate five new artificial data sets, using the exact same draws for the current period payoffs as were used to generate the original five artificial data sets. The only difference is that the estimated future component is substi­tuted for the true future component in forming the decision rule. The results in Table 22.6 indicate that the mean wealth losses from using the estimated decision rule range from five-hundredths to three-tenths of 1 percent. The percentage of choices that agree between agents who use the optimal versus the approximate rules ranges from 89.8 to 93.5 percent. These results suggest that our esti­mated polynomial approximations to the optimal decision rules are reason­ably accurate.

Figure 22.2 provides an alternative way to examine the quality of the polyno­mial approximation to the future component. This figure plots the value of the approximate and the true EMAX future components when evaluated at the mean

Table 22.6 Wealth loss when posterior polynomial approximation is used in place of true future component*

Data set Using Using

True EMAX** Posterior EMAX**

Occupation two's EMAX and polynomial future component

25.000

20.000

15.000

10.000

5,000

0

of each period's state vector.8 Each vertical axis corresponds to the value of the future component, and the horizontal axis is the period. Clearly, the approxima­tion reflects the true EMAX future component's main features. The fit of the polynomial seems relatively strong for each occupational alternative throughout the lifecycle. The fit is good for school in early periods, but begins to deteriorate later. One reason is that school is chosen very infrequently after the first five periods, so there is increasingly less information about its future component. A second reason is that the contemporaneous return begins to dominate the future component in alternative valuations. Consequently, each data point in later periods contains relatively less information about the future component's value.

Overall, however, these figures furnish additional evidence that the polynomial approximation does a reasonable job of capturing the key characteristics of the true future component.

2 Conclusion

This chapter described how to implement a simulation based method for infer­ence that is applicable to a wide class of dynamic multinomial choice models. The results of a Monte Carlo analysis demonstrated that the method works very well in relatively large state-space models with only partially observed payoffs where very high dimensional integrations are required. Although our discussion focused on models with discrete choices and independent and identically dis­tributed stochastic terms, the method can also be applied to models with mixed continuous/discrete choice sets and serially correlated shocks (see Houser, 1999).