A COMPANION TO Theoretical Econometrics

Empirical examples

We now turn to applications of some of these forecasting methods to the five US macroeconomic time series in Figures 27.1-27.5.2 In the previous notation, the series to be forecast, yt, is the series plotted in those figures, for example, for industrial production yt = 200 ln (IPt/IPt-6), while for the interest rate yt is the untransformed interest rate in levels (at an annual rate). The exercise reported here is a simulated out of sample comparison of six different forecasting models. All series are observed monthly with no missing observations. For each series, the initial observation date is 1959:1. Six-month ahead (h = 6) recursive forecasts of yt+6 are computed for t = 1971:3,..., 1996:6; because a simulated out of sample methodology was used, all models were re-estimated at each such date t.

Eight different forecasts are computed: (a) EWMA, where the parameter is estimated by NLS; (b) AR(4) with a constant; (c) AR(4) with a constant and a time trend; (d) AR where the lag length is chosen by BIC (0 < p < 12) and the decision to difference or not is made using the Elliott-Rothenberg-Stock (1996) unit root pretest; (e) NN with a single hidden layer and two hidden units; (f) NN with two hidden layers, two hidden units in the first layer, and one hidden unit in the second layer; (g) LSTAR in levels with three lags and Z t = yt - yt-d and (h) LSTAR in differences with three lags and Z t = yt - yt-6.

For each series, the simulated out of sample forecasts (b) and (e) are plotted in Figures 27.1-27.5. The root MSFEs for the different methods, relative to method (b), are presented in Table 27.1; thus method (b) has a relative root MSFE of 1.00 for all series. The final row of Table 27.1 presents the root mean squared forecast error in the native units of the series.

Several findings are evident. First, among the linear models, the AR(4) in levels with a constant performs well. This model dominates the AR(4) in levels with a

Table 27.1 Comparison of simulated out-of-sample linear and nonlinear forecasts for five US macroeconomic time series

Forecasting model Relative root mean squared

forecast errors

Sample period: monthly, 1959:1-1996:12; forecast period: 1971:3-1996:6; forecast horizon: six months.

Entries in the upper row are the root mean squared forecast error of the forecasting model in the indicated row, relative to that of model (b), so the relative root MSFE of model (b) is 1.00. Smaller relative root MSFEs indicate more accurate forecasts in this simulated out-of-sample experiment. The entries in the final row are the root mean squared forecast errors of model (b) in the native units of the series. The series are: the unemployment rate, the six-month rate of CPI inflation, the 90-day US Treasury bill annualized rate of interest, the six-month growth of IP, and the six-month growth of real manufacturing and trade inventories.

Sample period: monthly, 1959:1-1996:12; forecast period: 1971:3-1996:6; forecast horizon: six months.

Entries are relative root MSFEs, relative to the root MSFE of model (b) in Table 27.1 (AR(4) with constant in levels). The VAR specifications have lag lengths selected by BIC; the six-month ahead forecasts were computed by iterating one-month ahead forecasts. All forecasts are simulated out of sample. See the notes to Table 27.1.

constant and time trend, in the sense that for all series the AR(4) with a constant and time trend has an RMSFE that is no less than the AR(4) with a constant. Evidently, fitting a linear time trend leads to poor out-of-sample performance, a result that would be expected if the trend is stochastic rather than deterministic. Using BIC lag length selection and a unit root pretest improves upon the AR(4) with a constant for inflation, has essentially the same performance for industrial production and inventories, and exhibits worse performance for the unemploy­ment rate and the interest rate; averaged across series, the RMSFE is 0.99, indicat­ing a slight edge over the AR(4) with a constant on average.

None of the nonlinear models uniformly improve upon the AR(4) with a con­stant. In fact, two of the nonlinear models ((e) and (g)) are dominated by the AR(4) with a constant. The greatest improvement is by model (h) for inflation; however, this relative RMSFE is still greater than the AR(BIC) forecast for infla­tion. Interestingly, the very simple EWMA forecast is the best of all forecasts, linear and nonlinear, for the interest rate. For the other series, however, it does not get the correct long-run trend, and the EWMA forecasts are worse than the AR(4).

The final row gives a sense of the performance of these forecasts in absolute terms. The RMSFE of the unemployment rate, six months hence, is only 0.6 percentage points, and the RMSFE for the 90-day Treasury bill rate is 1.7 percent­age points. CPI inflation is harder to predict, with a six-month ahead RMSFE of 2.4 percentage points. Inspection of the graph of IP growth reveals that this series is highly volatile, and in absolute terms the forecast error is large, with a six-month ahead RMSFE of 6.2 percentage points.

Some of these points can be verified by inspection of the forecasts plotted in Figures 27.1-27.5. Clearly these forecasts track well the low frequency move­ments in the unemployment rate, the interest rate, and inflation (although the NN forecast does quite poorly in the 1990s for inflation). Industrial production and inventory growth has a larger high frequency component, which all these models have difficulty predicting (some of this high frequency component is just unpredictable forecast error).

These findings are consistent with the conclusions of the larger forecasting model comparison study in Stock and Watson (1999a). They found that, on aver­age across 215 macroeconomic time series, autoregressive models with BIC lag length determination and a unit root pretest performed well, indeed, outper­formed a range of NN and LSTAR models for six-month ahead forecasts. The autoregressive model typically improved significantly on no-change or EWMA models. Thus there is considerable ability to predict many U. S. macroeconomic time series, but much of this predictability is captured by relatively simple linear models with data-dependent determination of the specification.