A COMPANION TO Theoretical Econometrics

Model Selection

A difficult question when modeling economic behavior is to decide on what lags should be in the ARIMA model, the ARMA disturbance model, or the dynamic regression model. It is tempting to use hypothesis testing to help make model specification decisions based on the data, but as discussed by Granger, King, and White (1995), there are disadvantages in doing so. They and others recommend the use of a model selection procedure to make these decisions, the most com­mon of which are information criteria (IC). For each of the models under consid­eration, one calculates the maximized loglikelihood and then penalizes this value to take account of the number of free parameters in the model which we will denote by j. Akaike's (1973) IC (AIC) uses j as the penalty whereas Schwarz's (1978) Bayesian IC (BIC) uses j log(n)/2. There are a range of other IC procedures, but these two have become the most popular.

These days, BIC seems to be the favored procedure because it is consistent, which means that as the sample size goes to infinity, the probability that it will choose the correct model from a finite number of models goes to one. An unfortunate consequence of this property is that in small samples, BIC tends to wrongly choose underfitting models and can have very low probabilities of correctly selecting a model which has a large number of free parameters. AIC seems more balanced in this regard in small samples, but can suffer from a tendency to overfit in larger samples.

References

Akaike, H. (1973). Information theory and an extension of the maximum likelihood prin­ciple. In B. N. Petro and F. Csaki (eds.) 2nd International Symposium on Information Theory. Budapest: Akademiai Kiado, 267-81.

Ansley, C. F. (1979). An algorithm for the exact likelihood of a mixed autoregressive - moving average process. Biometrika 66, 59-65.

Ara, I. (1995). Marginal likelihood based tests of regression disturbances. Unpublished Ph. D. thesis, Monash University.

Baillie, R. T. (1996). Long memory processes and fractional integration in econometrics. Journal of Econometrics 73, 5-59.

Bartels, R. (1992). On the power function of the Durbin-Watson test. Journal of Econometrics 51, 101-12.

Beach, C. M., and J. MacKinnon (1978a). Full maximum likelihood procedure for regres­sion with autocorrelated errors. Econometrica 46, 51-8.

Beach, C. M., and J. MacKinnon (1978b). Full maximum likelihood estimation of second - order autoregressive error models. Journal of Econometrics 7, 187-98.

Box, G. E.P., and G. M. Jenkins (1970). Time Series Analysis, Forecasting and Control. San Francisco: Holden-Day.

Box, G. E.P., and G. M. Jenkins (1976). Time Series Analysis, Forecasting and Control, 2nd edn. San Francisco: Holden-Day.

Cochrane, D., and G. Orcutt (1949). Application of least squares regression to relationships containing autocorrelated error terms. Journal of the American Statistical Association 44, 32-61.

Cooper, D. M., and R. Thompson (1977). A note on the estimation of parameters of the autoregressive-moving average process. Biometrika 64, 625-8.

Corduas, M. (1986). The use of the marginal likelihood in testing for serial correlation in time series regression. Unpublished M. Phil. thesis, University of Lancaster.

Davidson, R., and J. G. MacKinnon (1993). Estimation and Inference in Econometrics. Oxford: Oxford University Press.

Durbin, J. (1970). Testing for serial correlation in least squares regression when some of the regressors are lagged dependent variables. Econometrica 38, 410-21.

Durbin, J., and G. S. Watson (1950). Testing for serial correlation in least squares regression

I. Biometrika 37, 409-28.

Durbin, J., and G. S. Watson (1951). Testing for serial correlation in least squares regression

II. Biometrika 38, 159-78.

Durbin, J., and G. S. Watson (1971). Testing for serial correlation in least squares regression

III. Biometrika 58, 1-19.

Farebrother, R. W. (1980). The Durbin-Watson test for serial correlation when there is no intercept in the regression. Econometrica 48, 1553-63 and 49, 227.

Godfrey, L. G. (1988). Misspecification Tests in Econometrics: The Lagrange Multiplier Principle and Other Approaches. Cambridge: Cambridge University Press.

Godfrey, L. G., and A. R. Tremayne (1988). Checks of model adequacy for univariate time series models and their application to econometric relationships. Econometric Reviews 7, 1-42.

Granger, C. W.J., M. L. King, and H. White (1995). Comments on testing economic theories and the use of model selection criteria. Journal of Econometrics 67, 173-87.

Hart, B. I. (1942a). Tabulation of the probabilities for the ratio of the mean square succes­sive difference to the variance. Annals of Mathematical Statistics 13, 207-14.

Hart, B. I. (1942b). Significance levels for the ratio of the mean square successive difference to the variance. Annals of Mathematical Statistics 13, 445-7.

Imhof, P. J. (1961). Computing the distribution of quadratic forms in normal variables. Biometrika 48, 419-26.

Inder, B. A. (1985). Testing for first-order autoregressive disturbances in the dynamic lin­ear regression model. Unpublished Ph. D. thesis, Monash University.

Inder, B. A. (1986). An approximation to the null distribution of the Durbin-Watson stat­istic in models containing lagged dependent variables. Econometric Theory 2, 413-28.

Johnston, J. (1972). Econometric Methods, 2nd edn. New York: McGraw-Hill.

Kalbfleisch, J. D., and D. A. Sprott (1970). Application of likelihood methods to models involving large numbers of parameters. Journal of the Royal Statistical Society B 32, 175-94.

King, M. L. (1981). The Durbin-Watson test for serial correlation: Bounds for regressions with trend and/or seasonal dummy variables. Econometrica 49, 1571-81.

King, M. L. (1985). A point optimal test for autoregressive disturbances. Journal of Eco­nometrics 27, 21-37.

King, M. L. (1987). Testing for autocorrelation in linear regression models: A survey. In M. L. King and D. E.A. Giles (eds.) Specification Analysis in the Linear Model, London: Routledge and Kegan Paul, 19-73.

King, M. L., and M. A. Evans (1988). Locally optimal properties of the Durbin-Watson test.

Econometric Theory 4, 509-16.

King, M. L., and M. McAleer (1987). Further results on testing AR(1) against MA(1) dis­turbances in the linear regression model. Review of Economic Studies 54, 649-63.

King, M. L., and P. X. Wu (1991). Small-disturbance asymptotics and the Durbin-Watson and related tests in the dynamic regression model. Journal of Econometrics 47, 145-52.

Laskar, M. R., and M. L. King (1998). Estimation and testing of regression disturbances based on modified likelihood functions. Journal of Statistical Planning and Inference 71, 75-92.

Levenbach, H. (1972). Estimation of autoregressive parameters from a marginal likelihood function. Biometrika 59, 61-71.

Nichols, D. F., A. R. Pagan, and R. D. Terrell (1975). The estimation and use of models with moving average disturbance terms: A survey. International Economic Review 16, 113-34.

Prais, S. J., and C. B. Winsten (1954). Trend estimators and serial correlation. Unpublished Cowles Commission Discussion Paper, University of Chicago.

Rahman, S., and M. L. King (1993). Testing for ARMA(1, 1) disturbances in the linear regression model. Australian Economic Papers 32, 284-98.

Rahman, S., and M. L. King (1998). Marginal-likelihood score-based tests of regression disturbances in the presence of nuisance parameters. Journal of Econometrics 82, 81-106.

Savin, N. E., and K. J. White (1977). The Durbin-Watson test for serial correlation with extreme sample sizes or many regressors. Econometrica 45, 1989-96.

Schwarz, G. W. (1978). Estimating the dimension of a model. Annals of Statistics 6, 461-4.

Silvapulle, P., and M. L. King (1991). Testing moving average against autoregressive dis­turbances in the linear regression model. Journal of Business and Economic Statistics 9, 329-35.

Thomas, J. J., and K. F. Wallis (1971). Seasonal variation in regression analysis. Journal of the Royal Statistical Society A 134, 57-72.

Tillman, J. A. (1975). The power of the Durbin-Watson test. Econometrica 43, 959-74.

Tse, Y. K. (1982). Edgeworth approximations in first-order stochastic difference equations with exogenous variables. Journal of Econometrics 20, 175-95.

Tunnicliffe Wilson, G. (1989). On the use of marginal likelihood in time series model estimation. Journal of the Royal Statistical Society B 51, 15-27.

Van der Leeuw, J. (1994). The covariance matrix of ARMA errors in closed form. Journal of Econometrics 63, 397-405.

Vinod, H. D. (1973). Generalization of the Durbin-Watson statistic for higher order auto­regressive processes. Communications in Statistics 2, 115-44.

Von Neumann, J. (1941). Distribution of the ratio of the mean square successive difference to the variance. Annals of Mathematical Statistics 12, 367-95.

Von Neumann, J. (1942). A further remark concerning the distribution of the ratio of the mean square successive difference to the variance. Annals of Mathematical Statistics 13, 86-8.

Wallis, K. F. (1972). Testing for fourth order autocorrelation in quarterly regression equa­tions. Econometrica 40, 617-36.