A COMPANION TO Theoretical Econometrics

Resampling the likelihood ratio statistic: bootstrap methods

The bootstrap is a data based simulation method for statistical inference. The bootstrap approach involves approximating the distribution of a function of the observed data by the bootstrap distribution of the quantity. This is done by substituting the empirical distribution function for the unknown distribution and repeating this process many times to obtain a simulated distribution. Its recent development follows from the requirement of a significant amount of computa­tional power. Obviously there is no advantage to utilizing bootstrap procedures when the exact sampling distribution of the test statistic is known. However, it has been demonstrated that when the sampling distribution is not known, the substitution of computational intensive bootstrap resampling can offer an improvement over asymptotic theory. The use of non-pivotal bootstrap testing procedures does not require the mean adjustment facilitated by (13.43) and (13.44). However, pivotal (or bootstrap-t) procedures require both mean and variance adjustments in order to guarantee asymptotic pivotalness.

Utilizing a parametric bootstrap we present below a simple algorithm for resampling the likelihood ratio statistic which we then use to construct the em­pirical distribution function of the test statistic. For the purpose of exposition the algorithm is presented for the non-pivotal bootstrap.

1. Generate R samples of size T by sampling from the fitted null model ft(0T).

2.

For each rth simulated sample, the pair (0T, yi (0T)) represent the parameter estimates obtained by maximizing the loglikelihoods

where yr(0T) denotes the rth bootstrap-sample conditional upon 0 = 0T. We then compute the simulated loglikelihood ratio statistic

Trf = Lf (0T) - Lg(Y * (0t)).

3. By constructing the empirical cdf of {Tf : 1 < r < R}, we can compare the ob­served test statistic, Tf = Lf(0T) - Lg(y*(0T)), with critical values obtained from the R independent (conditional) realizations of Tj. The p-value based upon the bootstrap procedure is given by24

X 1(Tfr > Tf)

Pr = , (13.46)

where 1(.) is the indicator function.

The bootstrap procedure outlined above simply resamples the likelihood ratio statistic without pivoting. There are a number of alternative test statistics which by using pivotal methods are conjectured to represent an improvement over classical first order methods (see for example, Beran (1988) and Hall (1988)). An evaluation of both the size and power properties of a number of simulation and bootstrap-based tests applied to linear versus loglinear regression models and a number of variants of threshold autoregressive models is provided in Kapetanios and Weeks (1999).

Notes

1 Therefore our focus is distinct from Chow (1981) who, in examining a similar prob­lem, assumes that the set of models under consideration contains a general model from which all other competing models may be obtained by the imposition of suitable parameter restrictions.

2 See, for example, Friedman and Meiselman (1963) on alternative consumption models, Barro (1977), Pesaran (1982b) and McAleer, Pesaran, and Bera (1990) on alternative explanations of the unemployment rate; Jorgenson and Siebert (1968), Dixit and Pindyck (1994) and Bernanke, Bohn, and Reiss (1988) on alternative models of investment behavior; and McAleer, Fisher, and Volker (1982) and Smith and Smyth (1991) on nonnested money demand functions.

3 An excellent survey article on nonnested hypothesis testing can be found in Gourieroux and Monfort (1994).

4 In cases where one or more elements of zt are discrete, as in probit or tobit specifica­tions cumulative probabality distribution functions can be used instead of probability density functions.

5 See Engle, Hendry, and Richard (1983).

6 A formalization of the concept of globally nonnested models can be found in Pesaran (1987). Also see Section 6.

7 Note that under Щ, E(yt) = E{exp(ln yt)} = exp(01 + 0.502).

8 See, Pesaran (1984, pp. 249-50).

9 There is substantial literature on nonnested tests of linear versus loglinear regres­sion models. Earlier studies include Aneuryn-Evans and Deaton (1980), Godfrey and Wickens (1981) and Davidson and MacKinnon (1985). In a more recent study Pesaran and Pesaran (1995) have examined the properties of a simulation-based variant of the Cox test.

10 A review of the model selection literature is beyond the scope of the present paper. See, for example, Leamer (1983) for an excellent review. A recent review focusing upon the selection of regressors problems is to be found in Lavergne (1998). Two excellent texts are Grasa (1989) and Linhart and Zucchini (1986). Maddala (1981) edited a special issue of the Journal of Econometrics which focuses on model selection.

11 For the case of the classical linear regression model examples of model selection criteria include Theil's )2, with more general loss functions based upon information criteria including Akaike's (1973) information criteria and Schwarz's (1978) Bayesian information criterion.

12 The concepts of globally and partially nonnested models are defined in Pesaran (1987).

13 See also Dastoor (1981) for further discussion.

14 The cases where ~Lf Ф 0 (respectively Ф 0) but nevertheless £f a0 = 0 (respectively ZgP0 = 0) are discussed in Pesaran (1987, p. 74).

15 See Pesaran (1974) for details of the derivations.

16 For example, it is not possible to test whether X = 1/2, which could have been of interest in assessing the relative weights attached to the two rival models.

17 For a proof see McAleer and Pesaran (1986).

18 Chao and Swanson (1997) provide some asymptotic results for the /-test in the case of nonnested models with 1(1) regressors.

19 Recall that the encompassing condition (13.37) for the regression coefficients implies the condition (13.38) for error variance encompassing but not vice versa.

20 Notice that the normality assumption is not needed and can be relaxed.

21 Similarly, the variance encompassing statistic for testing Hf EHg is given by roT - [6T + T-1a'T X MZX«T ].

22 See also McAleer (1983).

23 See McAleer and Pesaran (1986) for additional details.

24 If T is discrete then repeat values of T can occur requiring that we make an adjust­ment to (13.46).

References

Akaike, H. (1973). Information theory and an extension of the maximum likelihood prin­ciple. In N. Petrov and F. Csadki (eds.) Proceedings of the 2nd International Symposium on Information Theory. pp. 267-81. Budapest: Akademiai Kiado.

Altissimo, F., and G. L. Violante (1998). The nonlinear dynamics of output and unemploy­ment in the US. /ournal of Applied Econometrics (forthcoming).

Amemiya, T. (1985). Advanced Econometrics. Cambridge, MA: Harvard University Press.

Aneuryn-Evans, G., and A. S. Deaton (1980). Testing linear versus logarithmic regression models. Review of Economic Studies 47, 275-91.

Atkinson, A. (1970). A method for discriminating between models (with discussion). /our­nal of the Royal Statistical Society, B B32, 323-53.

Bahadur, R. R. (1960). Stochastic comparison of tests. Annals of Mathematics and Statistics. 31, 276 -95.

Bahadur, R. R. (1967). Rates of convergence of estimates and test statistics. Annals of Math­ematics and Statistics 38, 303-24.

Barro, R. (1977). Unanticipated money growth and unemployment in the United States. American Economic Review 67, 101-15.

Beran, R. (1988). Prepivoting test statistics: A bootstrap view of asymptotic refinements. /ournal of the American Statistical Association 83, 403.

Bernanke, B., H. Bohn, and P. Reiss (1988). Alternative nonnested specification tests of time-series investment models. /ournal of Econometrics 37, 293-326.

Chao, J. C., and N. R. Swanson (1997). Tests of nonnested hypotheses in nonstationary regressions with an application to modeling industrial production. Working Paper, Department of Economics, Pennsylvania State University.

Chow, G. C. (1981). A comparison of the information and posterior probability criteria for model selection. /ournal of Econometrics 16, 21-33.

Coulibaly, N., and B. Brorsen (1998). A Monte Carlo sampling approach to testing nonnested hypotheses: Monte Carlo results. Econometric Reviews 195-209.

Cox, D. (1961). Tests of separate families of hypothesis. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability.

Cox, D. (1962). Further results on tests of separate families of hypotheses. /ournal of Royal Statistical Society B24, 406-24.

Dastoor, N. (1981). A note on the interpretation of the Cox procedure for nonnested hypotheses. Economics Letters 8, 113-19.

Dastoor, N. K. (1983). Some aspects of testing nonnested hypotheses. Journal of Econometrics 21, 213-28.

Davidson, R., and J. MacKinnon (1981). Several tests for model specification in the pres­ence of alternative hypotheses. Econometrica 49, 781-93.

Davidson, R., and J. G. MacKinnon (1982). Some nonnested hypothesis tests and the rela­tions among them. Review of Economic Studies 49, 551-65.

Davidson, R., and J. G. MacKinnon (1985). Testing linear and loglinear regressions against Box-Cox alternatives. Canadian Journal of Economics 18, 499-517.

Davies, R. B. (1977). Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika 64(2), 247-54.

Deaton, A. S. (1982). Model selection procedures, or, does the consumption function exist? In G. C. Show, and P. Corsi (eds.) Evaluating the Reliability of Macroeconomic Models, pp. 43-65. New York: Wiley.

Dixit, A. V., and R. S. Pindyck (1994). Investment Under Uncertainty. Chichester, UK: Princeton University Press.

Duncan, A., and M. Weeks (1998). Nonnested models of labour supply with discrete choices. Working Paper, Department of Economics, University of York.

Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics 7, 1-26.

Engle, R., D. Hendry, and J. Richard (1983). Exogeneity. Econometrica 51, 277-304.

Ericsson, N. (1983). Asymptotic properties of instrumental variables statistics for testing nonnested hypotheses. Review of Economic Studies 50, 287-304.

Fisher, G., and M. McAleer (1979). On the interpretation of the Cox test in econometrics. Economic Letters 4, 145-50.

Fisher, G. R., and M. McAleer (1981). Alternative procedures and associated tests of sig­nificance for nonnested hypotheses. Journal of Econometrics 16, 103-19.

Friedman, M., and D. Meiselman (1963). The relative stability of monetary velocity and the investment multiplier in the United States 1897-1958. In Stabilization Policies. Englewood Cliffs, NJ: Commission on Money and Credit Research Study.

Ghysels, E., and A. Hall (1990). Testing nonnested Euler conditions with quadrature based method of approximation. Journal of Econometrics 46, 273-308.

Godfrey, L. G. (1983). Testing nonnested models after estimation by instrumental variables or least squares. Econometrica 51(2), 355-65.

Godfrey, L. G., and M. H. Pesaran (1983). Tests of nonnested regression models: Small sample adjustments and Monte Carlo evidence. Journal of Econometrics 21, 133-54.

Godfrey, L. G., and M. Wickens (1981). Testing linear and loglinear regressions for func­tional form. Review of Economic Studies 48, 487-96.

Gourieroux, C. (1982). Asymptotic comparison of tests for nonnested hypotheses by Bahadur's A. R.E. Discussion Paper 8215, CEPREMAP, Paris.

Gourieroux, C., and A. Monfort (1994). Testing nonnested hypotheses. In R. F. Engle, and D. L. McFadden (eds.) Handbook of Econometrics, Volume 4. Oxford: Elsevier.

Gourieroux, C., and A. Monfort (1995). Testing, encompassing, and simulating dynamic econometric models. Econometric Theory 11, 195-228.

Gourieroux, C., A. Monfort, and A. Trognon (1983). Testing nested or nonnested hypoth­eses. Journal of Econometrics 21, 83-115.

Granger, C. W.J., M. L. King, and H. White (1995). Comments on testing economic theories and the use of model selection criteria. Journal of Econometrics 67, 173-87.

Granger, C. W.J., and M. H. Pesaran (2000). A decision theoretic approach to forecast evalu­ation. In W. S. Chan, W. K. Li, and H. Tong (eds.), Statistics and Finance: An Interface. London: Imperial College Press.

Grasa, A. A. (1989). Econometric Model Selection: A New Approach. Spain: Kluwer Academic Publishers.

Hall, P. (1986). On the number of bootstrap simulations required to construct a confidence interval. The Annals of Statistics 14(4).

Hall, P. (1988). Theoretical comparison of bootstrap confidence intervals. Annals of Statis­tics 16, 927-53.

Hendry, D. F. (1993). Econometrics: Alchemy or Science? Oxford: Blackwell Publishers.

Hinkely, D. (1988). Bootstrap methods. Journal of the Royal Statistical Society, Series B 50, 321-37.

Jeong, J., and G. Maddala (1993). A perspective on application of bootstrap methods in econometrics. Handbook of Statistics 11, 573-605.

Jorgenson, D. W., and C. D. Siebert (1968). A comparison of alternative theories of corpor­ate investment behavior. American Economic Review 58, 681-712.

Kapetanios, G., and M. Weeks (1999). Nonnested models and the likelihood ratio statistic: A comparison of simulation and bootstrap-based tests. Department of Applied Eco­nomics Working Paper, University of Cambridge.

Kent, J. (1986). The underlying structure of nonnested hypothesis tests. Biometrika 7, 333-43.

King, M. L. (1983). Testing for autoregressive against moving average errors in the linear regression model. Journal of Econometrics 21, 35-51.

Kullback, S. (1959). Information Theory and Statistics. New York: Wiley.

Lavergne, P. (1998). Selection of regressors in econometrics: Parametric and nonparametric methods. Econometric Reviews 17(3), 227-73.

Leamer, E. E. (1983). Model choice and specification analysis. In Z. Griliches, and M. D. Intriligator (eds.) Handbook of Econometrics Volume 1. University of California, LA: North - Holland Publishing Company.

Li, H., and G. Maddala (1996). Bootstrapping time series models. Econometric Reviews 15(2), 115-58.

Linhart, H., and W. Zucchini (1986). Model Selection. New York: Wiley and Sons.

Maddala, G. S.E. (1981). Model selection. Special Issue of Journal of Econometrics 16(1).

McAleer, M. (1983). Exact tests of a model against nonnested alternative. Biometrika 70, 285-88.

McAleer, M. (1995). The significance of testing empirical nonnested models. Journal of Econometrics 67, 149-71.

McAleer, M., and S. Ling (1998). A nonnested test for the GARCH and E-GARCH models. Working Paper, Department of Economics, University of Western Australia.

McAleer, M., and M. H. Pesaran (1986). Statistical inference in nonnested econometric models. Applied Mathematics and Computation 20, 271-311.

McAleer, M. G., G. Fisher, and P. Volker (1982). Separate misspecified regressions and U. S. long run demand for money function. Review of Economics and Statistics 64, 572-83.

McAleer, M. J., M. H. Pesaran, and A. K. Bera (1990). Alternative approaches to testing nonnested models with autocorrelated disturbances: An application to models of U. S. unemployment. Communications in Statistics series A(19), 3619-44.

Mizon, G. E., and J. F. Richard (1986). The encompassing principle and its application to nonnested hypotheses. Econometrica 54, 657-78.

Orme, C. (1994). Nonnested tests for discrete choice models. Working Paper, Department of Economics, University of York.

Pereira, B. D.B. (1984). On the choice of a Weibull model. Journal of the Inter American Statistical Institute 26, 157-63.

Pesaran, M. H. (1974). On the general problem of model selection. Review of Economic Studies 41, 153-71.

Pesaran, M. H. (1981). Pitfalls of testing nonnested hypotheses by the Lagrange multiplier method. Journal of Econometrics 17, 323-31.

Pesaran, M. H. (1982a). Comparison of local power of altenative tests of nonnested regres­sion models. Econometrica.

Pesaran, M. H. (1982b). A critique of the proposed tests of the natural rate-rational expec­tations hypothesis. The Economic Journal 92, 529-54.

Pesaran, M. H. (1984). Asymptotic power comparisons of tests of separate parametric fam­ilies by Bahadur's approach. Biometrika 71(2), 245-52.

Pesaran, M. H. (1987). Global and partial nonnested hypotheses and asymptotic local power. Econometric Theory 3, 69-97.

Pesaran, M. H., and S. Deaton (1978). Testing nonnested nonlinear regression models. Econometrica 46, 677-94.

Pesaran, M. H., and B. Pesaran (1993). A simulation approach to the problem of computing Cox's statistic for testing nonnested models. Journal of Econometrics 57, 377-92.

Pesaran, M. H., and B. Pesaran (1995). A nonnested test of level differences versus log- differenced stationary models. Econometric Reviews 14(2), 213-27.

Pesaran, M. H., and S. Potter (1997). A floor and ceiling model of US output. Journal of Economic Dynamics and Control 21(4-5), 661-96.

Sawyer, K. R. (1983). Testing separate families of hypotheses: An information criterion. Journal of the Royal Statistical Society B 45, 89-99.

Sawyer, K. R. (1984). Multiple hypothesis testing. Royal Statistical Society B 46(3), 419-24.

Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics 6, 461-4.

Smith, M. A., and G. S. Maddala (1983). Multiple model testing for nonnested heteroskedastic censored regression models. Journal of Econometrics 21, 71-81.

Smith, M. D., and D. J. Smyth (1991). Multiple and pairwise nonnested tests of the influence of taxes on money demand. Journal of Applied Econometrics 6, 17-30.

Smith, R. J. (1992). Nonnested tests for competing models estimated by generalized method of moments. Econometrica 60, 973-80.

Smith, R. J. (1993). Consistent tests for the encompassing hypothesis. Document de Travail No. 9403, INSEE, Paris.

Vinod, H. (1993). Bootstrap methods: Applications in econometrics. Handbook of Statistics 11, 629-61.

Vuong, Q. H. (1989). Likelihood ratio tests for model selection and nonnested hypothesis. Econometrica 57(2), 307-33.

Walker, A. M. (1967). Some tests of separate families of hypotheses in time series analysis. Biometrika 54, 39-68.

Weeks, M. (1996). Testing the binomial and multinomial choice models using Cox's nonnested test. Journal of the American Statistical Association (Papers and Proceedings).

White, H. (1982). Regularity conditions for Cox's test of nonnested hypothesis. Journal of Econometrics 19, 301-18.

Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics. New York: John Wiley and Sons.