A COMPANION TO Theoretical Econometrics

Monte Carlo Test. Methods in. Econometrics

Jean-Marie Dufour and Lynda Khalaf [14]

1 Introduction

During the last 20 years, computer-based simulation methods have revolutionized the way we approach statistical analysis. This has been made possible by the rapid development of increasingly quick and inexpensive computers. Important innovations in this field include the bootstrap methods for improving standard asymptotic approximations (for reviews, see Efron, 1982; Efron and Tibshirani, 1993; Hall, 1992; Jeong and Maddala, 1993; Vinod, 1993; Shao and Tu, 1995; Davison and Hinkley, 1997; Horowitz, 1997) and techniques where estimators and forecasts are obtained from criteria evaluated by simulation (see Mariano and Brown, 1993; Hajivassiliou, 1993; Keane, 1993; Gourieroux, Monfort, and Renault, 1993; Gallant and Tauchen, 1996). An area of statistical analysis where such techniques can make an important difference is hypothesis testing which often raises difficult distributional problems, especially in view of determining appropriate critical values.

This paper has two major objectives. First, we review some basic notions on hypothesis testing from a finite-sample perspective, emphasizing in particular the specific role of hypothesis testing in statistical analysis, the distinction be­tween the level and the size of a test, the notions of exact and conservative tests, as well as randomized and non-randomized procedures. Second, we present a relatively informal overview of the possibilities of Monte Carlo test techniques, whose original idea originates in the early work of Dwass (1957), Barnard (1963) and Birnbaum (1974), in econometrics. This technique has the great attraction of providing provably exact (randomized) tests based on any statistic whose finite

sample distribution may be intractable but can be simulated. Further, the valid­ity of the tests so obtained does not depend on the number of replications employed (which can be small). These features may be contrasted with the boot­strap, which only provides asymptotically justified (although hopefully improved) large-sample approximations.

In our presentation, we will try to address the fundamental issues that will allow the practitioners to use Monte Carlo test techniques. The emphasis will be on concepts rather than technical detail, and the exposition aims at being intui­tive. The ideas will be illustrated using practical econometric problems. Examples discussed include: specification tests in linear regressions contexts (normality, independence, heteroskedasticity and conditional heteroskedasticity), nonlinear hypotheses in univariate and SURE models, and tests on structural parameters in instrumental regressions. More precisely, we will discuss the following themes.

In Section 2, we identify the important statistical issues motivating this eco­nometric methodology, as an alternative to standard procedures. The issues raised have their roots in practical test problems and include:

• an exact test strategy: what is it, and why should we care?;

• the nuisance-parameter problem: what does it mean to practitioners?;

• understanding the size/level control problem;

• pivotal and boundedly-pivotal test criteria: why is this property important?;

• identification and near non-identification: a challenging setting.

Further, the relevance and severity of the problem will be demonstrated using simulation studies and/or empirical examples.

Sections 3 and 4 describe the Monte Carlo (MC) test method along with various econometric applications of it. Among other things, the procedure is compared and contrasted with the bootstrap. Whereas bootstrap tests are asymptotically valid (as both the numbers of observations and simulated samples go to ^), a formal demonstration is provided to emphasize the size control property of MC tests. Monte Carlo tests are typically discussed in parametric contexts. Extensions to nonparametric problems are also discussed. The theory is applied to a broad spectrum of examples that illustrate the usefulness of the procedure. We con­clude with a word of caution on inference problems that cannot be solved by simulation. For convenience, the concepts and themes covered may be outlined as follows. •

• MC tests: breakthrough improvements and "success stories":

(a) The intractable null distributions problem (e. g. tests for normality, uni­form linear hypothesis in multi-equation models, tests for ARCH),

(b) MC tests or Bartlett corrections?,

(c) The case of unidentified nuisance parameters (test for structural jumps, test for ARCH-M);

• MC tests may fail: where and why? a word of caution.

We conclude in Section 5.