A COMPANION TO Theoretical Econometrics

Monte Carlo tests based on pivotal statistics

In the following, we briefly outline the MC test methodology as it applies to the pivotal statistic context and a right-tailed test; for a more detailed discussion, see Dufour (1995) and Dufour and Kiviet (1998).

Let S0 denote the observed test statistic S, where S is the test criterion. We assume S has a unique continuous distribution under the null hypothesis (S is a continuous pivotal statistic). Suppose we can generate N iid replications, Sj, j = 1,..., N, of this test statistic under the null hypothesis. Compute

In other words, NGn(S0) is the number of simulated statistics which are greater or equal to S0, and provided none of the simulated values Sj, j = 1, ..., N, is equal to S0, PN(S0) = N - NGn(S0) + 1 gives the rank of S0 among the variables S0, S1, . . . , SN-5 Then the critical region of a test with level a is:

Pn (S0) < a, (23.30)

where 0 < a < 1 and

N Gn(x) +1

N + 1

The latter expression gives the empirical probability that a value as extreme or more extreme than S0 is realized if the null is true. Hence pN(S0) may be viewed as a MC p-value.

Note that the MC decision rule may also be expressed in terms of Rn(S0). Indeed the critical region

N Gn (S0) + 1
N + 1

is equivalent to

RnS) > (N + 1) (1 - a) + 1. (23.32)

In other words, for 99 replications a 5 percent MC test is significant if the rank of S0 in the series S0, S1,..., SN is at least 96, or informally, if S0 lies in the series top 5 percent percentile. We are now faced with the immediate question: does the MC test just defined achieve size control?

If the null distribution of S is nuisance-parameter-free and a(N + 1) is an integer, the critical region (23.30) is provably exact, in the sense that

P(H0) [pN(S0) - a] = a

or alternatively

P(H0)[Rn(S0) > (N + 1) (1 - a) + 1] = a.

The proof is based on the following theorem concerning the distribution of the ranks associated with a finite dimensional array of exchangeable variables; see Dufour (1995) for a more formal statement of the theorem and related references.

Theorem 23.1 Consider an M x 1 vector of exchangeable real random vari­ables (Y1,..., YM) such that P[Y; = Y] = 0 for i Ф j, and let Rj denote the rank of Yj in the series Y1,..., YM. Then

where I(x) is the largest integer less than or equal to x.

If S is a continuous pivotal statistic, it follows from the latter result that P(Ho)[Rn(S0) > (N + 1) (1 - a) + 1].

Indeed, in this case, the observed test statistic and the simulated statistic are exchangeable if the null is true. Here it is worth recalling that the S;-s must be simulated imposing the null. Now using (23.33), it is easy to show that P(Ho)[Rn(So) > (N + 1) (1 - a) + 1] = a, provided N is chosen so that a(N + 1) is an integer.

We emphasize that the sample size and the number of replications are exp­licitly taken into consideration in the above arguments. No asymptotic theory has been used so far to justify the procedure just described.

It will be useful at this stage to focus on a simple illustrative example. Consider the Jarque and Bera normality test statistic,

1 (Sk)2 + 2^(Ku - 3)2 ,

in the context of the linear regression model Y = xp + u.6 The MC test based on JB and N replications may be obtained as follows.

• Calculate the constrained OLS estimates S, s and the associated residual й.

• Obtain the Jarque-Bera statistic based on s and й and denote it JB(0).

• Treating s as fixed, repeat the following steps for j = 1,..., N:

(a) draw an (n x 1) vector й( j) as iid N(0, s2);

(b) obtain the simulated independent variable #(j) = X0 + й( j);

(c) regress #(j) on X;

(d) derive the Jarque-Bera statistic JB( j) associated with the regression of #(j) on x.

• Obtain the rank Rn(JB(0)) in the series JB(0), JB(1),..., JB(N).

• Reject the null if Rn(JB(0)) > (N + 1) (1 - a) + 1.

Furthermore, an MC p-value may be obtained as pN(S0) = [N + 1 - Rn(S0)]/(N + 1). Dufour et al. (1998) show that the JB statistics can be computed from the stan­dardized residual vector й/s. Using (23.14), we see that

where w = u/a ~ N (0, 1) when u ~ N(0, a2In). It follows that the simulated statistics JB( j) may be obtained using draws from a nuisance-parameter-free (stan­dard normal) null distribution.