INTRODUCTION TO STATISTICS AND ECONOMETRICS
SIMPLE AGAINST COMPOSITE
We have so far considered only situations in which both the null and the alternative hypotheses are simple in the sense of Definition 9.1.1. Now we shall turn to the case where the null hypothesis is simple and the alternative hypothesis is composite.
We can mathematically express the present case as testing H0: 0 = 0O against Hi: 0 Є 01; where 0X is a subset of the parameter space. If 0j consists of a single element, it is reduced to the simple hypothesis considered in the previous sections. Definition 9.2.4 defined the concept of the most powerful test of size a in the case of testing a simple against a simple
(i +e,)2
 |
hypothesis. In the present case we need to modify it, because here the (3 value (the probability of accepting H0 when Нг is true) is not uniquely determined if 0j contains more than one element. In this regard it is useful to consider the concept of the power function.
DEFINITION 9.4.1 If the distribution of the sample X depends on a vector of parameters 0, we define the power function of the test based on the critical region R by
(9.4.1) (?(0) = P(X Є R I 0).
Using the idea of the power function, we can rank tests of a simple null hypothesis against a composite alternative hypothesis by the following definition.
DEFINITION 9.4.2 Let Qi(0) and <2г(0) be the power functions of two tests respectively. Then we say that the first test is uniformly better (or uniformly more powerful) than the second test in testing H(): 0 = 0O against
Hx 0 Є 0! if Qi(0o) = &(0o) and
(9.4.2) C? i(0) — Ог(0) for all 0 Є 0i
and
(9.4.3) Qi(9) > Ог(0) for at least one 0 Є 0X.
Note that £7(0o) Is what we earlier called a, and if 0] consists of a single element equal to 0b we have (?(0i) = 1 — (T
The following is an example of the power function. example 9.4.1 Let X have the density f(x) = і for 0 < X < 0,
= 0 otherwise.
We are to test H0: 0=1 against /7): 0 > 1 on the basis of one observation on X. Obtain and draw the power function of the test based on the critical region R = [0.75, oo).
By (9.4.1) we have
, 0 75
(9.4.4) Q(0) = P(X > 0.75 I 0) = 1 - “ •
U
Its graph is shown in Figure 9.7.
The following is a generalization of Definitions 9.2.4 and 9.2.5. This time we shall state it for size and indicate the necessary modification for level in parentheses.
DEFINITION 9.4.3 A test R is the uniformly most powerful (UMP) test of size (level) a for testing H0: 0 = 0O against Hf. 0 Є 0j if P(R 0O) = (<) a and for any other test Ri such that P{R | 0o) = (^) a, we have P(R I 0) > P(R I 0) for any 0 E 0j.
In the case where both the null and the alternative hypotheses are simple, the Neyman-Pearson lemma provides a practical way to find the most powerful test of a given size a. In the present case, where the alternative hypothesis is composite, the UMP test of a given size a may not always exist. The so-called likelihood ratio test, however, which may be thought of as a generalization of the Neyman-Pearson test, usually gives
 |
 |
 |
 |
FIGURE 9.7 Power function
the UMP test if a UMP test exists; even when it does not, the likelihood ratio test is known to have good asymptotic properties.
 |
 |
DEFINITION 9.4.4 Let L(x | 0) be the likelihood function and let the null and alternative hypotheses be H0: 0 = 0O and H{: 0 Є 0Ь where 0j is a subset of the parameter space 0. Then the likelihood ratio test of H0 against Hi is defined by the critical region
where c is chosen to satisfy P(A < c H0) = a for a certain specified value of a. Sup, standing for supremum, means the least upper bound and is equal to the maximum if the latter exists. Note that we have 0 < A < 1 because the subset of the parameter space within which the supremum is taken contains 0O.
Below we give several examples of the likelihood ratio test. In some of them the test is UMP, but in others it is not.
 |
 |
EXAMPLE 9.4.2 Let X be distributed as B(n, p). We are to test H0: p = po against Hi. p> p0, given the observation X = x. The likelihood function is L(x, p) = Cnxpx(l — p)n~x. If x/n < p0, clearly Л = 1, which means that H0 is accepted for any value of a less than 1. If x/n > p0, max^^ L(x, p) is attained at p = x/n. Therefore the critical region of the likelihood ratio test is given by
Taking the logarithm of both sides of (9.4.6) and dividing by —n, we obtain
(9.4.7) t log t + (1 - t) log (1 - t) ~ t log pQ - (1 - 0 log (1 - po)
> log c
n
where we have put t = x/n. Since it can be shown by differentiation that the left-hand side of (9.4.7) is an increasing function of t whenever t > po, it is equivalent to
(9.4.8) £ > d,
where d should be determined so as to make the probability of event
(9.4.8) under the null hypothesis (approximately) equal to a. (Note that c need not be determined.)
This test is UMP because it is the Neyman-Pearson test against any specific value of p > p0 (see Example 9.3.1) and because the test defined by (9.4.8) does not depend on the value of p.
EXAMPLE 9.4.3 Let the sample be ~ N(p, a2), і = 1, 2, . . . , n, where
9
с is assumed known. Let x, be the observed value of Xt. We are to test H0 p = |jl0 against Нг: p > p0. The likelihood ratio test is to reject H0 if
1 V 2
---- r (Xi - Po)
2cr2 ,-=i J
----------------------------- < c.
sup exp - -^2 X ~ Iх)2
L 2a,= 1
 |
If x < po, then Л — 1 because we can write E(x, — p)2 = 2(x; — x)2 + n( p — x)2; therefore, we accept H0. So suppose x > p0. Then the denominator of A attains a maximum at p = x. Therefore, we have
Therefore, since x > p,0, the likelihood ratio test in this case is characterized by the critical region
(9.4.11) x > d, where d is determined by P(X > d H0) = a.
For the same reason as in the previous example, this test is UMP.
EXAMPLE 9.4.4 The assumptions are the same as those of Example 9.4.3 except that Hi: p, Ф p0. Then the denominator in (9.4.9) is maximized with respect to the freely varying jjl, attaining its maximum at p. = x. Therefore we again obtain (9.4.10), but this time without the further constraint that x > p,0. Therefore the critical region is
(9.4.12) x - p,0| > d,
where d is determined by P(|X — p,0| > d H0) = a.
This test cannot be UMP, because it is not a Neyman-Pearson test against a specific value of p,.
Tests such as (9.4.8) and (9.4.11) are called onedail tests, whereas tests such as (9.4.12) are called two-tail tests. In a two-tail test such as (9.4.12) we could perform the same test using a confidence interval, as discussed in Section 8.2. From Example 8.2.2 the 1-а confidence interval of p is defined by |x — p,| < d, where d is the same as in (9.4.12). Therefore, H0 should be rejected if and only if p0 lies outside the confidence interval.
example 9.4.5 Consider the model of Example 9.3.3 and test H0: 0 = 0 against Hi. 0 > 0 on the basis of one observation x. If x ^ 0, then A = 1, so we accept H0. Therefore assume x > 0. Then the numerator of Л is equal to 1/[2(1 + x)2] and the denominator is equal to У2. Therefore the likelihood ratio test is to reject HQ if x > d, where d is chosen appropriately. This test is not UMP because it is not a Neyman-Pearson test, which was obtained in Example 9.3.3. That the UMP test does not exist in this case can be seen more readily by noting that the Neyman-Pearson test in this example depends on a particular value of 0.
EXAMPLE 9.4.6 Suppose X has a uniform density over [0, 0], 0 < 0 ^ 1. We are to test H0: 0 = y2 against H^. 0 Ф У2 on the basis of one observation
|
A
|
figure 9.8 Power function of a likelihood ratio test
x. Derive the likelihood ratio test of size У2, draw its power function, and show that it is UMP.
First, note that A = 0 for x Є [0.5, 1]; therefore, [0.5, 1] should be part of the critical region. Next assume that x Є [0, 0.5). Then we have
2
(9.4.13) A = — = 2x.
1/x
Therefore we reject H0 if 2x < c, where c should satisfy P(2X < c H0) = У2. This implies that с = У2. We conclude that the critical region is [0, 0.25] U [0.5, 1]. Its power function is depicted as a solid curve in Figure 9.8. To show that this is UMP, first note that [0.5, 1] should be part of any reasonable critical region, because this portion does not affect the size and can only increase the power. Suppose the portion A of [0, 0.25] is removed from the critical region and the portion В is added in such a way that the size remains the same. Then part of the power function shifts downward to the broken curve. This completes the proof.
In all the examples of the likelihood ratio test considered thus far, the exact probability of Л < c can be either calculated exactly or read from appropriate tables. There are cases, however, where P(A < c) cannot be easily evaluated. In such a case the following theorem is useful.
theorem 9.4.1 Let A be the likelihood ratio test statistic defined in
(9.4.5) . Then, —2 log A is asymptotically distributed as chi-square with the degrees of freedom equal to the number of exact restrictions implied by
H0. (For example, if there are r parameters and H0 specifies the values of all of them, the degrees of freedom are r.)
