INTRODUCTION TO STATISTICS AND ECONOMETRICS

CONFIDENCE INTERVALS

We shall assume that confidence is a number between 0 and 1 and use it in statements such as “a parameter 0 lies in the interval [a, b with 0.95 confidence,” or, equivalently, “a 0.95 confidence interval for 0 is [a, b].” A confidence interval is constructed using some estimator of the parameter in question. Although some textbooks define it in a more general way, we shall define a confidence interval mainly when the estimator used to construct it is either normal or asymptotically normal. This restriction is not a serious one, because most reasonable estimators are at least asymp­totically normal. (An exception occurs in Example 8.2.5, where a chi - square distribution is used to construct a confidence interval concerning a variance.) The concept of confidence or confidence intervals can be best understood through examples.

example 8.2.1 Let X, be distributed as і = 1, 2, . . . , n. Then

T = X ~ N[p, p{ — p)/ri. Therefore, we have

Let Z be N(0, 1) and define (8.2.2) yk = P(Z < k).

Then we can evaluate the value of 4k for various values of k from the standard normal table. From (8.2.1) we have approximately

(8.2.4) C

7k,

which reads “the confidence that p lies in the interval defined by the inequality within the bracket is yA” or “the 4k confidence interval of p is as indicated by the inequality within the bracket.”

Definition (8.2.4) is motivated by the observation that the probability that T lies within a certain distance from p is equal to the confidence that p lies within the same distance from an observed value of T. Note that this definition establishes a kind of mutual relationship between the estimator and the parameter in the sense that if the estimator as a random variable is close to the parameter with a large probability, we have a proportionately large confidence that the parameter is close to the observed value of the estimator. Equation (8.2.3) may be equivalendy written as

(8.2.5)

which may be further rewritten as

P[hiT) <p<hfiT)] = 4k,

f ^ 1 +- n

f

1 + — n

Similarly, (8.2.4) can be written as

(8.2.8) C[Ai(f) <p< h2(t) = y*.

The probabilistic statement (8.2.6) is a legitimate one, because it concerns a random variable T. It states that a random interval [/q(T), h^{JT)~ con­tains p with probability yk. Definition (8.2.8) is appealing as it equates the probability that a random interval contains p to the confidence that an observed value of the random interval contains p.

Let us construct a 95% confidence interval of p, assuming n = 10 and t = 0.5. Then, since jk = 0.95 when k = 1.96, we have from (8.2.8)

(8.2.9) C(0.2366 < p < 0.7634) = 0.95.

If n = 100 and t = 0.5, we have

(8.2.10) C(0.4038 < p < 0.5962) = 0.95.

Thus a 95% confidence interval keeps getting shorter as n increases—a reasonable result.

Next we want to study how confidence intervals change as k changes for fixed values of n and t. For this purpose, consider n(t — py/p{ — p) as a function of p for fixed values of n and t. It is easy to see that this function shoots up to infinity near p = 0 and 1, attains the minimum value of 0 at p = t, and is decreasing in the interval (0, t) and increasing in (t, 1). This function is graphed in Figure 8.1. We have also drawn horizontal lines whose ordinates are equal to k and k* . Thus the intervals (a, b) and (a*, b*) correspond to the yk and у** confidence intervals, respectively. By definition, confidence clearly satisfies probability axioms (1) and (2) if in (2) we interpret the sample space as the parameter space, which in this example is the interval [0, 1]. Moreover, Figure 8.1 shows that if interval 11 contains interval 12, we have С{1) ^ С(І2)■ This suggests that we may extend the definition of confidence to a larger class of sets than in (8.2.4), so that confidence satisfies probability axiom (3) as well. For example, (8.2.4) defines C(a < p < b) = у*, and C(a* < p < b*) = y**, and we may further define C[(a < p < a*) U (b* < p < b)] = у * — у**.

Confidence is not as useful as probability, however, because there are many important sets for which confidence cannot be defined, even after such an extension. For example, C(a < p < a*) cannot be uniquely determined from definition (8.2.4). This is definitely a shortcoming of the confidence approach. In Bayesian statistics we would be able to treat p as a random variable and hence construct its density function. Then we could

calculate the probability that p lies in a given interval simply as the area under the density function over that interval. This is not possible in the confidence approach, shown above. In other words, there is no unique function (“confidence density,” so to speak) such that the area under the curve over an interval gives the confidence of the interval as defined in

(8.2.4) . For, given one such function, we can construct another by raising the portion of the function over (0, t) and lowering the portion over (t, 1) by the right amount.

о

EXAMPLE 8.2.2 Let X* ~ iV(|Jt, cj ), і = 1, 2, . . . , n, where p, is unknown

2 — о

and a is known. We have T = X ~ Лі(р, а /п). Define

T ~ м-І

Therefore, given T = t, we define confidence

= Ik-

Thus the greater the probability that T lies within a certain distance from |x, the greater the confidence that p lies within the same distance from t. Note that (8.2.12) defines confidence only for intervals with the

о

center at t. We may be tempted to define N(t, a /n) as a confidence density for |x, but this is one among infinite functions for which the area under the curve gives the confidence defined in (8.2.12). For example, the function obtained by eliminating the left half of the normal density N(t, a /n) and doubling the right half will also serve as a confidence density.

Suppose that the height of the Stanford male student is distributed as N(p, 0.04) and that the average height of ten students is observed to be 6 (in feet). We can construct a 95% confidence interval by putting t = 6, a2 = 0.04, n = 10, and k = 1.96 in (8.2.12) as

Therefore the interval is (5.88 < p < 6.12).

EXAMPLE 8.2.3 Suppose that X* ~ 7V(p, a2), і = 1, 2, . . . , n, with both p and cr unknown. Let T = X be the estimator of p and let S = n — Xу be the estimator of a2. Then the probability distribution

of tn-1 = S 1(T — p) Vn — 1 is known and has been tabulated. It depends only on n and is called the Student’s t distribution with n — 1 degrees offreedom. See Theorem 5 of the Appendix for its derivation. Its density is symmetric around 0 and approaches that of N(0, 1) as и goes to infinity. Define

(8.2.14) lk = P(|tra_!| < k), where "у* for various values of k can be computed or read from the Student’s t table. Then we have

fT ~ Ph/w - 1 < ^

S У

Given T = t, S = s, we define confidence by

Consider the same data on Stanford students used in Example 8.2.2, but assume that a2 is unknown and estimated by. S'2, which is observed to be 0.04. Putting t = 6 and s = 0.2 in (8.2.16), we get

Therefore the 95% confidence interval of |x is (5.85 < |x < 6.15). Note that this interval is slightly larger than the one obtained in the previous example. The larger interval seems reasonable, because in the present example we have less precise information.

EXAMPLE 8.2.4 Let X,- ~ N(|xx, a2), і = 1, . . . , их, let Yt ~ N(|Xy, CT2), і = 1, . . . , nY, and assume that (X,) are independent of {Yt. Then, as shown in Theorem 6 of the Appendix,

(X Y) (jrx ILy) І ПХПу(пХ + Пу — 2)
^nxSx + nYSy ' nx + nY

Thus, given the observed values x, y, sx, and s2, we can define the y* confidence interval for jxx — xY by

where yk = P(tnx+ny-2 < Щ - The assumption that X and Y have the same variance is crucial, because otherwise (8.2.18) does not follow from equa­tion (11) of the Appendix. See Section 10.3.1 for a method which can be used in the case of crx Ф oy.

As an application of the formula (8.2.19), consider constructing a 0.95 confidence interval for the true difference between the average lengths of unemployment spells for female and male workers, given that a random sample of 35 unemployment spells of female workers lasted 42 days on the average with a standard deviation of 2.5 days, and that a random sample of 40 unemployment spells of male workers lasted 40 days on the

average with a standard deviation of 2 days. Since P(|f73| < 2) = 0.95, inserting k = 2, x = 42, у = 40, nx = 35, nY = 40, sx = (2.5)[5], and s2 = 22 into (8.2.19) yields

(8.2.20) C(0.9456 < |xx - |xy < 3.0544) = 0.95.

EXAMPLE 8.2.5 Let X,- ~ Х(|л, о2), і = 1, 2, . . . , n, with both p, and o2 unknown, as in Example 8.2.3. This time we want to define a confidence interval on a. It is natural to use the sample variance defined by S = n l'Lr-=i{Xi — X)2. Using it, we would like to define a confidence interval of the form

(8.2.21) a + bS2 < о2 < c + dS2, where we can get varying intervals by varying a, b, c, and d. A crucial question is, then, can we calculate the probability of the event (8.2.21) for various values of a, b, c, and d? We reverse the procedure and start out with a statistic of which we know the distribution and see if we can form an interval like that in (8.2.21). We begin by observing nS2/cr2 ~ Xn-i> given in Theorem 3 of the Appendix, and proceed as follows:

Given y, ki and &2 can be computed or read from the table of chi-square distribution.

As an application of the formula (8.2.23), consider constructing a 95% confidence interval for the true variance ct2 of the height of the Stanford male student, given that the sample variance computed from a random sample of 100 students gave 36 inches. Assume that the height is normally distributed. Inserting n = 100, s2 = 36, k = 74.22, and &2 = 129.56 into (8.2.23) yields the confidence interval (27.79, 48.50).

EXAMPLE 8.2.6 Besides the preceding five examples, there are many situations where T, as estimator of 0, is either normal or asymptotically normal. If, moreover, the variance of T is consistently estimated by some estimator V, we may define confidence approximately by

where Z is N(0,1) and t and v are the observed values of T and V, respectively. If the situations of Examples 8.2.1, 8.2.3, 8.2.4, or 8.2.5 actu­ally occur, it is better to define confidence by the method given under the respective examples, even though we can also use the approximate method proposed in this example.

As an application of the formula (8.2.26), consider the same data given at the end of Example 8.2.5. Then, by Theorem 4 of the Appendix,

(8.2.27) S2~A(ct2,ct4/50).

4 2

Estimating the asymptotic variance ct /50 by 36 /50 and using (8.2.26), we obtain an alternative confidence interval, (26.02,45.98), which does not differ greatly from the one obtained by the more exact method in Example 8.2.5.