INTRODUCTION TO STATISTICS AND ECONOMETRICS

LAWS OF LARGE NUMBERS AND CENTRAL LIMIT THEOREMS

Given a sequence of random variables {X;}, і = 1, 2, ... , define Xn = n lZf=lXi. A law of large numbers (LLN) specifies the conditions under which X„ — EXn converges to 0 in probability. This law is sometimes referred to as a weak law of large numbers to distinguish it from a strong law of large numbers, which concerns the almost sure convergence. We do not use the strong law of convergence, however, and therefore the distinction is unnecessary here.

In many applications the simplest way to show Xn — EXn —» 0 is to show

— — M

Xn — EXn —» 0 and then to apply Theorem 6.1.1 (Chebyshev). In certain situations it will be easier to apply

THEOREM 6.2.1 (Khinchine) Let {X;} be independent and identically distributed (i. i.d.) with XX; = p. Then Xn —> pc.

Note that the conclusion of Theorem 6.2.1 can be obtained from a different set of assumptions on {X,} if we use Theorem 6.1.1 (Chebyshev). For example, if {XJ are uncorrelated with EXt = jjl and VX, = a2, then Xn A |x; therefore, by Theorem 6.1.1, Xn A |x.

Now we ask the question, what is an approximate distribution of X„ when n is large? Suppose a law of large numbers holds for a sequence (XJ so that Xn — EXn A 0. It follows from Theorem 6.1.2 that Xn — EXn A 0. It is an uninteresting limit distribution, however, because it is degenerate. It is more meaningful to inquire into the limit distribution of Zn =

— І /л — —

(TXJ (Xn — EXn). For if the limit distribution of Zn exists, it should be nondegenerate, because VZn = 1 for all n. A central limit theorem (CLT) specifies the conditions under which Z„ converges in distribution to a standard normal random variable. We shall write Zn—>N( 0, 1). More precisely, it means the following: if Fn is the distribution function of Z„,

(6.2.1)

We shall state two central limit theorems—Lindeberg-Levy and Liapounov.

THEOREM 6.2.2 (Lindeberg-Levy) Let {XJ be i. i.d. with EX{ = |X and VXi = о2. Then Zn N(0, 1).

THEOREM 6.2.3 (Liapounov) Let {XJ be independent with EXt = |x„ and E(Xi ~ m,-|s) = mbl. If

then Zn —> N(0, 1).

These two CLTs are complementary: the assumptions of one are more restrictive in some respects and less restrictive in other respects than those of the other. Both are special cases of the most general CLT, which is due to Lindeberg and Feller. We shall not use it in this book, however, because its condition is more difficult to verify.

In the terminology of Definition 6.1.4, central limit theorems provide

— і /n —

conditions under which the limit distribution of Zn = (VX„) (Xn —

EXn) is N(0, 1). We now introduce the term asymptotic distribution, which means the “approximate distribution when n is large.” Given the mathe­matical result Zn A iV(0, 1), we shall make statements such as “the asymp­totic distribution of Zn is N{0, 1)” (written as Zn ~ N{0, 1)) or “the asymptotic distribution of Xn is N(EXn, TXJ.” This last statement may also be stated as “Xn is asymptotically normal with the asymptotic mean EXn and the asymptotic varianceVXn.” These statements should be regarded merely as more intuitive paraphrases of the result Zn AiV( 0, 1). Note that it would be meaningless to say that “the limit distribution of Xn is N(EXn, VXn).”