Introduction to the Mathematical and Statistical Foundations of Econometrics

B.2. Slutsky’s Theorem

Theorem 6.B.1 can be used to prove Theorem 6.7. Theorem 6.3 was only proved for the special case that the probability limit Xis constant. However, the general result of Theorem 6.3 follows straightforwardly from Theorems 6.7 and 6.B.3. Let us restate Theorems 6.3 and 6.7 together:

Theorem 6.B.4: (Slutsky’s theorem). Let Xn a sequence of random vectors in Kk converging a. s. (inprobability) to a (random or constant) vectorX. Let Ф(x) be an Rm-valued function on Kk that is continuous on an open (Borel) set B in Rk for which P(X є B) = 1). Then Ф(Xn) converges a. s. (inprobability) to Ф( X).

Proof: Let Xn ^ X a. s. and let {^, tX, P} be the probability space in­volved. According to Theorem 6.B.1 there exists a null set N1 such that lim„^Z Xn(ш) = X(of pointwise in ш є N1. Moreover, let N2 = {ш є ^ : X(ш) / B}. Then N2 is also a null set and so is N = N1 U N2 . Pick an ar­bitrary ш є ЙN. Because Ф is continuous in X(ш) it follows from standard calculus that lim„^Z Ф(Xn(«)) = Ф(X(ш)). By Theorem 6.B.1 this result im­plies that Ф(Xn) ^ ^(X) a. s. Because the latter convergence result holds along any subsequence, it follows from Theorem 6.B.3 that Xn ^p X implies ^(X„) ^p Ф(X). Q. E.D.

6.B.3. Kolmogorov’s Strong Law of Large Numbers

I will now provide the proof of Kolmogorov’s strong law of large numbers based on the elegant and relatively simple approach of Etemadi (1981). This proof (and other versions of the proof as well) employs the notion of equivalent sequences.

Definition 6.B.1: Two sequences of random variables, Xn and Yn, n > 1, are said to be equivalent if 1 P [Xn = Yn ] < z.

The importance of this concept lies in the fact that if one of the equivalent sequences obeys a strong law of large numbers, then so does the other one:

Lemma 6.B.1: IfXn and Yn are equivalent and (1 /n)J2"l1 Yj ^ д a. s., then (1/n)J2"j=1 Xj ^ д a. s.

Proof: Without loss of generality we may assume that д = 0. Let {Й,

P} be the probability space involved and let

TO

An = U [ю є П : Xm(ю) = Ym(ю)}.

m=n

Then P(An) <J2m=n P(Xm = Ym) ^ 0; hence, limn^cx, P(An) = 0 and thus P(n^U An) = 0. The latter implies that for each ю є ^[П^=1 An} there exists a natural number n*(«) such that Xn (ю) = Yn (ю) for all n > п*(ю) because, if not, there exists a countable infinite subsequence nm(ю), m = 1, 2, 3,... such that Xnk(ю)(ю) = Ynk(ю)(ю); hence, ю є An for all n > 1 and thus ю є п^=1 An. Now let N1 be the null set on which (1/n)J2n=1 Yj ^ 0 a. s. fails to hold, and let N = N1 U [nTO=1 An}. Because for each ю є йN, Xj (ю) and Yj (ю) differ for at most a finite number of j ’sandlimn^TO (1/n)YTj=1 Yj (ю) = 0, it follows also that limn^TO(1 /n)YTj=1 Xj(ю) = 0. Q. E.D.

The following construction of equivalent sequences plays a key role in the proof of the strong law of large numbers.

Lemma 6.B.2: Let Xn, n > 1, be i. i.d., with E[|Xn |] < to, and let Yn = Xn ■ I(|Xn | < n). Then Xn and Yn are equivalent.

Proof: The lemma follows from

[Xn = Yn] = £ P[|Xn | > n]

n = 1

TO

TO p

= ^ P[IX1I > n] < / P[|X1І > t]dt

n=1 0

TO

= j E[I(|X1| > t)]dt < E

Q. E.D.

Now let Xn, n > 1 be the sequence in Lemma 6.B.2, and suppose that (1/n)£n=1 max(0, Xj) ^ E[max(0, X1)] a. s. and (1/n)J^'j=1max(0, —Xj) ^ E[max(0, — X1)] a. s. Then it is easy to verify from Theorem 6.B.1, by taking the union of the null sets involved, that

1 max(0, Xj) / E[max(0, X1)] „ s

n j= max(0, — Xj) J E[max(0, — X1)]/ ' '

Applying Slutsky’s theorem (Theorem 6.B.4) with Ф(х, y) = x - y, we find that (1/n)YTj=1 Xj ^ E [X1] a. s. Therefore, the proof of Kolmogorov’s strong law of large numbers is completed by Lemma 6.B.3 below.

Lemma 6.B.3: Let the conditions of Lemma 6.B.2 hold, and assume in addition that P[Xn > 0] = 1. Then (1/п)^П=1 Xj ^ E[X1] a. s.

Proof: Let Z(n) = (1 /n)J2Yj and observe that

n n

var( Z (n)) < (1/ n2)J2 E[ Yj] = (1/ n2) £ E[x2 I (Xj < j)]

j=1 j=1

< n-1 E[X? I(X1 < n)]. (6.68)

Next let a > 1 and є > 0 be arbitrary. It follows from (6.68) and Chebishev’s inequality that

X>[|Z([an]) - E[Z([an])]| > є]

n=1

where [an ] is the integer part of an. Let k be the smallest natural number such that X1 < [ak], and note that [an] > an /2. Then the last sum in (6.69) satisfies

J2i(x1 < [an])/[an] n=1

hence,

Consequently, it follows from the Borel-Cantelli lemma that Z([an ]) - E[Z([an]) ^ 0 a. s. Moreover, it is easy to verify that E[Z([an]) ^ E[X1]. Hence, Z([an]) ^ E[X1] a. s.

For each natural number k > a there exists a natural number nk such that [ank] < k < [ank+1], and since the Xj’s are nonnegative we have

[ank ] [ank +1]

Ї^т+Г]z tt"n* 1) — Z <» — Z (Kk+1B • <«.7°)

The left-hand expression in (6.70) converges a. s. to E[X 1]/a as k — x, and the right-hand side converges a. s. to aE[X1]; hence, we have, with probabi­lity 1,

—E [X1] — liminf Z (k) — limsup Z (k) — a E [X1]^

a k—x k——x

In other words, if we let Z = liminfk—x Z (k), Z = limsupk—x Z (k), there ex­ists a null set Na (depending on a) such that for all ш є Na, E[X1]/a — Z (ш) — Z(rn) — a E [X1]. Taking the union N of Na over all rational a > 1, so that N is also a null set,8 we find that the same holds for all ш є ^N and all rational a > 1. Letting a I 1 along the rational values then yields limk—x Z(k) = Z(ш) = 2(ш) = E[X1] for all ш є ^N. Therefore, by The­orem 6.B.1, (1/n)Y^j=1Yj — E[X1] a. s., which by Lemmas 6.B.2 and 6.B.3 implies that (1/n)J^=1 Xj — E[X1]. a. s. Q. E.D.

This completes the proof of Theorem 6.6.

6.B.5. The Uniform Strong Law of Large Numbers and Its Applications Proof of Theorem 6.13: It follows from (6.63), (6.64), and Theorem 6.6 that

limsup sup

n—x 0 e©s (0.)

— 2 E

hence, (6.65) can now be replaced by

n

(1/n)J2 g( Xj, в)-E [g( X 1,0)]

j=1

(1/n)J2g(Xj, в) - E[g(X1,0)]

j=1

— e/2a. s. (6.71)

With e/2 replaced by 1/m, m > 1, the last inequality in (6.71) reads as follows:

Note that Ua£(i, x) Na is an uncountable union and may therefore not be a null set. Conse­quently, we need to confine the union to all rational a > 1, which is countable.

Let {^, &, P} be the probability space involved. For m = 1, 2, 3,.. .there exist null sets Nm such that for all ш є QNm,

and the same holds for all ш є ^ U|=1 Nk uniformly in m. If we get m ^то in (6.72), Theorem 6.13 follows.

Note that this proof is based on a seminal paper by Jennrich (1969).

An issue that has not yet been addressed is whether sup9є© |(1/n) YTj=1 g(Xj, 9) - E [g( X1, 9)| is a well-defined random variable. If so, we must have that for arbitrary y > 0,

n

(1/n)J2 g(Xj(ш),9) - E[g(X1,9)

j=1

However, this set is an uncountable intersection of sets in & and therefore not necessarily a set in & itself. The following lemma, which is due to Jennrich (1969), shows that in the case under review there is no problem.

Lemma 6.B.4: Let f (x, 9) be a real function on B x ©, B c Rk, © c Rm, where B is a Borel set and © is compact (hence © is a Borel set) such that for eachxinB, f (x,9) is continuous in 9 є ©,andforeach 9 є ©, f (x,9) isBorel measurable. Then there exists a Borel-measurable mapping 9 (x) : B ^ © such that f (x, 9(x)) = inf9є© f (x, 9); hence, the latter is Borel measurable itself. The same result holds for the “sup” case.

Proof: I will only prove this result for the special case k = m = 1, B = R, © = [0, 1]. Denote ©n = Un=1{0, 1/j, 2/j, ...,(j - 1)/j, 1}, and observe that ©n c ©n+1 and that ©+ = uTO=1 ©n is the set of all rational numbers in [0, 1]. Because ©n is finite, for each positive integer n there exists a Borel- measurable function 9n(x): R ^ ©n such that f (x, 9n(x)) = inf9© f (x, 9). Let 9(x) = liminfn^TO 9n(x). Note that 9(x) is Borel measurable. For each x there exists a subsequence nj (which may depend on x) such that 9 (x) = limj9nj(x). Hence, by continuity, f(x,9(x)) = limjf(x,9nj(x)) = limjinf9є@п. f (x,9). Now suppose that for some є > 0 the latter is greater or equal to є + inf9є04 f (x, 9). Then, because for m < nj, inf9є@^ f (x, 9) < inf9© f (x, 9), and the latter is monotonic nonincreasing in m, it follows that, for all n >1, inf9^._©nf (x,9) > є + inf9є@і f (x,9). It is not too hard to show, using the continuity of f (x,9) in 9, that this is not possible.

Therefore, f (x, 0(x)) = inf0e@t f (x, в); hence, by continuity, f (x, 0(x)) = infoe0 f(x, в). Q. E.D.

Proof of Theorem 6.14: Let {Й, P } be the probability space involved, and denote вп = в. Now (6.9) becomes Q(en) — Q(e0) a. s., that is, there exists a null set N such that for all ш e N,

lim Q(0n(ш)) = й(во)- (6.73)

n — ТО

Suppose that for some ш e ^N there exists a subsequence nm (ш) and an є > 0 such that infm>i \вПт(ш)(ш) — в0| > є. Then by the uniqueness condi­tion there exists a 8(ш) > 0 such that Q(00) - <2(впт(ш)(ш)) > 8(ш) for all m > 1, which contradicts (6.73). Hence, for every subsequence nm (ш) we have limm^TO Onm(ш)(ш) = во, which implies that limn—^(ш) = во.

Proof of Theorem 6.15: The condition Xn — c a. s. translates as follows: There exists a null set N1 such that for all ш e ^N1, limn—TO Xn(ш) = c. By the continuity of Ф on B the latter implies that limn—TO |Ф(Х„ (ш)) — Ф(с)| = 0and that for at most a finite number of indices n, Xn (ш) Є B. Similarly, the uniform a. s. convergence condition involved translates as follows: There exists a null set N2 such that for all ш e ^N2, limn—TOsupxeB|Фп(x, ш) — Ф(x)| — 0. Take N = N1 U N2. Then for all ш e QN,

limsup |Фп(Xn(ш), ш) — Ф(с)|

n—то

< limsup |Фп(Xn(ш), ш) — Ф(Xn(ш))|

n—то

+ limsup | Ф(Xn(ш)) — Ф(с)| < limsup sup | Фп(x, ш) — Ф^)|

n—то n — то xeB

+ limsup |Ф(Xn(ш)) — Ф(с)| = 0.

n—то