In this subsection we discuss LLNs for independent processes.
Theorem 18.12 (Kolmogorov's strong LLN for iid random variables) Let Zt be a sequence of identically and independently distributed (iid) random variables with E |Z1| < ro and EZ1 = p. Then Zn p (and hence Zn p) as n ^ ro.
We have the following trivial but useful corollary.
Corollary 6. Let Zt be a sequence of iid random variables, and let f be a Borel - measurable real function satisfying E |f(Z1)| < ro, then n_1 Xn=1 f (Zt) Ef (Z1) as
n ^ ro.
The corollary can, in particular, be used to establish convergence of sample moments of, say, order p to the corresponding population moment by choosing f(Zt) = Ztp.
We now derive the probability limit of the ordinary least squares estimator considered in Example 6 as an illustration.
Example 7. Assume the setup of Example 6. Assume furthermore that the processes xt and et are iid with Ex'2 = Qx, 0 < Qx < ro, E | et | < ro, and Eet = 0, and that the two processes are independent of each other. Then xtet is iid, has finite expectation and satisfies Extet = ExtEet = 0. Hence it follows from Theorem 18 that n1 X n=1 xtet 0. Corollary 6 implies n_1 X n=1 xt2 Qx. Applying Theorem 14
then yields Pn 0 + 0/Qx = 0.
The assumption in Theorem 18 that the random variables are identically distributed can be relaxed at the expense of maintaining additional assumptions on the second moments.
Theorem 19.13 (Kolmogorov's strong LLN for ID random variables) Let Zt be a sequence of independently distributed (ID) random variables with EZt = pt and
var(Zt) = о;2 < ro. Suppose X T=1 c2/12 < ro. Then Zn - <n 0 as n ^ ro.
The condition X n=1 о;2/12 < ro puts a restriction on the permissible variation in the о;2. For example, it is satisfied if the sequence о2 is bounded.