A COMPANION TO Theoretical Econometrics
Dependent processes
The following weak LLN follows immediately from Corollary 1. In contrast to the above LLNs this theorem does not require the variables to be independently distributed, but only requires uncorrelatedness.
Theorem 20. (Chebychev's weak LLN for uncorrelated random variables) Let Zt be a sequence of uncorrelated random variables with EZt = pt and var(Zt) = a2 < ro. Suppose var(Zn) = nr2 X n=1 °2 ^ 0 as n ^ ro, then Zn - <n -R 0.
The condition on the variance in Theorem 20 is weaker than the corresponding condition in Theorem 19 in view of Kronecker's lemma; see, e. g., Shiryayev (1984, p. 365). The condition is clearly satisfied if the sequence a2 is bounded.
A class of dependent processes that is important in econometrics and statistics is the class of martingale difference sequences. For example, the score of the maximum likelihood estimator evaluated at the true parameter value represents (under mild regularity conditions) a martingale difference sequence.
Definition 8. (Martingale difference sequence) Let Ft, t > 0, be a sequence of a-fields such that F0 C F1 C... C F. Let Zt, t > 1, be a sequence of random variables, then Zt is said to be a martingale difference sequence (wrt the sequence Ft), if Zt is Ft-measurable, E |Zt | < ro and
E(ZflF-1) = 0
for all t > 1.
We note that if Zt is a martingale difference sequence then E(Zt) = E(E(Zt | Ft-1)) = 0 by the law of iterated expectations. Furthermore, since E(Z tZ t+i) = E(Z tE(Z t+k | F+k-1)) = 0 for i > 1, we see that every martingale difference sequence is uncorrelated, provided the second moments are finite. We also note, if Zt is a martingale difference sequence wrt the a-fields Ft, then it is also a martingale difference sequence wrt the a-fields Rj where Rt is generated by {Zt, Zt-1, ..., Z1} and R, = (0, Q}.
We now present a strong LLN for martingale difference sequences.
Theorem 21.14 gale difference sequence with var(Zt) = a t <
ro. Suppose X П= 1at2/12 < ro. Then Zn 0 as n ^ ^.
The above LLN contains Kolmogorov's strong LLN for independent random variables as a special case (with Zt replaced by Zt - pt).
Many processes of interest in econometrics and statistics are correlated, and hence are not covered by the above LLNs. In the following we present a strong LLN for strictly stationary processes, which allows for a wide range of correlation structures.
Definition 9. (Strict stationarity) The sequence of random variables Zt, t > 1, is said to be strictly stationary if (Z1, Z2,..., Zn) has the same distribution as (Z1+k, Z2+k,..., Zn+i) for all i > 1 and n > 1.
Definition 10.15 (Invariance and ergodicity of strictly stationary sequences) Let Zt, t > 1, be a strictly stationary sequence.
(a) Consider the event
A = (ю C Q : (Z1(m), Z2(o>), ...) C B}
with B C B”, where B” are the Borel sets of R”. Then A is said to be invariant if
A = (ю C Q : (ZM(ra), Z2+i(«),...) C B}
for all i > 1.
(b) The sequence Zt is ergodic if every invariant event has probability one or zero.
We note that every iid sequence of random variables is strictly stationary and ergodic. Furthermore, if Zt is strictly stationary and ergodic, and g : R” ^ R is measurable, then the sequence Yt with Yt = g(Z t, Zt+1,...) is again strictly stationary and ergodic.
We can now give the following strong LLN, which is often referred to as the Ergodic Theorem. This theorem contains Kolmogorov's strong LLN for iid random variables as a special case.
Theorem 22.16 stationary and ergodic sequence with E |Z1|
< ” and EZ1 = p. Then Zn —> p as n ^ ”.
There is a large literature on LLNs for dependent processes beside the LLNs presented above. LLNs for weakly stationary processes, including linear processes and ARMA (autoregressive moving average) processes, can be found in Hannan (1970, ch. IV.3); see also Phillips and Solo (1992). Important classes of dependent processes considered in econometrics and statistics are a-mixing, ф-mixing, near epoch dependent and Lp-approximable processes. LLNs for such processes are discussed in some detail in, e. g., Davidson (1994, Part IV) and Potscher and Prucha (1997, ch. 6), and in the references given therein; see also Davidson and de Jong (1997) for recent extensions.
