Introduction to the Mathematical and Statistical Foundations of Econometrics
Distributions of Quadratic Forms of Multivariate Normal Random Variables
As we will see in Section 5.6, quadratic forms of multivariate normal random variables play a key role in statistical testing theory. The two most important results are stated in Theorems 5.9 and 5.10:
Theorem 5.9: Let X be distributed Nn(0, Y), where Y is nonsingular. Then XT£-1 X is distributed as x„.
Proof: Denote Y = (Y1,..., Yn)T = Y- /2X. Then Yis n-variate, standard normally distributed; hence, Y1,...,Yn are independent identically distributed (i. i.d.) N(0, 1), and thus, XT Y-1 X = YTY = Ynj=1 Y2 - x2. Q. E.D.
The next theorem employs the concept of an idempotent matrix. Recall from Appendix I that a square matrix M is idempotent if M2 = M. If M is also symmetric, we can write M = QA QT, where A is the diagonal matrix of eigenvalues of M and Q is the corresponding orthogonal matrix of eigenvectors. Then M2 = M implies A2 = A; hence, the eigenvalues of M are either 1 or 0. If all eigenvalues are 1, then A = I; hence, M = I. Thus, the only nonsingular symmetric idempotent matrix is the unit matrix. Consequently, the concept of a symmetric idempotent matrix is only meaningful if the matrix involved is singular.
The rank of a symmetric idempotent matrix M equals the number of nonzero eigenvalues; hence, trace(M) = trace( Q A QT) = trace(A QT Q) = trace(A) = rank(A) = rank(M), where trace(M) is defined as the sum of the diagonal elements of M. Note that we have used the property trace(AB) = trace(BA) for conformable matrices A and B.
Theorem 5.10: LetXbe distributed Nn (0, I), and let M be a symmetric idem - potent n x n matrix of constants with rank k. Then XJMX is distributed xf.
Proof: We can write
|
|
where Q is the orthogonal matrix of eigenvectors. Because Y = (Y1Yn )T = QTX ~ Nn(0, I), we now have
|
|
Q. E.D.
