Introduction to the Mathematical and Statistical Foundations of Econometrics

Conditional Distributions of Multivariate Normal Random Variables

Let Y be a scalar random variable and X be a k-dimensional random vector. Assume that

where mY = E(Y), mX = E(X), and

£yy = Var(Y), £yx = Cov(Y, X)

= E[(Y - E(Y))(X - E(X))T],

£xy = Cov(X, Y) = E(X - E(X))(Y - E(Y))

= £YX’ £XX = Var(X).

To derive the conditional distribution of Y, given X, let U = Y - a - втX, where a is a scalar constant and в is a k x 1 vector of constants such that E(U) = 0 and U and X are independent. It follows from Theorem 5.1 that

1 0ту

-в Ik)_ ■

The variance matrix involved can be rewritten as

Var (U^ — f^YY — £УХв — PT£XY + PT^XXP £YX — вT£XX Vх/ £xy — £ххв £XX

(5.6)

Next, choose в such that U and X are uncorrelated and hence independent. In view of (5.6), a necessary and sufficient condition for that is £XY — £ххв — 0; hence, в — £—X[£XY. Moreover, E(U) — 0 if a — gY — вTgX. Then

£yY — £Yxe — в T£xY + в Т£ххв — £yY — ^YX^XX^XY,

£yx — вT£XX — A, £XY — £ххв — 0,

and consequently

( 0 (£уу — £yx£—X1£xy 0t'

gx) 0 £xx,

Thus, U and X are independent normally distributed, and consequently E(U|X) — E(U) — 0. Because Y — a + вTX + U, we now have E(Y|X) — a + вT (E(X|X)) + E(U|X) — a + вTX. Moreover, it is easy to verify from

(5.6) that the conditional density of Y, given X — x, is

/(y|x) — exp [- 1(y — a — в T x )2/a»],

au*J 2n

where &U — £yy — £yx £xx £xy-

Furthermore, note that aU is just the conditional variance of Y, given X:

al — var(Y|X) — E [(Y — E(Y|X))2|X]. These results are summarized in the following theorem.

Theorem 5.5: Let

( My (£yy £yx

gx) ’ V£xy £xxJ J ’

where Y є К, X є Kk, and £хх is nonsingular. Then, conditionally onX, Y is normally distributed with conditional expectation E(Y |X) — a + вTX, where в — £Xy£XY and a — gY — вTgX, and conditional variance var(Y|X) — £yy — £yx£—x£xy.

The result in Theorem 5.5 is the basis for linear regression analysis. Suppose that Y measures an economic activity that is partly caused or influenced by other economic variables measured by the components of the random vector X. In applied economics the relation between Y, called the dependent variable, and the components of X, called the independent variables or the regressors,
is often modeled linearly as Y = a + fTX + U, where a is the intercept, в is the vector of slope parameters (also called regression coefficients), and U is an error term that is usually assumed to be independent of X and normally N(0, a2) distributed. Theorem 5.5 shows that if Y and X are jointly normally distributed, then such a linear relation between Y and X exists.