Advanced Econometrics Takeshi Amemiya
Model 1 with Normality
In this section we shall consider Model 1 with the added assumption of the joint normality of u. Because no correlation implies independence under normality, the {w,} are now assumed to be serially independent. We shall show that the maximum likelihood estimator (MLE) of the regression parameter is identical to the least squares estimator and, using the theory of the Cramer - Rao lower bound, shall prove that it is the best unbiased estimator. We shall also discuss maximum likelihood estimation of the variance a2.
1.1.4 Maximum Likelihood Estimator
Under Model 1 with normality we have у ~ N(Xfi, o2I). Therefore the likelihood function is given by3
L = (2я<72Гт/2 exp [-0.5<Г2(у - ХД)'(У - X0)]. (1.3.1)
Taking the logarithm of this equation and ignoring the terms that do not depend on the unknown parameters, we have
log L = —у log a2 — 2^ (у — ХД)'(у — Xfl). (1.3.2)
Evidently the maximization of log L is equivalent to the minimization of (y — Xfl)' (y — Xfi), so we conclude that the^maximum likelihood estimator of I is the same as the least squares estimator ft obtained in Section 1.2. Putting Д into (1.3.2) and setting the partial derivative of log L with respect to a2 equal to 0, we obtain the maximum likelihood estimator of <r2:
a2 = T_1u'fl, (1.3.3)
where fi = у — хД This is identical to what in Section 1.2.1 we called the least squares estimator of a2.
The mean and the variance-covariance matrix offt were obtained in Section 1.2.3. Because linear combinations of normal random variables are normal, we have under the present normality assumption
Д ~ ЛЧД<72(Х'Х)-Ч. (1.3.4)
The mean of a2 is given in Eq. (1.2.17). We shall now derive its variance under the normality assumption. We have
Ди' Mu)2 = Em ' Muu' Mu (1.3.5)
= tr (M£[(u' Mu)uu' ]}
= <74 tr {M[2M + (tr M)I]}
= <r4[2 tr M + (tr M)2]
= oA[2(T-K) + {J-K?],
where we have used Ей] = 0 and EuAt = 3<r4 since и, ~ N(0, a2). The third equality in (1.3.5) can be shown as follows. If we write the t,5th element of M as mu, the ijth element of the matrix £(u'Mu)uu' is given by 2£.! 2J_ і mtsEu, us щ u}. Hence it is equal to 2aAmy if і Ф j and to 2o4mu + aA, m„ if і = j, from which the third equality follows. Finally, from (1.2.17)
 |
 |
and (1.3.5),
Another important result that we shall use later in Section 1.5, where we discuss tests of linear hypotheses, is that
u'Mu,
0.2 ~ Хт-Ю
which readily follows from Theorem 2 of Appendix 2 because M is indempo - tent with rank T — K. Because the variance of 2(T — K) by Theorem 1
of Appendix 2, (1.3.6) can be derived alternatively from (1.3.7).
