Advanced Econometrics Takeshi Amemiya
An Alternative Derivation of the Constrained Least Squares Estimator
Define а К X (К — q) matrix R such that the matrix (Q, R) is nonsingular and R' Q = 0. Such a matrix can always be found; it is not unique, and any matrix
that satisfies these conditions will do. Then, defining A = (Q, R)', у = АД and Z = XA-1, we can rewrite the basic model (1.1.4) as
y = Xfi+u (1.4.7)
= XA-'A/H-ii = Zy + u.
If we partition у = (yj, У2)', where Уі = Q'fi and y2 = R'fi, we see that the constraints (1.4.1) specify y, and leave y2 unspecified. The vector y2 hasK— q elements; thus we have reduced the problem of estimating К parameters subject to q constraints to the problem of estimating К — q free parameters. Using y, = c and
A"1 = [Q(Q'Q)_I, R(R'R)_1], (1.4.8)
we have from (1.4.7)
у - XCKQ'Qr'c = XR(R'R)-1y2 + u. (1.4.9)
Let y2 be the least squares estimator of y2 in (1.4.9):
y2 - R'R(R, X'XR)~1R, X,[y - XQfQ'Q)"^]. (1.4.10)
Now, transforming from у back to fi by the relationship fi — A-1y, we obtain the CLS estimator of fi:
fi= RfR'X'XRj-'R'X'y
+ [I - R(R, X'XR)_1R'X'X]Q(Q'Q)_1c.
Note that (1.4.11) is different from (1.4.5). Equation (1.4.5) is valid only if X'X is nonsingular, whereas (1.4.11) can be defined even if X'X is singular provided that R' X ’ XR is nonsingular. We can show that if X' X is nonsingular, (1.4.11) is unique and equal to (1.4.5). Denote the right-hand sides of (1.4.5) and (1.4.11) by Д and fi2, respectively. Then it is easy to show
[Rq,,X](A-&) = 0. (1.4.12)
Therefore fii = fi2 if the matrix in the square bracket above is nonsingular. But we have
R'X'X"]|"_ _."l _ TR'X'XR R'X'XQ]
where the matrix on the right-hand side is clearly nonsingular because non
singularity of X'X implies nonsingularity of R'X'XR. Because the matrix [R, Q] is nonsingular, it follows that the matrix
is nonsingular, as we desired.
