 A COMPANION TO Theoretical Econometrics

# Estimation methods designed specifically for collinear data

A number of estimation methods have been developed to improve upon the least squares estimator when collinearity is present. We will briefly discuss two, ridge regression and principal components regression, if only to warn readers about their use.

The ridge family of estimators is

b(k) = (X'X + kI )-1X'y, (12.14)

where k is a suitably chosen constant. When k = 0 then the ridge estimator is just the OLS estimator of p. For nonstochastic values of k > 0 the ridge estimator is biased, but has smaller variances than the least squares estimator. It achieves the variance reduction by "shrinking" the least squares estimates towards zero. That is, the (Euclidean) length of the ridge estimator is smaller than that of the least squares estimator. Choosing k is important since some values result in reductions of overall mean square error and others do not. Unfortunately, picking a value of k that assures reduction in overall MSE requires knowledge of P and a2, the original object of the regression analysis. Numerous methods for selecting k based on the data have been proposed, but choosing k using data makes k random, and completely alters the statistical properties of the resulting "adaptive" ridge estimator (Hoerl, Kennard, and Baldwin, 1975; Lawless and Wang, 1976). Finite sample inference using the ridge estimator is hindered by dependence of its sampling distribution on unknown parameters. There is a huge statistics liter­ature on the ridge estimator, but the fundamental problems remain and we cannot recommend this estimator.

Principal components regression (Fomby et al., 1984, pp. 298-300) is based upon eigenanalysis. Recall that the (K x K) matrix C, whose columns are the eigenvectors of X'X, is an orthogonal matrix, such that C'C = CC' = I. The T x K matrix Z = XC is called the matrix of principal components of X. The ith column of Z, zj = Xcj, is called the ith principal component. From equation (12.5) zj has the property that zZ = Xj.

The "principal components" form of the linear regression model is

y = хв + e = XCC'P + e = Z0 + e, (12.15)   where Z = XC and 0 = C'p. The new set of explanatory variables Z are linear transformations of the original variables, and have the property that Z'Z = Л = diag(X 1, X2,..., XK), where the Xk are the ordered (in decreasing magnitude) eigenvalues of XX. If we apply least squares to the transformed model we obtain 0 = (Z'Z)-1Z'y, which has covariance matrix cov(0) = o2(Z'Z)-1 = о2Л-1, so that var(0k) = a2/Xk. If the data are collinear then one or more of the eigenvalues will be near zero. If XK ~ 0 then the eigenvector zK ~ 0, and consequently it is difficult to estimate 0K precisely, which is reflected in the large variance of its estimator, var(0K) = o2/XK. Principal components regression deletes from equation (12.15) the zk associated with small eigenvalues (usually based upon tests of significance, or some other model selection criterion, such as AIC or BIC). Partition the trans­formed model as y = Z0 + e = Z101 + Z202 + e. Dropping Z2, which contains the zk to be deleted, and applying OLS yields 01 = (Z1 Z1)-1Z1y. The principal compon­ents estimator of в is obtained by applying an inverse transformation

The properties of this estimator follow directly from the observation that it is equivalent to the RLS estimator of в obtained by imposing the constraints C, в = 0. Thus the principal components estimator bpc is biased, but has smaller variances than the OLS estimator. The data based constraints C, в = 0 generally have no economic content, and are likely to induce substantial bias. One positive use of principal components regression is as a benchmark. The J constraints C, в = 0 have the property that they provide the maximum variance reduction of any set of J linear constraints (Fomby, Hill, and Johnson, 1978). Thus researchers can measure the potential for variance reduction using linear constraints.

Добавить комментарий

## A COMPANION TO Theoretical Econometrics

### Normality tests

Let us now consider the fundamental problem of testing disturbance normality in the context of the linear regression model: Y = Xp + u, (23.12) where Y = (y1, ..., …

### Univariate Forecasts

Univariate forecasts are made solely using past observations on the series being forecast. Even if economic theory suggests additional variables that should be useful in forecasting a particular variable, univariate …

### Further Research on Cointegration

Although the discussion in the previous sections has been confined to the pos­sibility of cointegration arising from linear combinations of I(1) variables, the literature is currently proceeding in several interesting …

## Как с нами связаться:

Украина:
г.Александрия
тел./факс +38 05235  77193 Бухгалтерия
+38 050 512 11 94 — гл. инженер-менеджер (продажи всего оборудования)

+38 050 457 13 30 — Рашид - продажи новинок
e-mail: msd@msd.com.ua
Схема проезда к производственному офису:
Схема проезда к МСД

Партнеры МСД

## Контакты для заказов шлакоблочного оборудования:

+38 096 992 9559 Инна (вайбер, вацап, телеграм)
Эл. почта: inna@msd.com.ua

За услуги или товары возможен прием платежей Онпай: Платежи ОнПай