A COMPANION TO Theoretical Econometrics

The Concept of an Artificial Regression

Consider a fully parametric, nonlinear model that is characterized by a para­meter vector 0 which belongs to a parameter space 0 C R k and which can be estimated by minimizing a criterion function Q(0) using n observations. In the case of a nonlinear regression model estimated by nonlinear least squares, Q(0) would be one half the sum of squared residuals, and in the case of a model estim­ated by maximum likelihood, Q(0) would be minus the loglikelihood function.

If an artificial regression exists for such a model, it always involves two things: a regressand, r(0), and a matrix of regressors, R(0). The number of regressors for the artificial regression is equal to k, the number of parameters. The number of "observations" for the artificial regression is often equal to n, but it may also be equal to a small integer, such as 2 or 3, times n. We can write a generic artificial regression as

r(0) = R(0)b + residuals,

where b is a k-vector of coefficients. "Residuals" is used here as a neutral term to avoid any implication that (1) is a statistical model. The regressand and regressors in (1) can be evaluated at any point 0 C 0, and the properties of the artificial regression will depend on the point at which they are evaluated. In many cases, we will want to evaluate (1) at a vector of estimates 0 that is root-n consistent. This means that, if the true parameter vector is 0O C 0, then 0 approaches 0O at a rate proportional to n~1/2. One such vector that is of particular interest is 0, the vector of estimates which minimizes the criterion function Q(0).

For (1.1) to constitute an artificial regression, the vector r(0) and the matrix R(0) must satisfy certain defining properties. These may be stated in a variety of ways, which depend on the class of models to which the artificial regression is intended to apply. For the purposes of this chapter, we will say that (1.1) is an artificial regression if it satisfies the following three conditions:

if b denotes the vector of estimates from the artificial regression (1.1) with regressand and regressors evaluated at 0, then

0 + b = 0 + op(n 1/2).

Many artificial regressions actually satisfy a stronger version of condition (1):

g(0) = - RT(0)r(0), (1.1')

where g(0) denotes the gradient of the criterion function Q(0). Clearly, condition (1.1') implies condition (1), but not vice versa. The minus sign in (1.1') is due to the arbitrary choice that the estimator is defined by minimizing Q(0) rather than maximizing it.

Condition (2) has been written in a particularly simple form, and some non­standard artificial regressions do not actually satisfy it. However, as we will see, this does not prevent them from having essentially the same properties as artificial regressions that do satisfy it.

Condition (3), which is perhaps the most interesting of the three conditions, will be referred to as the one-step property. It says that, if we take one step from an initial consistent estimator 0, where the step is given by the coefficients b from the artificial regression, we will obtain an estimator that is asymptotically equival­ent to 0.

The implications of these three conditions will become clearer when we study specific artificial regressions in the remainder of this chapter. These conditions differ substantially from the conditions used to define an artificial regression in Davidson and MacKinnon (1990), because that paper was concerned solely with artificial regressions for models estimated by maximum likelihood.