Advanced Econometrics Takeshi Amemiya

Classical Least Squares Theory

In this chapter we shall consider the basic results of statistical inference in the classical linear regression model—the model in which the regressors are inde­pendent of the error terrn and the error term is serially uncorrelated and has a constant variance. This model is the starting point of the study; the models to be examined in later chapters are modifications of this one.

1.1 Linear Regression Model

In this section let us look at the reasons for studying the linear regression model and the method of specifying it. We shall start by defining Model 1, to be considered throughout the chapter.

1.1.1 Introduction

Consider a sequence of К random variables (y„ x^, x3t,. . . , x^,), f = 1, 2,. . . ,T. Define a Г-vector у = (у,, y2> ■ • • .Уг)', a (K-iy vector xf = (x2l, x3(). . . ,xKt) and a [(AT— 1)X 7]-vector x* = (xf', xf. . . , xf')'. Suppose for the sake of exposition that the joint density of the variables is given by f(y, x*, в), where 0 is a vector of unknown parame­ters. We are concerned with inference about the parameter vector 0 on the basis of the observed vectors у and x*.

In econometrics we are often interested in the conditional distribution of one set of random variables given another set of random variables; for exam­ple, the conditional distribution of consumption given income and the condi­tional distribution of quantities demanded given prices. Suppose we want to know the conditional distribution of у given x*. We can write the joint density as the product of the conditional density and the marginal density as in

ЛУ» x*, 0) =/(y|x* 0, )/(x*, 02). (1.1.1)

and 62. The vector у is called the vector of dependent or endogenous variables, and the vector x* is called the vector of independent or exogenous variables.

In regression analysis we usually want to estimate only the first and second moments of the conditional distribution, rather than the whole parameter vector ву. (In certain special cases the first two moments characterize 6y completely.) Thus we can define regression analysis as statistical inference on the conditional mean E(y |x*) and the conditional variance-covariance matrix F(y|x*). Generally, these moments are nonlinear functions of x*. However, in the present chapter we shall consider the special case in which E(y,x*) is equal to E(y,xf) and is a linear function of xf, and F(y|x*) is a constant times an identity matrix. Such a model is called the classical (or standard) linear regression model or the homoscedastic (meaning constant variance) linear regression model. Because this is the model to be studied in Chapter 1, let us call it simply Model 1.