INTRODUCTION TO STATISTICS AND ECONOMETRICS

BIVARIATE REGRESSION MODEL

10.1 INTRODUCTION

In Chapters 1 through 9 we studied statistical inference about the distri­bution of a single random variable on the basis of independent observa­tions on the variable. Let {Xt}, t = 1, 2, . . . , T, be a sequence of inde­pendent random variables with the same distribution F. Thus far we have considered statistical inference about F based on the observed values {xt} of {X,}.

In Chapters 10, 12, and 13 we shall study statistical inference about the relationship among more than one random variable. In the present chap­ter we shall consider the relationship between two random variables, x and y. From now on we shall drop the convention of denoting a random variable by a capital letter and its observed value by a lowercase letter because of the need to denote a matrix by a capital letter. The reader should determine from the context whether a symbol denotes a random variable or its observed value.

By the inference about the relationship between two random variables x and y, we mean the inference about the joint distribution of x and y. Let us assume that x and у are continuous random variables with the joint density function f{x, y). We make this assumption to simplify the following explanation, but it is not essential for the argument. The problem we want to examine is how to make an inference about f(x, y) on the basis of independent observations {xt} and [yt], t = 1, 2, . . . , T, on x and y. We call this bivariate (more generally, mutivariate) statistical analysis. Bivariate regression analysis is a branch of bivariate statistical analysis in which

attention is focused on the conditional density of one variable given the other, say, f(y I x). Since we can always write f(x, y) = f(yx)f (x), regres­sion analysis implies that for the moment we ignore the estimation off (x).

Regression analysis is useful in situations where the value of one vari­able, y, is determined through a certain physical or behavioral process after the value of the other variable, x, is determined. A variable such as у is called a dependent variable or an endogenous variable, and a variable such as x is called an independent variable, an exogenous variable, or a regressor. For example, in a consumption function consumption is usually regarded as a dependent variable since it is assumed to depend on the value of income, whereas income is regarded as an independent variable since its value may safely be assumed to be determined independently of consumption. In situations where theory does not clearly designate which of the two vari­ables should be the dependent variable or the independent variable, one can determine this question empirically. It is wise to choose as the inde­pendent variable the variable whose values are easier to predict.

Thus, we can state that the purpose of bivariate regression analysis is to make a statistical inference on the conditional density f(y x) based on independent observations of x and y. As in the single variate statistical inference, we may not always try to estimate the conditional density itself; instead, we often want to estimate only the first few moments of the density—notably, the mean and the variance. In this chapter we shall assume that the conditional mean is linear in x and the conditional variance is a constant independent of x.

We define the bivariate linear regression model as follows:

(10.1.1) y, = a + $xt + ut, t = 1, 2, . . . , T,

where {yt} are observable random variables, xt) are known constants, and {ut} are unobservable random variables which are i. i.d. with Eut = 0 and Vut = a. Here, a, p, and <x are unknown parameters that we wish to estimate. We also assume that xt is not equal to a constant for all t. The linear regression model with all the above assumptions is called the clas­sical regression model.

Note that we assume {xt} to be known constants rather than random variables. This is equivalent to assuming that (10.1.1) specifies the mean and variance of the conditional distribution of у given x. We shall continue to call x, the independent variable. At some points in the subsequent discussion, we shall need the additional assumption that {ut} are normally distributed. Then (10.1.1) specifies completely the conditional distribu­tion of у given x.

The assumption that the conditional mean of у is linear in x is made for the sake of mathematical convenience. Given a joint distribution of x and у, E(y I x) is, in general, nonlinear in x. Two notable exceptions are the cases where x and у are jointly normal and x is binary (that is, taking only two values), as we have seen in Chapters 4 and 5. However, the linearity assumption is not so stringent as it may seem, since if E{y* | x*) is nonlinear in x*, where y* and x* are the original variables (say, con­sumption and income), it is possible that E(y x) is linear in x after a suitable transformation—such as, for example, у = log y* and x = log x*. The linearity assumption may be regarded simply as a starting point. In Section 13.4 we shall briefly discuss nonlinear regression models.

Our assumption concerning {ut} may also be regarded as a starting point. In Chapter 13 we shall also briefly discuss models in which {ut} are serially correlated (that is, Eutus Ф 0 even if t Ф s) or heteroscedastic (that is, Vut varies with t).

We have used the subscript t to denote a particular observation on each variable. If we are dealing with a time series of observations, t refers to the tth period (year, month, and so on). But in some applications t may represent the tth person, tth firm, fth nation, and the like. Data which are not time series are called cross-section data.