INTRODUCTION TO STATISTICS AND ECONOMETRICS
Estimators in General
We may sometimes want to estimate a parameter of a distribution other than a moment. An example is the probability (pi) that the ace will turn up in a roll of a die. A “natural” estimator in this case is the ratio of the number of times the ace appears in n rolls to n—denote it by p. In general, we estimate a parameter 0 by some function of the sample. Mathematically we express it as
(7.1.1) 0 = ф(Х], X2, . . . , Xn).
We call any function of a sample by the name statistic. Thus an estimator is a statistic used to estimate a parameter. Note that an estimator is a random variable. Its observed value is called an estimate.
The pi just defined can be expressed as a function of the sample. Let Xi be the outcome of the zth roll of a die and define У, = 1 if X* = 1 and Yi = 0 otherwise. Then pi = (1/?г)ХГ=іУ. Since T, is a function of Xt (that is, Yi is uniquely determined when X, is determined), pi is a function of Xj, X2, . . . , Xn. In Section 7.3 we shall learn that pi is a maximum likelihood estimator.
We stated above that the parameter pi is not a moment. We shall show that it is in fact a function of moments. Consider the following six identities:
6
(7.1.2) EXk = ^jkpj, * = 0, 1, 2, ... ,5,
j= і
where pj = P(X = j), j = 1, 2, . . . , 6. When k = 0, (7.1.2) reduces to the identity which states that the sum of the probabilities is unity, and the remaining five identities for k = 1, 2, . . . , 5 are the definitions of the first five moments around zero. We can solve these six equations for the six unknowns {pj} and express each pj as a function of the five moments. If we replace these five moments with their corresponding sample moments, we obtain estimators of {pj}. This method of obtaining estimators is known as the method of moments. Although, as in this case, the method of moments estimator sometimes coincides with the maximum likelihood estimator, it is in general not as good as the maximum likelihood estimator, because it does not use the information contained in the higher moments.