Advanced Econometrics Takeshi Amemiya

Statistical Decision Theory

We shall briefly explain the terminology used in statistical decision theory. For a more thorough treatment of the subject, the reader should consult Zacks (1971). Statistical decision theory is a branch of game theory that analyzes the game played by statisticians against nature. The goal of the game for statisti­cians is to make a guess at the value of a parameter (chosen by nature) on the basis of the observed sample, and their gain from the game is a function of how close their guess is to the true value. The major components of the game are 0, the parameter space; Y, the sample space; and D, the decision space (the totality of functions from Y to 0). We shall denote a single element of each space by the lowercase letters 0, у, d. Thus, if у is a particular observed sample (a vector of random variables), d is a function of у (called a statistic or an estimator) used to estimate 0. We assume that the loss incurred by choosing d when the true value of the parameter is 0 is given by the loss function L(d, 0).

We shall define a few standard terms used in statistical decision theory.

Risk. The expected loss EyL(d, 0) for which the expectation is taken with respect to у (which is implicitly in the argument of the function d) is called the risk and is denoted by jR(d|0).

Uniformly smaller risk. The estimator2 d, has a uniformly smaller risk than the estimator d2 if R(<Ai) Ш i?(d2|0) for all 0 Є 0 and R(&i) < R(i2) for at least one 0 Є 0.

Admissible. An estimator is admissible if there is no d in D that has a uniformly smaller risk. Otherwise it is called inadmissible.

Minimax. The estimator d* is called a minimax estimator if

max /?(d*|0) = min max i?(d|0).

0ЄЄ ІЄЛ 0ЄЄ

The minimax estimator protects the statistician against the worst pos­sible situation. If maxeee. R(d|0) does not exist, it should be replaced with supeee 7?(d|0) in the preceding definition (and min with inf).

Posterior risk. The expected loss EeL{&, 0) for which the expectation is taken with respect to the posterior distribution of в given у is called the posterior risk and is denoted by 7?(d|y). It obviously depends on the particular prior distribution used in obtaining the posterior distribution.

Bayes estimator. The Bayes estimator, given a particular prior distribution, minimizes the posterior risk 7?(d|y). If the loss function is quadratic, namely, L(d, 0) = (d — 0)' W(d — 0) where W is an arbitrary nonsingular matrix, the posterior risk Ев{й — 0)'W(d — 0) is minimized at d = Ee0, the posterior mean of 0. An example of the Bayes estimator was given in Section 1.4.4.

Regret. Let i?(d|0) be the risk. Then the regret W(d|0) is defined by

Щd|0) = 7?(d|0) - min i?(d|0).

лев

Minimax regret. The minimax regret strategy minimizes maxeee lF(d|0) with respect to d.

Some useful results can be stated informally as remarks rather than stating them formally as theorems.

Remark 2.1.1. A Bayes estimator is admissible.

Remark 2.1.2. A minimax estimator is either a Bayes estimator or the limit of a sequence of Bayes estimators. The latter is called a generalized Bayes estimator. (In contrast, a Bayes estimator is sometimes called a proper Bayes estimator.)

Remark 2.1.3. A generalized Bayes estimator with a constant risk is minimax.

Remark 2.1.4. An admissible estimator may or may not be minimax, and a minimax estimator may or may not be admissible.