Advanced Econometrics Takeshi Amemiya
Statistical Decision Theory
We shall briefly explain the terminology used in statistical decision theory. For a more thorough treatment of the subject, the reader should consult Zacks (1971). Statistical decision theory is a branch of game theory that analyzes the game played by statisticians against nature. The goal of the game for statisticians is to make a guess at the value of a parameter (chosen by nature) on the basis of the observed sample, and their gain from the game is a function of how close their guess is to the true value. The major components of the game are 0, the parameter space; Y, the sample space; and D, the decision space (the totality of functions from Y to 0). We shall denote a single element of each space by the lowercase letters 0, у, d. Thus, if у is a particular observed sample (a vector of random variables), d is a function of у (called a statistic or an estimator) used to estimate 0. We assume that the loss incurred by choosing d when the true value of the parameter is 0 is given by the loss function L(d, 0).
We shall define a few standard terms used in statistical decision theory.
Risk. The expected loss EyL(d, 0) for which the expectation is taken with respect to у (which is implicitly in the argument of the function d) is called the risk and is denoted by jR(d|0).
Uniformly smaller risk. The estimator2 d, has a uniformly smaller risk than the estimator d2 if R(<Ai ) Ш i?(d2|0) for all 0 Є 0 and R(&i ) < R(i2 ) for at least one 0 Є 0.
Admissible. An estimator is admissible if there is no d in D that has a uniformly smaller risk. Otherwise it is called inadmissible.
Minimax. The estimator d* is called a minimax estimator if
max /?(d*|0) = min max i?(d|0).
0ЄЄ ІЄЛ 0ЄЄ
The minimax estimator protects the statistician against the worst possible situation. If maxeee. R(d|0) does not exist, it should be replaced with supeee 7?(d|0) in the preceding definition (and min with inf).
Posterior risk. The expected loss EeL{&, 0) for which the expectation is taken with respect to the posterior distribution of в given у is called the posterior risk and is denoted by 7?(d|y). It obviously depends on the particular prior distribution used in obtaining the posterior distribution.
Bayes estimator. The Bayes estimator, given a particular prior distribution, minimizes the posterior risk 7?(d|y). If the loss function is quadratic, namely, L(d, 0) = (d — 0)' W(d — 0) where W is an arbitrary nonsingular matrix, the posterior risk Ев{й — 0)'W(d — 0) is minimized at d = Ee0, the posterior mean of 0. An example of the Bayes estimator was given in Section 1.4.4.
Regret. Let i?(d|0) be the risk. Then the regret W(d|0) is defined by
Щd|0) = 7?(d|0) - min i?(d|0).
лев
Minimax regret. The minimax regret strategy minimizes maxeee lF(d|0) with respect to d.
Some useful results can be stated informally as remarks rather than stating them formally as theorems.
Remark 2.1.1. A Bayes estimator is admissible.
Remark 2.1.2. A minimax estimator is either a Bayes estimator or the limit of a sequence of Bayes estimators. The latter is called a generalized Bayes estimator. (In contrast, a Bayes estimator is sometimes called a proper Bayes estimator.)
Remark 2.1.3. A generalized Bayes estimator with a constant risk is minimax.
Remark 2.1.4. An admissible estimator may or may not be minimax, and a minimax estimator may or may not be admissible.
