INTRODUCTION TO STATISTICS AND ECONOMETRICS
WHAT IS PROBABILITY?
As a common word in everyday usage, probability expresses a degree of belief a person has about an event or statement by a number between zero and one. Probability also has a philosophical meaning, which will not be discussed here. The two major schools of statistical inference—the Bayesian and the classical—hold two different interpretations of probability. The Bayesian (after Thomas Bayes, an eighteenth-century English clergyman and probabilist) interpretation of probability is essentially that of everyday usage. The classical school refers to an approach that originated at the beginning of the twentieth century under the leadership of R. A. Fisher and is still prevalent. The classical statistician uses the word probability only for an event which can be repeated, and interprets it as the limit of the empirical frequency of the event as the number of repetitions increases indefinitely. For example, suppose we toss a coin n times, and a head comes up r times. The classical statistician interprets the probability of heads as a limit (in the sense that will be defined later) of the empirical frequency r/n as n goes to infinity. Because a coin cannot be tossed infinitely many times, we will never know this probability exactly and can only guess (or estimate) it.
To consider the difference between the two interpretations, examine the following three events or statements:
(1) A head comes up when we toss a particular coin.
(2) Atlantis, described by Plato, actually existed.
(3) The probability of obtaining heads when we toss a particular coin is У2.
A Bayesian can talk about the probability of any one of these events or statements; a classical statistician can do so only for the event (1), because only (1) is concerned with a repeatable event. Note that (1) is sometimes true and sometimes false as it is repeatedly observed, whereas statement (2) or (3) is either true or false as it deals with a particular thing—one of a kind. It may be argued that a frequency interpretation of (2) is possible to the extent that some of Plato’s assertions have been proved true by a later study and some false. But in that case we are considering any assertion of Plato’s, rather than the particular one regarding Atlantis.
As we shall see in later chapters, these two interpretations of probability lead to two different methods of statistical inference. Although in this book I present mainly the classical method, I will present Bayesian method whenever I believe it offers more attractive solutions. The two methods are complementary, and different situations call for different methods.
