eBook
Available on Compatible NOOK devices, the free NOOK App and in My Digital Library.
Related collections and offers
Overview
Product Details
| ISBN-13: | 9780486137100 |
|---|---|
| Publisher: | Dover Publications |
| Publication date: | 08/29/2012 |
| Series: | Dover Books on Mathematics |
| Sold by: | Barnes & Noble |
| Format: | eBook |
| Pages: | 352 |
| File size: | 9 MB |
Read an Excerpt
The Foundations of Statistics
By Leonard J. Savage
Dover Publications, Inc.
Copyright © 1972 Dover Publications, Inc.All rights reserved.
ISBN: 978-0-486-13710-0
CHAPTER 1
Introduction
1 The role of foundations
It is often argued academically that no science can be more secure than its foundations, and that, if there is controversy about the foundations, there must be even greater controversy about the higher parts of the science. As a matter of fact, the foundations are the most controversial parts of many, if not all, sciences. Physics and pure mathematics are excellent examples of this phenomenon. As for statistics, the foundations include, on any interpretation of which I have ever heard, the foundations of probability, as controversial a subject as one could name. As in other sciences, controversies over the foundations of statistics reflect themselves to some extent in everyday practice, but not nearly so catastrophically as one might imagine. I believe that here, as elsewhere, catastrophe is avoided, primarily because in practical situations common sense generally saves all but the most pedantic of us from flagrant error. It is hard to judge, however, to what extent the relative calm of modern statistics is due to its domination by a vigorous school relatively well agreed within itself about the foundations.
Although study of the foundations of a science does not have the role that would be assigned to it by naive first-things-firstism, it has a certain continuing importance as the science develops, influencing, and being influenced by, the more immediately practical parts of the science.
2 Historical background
The concept and problem of inductive inference have been prominent in philosophy at least since Aristotle. Mathematical work on some aspects of the problem of inference dates back at least to the early eighteenth century. Leibniz is said to be the first to publish a suggestion in that direction, but Jacob Bernoulli's posthumous Ars Conjectandi (1713) [B12] seems to be the first concerted effort. This mathematical work has always revolved around the concept of probability; but, though there was active interest in probability for nearly a century before the publication of Ars Conjectandi, earlier activity seems not to have been concerned with inductive inference.
In the present century there has been and continues to be extraordinary interest in mathematical treatment of problems of inductive inference. For reasons I cannot and need not analyze here, this activity has been strikingly concentrated in the English-speaking world. It is known under several names, most of which stress some aspect of the subject that seemed of overwhelming importance at the moment when the name was coined. "Mathematical statistics," one of its earliest names, is still the most popular. In this name, "mathematical" seems to be intended to connote rational, theoretical, or perhaps mathematically advanced, to distinguish the subject from those problems of gathering and condensing numerical data that can be considered apart from the problem of inductive inference, the mathematical treatment of which is generally relatively trivial. The name "statistical inference" recognizes that the subject is concerned with inductive inference. The name "statistical decision" reflects the idea that inductive inference is not always, if ever, concerned with what to believe in the face of inconclusive evidence, but that at least sometimes it is concerned with what action to decide upon under such circumstances. Within this book, there will be no harm in adopting the shortest possible name, "statistics."
It is unanimously agreed that statistics depends somehow on probability. But, as to what probability is and how it is connected with statistics, there has seldom been such complete disagreement and breakdown of communication since the Tower of Babel. There must be dozens of different interpretations of probability defended by living authorities, and some authorities hold that several different interpretations may be useful, that is, that the concept of probability may have different meaningful senses in different contexts. Doubtless, much of the disagreement is merely terminological and would disappear under sufficiently sharp analysis. Some believe that it would all disappear, or even that they have themselves already made the necessary analysis.
Considering the confusion about the foundations of statistics, it is surprising, and certainly gratifying, to find that almost everyone is agreed on what the purely mathematical properties of probability are. Virtually all controversy therefore centers on questions of interpreting the generally accepted axiomatic concept of probability, that is, of determining the extramathematical properties of probability.
The widely accepted axiomatic concept referred to is commonly ascribed to Kolmogoroff [K7] and goes by his name. It should be mentioned that there is some dissension from it on the part of a small group led by von Mises [V2]. There are also a few minor technical variations on the Kolmogoroff system that are sometimes of interest; they will be discussed in § 3.4.
I would distinguish three main classes of views on the interpretation of probability, for the purposes of this book, calling them objectivistic, personalistic, and necessary. Condensed descriptions of these three classes of views seem called for here. If some readers find these descriptions condensed to the point of unintelligibility, let them be assured that fuller ones will gradually be developed as the book proceeds.
Objectivistic views hold that some repetitive events, such as tosses of a penny, prove to be in reasonably close agreement with the mathematical concept of independently repeated random events, all with the same probability. According to such views, evidence for the quality of agreement between the behavior of the repetitive event and the mathematical concept, and for the magnitude of the probability that applies (in case any does), is to be obtained by observation of some repetitions of the event, and from no other source whatsoever.
Personalistic views hold that probability measures the confidence that a particular individual has in the truth of a particular proposition, for example, the proposition that it will rain tomorrow. These views postulate that the individual concerned is in some ways "reasonable," but they do not deny the possibility that two reasonable individuals faced with the same evidence may have different degrees of confidence in the truth of the same proposition.
Necessary views hold that probability measures the extent to which one set of propositions, out of logical necessity and apart from human opinion, confirms the truth of another. They are generally regarded by their holders as extensions of logic, which tells when one set of propositions necessitates the truth of another.
After what has been said about the intensity and complexity of the controversy over the probability concept, you must realize that the short taxonomy above is bound to infuriate any expert on the foundations of probability, but I trust it may do the less learned more good than harm.
The great burst of statistical research in the English-speaking world in the present century has revolved around objectivistic views on the interpretation of probability. As will shortly be explained, any purely objectivistic view entails a severe difficulty for statistics. This difficulty is recognized by members of the British-American School, if I may use that name without its being taken too literally or at all nationalistically, and is regarded by them as a great, though not insurmountable, obstacle; indeed, some of them see it as the central problem of statistics.
The difficulty in the objectivistic position is this. In any objectivistic view, probabilities can apply fruitfully only to repetitive events, that is, to certain processes; and (depending on the view in question) it is either meaningless to talk about the probability that a given proposition is true, or this probability can be only 1 or 0, according as the proposition is in fact true or false. Under neither interpretation can probability serve as a measure of the trust to be put in the proposition. Thus the existence of evidence for a proposition can never, on an objectivistic view, be expressed by saying that the proposition is true with a certain probability. Again, if one must choose among several courses of action in the light of experimental evidence, it is not meaningful, in terms of objective probability, to compute which of these actions is most promising, that is, which has the highest expected income. Holders of objectivistic views have, therefore, no recourse but to argue that it is not reasonable to assign probabilities to the truth of propositions or to calculate which of several actions is the most promising, and that the need expressed by the attempt to set up such concepts must be met in other ways, if at all.
The British-American School has had great success in several respects. The number of its adherents has rapidly increased. It has contributed many procedures of strong intuitive appeal and (one feels) of lasting worth. These have found widespread application in many sciences, in industry, and in commerce. The success of the school may pragmatically be taken as evidence for the correctness of the general view on which it is based. Indeed, anyone who overthrows that view must either discredit the procedures to which it has led, or show, as I hope to show in this book, that they are on the whole consistent with the alternative proposed.
Some, I among them, hold that the grounds for adopting an objectivistic view are not overwhelmingly strong; that there are serious logical objections to any such view; and, most important of all, that the difficulty a strictly objectivistic view meets in statistics reflects real inadequacy.
3 General outline of this book
This book presents a theory of the foundations of statistics which is based on a personalistic view of probability derived mainly from the work of Bruno de Finetti, as expressed for example in [D2]. The theory is presented in a tentative spirit, for I realize that the serious blemishes in it apparent to me are not the only ones that will be discovered by critical readers. A theory of the foundations of statistics that appears contrary to the teaching of the most productive statisticians will properly be regarded with extraordinary caution. Other views on probability will, of course, be discussed in this book, partly for their own interest and partly to explain the relationship between the personalistic view on which this book is based and other views.
The book is organized into seventeen chapters, of which the present introduction is the first. Chapters 2–7 are, so to speak, concerned with the foundations at a relatively deep level. They develop, explain, and defend a certain abstract theory of the behavior of a highly idealized person faced with uncertainty. That theory is shown to have as implications a theory of personal probability, corresponding to the personalistic view of probability basic to this book, and also a theory of utility due, in its modern form, to von Neumann and Morgenstern [V4].
There is a transition, occurring in Chapter 8 and maintained throughout the rest of the book, to a shallower level of the foundations of statistics; I might say from pre-statistics to statistics proper. In those later chapters, it is recognized that the theory developed in the earlier ones is too highly idealized for immediate application. Some compromises have to be made, and the appropriate ones are sought in an analysis of some of the inventions and ideas of the British-American School. It will, I hope, be demonstrated thereby that the superficially incompatible systems of ideas associated on the one hand with a personalistic view of probability and on the other with the objectivistically inspired developments of the British-American School do in fact lend each other mutual support and clarification.
CHAPTER 2Preliminary Considerations on Decision in the Face of Uncertainty
1 Introduction
Decisions made in the face of uncertainty pervade the life of every individual and organization. Even animals might be said continually to make such decisions, and the psychological mechanisms by which men decide may have much in common with those by which animals do so. But formal reasoning presumably plays no role in the decisions of animals, little in those of children, and less than might be wished in those of men. It may be said to be the purpose of this book, and indeed of statistics generally, to discuss the implications of reasoning for the making of decisions.
Reasoning is commonly associated with logic, but it is obvious, as many have pointed out, that the implications of what is ordinarily called logic are meager indeed when uncertainty is to be faced. It has therefore often been asked whether logic cannot be extended, by principles as acceptable as those of logic itself, to bear more fully on uncertainty. An attempt to extend logic in this way will be begun in this chapter, differing in two important respects from most, but not all, other attempts.
First, since logic is concerned with implications among propositions, many have thought it natural to extend logic by setting up criteria for the extent to which one proposition tends to imply, or provide evidence for, another. It seems to me obvious, however, that what is ultimately wanted is criteria for deciding among possible courses of action; and, therefore, generalization of the relation of implication seems at best a roundabout method of attack. It must be admitted that logic itself does lead to some criteria for decision, because what is implied by a proposition known to be true is in turn true and sometimes relevant to making a decision. Should some notion of partial implication be demonstrably even better articulated with decision than is implication itself, that would be excellent; but how is such a notion to be sought except by explicitly studying decision? Ramsey's discussion in [R1] of the point at issue here is especially forceful.
Second, it is appealing to suppose that, if two individuals in the same situation, having the same tastes and supplied with the same information, act reasonably, they will act in the same way. Such agreement, belief in which amounts to a necessary (as opposed to a personalistic) view of probability, is certainly worth looking for. Personally, I believe that it does not correspond even roughly with reality, but, having at the moment no strong argument behind my pessimism on this point, I do not insist on it. But I do insist that, until the contrary be demonstrated, we must be prepared to find reasoning inadequate to bring about complete agreement. In particular, the extensions of logic to be adduced in this book will not bring about complete agreement; and whether enough additional principles to do so, or indeed any additional principles of much consequence, can be adduced, I do not know. It may be, and indeed I believe, that there is an element in decision apart from taste, about which, like taste itself, there is no disputing.
The next four sections of this chapter build up a formal model, or scheme, of the situation in which a person is faced with uncertainty; the final two, in terms of this model, motivate and state some of the few principles that seem to me entitled to be taken as postulates for rational decision.
2 The person
I am about to build up a highly idealized theory of the behavior of a "rational" person with respect to decisions. In doing so I will, of course, have to ask you to agree with me that such and such maxims of behavior are "rational." In so far as "rational" means logical, there is no live question; and, if I ask your leave there at all, it is only as a matter of form. But our person is going to have to make up his mind in situations in which criteria beyond the ordinary ones of logic will be necessary. So, when certain maxims are presented for your consideration, you must ask yourself whether you try to behave in accordance with them, or, to put it differently, how you would react if you noticed yourself violating them.
It is brought out in economic theory that organizations sometimes behave like individual people, so that a theory originally intended to apply to people may also apply to (or may even apply better to) such units as families, corporations, or nations. In view of this possibility, economic theorists are sometimes reluctant to use the word "person," or even "individual," for the behaving units to which they refer; but for our purpose "person" threatens no confusion, though the possibility of using it in an extended sense may well be borne in mind.
3 The world, and states of the world
A formal description, or model, of what the person is uncertain about will be needed. To motivate this formal description, let me begin informally by considering a list of examples. The person might be uncertain about:
1. Whether a particular egg is rotten.
2. Which, if any, in a particular dozen eggs are rotten.
3. The temperature at noon in Chicago yesterday.
4. What the temperature was and will be in the place now covered by Chicago each noon from January 1, 1 A.D., to January 1, 4000 A.D.
5. The infinite sequence of heads and tails that will result from repeated tosses of a particular (everlasting) coin.
6. The complete decimal expansion of π.
7. The exact and entire past, present, and future history of the universe, understood in any sense, however wide.
(Continues...)
Excerpted from The Foundations of Statistics by Leonard J. Savage. Copyright © 1972 Dover Publications, Inc.. Excerpted by permission of Dover Publications, Inc..
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
Table of Contents
1. INTRODUCTION1. The role of foundations
2. Historical background
3. General outline of this book
2. PRELIMINARY CONSIDERATIONS ON DECISION IN THE FACE OF UNCERTAINTY
1. Introduction
2. The person
3. "The world, and states of the world"
4. Events
5. "Consequences, acts, and decisions"
6. The simple ordering of acts with respect to preference
7. The sure-thing principle
3. PERSONAL PROBABILITY
1. Introduction
2. Qualitative personal probability
3. Quantitative personal probability
4. Some mathematical details
5. "Conditional probability, qualitative and quantitative"
6. The approach to certainty through experience
7. Symmetric sequences of events
4. CRITICAL COMMENTS ON PERSONAL PROBABILITY
1. Introduction
2. Some shortcomings of the personalistic view
3. Connection with other views
4. Criticism of other views
5. The role of symmetry in probability
6. How can science use a personalistic view of probability?
5. UTILITY
1. Introduction
2. Gambles
3. "Utility, and preference among gambles"
4. The extension of utility to more general acts
5. Small worlds
6. Historical and critical comments on utility
6. OBSERVATION
1. Introduction
2. What an observation is
3. "Multiple observations, and extensions of observations and of sets of acts"
4. Dominance and admissibility
5. Outline of the design of experiments
7. PARTITION PROBLEMS
1. Introduction
2. Structure of (twofold) partition problems
3. The value of observation
4. "Extension of observations, and sufficient statistics"
5. Likelihood ratios
6. Repeated observations
7. Sequential probability ratio procedures
8. "Standard form, and absolute comparison between observations"
8. STATISTICS PROPER
1. Introduction
2. What is statistics proper?
3. Multipersonal problems
4. The minimax theory
9. INTRODUCTION TO THE MINIMAX THEORY
1. Introduction
2. The behavioralistic outlook
3. Mixed acts
4. Income and loss
5. "The minimax rule, and the principle of admissibility"
6. Illustrations of the minimax rule
7. Objectivistic motivation of the minimax rule
8. Loss as opposed to negative income in the minimax rule
10. A PERSONALISTIC REINTERPRETATION OF THE MINIMAX THEORY
1. Introduction
2. A model of group decision
3. "The group minimax rule, and the group principle of admissibility"
4. Critique of the group minimax rule
11. THE PARALLELISM BETWEEN THE MINIMAX THEORY AND THE THEORY OF TWO-PERSON GAMES
1. Introduction
2. Standard games
3. Minimax play
4. Parallelism and contrast with the minimax theories
12. THE MATHEMATICS OF MINIMAX PROBLEMS
1. Introduction
2. Abstract games
3. Bilinear games
4. An example of a bilinear game
5. Bilinear games exhibiting symmetry
13. OJBECTIONS TO THE MINIMAX RULES
1. Introduction
2. A confusion between loss and negative income
3. Utility and the minimax rule
4. Almost sub-minimax acts
5. The minimax rule does not generate a simple ordering
14. THE MINIMAX THEORY APPLIED TO OBSERVATIONS
1. Introduction
2. Recapitulation of partition problems
3. Sufficient statistics
4. "Simple dichotomy, an example"
5. The approach to certainty
6. Cost of observation
7. Sequential probability ratio procedures
8. Randomization
9. Mixed acts in statistics
15. POINT ESTIMATION
1. Introduction
2. The verbalistic concept of point estimation
3. Examples of problems of point estimation
4. Criteria that have been proposed for point estimates
5. A behavioralistic review of the criteria for point estimation
6. "A behavioralistic review, continued"
7. "A behavioralistic review, concluded"
16. TESTING
1. Introduction
2. A theory of testing
3. Testing in practice
17. INTERVAL ESTIMATION AND RELATED TOPICS
1. Estimates of the accuracy of estimates
2. Interval estimation and confidence intervals
3. Tolerance intervals
4. Fiducial probability
APPENDIX 1. EXPECTED VALUE
APPENDIX 2. CONVEX FUNCTIONS
APPENDIX 3. BIBLIOGRAPHIC MATERIAL
APPENDIX 4. BIBLIOGRAPHIC SUPPLEMENT
TECHNICAL SYMBOLS
AUTHOR INDEX
GENERAL INDEX