Measuring Up: What Educational Testing Really Tells Us

by Daniel Koretz

Overview

How do you judge the quality of a school, a district, a teacher, a student? By the test scores, of course. Yet for all the talk, what educational tests can and can’t tell you, and how scores can be misunderstood and misused, remains a mystery to most. The complexities of testing are routinely ignored, either because they are unrecognized, or because they may be—well, complicated.

Inspired by a popular Harvard course for students without an extensive mathematics background, Measuring Up demystifies educational testing—from MCAS to SAT to WAIS, with all the alphabet soup in between. Bringing statistical terms down to earth, Daniel Koretz takes readers through the most fundamental issues that arise in educational testing and shows how they apply to some of the most controversial issues in education today, from high-stakes testing to special education. He walks readers through everyday examples to show what tests do well, what their limits are, how easily tests and scores can be oversold or misunderstood, and how they can be used sensibly to help discover how much kids have learned.

Product Details

ISBN-13:	9780674035218
Publisher:	Harvard University Press
Publication date:	09/15/2009
Pages:	368
Product dimensions:	5.40(w) x 8.10(h) x 1.10(d)

About the Author

Daniel Koretz is Professor of Education at Harvard University.

Table of Contents

Prologue     1
If Only It Were So Simple     5
What Is a Test?     16
What We Measure: Just How Good Is the Sample?     35
The Evolution of American Testing     46
What Test Scores Tell Us about American Kids     74
What Influences Test Scores, or How Not to Pick a School     113
Error and Reliability: How Much We Don't Know What We're Talking About     143
Reporting Performance: Standards and Scales     179
Validity     215
Inflated Test Scores     235
Adverse Impact and Bias     260
Testing Students with Special Needs     281
Sensible Uses of Tests     315
Notes     335
Acknowledgments     345
Index     347

What People are Saying About This

Here we are, lost in Testland, bombarded by data about how well or poorly we or our kids have done on the latest exam. What do test results mean? Every expert has a different explanation. What to do? Read Daniel Koretz's new book, as soon as possible. Never have I seen a clearer or more sensible exploration of our testing frenzy. I thought one chapter, "What Influences Test Scores, or How Not to Pick a School," was all by itself worth the price of the book. Read it and relax.

Jay Mathews

Here we are, lost in Testland, bombarded by data about how well or poorly we or our kids have done on the latest exam. What do test results mean? Every expert has a different explanation. What to do? Read Daniel Koretz's new book, as soon as possible. Never have I seen a clearer or more sensible exploration of our testing frenzy. I thought one chapter, "What Influences Test Scores, or How Not to Pick a School," was all by itself worth the price of the book. Read it and relax. --(Jay Mathews, Washington Post education reporter and columnist)

H.D. Hoover

This is the most easily understood presentation I know of the deceptively complex world of educational testing, and the most important current issues. It should be welcomed with relief by a very broad audience, much of which is ignored in most presentations on testing. I would love to see it used in courses for virtually all future administrators, policy makers, and teachers. Anyone directing testing programs in school districts and states will find this invaluable when they have to explain what they're doing. This book is badly needed. --(H.D. Hoover, Professor Emeritus, University of Iowa)

Editorial Reviews

Test scores are objective, scientific, and easy to understand—so what's the problem? It turns out that there are a lot of problems and that we would do well to try and understand them better. Daniel Koretz's Measuring Up is an excellent place to start. The book is hard to classify. It is too sophisticated to be called a primer. There are no equations, so it can't be a measurement book. (Also, it is entertaining to read.) It says good things about testing and test use and takes apart some arguments of testing opponents, so it can't be an anti-testing book. But, it raises profound challenges to the interpretation of score trends on high-stakes tests, to the meaning of achievement trend and gap reports in terms of percent proficient, to the interpretation of crossnational achievement comparisons, and to popular assumptions about testing of students in special populations (including some assumptions written into law). So, it can't be a protesting book, either...He does a great service by clarifying measurement principles in the context of widespread testing uses and misuses.

Science - Edward Haertel

The best explanation of standardized testing is Daniel Koretz's Measuring Up: What Educational Testing Really Tells Us.

New York Review of Books - Diane Ravitch

Deconstructs the complexities of achievement testing for the educational layman.

Education Week

Here we are, lost in Testland, bombarded by data about how well or poorly we or our kids have done on the latest exam. What do test results mean? Every expert has a different explanation. What to do? Read Daniel Koretz's new book, as soon as possible. Never have I seen a clearer or more sensible exploration of our testing frenzy. I thought one chapter, "What Influences Test Scores, or How Not to Pick a School," was all by itself worth the price of the book. Read it and relax.

Jay Mathews

This is the most easily understood presentation I know of the deceptively complex world of educational testing, and the most important current issues. It should be welcomed with relief by a very broad audience, much of which is ignored in most presentations on testing. I would love to see it used in courses for virtually all future administrators, policy makers, and teachers. Anyone directing testing programs in school districts and states will find this invaluable when they have to explain what they're doing. This book is badly needed.

H.D. Hoover

Koretz has written the book on educational testing most educators and educational policy makers have been waiting for, even if they don't know it. In a culture defined by whether one is attacking or defending the messenger, the author's endeavor is to explain what educational testing does, and does not, reveal about how students and their schools are performing...For someone looking for a good lay explanation of essential topics such as score reliability and validity, measurement error, and the relationship between high-stakes testing and score inflation, this is the book. The style is eminently readable and the topics are profoundly important.

Choice - D. E. Tanner

Every parent who uses league tables as a basis for placing his or her child in a school, whether in the U.S. or anywhere else, should read this book.

Times Higher Education Supplement - Lee Harvey

Here we are, lost in Testland, bombarded by data about how well or poorly we or our kids have done on the latest exam. What do test results mean? Every expert has a different explanation. What to do? Read Daniel Koretz's new book, as soon as possible. Never have I seen a clearer or more sensible exploration of our testing frenzy. I thought one chapter, "What Influences Test Scores, or How Not to Pick a School," was all by itself worth the price of the book. Read it and relax.--Jay Mathews, Washington Post education reporter and columnist

Deconstructs the complexities of achievement testing for the educational layman.-- "Education Week" (5/21/2008 12:00:00 AM)

Every parent who uses league tables as a basis for placing his or her child in a school, whether in the U.S. or anywhere else, should read this book.--Lee Harvey "Times Higher Education Supplement" (8/28/2008 12:00:00 AM)

Koretz has written the book on educational testing most educators and educational policy makers have been waiting for, even if they don't know it. In a culture defined by whether one is attacking or defending the messenger, the author's endeavor is to explain what educational testing does, and does not, reveal about how students and their schools are performing...For someone looking for a good lay explanation of essential topics such as score reliability and validity, measurement error, and the relationship between high-stakes testing and score inflation, this is the book. The style is eminently readable and the topics are profoundly important.--D. E. Tanner "Choice" (12/1/2008 12:00:00 AM)

Test scores are objective, scientific, and easy to understand--so what's the problem? It turns out that there are a lot of problems and that we would do well to try and understand them better. Daniel Koretz's Measuring Up is an excellent place to start. The book is hard to classify. It is too sophisticated to be called a primer. There are no equations, so it can't be a measurement book. (Also, it is entertaining to read.) It says good things about testing and test use and takes apart some arguments of testing opponents, so it can't be an anti-testing book. But, it raises profound challenges to the interpretation of score trends on high-stakes tests, to the meaning of achievement trend and gap reports in terms of percent proficient, to the interpretation of crossnational achievement comparisons, and to popular assumptions about testing of students in special populations (including some assumptions written into law). So, it can't be a protesting book, either...He does a great service by clarifying measurement principles in the context of widespread testing uses and misuses.--Edward Haertel "Science" (1/2/2009 12:00:00 AM)

The best explanation of standardized testing is Daniel Koretz's Measuring Up: What Educational Testing Really Tells Us.--Diane Ravitch "New York Review of Books" (3/8/2012 12:00:00 AM)

This is the most easily understood presentation I know of the deceptively complex world of educational testing, and the most important current issues. It should be welcomed with relief by a very broad audience, much of which is ignored in most presentations on testing. I would love to see it used in courses for virtually all future administrators, policy makers, and teachers. Anyone directing testing programs in school districts and states will find this invaluable when they have to explain what they're doing. This book is badly needed.--H.D. Hoover, Professor Emeritus, University of Iowa

From the Publisher