"Campbell's Law"

I just finished the chapter on accountability in The Death and Life of the Great American School System. So many times I found myself nodding vigorously...wanting to highlight every word...It's just comforting to know there's research that supports what I've been thinking about testing all along: standardized tests are flawed and limited, they're not intended to be used as the sole measure on which important decisions are based, there are too many confounding variables that affect test scores (i.e., language background, socioeconomic status, student motivation, parental engagement, attendance...just to name a few), and test-prep is taking over classrooms across America...superseding health, physical education, the arts, literature, government, history, and science. We're producing a generation of students who can (maybe) perform well on standardized, multiple choice tests in reading and math. Period. And we're calling that an education.

Ravitch cites Campbell's Law: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social process it is intended to monitor." (italics mine) In other words, "When faced with demands to satisfy a single measure, people strive to satisfy that measure but neglect the other, perhaps more important goals of the organization." This phenomenon is known as goal-distortion...and Ravitch (drawing on the work of Harvard psychometrician Daniel Koretz) gives several examples from different disciplines...When the state of New York started issuing scorecards reporting mortality rates for surgeons, most cardiologists stopped performing surgery on critically ill patients to avoid getting a bad score. As a result some patients who might have survived were turned away. Also, when the airline industry was required to report on-time arrivals, they manipulated the expected duration of flights to compensate. As a result the on-time statistics became meaningless. Ravitch and Koretz believe this is what NCLB, testing, and "punitive accountability" are doing to education. Ravitch states, "The pressure to increase test scores is likely to produce higher scores, whether by coaching or cheating or manipulating the pool of test takers. As long as the state or district continues to report good news about student performance, the public seems satisfied, and the media usually sees no reason to investigate whether the gains are real...or meaningful." Fascinating!

Ravitch goes on to say, "Accountability as we know it now is not helping our schools. Its measures are too narrow and imprecise, and its consequences too severe." She says a good system would include other measures besides test scores...things like grades, teacher evaluations, student work, attendance, and graduation rates. A good system would also take into account what schools and districts are providing in terms of resources, class sizes, well-educated teachers, and a well-rounded curriculum. It would also include regular external inspections conducted by trained observers. In Ravitch's system, a low-scoring school would not be closed down, disrupting hundreds of lives, with the hopes that a new and improved school would spring up in its place. Instead, that school would receive additional support. Decisive and consistent steps would be taken to see it improve. Closing a school would be a worst-case scenario...an epic failure on the part of not only the people who work in that building, but also on every district official.

So...I'm wondering...what do we have to do to make that a reality???