Achievement and Rats

Student achievement -- in American education parlance the term has become synonymous with "test scores." We measure the progress of schools, states, and even the nation using test scores. We judge the success of programs and policies from scores (as the only satisfactory evidence). We evaluate teachers using scores of their students, or more often esoteric statistics based on them. This narrowly prescribed definition of student achievement is hard-wired into all parts of American education, making us unique among nations.

Relabeling test scores as student achievement was brilliant marketing. Who can be against achievement? Indeed, this narrow definition of student achievement is employed by all sides of reform, including those who oppose tests (but who can't help citing test scores when the opportunity comes along to bash an opponent).

It wasn't always this way. Student achievement used to mean more than test scores. In Ancient Greece, achievement was sophisticated--education prepared leaders, socialized people, promoted ethics, taught discipline, engendered curiosity and the love of knowledge. The Enlightenment produced intellectuals such as John Locke and Jean Jacques Rousseau who promoted universal education and defined achievement broadly because of this. In the twentieth century, progressive educators like John Dewey promoted student-centered education, for which achievement was individual to each student and meant not only knowledge but understanding as well. And throughout the twentieth century, scholars refined what achievement should mean, for example, Bloom's cognitive taxonomy of six skill levels--knowledge, comprehension, application, analysis, synthesis, and evaluation; few of these are measured by tests. Until the past few decades, tests were used as invaluable tools, to diagnose and provide feedback, to teachers and students alike. Achievement and test scores were not the same.

Policy makers honed the new definition of achievement with good intentions--to weed out weak teachers, to close failing schools, to foster only reform that worked, and generally to make education accountable. They saw data from tests as a potent weapon to accomplish these things. But that redefinition has had unintended consequences.

In the past few years, large-scale cheating has been reported in more than 37 states and the District of Columbia. Tests cover only a portion of what students learn, and if the stakes are high, teachers and administrators will focus only on that portion. The craft of teaching is complicated--the Socratic Method, student-centered classrooms, active learning--but when higher test scores are the only goal the craft is reduced to test-prep.

Education research has been profoundly affected by narrowing achievement's meaning. Here is an excerpt from a recent paper by a well-known education researcher.

In comparison to classrooms of students elsewhere with similar baseline achievement and demographics, a teacher's achievement gain in one year is correlated at a rate of .48 in math and .36 in English language arts (ELA), with the average growth of students in another year. Such volatility notwithstanding, a track record of achievement gains is a more reliable predictor of the gains of future students than classroom observations or student surveys.
Thomas Kane, 2012

Of course, this is a statement about test scores, not education. Research used to be driven by analysis and philosophy; today it has become arcane computations.

Reform is affected as well. One can improve education without immediately improving test scores. Fostering creativity and imagination may not change scores, for example. Cultivating better attitudes about a subject may have no effect on scores. Inspiring students to be life-long learners will pay off, but perhaps not this year or even next, and it won't necessarily change scores. Judging reform by test scores alone impoverishes education.

Even aspirations are diminished. Einstein is famous for promoting curiosity combined with critical thinking--what he refers to as "imagination."

Imagination is more important than knowledge. For knowledge is limited, whereas imagination embraces the entire world, stimulating progress, giving birth to evolution. It is, strictly speaking, a real factor in scientific research.
Albert Einstein, Cosmic Religion, p. 97, 1931

You don't measure imagination with tests. The new definition of achievement makes aspirations shallow, dull, and uninspiring.

But the most serious effect of redefining achievement is this: Teachers are demoralized and the more accomplished more so. Accomplished teachers--the ones who know their subject and craft most deeply--recognize the damage done to their students by narrowing achievement's meaning. Inexperienced or indifferent teachers may be willing to convert their classrooms into test-prep, but accomplished teachers are not. Eventually they leave. When they do, we lose not only those great teachers but future great teachers as well.

Increased cheating, a narrowed curriculum, limited pedagogy, arcane research, impoverished reform, diminished aspirations, and demoralized teachers--these are the consequences of redefining achievement. The goal of the new definition was to make education more accountable, but tests measure only certain things and do not measure many others. The "others" are not merely frills--they are the heart of education.

In colonial Vietnam, the city of Hanoi was overrun by rats. The French set out to solve the problem by offering monetary rewards to kill the rats. People had only to submit the tails as evidence. But rat catchers soon realized that larger rat populations meant larger profits, and since tailless rats reproduced as well as others, catchers set the rats free after cutting off their tails. The number of rats (many sans tails) grew even more rapidly, making things worse. Eventually, the program was stopped.

There is a lesson here.