What if Evidence Doesn't Match Ideology?


Photo by Michael Hummel

Several years ago when the Conservative Party was first coming into office in the U.K., I had an opportunity to meet with a High Government Official. He had been told that I was a supporter of phonics in early reading, and that was what he wanted to talk about. We chatted amicably for some time about our agreement on this topic.

Then the Great Man turned to another topic. What did I think about the evidence on ability grouping?

I explained that the evidence did not favor ability grouping, and was about to explain why when he cut me off with the internationally understood gesture meaning, "I'm a very busy and important person. Get out of my office immediately." Ever since then, the British government has gotten along just fine without my advice.

What the Great Man was telling me, of course, is the depressing reality of why it is so difficult to change policy or practice with evidence. Most people value research when it supports the ideological position they already had, and reject research when it does not. The result is that policy and practice remain an ideological struggle, little influenced by the actual findings of research. Advocates of a given position seek evidence to throw at their opponents or to defend themselves from evidence thrown at them by the "other side." And we all too often evaluate evidence based on the degree to which it corresponds to our pre-existing beliefs rather than re-evaluating our beliefs in light of evidence. I recall that at a meeting of Institute of Education Sciences (IES) grantees, a respected superintendent spoke to the whole assemblage and, entirely without irony or humor, defined good research as that which confirms his beliefs, and bad research as that which contradicts his beliefs.

A scientific field only begins to move forward when researchers and users of the research come to accept research findings whether or not they support their previous beliefs. Not that this is easy. Even in the most scientific of fields, it usually takes a great deal of research over an extended time period to replace a widely accepted belief with a contrary set of findings. In the course of unseating the old belief, researchers who dare to go against the current orthodoxy have difficulty finding an audience, funding, promotions, or respect, so it's a lot easier to go with the flow. Yet true sciences do change their minds based on evidence, even if they must often be dragged kicking and screaming to the altar of knowledge. One classic example I've heard of involved the bacterial origin of gastric ulcers. Ulcers were once thought to be caused by stress, until an obscure Australian researcher deliberately gave himself an ulcer by drinking a solution swarming with gastric bacteria. He then cured himself with a drug known to kill those bacteria. Today, the stress theory is gone and the bacteria theory is dominant, but it wasn't easy.

Education researchers are only just beginning to have enough confidence in our own research to expect policy makers, practitioners, and other researchers to change their beliefs on the basis of evidence. Yet education will not be an evidence-driven field until evidence begins to routinely change beliefs about what works for students and what does not. We need to change thinking not only about individual programs or principles, but about the role of evidence itself. This is one reason that it is so important that research in education be of impeccable quality, so that we can have confidence that findings will replicate in future studies and will generalize to many practical applications.

A high government official in health would never dismiss research on gastric ulcers because he or she still believed that ulcers are caused by stress. A high government official in agriculture would never dismiss research on the effects of certain farming methods on soil erosion. In the U.S., at least, our Department of Education has begun to value evidence and to encourage schools to adopt proven programs and practices, but there is a long way to go before education joins medicine and agriculture in willingness to recognize and promote findings of rigorous and replicated research. We're headed in the right direction, but I have to admit that the difficulties getting there are giving me one heck of an ulcer.*

*Just kidding. I'm fine.

This blog is sponsored by the Laura and John Arnold Foundation