Columbus and Replicability


Happy Columbus Day!

Columbus is revered among researchers because:
  1. He didn't know where he was going;
  2. He didn't know where he was when he got there; and
  3. He did it all on government money.

Columbus gets a lot of abuse these days, and for good reason. He was a terrible person. However, people also say that he didn't actually discover America. Leif Erikson had been here earlier, they say, and of course the Indians were already here.

What Columbus did discover was not America per se, but a replicable and openly published route to America. And that's what made him justifiably famous. In research, as in discovery, what matters is replicability, the ability to show that you can do something again, and to tell others how they can do the same. Columbus was indisputably the first to do that (Leif Erikson kept his voyage secret).

Replicability is the hallmark of science. In science, if you can't do it again, it didn't happen. In fact, there is a popular science humor magazine called the Journal of Irreproducible Results, named for this principle.

As important as replication is in all of science, it is rare in educational research. It's difficult to get funding to do replications, and if you manage to replicate a finding, journal editors are likely to dismiss it ("What does this add to the literature?" they say). Yet as evidence-based reform in education advances, the need for replication increases. This is a problem because, for example, the majority of programs with at least one study that met What Works Clearinghouse standards had exactly one study that did so.

Soon, results will become available for the first and largest cohort of projects funded by the Investing in Innovation (i3) program. Some of these will show positive effects and some will show outcomes close enough to significance to be worth trying again. I hope there will be opportunities for these programs to replicate and hopefully improve their outcomes, so we can expand our armamentarium of replicable and effective approaches to enduring problems of education.

We really should celebrate Columbus Day on November 3rd when Columbus returned to the New World. The day he reached the New World was a significant event, but it wasn't really important until he showed that he (and anyone else) could do it again.