When you go into a pharmacy to purchase a drug, you have confidence that the drug is (a) safe and (b) does what the packaging claims it does. The reason you can be confident is that pharmaceutical companies are obliged to run scientifically controlled studies to make sure that any drug that goes on sale meets those two crucial requirements.
When you purchase an educational software package, however, you cannot be so confident. The developer may claim to have conducted studies that support its claims, and in some cases that is true. But good scientific studies are expensive and take time (often several months or a year or more), so in many cases the claim is little more than marketing hype. “So what?” you may think, “the worse that can happen is that my child does not master fractions after using this app. It’s not like the drug industry, where lives may be at stake.”
True, using a poor educational product won’t kill or maim your child, but it could still inflict life changing harm. A product that leaves a child thinking they suck at math and will never get it could, if not countered, result in a whole range of future careers (say in science or technology) being inaccessible. We tend to overlook harm done to people’s minds because we can’t actually see the damage. But it can be no less life altering than a drug that leaves a child paralyzed in one arm.
Until recently, these was little anyone could do about this state of affairs. Though education researchers had been conducting scientific studies for many decades, they were for the most part on a fairly small scale, one or a handful of classes in a small number of schools, and they were riddled with many factors that could confound the results. Not the least of those confounding factors is the danger that what is measured may be the teacher as much as, or more than, the students.
Another potential confounder is the Hawthorne effect, an observable phenomenon that people’s behavior and performance is affected simply by being part of a study. This can be countered to some extent by using comparison groups in the study, which immediately doubles the study size. But in education research, comparison groups introduce other confounding factors, and at the very least require that the comparison group be taught by the same teacher (ideally at roughly the same time of day) as the group receiving the intervention.
To take another example, news articles often cite the large number of successful people who as children attended a Montessori school, a figure hugely disproportionate to the relatively small number of such schools. Now, it may well be the case that the Montessori educational principles are good, but it’s also true that such schools are magnets for passionate, dedicated teachers and the pupils that attend them do so because they have parents who go out of their way to enroll their offspring in such a school, and already raise their children in a learning-rich home environment.
In the case of mathematics (my area), still another, highly significant problem faces researcher if the if the students’ performance is measured by answering quiz questions that involve entering answers that are either right or wrong. What such tests miss is the thinking process that led to the answer. Just as the doctor taking your temperature can indicate you are sick, but won’t tell you what illness you have, so too the grade you are awarded from taking a multiple-choice or single-numerical answer quiz can provide crude information about what you have learned but gives little insight into how your thinking or problem solving capacity has improved.
Until recently, the only way to get that kind of deep measurement was for a trained expert to sit alongside the student and talk with them at length as the student worked through various test problems. On some occasions, that hugely labor intensive process led to a discovery that a student thought to be doing well was in fact doing very badly, a classic example being known colloquially as Benny’s Rules.
Nevertheless, learning science has progressed, which is why today’s education community can make recommendations with some degree of confidence. But by far the biggest leap forward is just getting underway.
Digital technology, including Internet connectivity, offers an opportunity to carry out medical-research-like, large-scale, control-group studies of classroom learning that can significantly mitigate the “teacher effect” the “home effect”, and various other confounders, allowing useful studies of different educational techniques to be carried out. By collecting the right data, Big Data analytic techniques can detect patterns that cut across the wide range of teacher-teacher and family-family variation, allowing useful educational conclusions to be drawn.
Even more exciting, if a sufficiently significant part of the actual learning is done in a digital environment, where every action can be captured, and if students’ progress is measured, not by a quiz at the end, but by their actions throughout the learning process—all of which is possible for a many topics in my subject of mathematics—then we are one step closer to that ideal of the at-length, expert–student interaction, and one step further removed from falling victim to Benny’s Rules.
If all of this sounds to you like scientists talking to one another behind ivy-covered walls in research universities, then think again. Yes, there is a lot of that talking going on. But the results are finding their way into educational products you can buy now, with many more on the way.
Last year, Digital Promise, an independent, bipartisan nonprofit organization, authorized by Congress in 2008, reported on its first campaign to identify companies that are leading the way in building products based on, and being validated by, scientific research. I was very proud that my own early-stage education technology company, BrainQuake, came out top in one of the three categories.
The government’s commitment to getting solid research into educational technology development can also be seen in the Small Business Innovation Research (SBIR) program run by the US Department of Education’s Institute of Education Sciences, which I described in a previous Huffington Post article.
Learning research does not get the same degree of media coverage as, say, medical research or space exploration or the development of new electronic devices. Yet it can, has, does, and will affect every single child in our school system. A nation that does not keep abreast of developments in learning science is a nation that will fall behind nations that do—or end up having to import critical talent from other countries. Traditionally, the USA has maintained its lead in global innovation by doing both. We would be wise to continue along that path.
Some of this article is adapted from a more detailed account of the new science of learning I published on my blog profkeithdevlin.org.