The first head of the Institute of Education Sciences, Russ Whitehurst set a gold standard for research particularly around the use of randomized controlled trials (RCT). There is obvious benefit to setting a high bar for evidence but we often need better evidence faster and cheaper than is possible through RCT.
For example, an MDRC study released last month was a decade late in reporting what was obvious in 2002 and it missed the point by making the wrong comparison. It compared new schools to other options in New York rather than comparing the new schools to the schools they replaced. They sacrificed efficacy for validity.
I'm optimistic about the shift to personal digital learning not only for the customization and motivation students will experience but for the tools that it will provide to teachers. Big Data will allow interested teachers to become data scientists and participate in large scale experiments in real time.
Ian Eslick shares my concerns about the limitations of RCT. He has been supporting aggregation of self-experiments in health. A growing percentage of adults read health information found on social media and they put it to work. Ian "aims to bridge the massive gap between clinical research and anecdotal evidence by putting the tools of science into the hands of patients." Eslick adds, "A recommender system will aggregate experimental outcomes and background information from many patients to recommend experiments for each individual. Unusual interventions that succeed over many trials become evidence to motivate future clinical research."
Ian is interested in efficacy not validity. He's willing to be wrong for a while but wants to make quick practical progress. He wants to help people find treatments that work for them -- fast. "As individuals we make the best progress when our hypotheses are well founded and the conclusions drawn from our experiments are accurate. As a community we make the best progress when we can share our ideas in ways that accurately predict how someone else will fare."
With the dynamic nature of education technology, it strikes me that encouraging teachers and schools to self-experiment would (in many cases) be a better, faster, cheaper alternative to RCT. A site like Personal Experiments could suggest trials, make connections, and et schools share outcomes.
We need an order of magnitude increase in education R&D. The recent increase in venture capital is overdue. More grant funding for innovation is becoming available. It's exciting to watch hundreds of startups modeling iterative development. The same old wait-a-decade for a BFO doesn't cut it any more. It's time for innovate in research and evaluation. Eslick's pioneering work in health is an example of the fresh thinking we need in education.