Sometimes we get so wrapped up in the new that we forget the old. Most of us are inclined to trust that new ways are iteratively better, but useful things can sometimes get lost amid the speed of modern advancements. Do you remember the early Amstrad word processors, with their flickering green screens and easy templates for letters and articles? On occasion, I would have eagerly swapped my 1990s talking Microsoft paperclip for that good old green screen. Well, maybe that's nostalgia talking. But how about the way we used to learn math in school, through long division and multiplication, as opposed to the current "chunking" or "grinding" craze? Or the quality of the CD compared with the average modern MP3 file?
Moving on is fine, but notwithstanding my "I love tech" mug, sometimes I question whether some of the new ways really are better than the old. The hazard is that we may be so keen to evolve and accept the new that we occasionally forget the things we got right before.
This can happen in academic research too. In medicine and epidemiology things move on quickly. Quicker processors mean new methods -- Big Data, computational epigenetics, and stacks of computer cores all dutifully crunching the latest research data while scientists catch up on email or use the newly liberated time to update their blogs. All very reasonable, so long as, in the meantime, we have remembered to tell the computer exactly how to crunch, parse, chunk, and grind.
A while back a colleague and I were having lunch and looking over some data resulting from something called an ROC curve. These curves were initially developed in World War II to help "diagnose" allied aircraft and submarines so that Allied forces shot at the right things and didn't unnecessarily reduce a population of large, water-bound mammals. Later, in the 1980s, the curves were adopted by epidemiologists to help them decide at what point an individual can be said to have recovered from a health complaint. I won't bore you with all the techy details, in case statistics isn't your passion, but basically it all comes down to choosing a point on a curve to determine when recovery has occurred. For many chronic conditions epidemiologists agree that the correct point to choose is the point closest to the top-left corner of this curve. As we stopped to think about it, it struck us as obvious that the way to choose this point is by using Pythagoras' theorem. We had a point in two-dimensional space, with coordinates that describe two sides of a right-angled triangle. The distance to the top-left corner is then the hypotenuse.
This wasn't really a "eureka" moment -- not by any stretch -- but what concerned us was that this was not what our epidemiologist peers and colleagues were doing. None of them was. For example, methods of defining the point based on drawing a tangent line had crept into research articles and were being repeated by those applying the work in new settings. It was just the way it was being done, and no one had stopped to question it.
We set about exploring the implications of our insight and how it might change conclusions in research. We conducted several experiments using real trial data and also considered the theoretical implications, and it turned out that it matters -- a lot.
So the moral of the story is that before you throw out the old stuff in the attic, just go through it one last time; there might be something in there that is still relevant and useful. But if you are in the business of using ROC curves and determining minimally important change (MIC), then you should probably read the full, gory, techie details in our open-access paper.
Open-Access Paper:
Froud R, Abel, G. "Using ROC curves to choose minimally important change thresholds when sensitivity and specificity are valued equally: the forgotten lesson of Pythagoras. Theoretical considerations and an example application of change in health status." PLOS One 2014; DOI: 10.1371/journal.pone.0114468
Stata Module:
"ROCMIC: Stata module to estimate minimally important change (MIC) thresholds for continuous clinical outcome measures using ROC curves," available at https://ideas.repec.org/c/boc/bocode/s457052.html, or type "ssc install rocmic" at the stata prompt.