If you’re a sports fan like me, then you were glued to the couch watching the Final Four and National Championship game this past week. March Madness is one of the greatest spectacles in sport. It rarely disappoints, and this year was no different. Congrats to the Tarheels, by the way!
As I watched the championship game, my mind began to wander, as it often does. With the officials taking serious heat for playing a more visible (and audible) role in the game than usual, I couldn’t help but think: What if there were no referees? What if players called all the fouls, both on themselves and against their opponents? Chaos—complete and utter disorder would surely ensue. Would we be able to trust the results of a game without impartial referees?
Bridging my passions for sports and science, my mind continued to wander and landed on the field of nutrition science, because #foodnerd. I thought to myself, “Maybe nutrition science needs referees.” I quickly snapped out of it, realizing that nutrition science already has them: They’re called peer reviewers! Peer reviewers are referees without whistles. And thank goodness for them.
Imagine if science had no review process. No referees. How would the quality of methods, results, and evidence be evaluated? It would be near impossible. Players can’t be trusted to call fouls on themselves (except sometimes in golf), much the same way researchers are often blind to flaws in their methodology, analyses or interpretation of results. But that’s OK. It’s human nature, and precisely what peer review is designed to correct for. It’s a critical component of science evolution.
Sometimes preliminary research findings are presented prior to peer review. It happens regularly at scientific meetings in the form of invited presentations and/or posters. These meetings are a valuable outlet to quickly share new results among scientists, showcase new techniques and methods, and discuss the relevance of emerging data and how it compares to the established literature. It’s also a great opportunity for collaboration, conversation, and debate within the scientific community—“within” being the operative word here.
Some might argue that unpublished data should remain within the scientific community until it’s gone through the peer review wringer. Dennis Bier, MD, Professor of Pediatrics-Nutrition at Baylor College of Medicine and editor of the American Journal of Clinical Nutrition, expands on this concept. “What is presented at a conference or in an abstract offers too little detail," he said. "They can be thought of as advertisements or commercials trying to sell their findings to peers. These findings need to stay in the scientific community until they are ready for prime time.”
As fast as news travels these days, it’s probably an unrealistic ideal. At a minimum, however, preliminary results must be couched and communicated as such.
Fortunately, the scientific community understands the nature of the peer review process, and for the most part, is careful not to get too far ahead of it. But there are times when press releases and media shine the spotlight on data before it’s ready for prime time. The release of unpublished findings presented as new validated results can stir controversy when, in fact, there are none.
But science doesn’t stop at peer review. Just because a study survives peer review and is published in a scientific journal does not mean it automatically becomes dogma. Single studies should then be compared to the body of evidence and established literature. Dr. Bier echoes these thoughts, stating, “Peer review is an imperfect process, but it’s the first important step in getting an objective read on what the data say. But peer review can miss flawed methods, data, and reporting. It’s a fact of life: sometimes referees miss a call. That’s why sports referees use instant replay. Science refs don’t have instant replay to lean on.”
I’m proud to be a self-proclaimed #foodnerd. Few things frustrate me more than reading a story about new research, only to discover that there’s no link to the actual study being discussed. If there’s no link, does the study even exist? If it doesn’t, let’s call a timeout until it does and resume play once science referees have had their say.
Kris Sollid, RD, contributed to this post.
