The mechanism used by scientists to decide what research should be funded by public and/or private sources, and then what research is published in a reputable scientific journal is a long-standing peer review system. This process subjects science ideas, and outcomes, to independent in-depth scrutiny by other highly-qualified qualified scientific experts (peers) before they are made public and ultimately used as a base upon which more science knowledge can be built. Peer review is the foundation upon which science stands. The evaluation of work by individuals of similar (and often higher) competence to their peers constitutes a form of self-regulation by members of a profession within the relevant field. Such rigorous review is intended to maintain standards of quality, improves performance, provides credibility, and provides important guidance for informed decision-making by funders and journal editors. It is estimated that, globally, over a million scientific research ideas are reviewed annually as scientists seek resources to initiate and/or continue work on ideas leading to new discoveries, and publish the outcomes of their research. The peer-review system is an essential arbiter of scientific quality.
Peer-review is, however, not without its flaws and, given its important role in 'regulating', it justifiably deserves scrutiny. It is often labeled by scientists as conservative, cumbersome, capricious and intrusive. Something that even the most fervent supporters of peer review will agree on is that the process takes a long time. While methodical attention to detail is of great importance, some would argue that the process slows down advances in scientific and medical knowledge and that in this day and age of instant access to information, peer review seems dated. More significantly, critics note that the system wrongly rejects valid science while too often wrongly accepting scientifically flawed thinking. Even more importantly, in my opinion, the usually very conservative peer-review system too often makes recommendations that maintain the scientific status quo rather than takes a visionary approach to research ideas that challenge conventional wisdom. This often works against 'high risk, high reward' research as a conservative consensus shows reduced tolerance for genuinely new scientific approach, findings and interpretations."
My own professional life has been intertwined with the peer-review system, beginning with the first professional manuscript I published as an undergraduate student, through a long research career writing successful and unsuccessful grant proposals, as an academic administrator working with faculty affirmed by, or frustrated with, peer-reviews of their work, as a panelist for many proposal review committees for national and international funding agencies, as Editor of a major international journal, and finally as the President of a private foundation that funds science. I have seen, and experienced, the good and the bad of the system and -- overall and despite its short comings -- I continue to find it an important mechanism of validation for science and scientists. And, along with almost every other scientist in the world, I also willingly accept the responsibilities that come with being a reviewer. Excellent peer review provides an important due diligence and it requires a considerable time commitment.
The peer review system, particularly in the NSF, is now coming under scrutiny by Congress. Members of the House of Representatives Committee on Science, Space and Technology believe that every NSF proposal should be required to include a statement of how the research, if funded, would "directly benefit the American people." Sponsored by Rep. Lamar Smith (R-TX), this 'High Quality Research Act' would strip the peer-review requirement from the National Science Foundation (NSF) grant process, inserting a new set of funding criteria that is significantly less transparent and not inclusive of the opinions of independent experts. Smith notes that "... he was not trying to micromanage the $7 billion agency but that NSF needs to do a better job of deciding what to fund given the low success rates for grant applicants and a shrinking federal budget."
Several grants, primarily in the social sciences, but also some in the broader portfolio of the NSF, that have been awarded in recent years seemed to trigger this action. They were identified by the Committee (in the opinion of the Committee) as grants that should not have been made. This type of 'concern' by legislators is not rare. The targeting of individual research projects with frivolous-sounding titles is a long tradition in Washington, reaching back decades to the infamous 'Golden Fleece' awards presented monthly by Wisconsin Senator William Proxmire (D-WI, 1975-1987) to identify what he personally viewed as wasteful government spending.
In the past months, however, things have taken a dramatic turn, when Congress approved an amendment by Senator Tom Couburn (R-OK) to a spending bill; the language would stop funding in political science research at NSF unless the Director certifies that the research addresses economic or national security interests. The Committee further noted that they recognize "... that we might be able to improve the process by which the NSF makes its funding decisions" and thus measure any NSF grant proposal based upon the likely contribution "to the national interest." The peer review system itself came under the scrutiny of the Committee as they probed the question as to how "we" (the Committee) could insure that those (independent peer review panel members) might be persuaded to focus on more 'helpful subjects.' The Committee would also require the NSF's director to certify that all accepted research proposals are: "in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science; ... answers questions or solves problems that are of utmost importance to society at large; and not duplicative of other research projects being funded by the Foundation or other Federal science agencies." The draft bill also requires that the NSF director report to Congress how the same criteria can be applied to "other Federal science agencies."
Presidential Science Advisor John Holdren correctly, and repeatedly, defended peer review as a mechanism well-suited for funding research, both in general and as practiced at the NSF. And, importantly and with well-reasoned insight, he indicated that it was a dangerous thing for Congress, or other non-scientific and non-expert bodies, to specify in detail what constitutes fundamental research and what types of fundamental research should be funded by the NSF and/or other US agencies.
I fully agree with Holden's perspective and position. The peer review system, even with its innate imperfections, is an essential arbiter of scientific quality and integrity. In addition to the problem of stripping out a transparent, peer-review process, the new standards proposed by Congress also discount the importance of research overlap, which is a critically important component of the scientific process. Without overlapping research, scientists cannot independently verify and subsequently extend experimental results from other laboratories.
Quality peer review is not only vitally important to the scientific community relative to funding and the dissemination of new knowledge through journal publications, it is also important to validating the public's faith in, and understanding of, new, innovative scientific advancements and discoveries. Such new developments in science, technology and medicine are frequently the subject of news headlines and public discussion. With increasing amounts of scientific information being put into the public domain, and a growing number of organizations and social media mechanisms involved in disseminating, promoting and discussing scientific research and the impact of research results, it can be difficult to judge which research claims should be taken seriously vs. which are frivolous 'scares' seeking sensational headlines. Sometimes scientists are reported as saying conflicting things. How do we know what to believe?
Figuring out scientific credibility is a tricky job and since most of us (even scientists) are not experts in all sciences, we rely on others to tell us and validate "scientific truths." Despite what should be a focus on the content of scientific claims in the public world we most often judge scientific news by who reported it, who 'stands' behind it, who funded it, and who might benefit from the research outcomes. The deep and thorough peer-review (with all its foibles) of government and private foundation supported research, and its subsequent publication in peer-reviewed national and international journals should hold sway in the court of public opinion and help all of us to make value judgments on science outcomes as they become public.
My hope is that Congress will make the right decision and not interfere with, nor disband, the peer review system now in place for the NSF, NIH and other federal agencies. By interfering in this process, favoritism in science funding will abound, faith in the outcomes and value-added of scientific discovery will be virtually impossible to judge for scientists and the public alike, and the U.S. will inevitably fall even further behind the rest of the world in innovation as well as science and technology achievements.