Peer Review Is Time-Tested to Withstand Political Fashion Trends

By Dr. Ed Liebow, Executive Director, American Anthropological Association

Expert peer review is the best way to judge research proposals and their scientific merit. It is time-tested to withstand political fashion trends and to bring to the forefront the most promising scholarship. Expert peer review helps determine what public investments should be made in proposed research by gauging how best to fill our gaps in knowledge. Peer review also recognizes that the steps from basic scientific discovery to societal payoffs follow a winding path. For most of society's complex problems, there is no magic bullet solution that will come from a single research project's results. The absence of an immediate payoff, however, should not be equated with an absence of scientific merit.

In an unpersuasive defense of yet another legislative proposal making its way through the House Committee on Science, Space and Technology, Committee Chair Lamar Smith (R-Texas) and the Subcommittee on Research Chairman Larry Buschon (R-Ind.) introduced the FIRST Act (HR4186). The FIRST Act argues for re-examining grant funding priorities at the NSF and reducing the agency's overall budget and making monstrous cuts to the agency's Directorate for Social, Behavioral and Economic Sciences. Rather than trusting the peer review process, Chairmen Smith and Buschon prefer that Congress dictate what is valuable science. This is a slippery slope, especially for the social and behavioral sciences.

In the last session of Congress, Rep. Smith put forward a false argument that the American public has no interest in supporting research in the social and behavioral sciences unless it can be shown to contribute directly to U.S. national security or domestic job creation. This time around, he is less specific about what is judged to be valuable social science research, but Rep. Smith still argues that research projects focusing on what drives consumer behavior to unsustainable choices or on the policy implementation impacts of shifting balance of power between the judiciary, legislature and executive branches of government are somehow of questionable public value.

Cherry-picking grants awarded by the National Science Foundation is not an effective use of time or money. Investment in scientific research has never been more important than the present. Prudent public investments in research and development enhance American competitiveness, and we can be confident that these investments will eventually lead to quality of life enhancements afforded by innovative technologies like GPS and iPhones. Just as important, these investments also help us make informed, sustainable choices about energy and environmental policy, and provide evidence for the most enduring ways to overcome social injustices. While some members of Congress look at grant titles and question whether the grants are worthy of public investment, they miss a key point about how science works. Science advances human understanding most often by moving at a marathoner's pace, not a sprinter's. We make hypotheses based on our current understanding, then test these hypotheses with observations, and use that information to lead to revised understandings.

A number of scholars, including my colleagues and I, have spent considerable time thinking about the public impacts of publicly supported basic research (Engel-Cox et al. 2008; Liebow et al. 2009). After examining reams of grant data, publications and responses to surveys with scholars, policy-makers and industry representatives, we identified several complex pathways by which publicly supported research not only leads to better science, but also measurable improvements in quality of life. These include distinct pathways to benefit government policy-makers, scholars and their institutions, communities, as well as business and industry.

The public benefits of basic social science research go far beyond jobs and national security. They include smarter decisions about investing in schools and social programs, setting health targets, providing guidance for regulatory and clinical guidelines, commercial development and availability of products and processes, encouraging consumer behavior for improved health and safety, less waste and more sustainable resource use, health and social welfare gain and national economic benefit from commercial development and user-friendly design, environmental quality and sustainability, improved healthy aging, improved international balance of trade, and energy independence. Not to be overlooked are the contributions that researchers make to a general reservoir of knowledge, whose accumulated contents are available for discovery and innovative applications in unexpected ways.

In other words, it would be unwise to assume that research projects, to be considered worthy of public support, must have a clear and direct path to improving jobs and national security, or a practical application that can be seen ahead of time. Since the path is often indirect, it would be equally unwise to leave the job of judging the scientific merits of research proposals to people who cannot see around the corner to a range of possible benefits from original research.

But back to Rep. Smith and Rep. Buschon and their latest legislative proposal. By alluding to the questionable quality of individual grants -- solely based, it appears, on the grants' titles -- they are short-circuiting the peer review process. Rather than leaving the job of judging scientific importance to scientists, they are claiming that Congress is in a better position to decide which gaps in knowledge need to be filled.

To the contrary, peer review takes these judgments about the return on scientific investment out of the realm of politics, and places them where they belong, in the realm of advancing human understanding, so that such understanding can be applied to tackling the world's most pressing problems.

Rep. Smith recently criticized NSF for awarding grants to study how automobiles are marketed to consumers in China and about the history of legal cases involving slavery in Peru. China, a country of 1.5 billion consumers who are literally choking on their new-found consumer power, is worth further study that can provide detailed observations that gain us a better understanding of what fuels changing consumption. Such study may also potentially provide U.S. industry with valuable information about business opportunities in the Chinese industry.

Another study that has gotten a bee stuck in Rep. Lamar's bonnet is that of 17th century Peruvian legal papers is an excellent case, as it aims to establish through empirical means a basis for determining how courts and the legal systems in different countries may have softened or accentuated the injuries and injustices perpetuated by legalized slavery. And, in a world where the balance of power between the legislative, executive and judiciary branches of government is quite different from country to country, historical studies can be extremely valuable in helping us understand in an impartial, empirical fashion the impact that courts have had in setting social policy.

The direct line of sight from research findings to the next iPhone may not always be apparent, but it is precisely the open-ended nature of scientific inquiry that leads us to important discoveries. In a recent essay on the "useful pursuit of useless knowledge," Ashutosh Jogalekar argues that, in fact, it is highly practical to support research despite its lack of immediately apparent applications:

The case for basic research thus boils down to a practical consideration: the recognition that a stubborn sea of scientific possibilities will yield its secrets only to the one who casts her net the widest, takes the biggest risks, makes the most unlikely and indirect connections, pursues a path of discovery for the sheer pleasure of it. Even from a strictly practical viewpoint, you encourage pure research because you want to maximize the probability of a hit in the face of uncertainty about the landscape of facts.

We scientists are grateful for the public support we receive for our research. We appreciate the importance of using taxpayers' funds wisely, as we are taxpayers too. We are also convinced that there is no adequate substitute for the expert peer review process in assessing the adequacy of proposed research and its contributions to science and society. And as Edmund Burke said, those who do not study history are doomed to repeat it.