The idea that all work in science is reviewed to keep the process honest is also something of a myth. In reality, the number of findings from one scientist that get checked by others is quite small. Most scientists are too busy, research funds are too limited, and the pressure to produce new work is too great for this type of review to occur very often. What occurs instead is a system of "peer review," in which panels of experts are convened to pass judgment on the work of other researchers. Peer review is used mainly in two situations: during the grant approval process to decide which research should get funding, and after the research has been completed to determine whether the results should be accepted for publication in a scientific journal.

Like the myth of the scientific method, peer review is also a fairly new phenomenon. It began as an occasional, ad hoc practice during the middle of the nineteenth century but did not really become established until World War I, when the federal government began supporting scientists through the National Research Council. As government support for science increased, it became necessary to develop a formal system for deciding which projects should receive funding.

In some ways, the system of peer review functions like the antithesis of the scientific method described above. Whereas the scientific method assumes that "experiment is supreme" and purports to eliminate bias, peer review actually imposes the bias of peer reviewers on the scientific process, both before and after experiments are conducted. This does not necessarily mean that peer review is a bad thing. In some ways, it is a necessary response to the empiricist limitations of the scientific method as it is commonly defined. However, peer review also institutionalizes conflicts of interest and a certain amount of dogmatism. In 1994, the General Accounting Office of the U.S. Congress studied the use of peer review in government scientific grants and found that reviewers often know applicants and tend to give preferential treatment to the ones they know. Women and minorities have charged that the system constitutes an "old boys' network" in science that denies them equal opportunity. The system also stacks the deck in favor of older, established scientists and against younger, more independent researchers. The process itself creates multiple opportunities for conflict of interest to arise. Peer reviewers are often anonymous, which means that they do not have to face the researchers whose work they judge. Also, the realities of science in today's specialized world means that peer reviewers are often either colleagues or competitors of the scientist whose work they review. In fact, observes science historian Horace Freeland Judson, "the persons most qualified to judge the worth of a scientist's grant proposal or the merit of a submitted research paper are precisely those who are the scientist's closest competitors."

"The problem with peer review is that we have good evidence on its deficiencies and poor evidence on its benefits," the British Medical Journal observed in 1997. "We know that it is expensive, slow, prone to bias, open to abuse, possibly anti-innovatory, and unable to detect fraud. We also know that the published papers that emerge from the process are often grossly deficient."

In theory, the process of "peer review" offers protection against scientific errors and bias. In reality, competition for research funding is more powerful than peer review in determining which scientific voices win acceptance. This has become particularly noticeable in recent years as declining public funding for research has coincided with increasing corporate funding of universities and other scientific institutions.

Sheldon Rampton and John Stauber, Trust Us We're Experts! How Industry Manipulates Science and Gambles with Your Future, Tarcher/Putnam, 2001, chapter 8.

Further References:

Brian Martin, Peer review as scholarly conformity, chapter 5 in Suppression Stories, Fund for Intellectual Dissent, Wollongong, 1997.

Tom Abate, What's the verdict on peer review?, 21st C, no. 1.1, Spring 1995.

Alfred Yankauer, Who are the peer reviewers and how much do they review?, JAMA, The Journal of the American Medical Association, v263 n10, March 9, 1990, pp. 1338-40.

...back to top