Science and Uncertainty

articlesArrowBack

DividerScience and Regulation

On the Inevitability of Being Wrong

Steven D. Jellinek

Steven D. Jellinek is assistant administrator for pesticides and toxic substances for the Environmental Protection Agency. This is an adaptation of a talk he gave in March to the New York Academy of Sciences workshop on Management of Assessed Risk for Carcinogens.

The U.S. Environmental Protection Agency (EPA) administers two laws that require risk-benefit balancing: the Toxic Substances Control Act (TSCA), which gives the agency broad authority to control unreasonable risks of commercial chemical substances, and the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), a pesticide registration law also aimed at protecting public health and the environment from unreasonable risks. These laws both involve what I call "the inevitability of being wrong," which explains why scientists and regulators approach chemical problems differently, and why a regulator's approach to chemical problems must be different from that of scientists.

Scientists tend to be cautious about accepting a new theory or evidence as valid. When confronted with a great deal of uncertainty, they avoid drawing conclusions and instead call for additional careful study and research. There's always the chance that others may beat them to the result, but this risk is worth taking compared with being "wrong" or misleading the scientific profession. This approach is entirely consistent with generations of scientific tradition and training&emdash;hypotheses must be carefully developed, tested, and proven; cause and effect must be demonstrated.

Unfortunately, regulators don't have the luxury of putting off decisions until certainty arrives. Enormous scientific uncertainty surrounds the potential risks and benefits of most chemicals, yet these risks and benefits exist. Therefore, a regulator's every action&emdash;or inaction&emdash;represents a decision of some kind. Postponing action on a chemical until there is better information is a decision; taking precautionary action in the meantime is a decision; and delaying action because of limited resources or other priorities is also a decision. Whatever way they decide, regulators run the risk of making the wrong decision in the midst of pervasive uncertainty. The law of averages says that they won't be right every time. Therein lies the inevitability of being wrong.

Too Late to Act

If, for example, regulators decide to wait ten years until better data are in on whether chlorofluoro-carbons deplete the earth's ozone layer, either the theory will be wrong and there will be no harm done or the theory will be right and it will be too late to do anything about it. In the face of such uncertainty, should precautionary action be taken while science tries to find the right answers, or should more data be gathered to inform possible future action?

Let's consider the traditional 5 percent significance level. Scientists, as a rule, are uncomfortable with anything much higher than a 5 percent chance that their results are wrong. The lower the "p" value, the happier the scientists. The significance level is the probability of a false positive &emdash;the chance of a false alarm. A one in twenty chance is not very frequent; a one in ten chance is notably more frequent. From a public policy point of view, on the other hand, a "p" value of 50 percent&emdash;a one in two chance of a false positive&emdash; might sometimes be more appropriate, although many scientists would find this difficult to accept. But think about the matter from the regulator's point of view.

When it comes to toxic chemicals, the potential cost of a false negative (that is, insufficient precautionary action) may far exceed the potential cost of a false positive (too much precautionary action). Toxic effects are difficult to find, like needles in a haystack. Even after conducting an expensive test with a 5 percent chance of a false positive, there still could be a 50 percent chance of a false negative for a major effect deserving precautionary action. To resolve this difference in perspective, tests could routinely include analyses of the chance of a false negative. These calculations are currently scarce indeed.

Innocent Until Proven Guilty?

It is often said that it is possible to prove a positive but impossible to prove a negative, and therefore the burden of proof should rest on those who suspect a chemical to be carcinogenic or otherwise harmful. In other words, a chemical should be presumed innocent until proven guilty. If the evidence is suggestive but insufficient, there is a hung jury and judgment is "reserved." In this spirit, a recent Science article quotes a spokesperson for the chlorofluorocarbon industry as saying that the CFC depletion potential is still a "hypothesis," and no further precautionary action should be taken until there is conclusive proof. In placing a high value on civil liberty, we generally believe that acquittal of the guilty (false negative) is less harmful than conviction of the innocent (false positive). But when toxic substances are involved, the idea of "innocent until proven guilty" does not make much sense.


Source: Technolgy Review, Aug/Sept,1980, p.8.

Back...

Divider