In the recent unanimous decision in Matrixx Initiatives v. Siracusano, the U.S. Supreme Court applied the “fair preponderance of the evidence” standard of proof used for civil matters, in which a particular conclusion is deemed “more likely than not” to be justified. At issue was whether Matrixx had violated federal securities laws by failing to disclose to shareholders sporadic reports of anosmia associated with the use of its Zicam nasal spray before the Food and Drug Administration (FDA) issued a warning about that association in 2009. The question before the Court was not whether the drug caused the loss of smell, but rather whether the company failed to provide material information to the investor plaintiffs that would have led a “reasonable shareholder” to alter his or her investment strategy. The initial trial court was persuaded by the company’s primary argument that the evidence suggesting that its product caused anosmia did not reach statistical significance and therefore should not have been considered material. In upholding the ruling of the appellate court, which had reversed the trial court’s decision, the Supreme Court ruled that whether or not it was considered statistically significant, the information about the seemingly infrequent occurrences of loss of smell after use of the product was indeed material to investors. Speaking for the undivided Court, Justice Sonia Sotomayor also acknowledged that the mere existence of reports of adverse events associated with a drug does not prove causality — but asserted that such a high level of proof did not have to be achieved. Similarly, under the Code of Federal Regulations for the FDA, warnings and precautions regarding the safety of drugs must be revised to include information on “a clinically significant hazard as soon as there is reasonable evidence of a causal association with a drug; a causal relationship need not have been definitely established.” There is no requirement for statistical significance.

Clinicians are well aware that to be considered material, information regarding drug safety does not have to reach the same level of certainty that we demand for demonstrating efficacy. We understand that clinical trials that are designed to prove that a drug is effective use preplanned statistical analyses focused on a specific, carefully defined and adjudicated primary end point. Moreover, the number of subjects who will have to experience this targeted event for researchers to adequately test whether it occurs at the same rate as it does in a comparison group (the trial’s statistical power) is also established before the study begins. This same carefully constructed statistical framework is not, and understandably cannot be, used for evaluating unplanned and uncommon adverse events. When studying safety, we search for signals of imbalances and attempt to piece together multiple underpowered comparisons to obtain a better estimate of the risk. Sorting the wheat of true adverse drug effects from the chaff of biologic variability and chance associations is exceedingly difficult. A staggering and increasing number of reports are received by the FDA’s Adverse Event Reporting System (AERS) each year — more than half a million in 2009.