Reference

Kaplan, R. M., & Irvin, V. L. (2015). Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. PLOS ONE, 10(8), e0132382. https://doi.org/10.1371/journal.pone.0132382


Yellow: Interesting

Highlight ( Page 2)

supplements funded between 1970–2012

Highlight ( Page 5)

A total of 55 trials were analyzed

Highlight ( Page 6)

Fig 1 plots the relative risks of the primary outcome by the publication year of the main outcome paper. Because it was an extreme outlier, the CAST study is excluded from the figure. Prior to publication in 2000, studies often showed benefits of treatments with the notable exception of CAST (not shown in figure). Following 2000, confidence intervals for relative risk ratios included 1.0 in all cases, with the exceptions of the PREVENT and the SANDS trials (benefit) and the Women’s Health Initiative (Harm). In addition, the variability in RRs was considerably reduced after the year 2000 (Fig 2).

Highlight ( Page 6)

more recent trials may have evaluated their treatment drug against clinically effective alternatives instead of placebos

Highlight ( Page 8)

A final explanation for the trend toward null reports is that current authors face greater constraints in reporting the results of their studies. In our review, the year 2000 marks the beginning of a natural experiment. After the year 2000, all (100%) of large NHLBI were registered prospectively in ClinicalTrials.Gov prior to publication. Prior to 2000 none of the trials (0%) were prospectively registered.

Highlight ( Page 8)

Prospective declaration of the primary outcome variable is important because it eliminates the possibility of selecting for reporting an outcome among many different measures included in the study

Highlight ( Page 8)

Had the prospective declaration of a primary outcome not have been required, it is possible that the number of positive studies post-2000 would have looked very similar to the pre-2000 period.

Highlight ( Page 8)

Beginning in approximately 2000, the likelihood of showing a significant benefit in large NHLBI funded studies declined. Among the explanations we evaluated, the requirement of prospective registration in Clinicaltrials.gov is most strongly associated with the observed trend toward null clinical trials. The decline is not easily explained by the increased use of active comparators or a decline in industry sponsorship.

Highlight ( Page 8)

It is widely noted that journals favor publication of statistically significant findings[2]. Bias in favor of publishing positive outcomes is not a likely explanation for our results. We focused on large trials because previous analyses by NHLBI reported that 97% of trials with annual budgets over $500,000/year were published [3], thus removing publication bias as a rival explanation

Highlight ( Page 9)

It has been argued that there have been few efficacious drugs in the pipeline[4,5]. Since about 1998, there has been a systematic decline in the number approvals for new cardiovascular drugs[6].

Highlight ( Page 9)

Our analysis is limited to large NHLBI-funded trials and to studies on cardiovascular outcomes in adults. We focused on NHLBI because the Institute has championed transparency and allowed us full access to all trials.

Highlight ( Page 10)

We cannot say that trend toward null trials to preregistration in ClinicalTrials.gov is causal. Our analysis included only a small number of trials and the design of the study does not allow causal inferences.

Highlight ( Page 10)

The transparency of RCTs is likely to have improved following the FDA Modernization Act of 1997, which created the ClinicalTrials.gov registry[8], a service that required registration of studies that test drugs, biologics, or devices for the treatment of serious or life threatening diseases[9–11]. Registered studies must provide: the study’s purpose, recruitment status, design, eligibility criteria, locations and pre-specified primary and secondary outcomes[11].