Harlan Krumholz of Yale University School of Medicine and Eric Peterson of Duke University Medical Center write that a recent study published in JAMA includes “surprising and discomforting” findings on “just how infrequently data reanalysis has occurred in medical research.” A search of the research literature from 1966 to present revealed only 37 reports that met the criteria as an RCT (randomized clinical trial) reanalysis.
And of these few reanalyses, 84% had overlapping authors from the original report. “Thus, reanalyses are not only rare, but the majority that were reported were not fully independent of the original research group,” Krumholz and Peterson write.
Thirty-five percent of published reanalyses also “led to changes in findings that implied conclusions different from those of the original article about the types and number of patients who should be treated.”
In addition, other research has found discordance between what is reported in ClinicalTrials.gov and what is published in journal articles, the authors say. One study showed that in a random sample of Phase III and IV trials, in 15% the primary endpoint in the main article was different from the primary endpoint the trial reported in ClinicalTrials.gov.
The authors call for “all RCTs, their prespecified study protocols and analytic plans, and their results” to be registered and reported to the medical community, “fulfilling the ethical promise made to those enrolled in the scientific experiment… Full availability of trial registration data is essential to allow peer reviewers and journals to monitor trial protocols and analytic plans to ensure consistency and thereby reduce some of the variation that may occur in the reporting of results, particularly with respect to primary, secondary, and exploratory outcomes.”
EU Trial Transparency
The call for more transparency comes as the EU is grappling with how to release information on trials, though some regulators, such as the UK’s NICE, say they will go around the companies to obtain trial data.
Peterson told Outsourcing-Pharma.com, “I think the EU regulatory changes are doing a lot. Without these, I suspect few pharma companies would be hustling to open up their trial databases to researchers. On the other hand, the requirements alone probably don't go far enough.”
He added that there needs to be an easier way for researchers to get to the data because currently there’s a “limited ability to look across or combine data from several studies and certainly no support for the costs of reanalysis.”
Regardless of what the early efforts of the EU can accomplish, both of the authors of the editorial make it clear that the culture around clinical research will need to change for there to be significant improvements in standards.
“Fields outside of medicine have already embraced sharing experimental data, as have the basic biological sciences within medicine. The culture of clinical research in medicine will need to evolve for open science to succeed,” the authors conclude. “Rather than the rare exception, open science and replication should become the standard for all trials and especially those that have high potential to influence practice.”