In order to compare the validity of sponsor-submitted results posted on ClinicalTrials.gov with corresponding information on Drugs@FDA, researchers Lisa Schwartz and Steven Woloshin from The Dartmouth Institute for Health Policy and Clinical Practice, collaborated with researchers from the U.S. National Library of Medicine (NLM).
Woloshin, MD, Professor, Medicine and Community & Family Medicine, Dartmouth Institute, told us that “Recent studies attempting to validate posted results by comparing them to corresponding peer reviewed medical journal publications suggest that discrepancies are relatively common.”
However, in their research, Woloshin explained they tried to validate posted results against “an arguably better gold standard.”
According to Woloshin, this standard, the drug approval packages from the FDA, which includes the medical and statistical reviews posted on the drugs@fda.gov website, has several advantages over peer reviewed publications.
Mainly, unlike medical journal editors and peer reviewers, the FDA has access to the individual participant data from the trials.
“This means FDA can see all the trials and all the outcomes (avoiding selective publication) and it means FDA can independently reanalyze according to what they believe to be the best statistical practices (data can be analyzed in many ways - and different decisions, for example, how to account for missing data, can yield very different results),” said Woloshin.
Conclusions
The researchers were “pleasantly surprised” at how close primary outcome information in ClinicalTrials.gov posts were to the FDA review materials. Yet, Woloshin noted that it’s important to be clear that "primary outcomes" mean those outcomes that are the main focus of the trial, “but these are not necessarily the most important outcomes,” he added.
For example, companies may designate number of pounds lost in a weight loss trial as primary, but the percent of people who lose a meaningful amount of weight, which might be a secondary outcome, “is arguably much more important,” explained Woloshin.
Ultimately, according to the researchers, the takeaway is that ClinicalTrials.gov and FDA reviews serve different purposes.
Specifically, Woloshin explained that ClinicalTrials.gov is meant to provide a complete reporting of all summary data as planned in the study protocols, while FDA reviews provide FDA's take: “independent, expert assessment of the evidence as analyzed for the purposes of regulatory decision making,” added Woloshin.
The researchers concluded that even if the numbers reported on ClinicalTrials.gov were accurate, there would still be question surrounding trial design, conduct, or analysis – which could affected trial result conclusions assessed in Drugs@FDA.
“For example, reviewers may have raised questions about the appropriateness of an active comparator used (or its dose) or about unvalidated outcome measures,” the researchers said.
“These sources are complementary,” added Woloshin. “ClinicalTrials gives ‘all the data,’ FDA gives context for interpreting the evidence.”
Moving forward, the researchers believe that all sponsors can be much clearer about which secondary outcomes are pre-specified and which were exploratory. “Data discrepancies varied some across companies,” said Woloshin, but “all companies can work hard to make sure their information is accurate and complete.”