The claim is based on analysis of published papers which suggests – according to authors Leonard Freedman, Iain Cockburn and Timothy Simcoe – that half of all preclinical studies have errors that mean they cannot be repeated by other researchers.
If Freedman and his colleagues are right – they admit their estimates are based on a small number of papers – it would mean $28bn (€24.8bn) is spent on irreproducible preclinical studies each year, based on a total annual spend of $56bn reported in 2013 AAAS research.
The authors stress their aim is to highlight the potential economic impact of irreproducible studies to stimulate better trial design and concede that such research can still provide useful information.
They also suggest irreproducibility impacts drug development, explaining that it is normal for drugmakers repeat academic research before initiating product development in a process that can take two years and cost as much as $2m.
“While industry will continue to replicate external studies for their own drug discovery process, a substantially improved preclinical reproducibility rate would derisk or result in an increased hit rate on such investments, both increasing the productivity of life science research and improving the speed and efficiency of the therapeutic drug development processes.”
Solutions
The reproducibility of preclinical research is also focus for the US National Institutes of Health (NIH), which recently introduced reporting guidelines for journals and scientists.
Freedman and his team welcome such efforts as positive, but suggest “compliance levels and their impact to improve reproducibility have been disappointing” adding that scale of the challenge means “there is no single magic bullet solution to the problem.”
In clinical research, initiatives like CONSORT have improved reproducibility. However, according to the authors, such measures cannot simply be applied to preclinical studies without a dramatic increase in the cost and time.