 |
| NIH Director Francis Collins |
It's a well-told problem in the industry: Over and over, preclinical researchers do something unprecedented in an animal model, fire off a press release touting yet another "breakthrough," and then quietly watch as their colleagues fail to reproduce those findings. Now, thanks to some high-profile failings in some of the world's leading journals, the call for reform has reached a fever pitch, and National Institutes of Health Director Francis Collins has taken up the baton.
In an essay published in Nature, Collins and Principal Deputy Director Lawrence Tabak write that "the checks and balances that once ensured scientific fidelity have been hobbled" over the past few years, as researchers strive to be provocative at the cost of explaining their methodology. That shoot-for-the-moon paradigm is only made worse by funding agencies, academic centers and journals, the two write, as many science financiers encourage hyperbole in hopes of snagging headlines.
And while the problem starts at the preclinical level, it can quickly trickle up into the drug development world. As Reuters reports, Amgen ($AMGN) found in 2011 that its researchers could confirm just 6 of the 53 breakthrough cancer studies they vetted, while Bayer said in 2012 that of 67 landmark studies in oncology, women's health and cardiovascular disease, it could verify only 14.
So what's the nation's largest bankroller of biomedical research going to do about the problem? NIH chalks up at least part of the reproducibility gap to poor training, so the organization is developing a program to educate researchers on good experimental design and transparent conduct. Furthermore, NIH is considering instituting a tougher rubric for potential fundees, more closely reviewing each application for scientific basis and viability.
Still, as Collins and Tabak admit, these efforts alone won't solve the problem of irreproducible findings, so NIH is calling on journals, academics and industry magnates to help right the ship of research. The organization is encouraging journals to devote more space to failed studies and corrections to earlier results, asking universities to amend their priorities so as not to incentivize hasty publication in the interest of tenure, and encouraging researchers to spend more time explaining the details of their methods.
Meanwhile, the President's Council of Advisors on Science and Technology is gearing up for hearings on the issue, planning to interview researchers and journal editors to get a better grasp of the problem before potentially making policy recommendations.
But many in the field believe the epidemic is too widespread to be solved by mere institutional reforms. As TetraLogic ($TLOG) CSO C. Glenn Begley told Reuters, the culture of overstatement is deeply rooted in the research community.
"The real problem is that scientists are reluctant to speak up about studies that won't replicate because there is so much to lose," Begley said. "If I criticize you, and you review my next grant application, you might [take revenge]. That's why people are afraid to say the reason they couldn't replicate a study is that it was just plain wrong."
- read Collins and Tabak's essay
- check out the Reuters story