The head of the US Food and Drug Administration (FDA) and other speakers at a recent conference sponsored by the Friends of the National Library of Medicine suggested the reproducibility of biomedical research might improve if scientists could agree on a definition of “reproducibility” and if more information about preclinical trial research findings was available and shared.

FDA commissioner Robert Califf, MD, suggested there may be a need for a clearinghouse database to track basic science research that occurs prior to the start of clinical trials.

Robert Califf, MD, the commissioner of the FDA, suggested there may be a need for a clearinghouse database that could keep track of the basic science research that occurs prior to the start of clinical trials.

Califf is interested in the feasibility of “something like a ClinicalTrials.gov for preclinical work.” ClinicalTrials.gov provides standardized information about in-progress clinical trials, and findings about the efficacy and side effects of FDA-approved drugs and medical products.

Hypothetically, Califf said, the proposed project could provide access to data from the extensive preclinical research that involves lab animals or cells growing in laboratory dishes.

Califf explained a database of preclinical information also might improve research transparency as well as address occasional inabilities to reproduce some clinical research findings. Currently there is no organized clearinghouse of preclinical information accessible to physicians, scientists, researchers, and others around the world.

In another session during the two-day conference, which featured national experts on research reproducibility and transparency, some panelists noted the disagreement among scientists about how to define “reproducibility.”

The term “reproducible” sometimes describes when consistent results occur in repeated scientific studies despite small variations within an experimental set-up. Ferric Fang, a microbiologist from the University of Washington, noted, however, that some scientists believe reproducibility refers only to the ability to achieve the same results within the same lab.

Fang explained these and other disagreements about what “reproducibility” means make it difficult for scientists to respond when peers allege irreproducible research methods and findings. Such disagreements, he pointed out, impede experts’ ability to find common ground where honest differences and possible solutions about research inconsistencies can be considered.

Ferric Fang, MD, a microbiologist from the University of Washington, reported on the reproducibility of research.

Fang clarified he and the other panelists were discussing research discrepancies when there are no allegations of fraud or deceit by the original investigators. He noted the current debate about reproducibility is ethically principled and should not be confused with concerns about occasional scientific fraud and dishonesty.

Several conference attendees and speakers acknowledged it is rare for scientists and physicians to publicly discuss research reproducibility. Several conference speakers also praised PubMed Commons, which is PubMed’s effort to create a dialogue among scientists about the veracity, generalizability, and inferences drawn from research results.

PubMed Commons, published by NLM, has its own website where scientists occasionally discuss the merits of each others’ research findings. Designed to boost reproducibility by fostering expert criticism of peer research, PubMed Commons was recently promoted from a trial run to a permanent part of PubMed.

Individual entries within PubMed indicate if there is discussion about the article within PubMed Commons—and a link is provided.

An entry in PubMed includes a link when comments associated with an article are available.

Toward the end of the conference, several attendees urged the Friends of the National Library of Medicine–a private foundation that supports NLM’s activities–to continue the topic of research reproducibility in the future and to focus on access to and even criticism of the clinical research that occurs after a drug or medical product is approved and marketed.

By Rob Logan, PhD, senior staff, US National Library of Medicine

More Information
Best Practices of Biomedical Research: Improving Reproducibility and Transparency of Preclinical Research [conference recording]
To Your Health: NLM Update–“The Road to Reproducible Research?” [podcast]