Well before the COVID-19 pandemic, many physicians were concerned about problems that could arise from papers being published on preprint servers before undergoing a thorough peer-review process.
In 2017, Howard Bauchner, MD, JAMA editor-in-chief and senior vice president of AMA scientific publications and multimedia applications, penned an editorial saying that “sacrificing adequate and thoughtful peer review and editorial assessment is a mistake for research in medicine.”
Fast forward to today’s pandemic. The hunt for a COVID-19 treatment is moving at warp speed and there’s a voracious demand for the latest information on discoveries. Mirroring that atmosphere, papers posted to—and traffic on—preprint servers has skyrocketed.
MedRxiv, a preprint server for health sciences, and bioRxiv, a preprint server for biology, combined saw the number of papers jump to more than 1,400 in April, about double the 717 on the preprint servers in March. The usage on the sites grew too, with medRxiv experiencing nearly 6.9 million abstract views in March, up from slightly more than 65,000 views in December.
The intense interest has sometimes resulted in papers going up on a preprint server one day to then have major daily newspapers reporting on the studies the next day.
For example, some seized on a study that raised the possibility of hydroxychloroquine taken with azithromycin as possible COVID-19 treatments. However, those who read the full study were concerned that it didn’t fully support its conclusions, the AMA’s Chief Health & Science Officer Mira Irons, MD, said.
Emerging research in the form of clinical trials are not supportive of their use in COVID-19 and have raised safety concerns, including cardiac risks, but the paper had already taken on a life of its own. The AMA joined the American Pharmacists Association and American Society of Health-System Pharmacists in issuing a joint statement on inappropriate ordering, prescribing or dispensing of medications to treat COVID-19.
“It’s an example of the potential harm that can result because of not critically evaluating the paper and whether the design of the study and the results fully support the conclusions,” Dr. Irons said.
Physicians—and anyone else—looking at studies on preprint servers should bring along a healthy dose of skepticism, she explained.
“You have to really understand what you are looking at,” Dr. Irons said. “It requires more skepticism than you might have when reading a study that has undergone the peer-review process and more personal responsibility to really read the entire paper—and not just the abstract—and act as a reviewer might to determine if the methods and the results support the conclusion—because you can’t assume that anyone else has done that important process for you.”
You can stay up to speed on the AMA’s COVID-19 advocacy efforts and track the fast-moving pandemic with the AMA's COVID-19 resource center, which offers a library of the most up-to-date resources from JAMA Network™, the Centers for Disease Control and Prevention, and the World Health Organization.
Peer review’s value
Many busy physicians don’t have time to sit and read an entire journal article. Instead, they often just look at the abstract, Dr. Irons said.
When it’s gone through the peer-review process, that three-paragraph abstract has been vetted to back up what’s in the paper. Multiple independent, unbiased reviewers who are experts in the field examine the paper’s methods and conclusions to determine whether everything supports the authors’ case. And there is a give-and-take process for reviewers to ask authors questions. Finally, medical journal editors ensure everything is communicated correctly, has context and doesn’t lead readers to conclude something the authors didn’t intend.
“I look at peer review as an independent analysis. It’s like a quality control,” Dr. Irons said.
That verification and review may be underway for a preprint article, Dr. Irons said, but physicians looking at preprints need to understand the peer-review process hasn’t yet been performed.
“You have to read the entire paper from beginning to end and do your own due diligence before just accepting the authors’ conclusion,” she said.
Dr. Irons said physicians should go to medical journals’ websites during the pandemic, as many studies are published online before they come across physicians’ desks in print editions.
Read this original investigation published May 11 in JAMA, “Association of Treatment With Hydroxychloroquine or Azithromycin With In-Hospital Mortality in Patients With COVID-19 in New York State.”
Journals expedite peer review
In the 2017 JAMA editorial, “The Rush to Publication An Editorial and Scientific Mistake,” Dr. Bauchner notes that the JAMA Network™ in the past five years had halved the time it took from submission to publication and that journal editors expedite peer review when articles make critically important contributions to clinical care.
That turnaround has been shortened even further at JAMA during the pandemic, while still maintaining accuracy and reliability. Dr. Bauchner said COVID-19 manuscripts for the AMA’s JAMA Network of journals are now turned around in two to four days instead of the typical week to 10 days. Physicians can also find resources at the JAMA Network COVID-19 resource center.
