How science communication is failing during COVID-19

Don’t put too much stock in reports in the general news media about COVID-19 treatments and vaccines. Many news reports come from press releases and other corporate announcements, not authoritative scientific studies.

That’s some of the advice set forth in a recent JAMA Viewpoint essay by Richard Saitz, MD, MPH, associate editor of JAMA, and Gary Schwitzer, publisher of HeatlhNewsReview.org, in which they noted the unreliability and scientific inaccuracies of recent news reports about COVID-19 medical research.

"Government reports, journalism, talk shows and public relations news releases from industry and academic institutions have often failed to communicate the results of studies well, and these failures have important consequences,” the authors wrote.

Failures of science communication during the pandemic have included:

• A focus on single-study results without the context of other studies or acknowledgment that single studies are rarely definitive.
• Overemphasis on results, particularly relative effects, without recognition of important limitations.
• Communications based on incomplete reports of studies and reports of studies that have not been adequately reviewed.

You can stay up to speed on the AMA’s COVID-19 advocacy efforts and track the fast-moving pandemic with the AMA's COVID-19 resource center, which offers a library of the most up-to-date resources from JAMA Network™, the Centers for Disease Control and Prevention, and the World Health Organization. 

The article cites recent reporting about three of the most talked about COVID-19 treatments: remdesivir, dexamethasone and hydroxychloroquine.

The first remdesivir news stories in April identified a small study conducted with 53 patients who needed supplemental oxygen.

“Although a news release from the manufacturer of the medication mentioned limitations and stated that safety and efficacy were unknown, the headline of the release stated, ‘Remdesivir treatment resulted in clinical improvement.’”

The headline strongly suggested cause and effect and was an inappropriate description of the results of a small observational study, wrote Dr. Saitz and Schwitzer. And later, somewhat larger studies highlighted more positive results that were not statistically significant.

Similarly, in June, a university news release about a small study of dexamethasone was picked up by the World Health Organization and The New York Times, which reported the results in dramatic positive terms.

Find out what doctors must know about medical preprints.

“The source for these announcements was a news release, not an abstract, preprint, or peer-reviewed article,” notes the JAMA essay. “Steroids, including dexamethasone, have been studied for community-acquired pneumonia, with earlier studies suggesting mortality benefit that was not confirmed in later systematic reviews.”

Hydroxychloroquine is one of the most controversial treatments, earning a Food and Drug Administration emergency use authorization after a small, open-label study in March and then losing the authorization after subsequent larger, more sophisticated studies.

“News stories and social media reports took readers on a roller-coaster ride, alternately reporting efficacy, lack of efficacy, and harm, reporting dutifully on the results of each latest study,” Dr. Saitz and Schwitzer noted.

The authors recommend that news reports about single studies should be matter of fact and favor reporting of main outcomes and absolute risks, specify patient populations, and highlight limitations in validity and generalizability. All such news reports should include a note of caution that single studies are rarely definitive.

News reports “should also include the views of other independent experts in the field without conflicts of interest. Included in those expert views should be context, including comparison with the findings from other studies (of the reported and other treatments), the relative weight that should be given to the current vs. other studies, and how such treatments are usually studied,” says the JAMA essay.

Find out how JAMA Network has quickened its pace to deliver critical COVID-19 information.

The top editors of JAMA recently explained in an editorial why peer review is a must when it comes to evaluation of scientific manuscripts.  

“Peer review is essential in science. The substantive evaluation by and opinions from reviewers with subject matter knowledge and with methodological and statistical expertise are invaluable in assessing the scientific rigor and plausibility of study findings, and help to ensure that the presentation of results accurately reflects the data,” wrote JAMA Editor-in-Chief Howard Bauchner, MD, Executive Editor Phil B. Fontanarosa, MD, MBA, and Deputy Editor Robert M. Golub, MD.

“But peer review is only one part of the process of ensuring that published research is of high quality,” the JAMA editors added. “Authors must be honest; they cannot fabricate, falsify, or misrepresent data, all of which may not be detectable during review. Ultimately, though, editors, who are the final arbiters of what is published, must carefully assess the scientific and clinical aspects of the study, consider the opinions of peer reviewers and the responsiveness of authors, and exercise judgment in deciding what to publish.”