Newspapers and other online media competing for views and clicks often use exaggerated headlines and sometimes ‘fake’ news to lure readers. While these ‘false’ headlines serve as a ‘click-bait’, the direct harm it causes people is generally limited. But the same cannot be said of the use of ‘spin’ in medical and research papers, where the exaggerated content could lead to wrong assumptions and dangerous diagnoses by doctors.
A recent study, which investigated the use of ‘spin’ in medical research papers, found the use of hyperbole in more than half of the abstracts they analyzed. The impact of these exaggerations on the decisions made by doctors is a serious cause for concern, said the researchers behind the study, as many doctors base their decisions from only reading the abstract.
The study chose to look at ‘spin’ in research and medical abstracts because they summarize the entire paper, and doctors often use them to help inform medical decisions. The researchers defined ‘spin’ as the deliberate use of specific reporting strategies to highlight that an experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results.
The study looked at scientific papers from the top six psychiatry and psychology journals from 2012–2017. Specifically, the researchers focused on randomized controlled trials with “nonsignificant primary endpoints.” The primary endpoint of a study is the main result of the study, and “nonsignificant” in this context means that, statistically, the team did not find enough evidence to back up their theory.
Spin comes in many forms, including selectively reporting outcomes, wherein the authors only mention certain results; or hacking, wherein researchers run a series of statistical tests but only publish the figures from tests that produce significant results; or use inappropriate or misleading statistical measures
In total, they analyzed the abstracts of 116 papers. Of these, 56 percent showed evidence of spin. This included spin in 2 percent of titles, 21 percent of the results section of the abstract, and 49 percent in the conclusion section of the abstract. In 15 percent of the papers, spin was present in both the results and conclusion sections of the abstracts.
Perhaps surprisingly, the researchers found no evidence that having financial backing from industry increased the likelihood of spin.
The findings are concerning. Although spin in the news media in general is worrying in itself, doctors use research papers to help steer clinical decisions. As the authors write: “Researchers have an ethical obligation to honestly and clearly report the results of their research.Adding spin to the abstract of an article may mislead physicians who are attempting to draw conclusions about a treatment for patients. Most physicians only read the article abstract the majority of the time.”
To drive home the implications of fabricating abstracts on decisions by doctors, the research team collected abstracts from the field of cancer research. All were randomized controlled trials with a statistically nonsignificant primary outcome. All abstracts included spin. The researchers also created second versions of these abstracts in which they removed the spin. They then recruited 300 oncologists as participants and gave half of them an original abstract with spin, and the other half the abstract with no spin.
Worryingly, the doctors who read the abstracts with spin rated the intervention covered in the paper as more beneficial. Though the study does have some limitations, the authors conclude that “authors, journal editors, and peer reviewers should continue to be vigilant for spin to reduce the risk of biased reporting of trial results.”