Misrepresentation of ADHD in scientific journals and in the mass media

The scientific community often discusses the misrepresentation of health news by the media. A less discussed subject is misrepresentation of data in the scientific literature. Gonon, Bezard and Boraud used their knowledge about ADHD to find misrepresentations of data in scientific literature and mass media, and found that the misrepresentation problem often begins in the scientific literature.

1. Internal inconsistencies

The good news is that only 2 out of about 360 papers (Barbaresi et al and Volkow et al) had "obvious discrepancies" between results and their authors' stated conclusions.

The bad news is that both papers had been covered by the media, who mostly accepted their conclusions as gospel. Gonon et al say that in the 40 mass media articles they'd read about the Volkow et al. paper, "We have never read a mitigating statement saying that their results are open to the opposite interpretation although the authors explicitly raised this

possibility in their result section." Out of 21 the articles written about Barbaresi et al's paper, only The Guardian's article questioned the conclusions. More than that: out of the 30 times the Volkow et al paper was cited in scientific papers, in 20 the authors quoted its conclusion without pointing out the discrepancies.

2. Fact omission

It goes like this:

Summary: A totally controls B!

Result section: A controls B if C is present and D isn't.

In this part, the authors focused on papers dealing with "the association between alleles of the gene coding for the D4 dopamine receptor (DRD4) and ADHD." According to the authors, previous research has shown that while there is an association between higher frequency of a certain DRD4 allele and ADHD, it only occures in 23% of ADHD patients, as opposed to 17% of the control population. Out of 117 papers about ADHD research done in humans that mentioned the DRD4-ADHD connection, 74 mentioned the association in their summaries, but only 19 of those also mentioned the conferred small risk. All 25 papers which mentioned the association but didn't present data on it had the misrepresentation in their summaries. In review papers, out of 43 summaries, only 6 mentioned that the allele confer only a small risk.

The DRD4 gene, ADHD and the mass media - Media outlets have been known for their tendency toward genetic determinism (the "gay gene" for example) and so were quick to adopt the view that ADHD is "genetic". Out of 170 articles between 1996-2009, 168 mentioned that the DRD4 gene is significally associated with ADHD and out of those, 117 didn't mention the small risk and/or presented the raw data. 26 articles mentioned the 1.2 to 1.34 odd ratio but also stated there's a strong connection between the gene and ADHD. The authors' conclusion is that 82% of the articles misrepresented the association, a rate similar to that observed in the scientific literature.

3. Extrapolating basic and pre-clinical findings to new therapeutic prospects ("Hi, it worked on mice!")

The authors surveyed 101 papers dealing with the mouse brain for 3 common overstatements, and found that 56 overstated their conclusions. 23 even fantasized extrapolated about new therapeutic prospects. Naturally, those 23 papers were published in higher-impact journals and the overstatements made their way to the mass media. Out of 63 mass media articles, only 11 contained migtated comments.

Limitations

The authors consider their work to be qualitative rather than quantitative, since the selection of papers in the first case was not systematic. In the second and third cases the papers were selected after a systematic search, but the authors only highlighted one aspect of misrepresentation in each case. While the results correlate with misrepresentation in the mass media, there's no way to determine causation.

In conclusion

When I was young and working on a Biology degree, my (great) professor read us an abstract and said something along the lines of "They added that definitive conclusion in the end so the paper will be published in a better journal". While anecdotes aren't data, it does seem that scientists sometimes overstate their results in order to be published in higher rank journals.

It's easy to blame the mass media whenever people put on their tin hats, but the responsibility also falls on scientists to report their findings as accurately as possible, even outside the result section.

Gonon, F., Bezard, E., & Boraud, T. (2011). Misrepresentation of Neuroscience Data Might Give Rise to Misleading Conclusions in the Media: The Case of Attention Deficit Hyperactivity Disorder PLoS ONE, 6 (1) DOI: 10.1371/journal.pone.0014618