Skip to content

Single study syndrome – clutching at convenient confirmation

October 23, 2012
The notion that the latest scientific information trumps all is a red flag.

How to recognise this tactic

This tactic shows up when a person who has a vested interest in a particular point of view pounces on some new finding which seems to either support or threaten that point of view. It’s usually used in a context where the weight of evidence is against the perpetrator’s view.


… “single-study syndrome” … tends to turn up whenever a political agenda is threatened or supported by a specific line of scientific inquiry. “Some new finding, however tentative, gets highlighted while the broader suite of research on a tough subject is downplayed or ignored, …”

Mark Follman, quoting  Andrew Revkin, American science writer, 2011.

Why do people use this tactic?

People use this tactic to try to convince you that the evidence supports their position. They hope you are fooled into believing that, because a single new study is hot off the presses, its evidence supersedes all previous evidence.

What’s wrong with this tactic

In science, it’s the overall weight of evidence that matters. Unless a new study turns up evidence that falsifies most of the existing evidence, it can’t be considered any more influential than other studies. In most cases, it should be regarded as simply another piece in the jigsaw of evidence, especially while it is still subject to post-publication peer review and possible attempts at replication. It’s not uncommon for faults to be found in recent studies that render the results worthless.

What to do when confronted by this tactic

Don’t let yourself be talked into accepting the latest study as the final word on a topic, even if it supports your own viewpoint. Ask (or find out) how the study fits in with the full spectrum of evidence on the topic. You may be able to find systematic reviews on the topic to help with this. To get some idea of how influential a new study will be, try to tap into the informal post-publication reviews that invariably appear on respectable blogs. Remember that scientific models are tentative and withhold judgement until the quality of the evidence is established.

Variations and related tactics

This tactic derives from confirmation bias, the human tendency to favour information that confirms existing beliefs. It has similar characteristics to the anchoring effect, but in anchoring, the single study is responsible for establishing the initial bias.

Examples

  • A recent animal study on the health effects of GM maize was immediately jumped on as the last word on the topic by opponents of GM food (for example, see here) because it claimed to show that the maize produced cancers. Almost immediately, the validity of the study was questioned (here, here, here, here, here), as was the unusual behaviour of the authors in publicising the study (here). The study has now been widely discredited. Animal studies have shown that there are few health concerns associated with GM foods, although one recent review explained that much more investigation is needed.
  • In August 2011, climate-deniers seized on a study from CERN that in their eyes proved that climate change is driven by cosmic rays. Multiple independent lines of evidence show, of course, that the earth’s climate is changing as a result of human-produced greenhouse gases. Although the study shed some light on the role of cosmic rays in aerosol formation, in the words of its lead author, “it actually says nothing about a possible cosmic-ray effect on clouds and climate, but it’s a very important first step.” As Skeptical Science explains:

In reality, the CERN experiment only tests the bolded step in this list of requirements for cosmic rays to be causing global warming:

  1. Solar magnetic field must be getting stronger
  2. The number of cosmic rays reaching Earth must be dropping
  3. Cosmic rays must successfully seed clouds, which requires:
    1. Cosmic rays must trigger aerosol (liquid droplet) formation
    2. These newly-formed aerosols must grow sufficiently through condensation to form cloud-condensation nuclei (CCN)
    3. The CCN must lead to increased cloud formation
  4. Cloud cover on Earth must be declining

Update 2013/11/12: This article by Dana Nuccitelli explains how a barrage of new papers now show that variations in cosmic rays are not responsible for current global warming. Here‘s another by Rasmus E Benestad at RealClimate.

  • In September 2012, the Annals of Internal Medicine published a Stanford University systematic review into the safety and nutritional value of organic foods. It found that:

The published literature lacks strong evidence that organic foods are significantly more nutritious than conventional foods. Consumption of organic foods may reduce exposure to pesticide residues and antibiotic-resistant bacteria.

Another systematic review published 2 years earlier had likewise found no difference in nutritional value between organic and conventional foods. Nevertheless, the Stanford study provoked criticism from supporters of organic food, to the extent that a petition calling for retraction was organised. Even if the paper were retracted (and there are no reasons why it would be), this would have little effect because the papers that were analysed in the study still convey the same message.

  • Update 2012/11/21: Here’s an interesting discussion at Climate Progress about a paper called “Little change in global drought over the past 60 years” which appeared in Nature recently. It clashes with the existing literature on the topic and drew immediate criticism from other climate scientists.
  • Further update 2013/01/04: Here’s an article by David Freedman describing the harmful effects of single-study syndrome on personal-health journalism.
  • Update 2013/04/15: A recent study (here) gained a huge amount of publicity because it claimed to show once and for all that the Mediterranean diet prolonged life. However, major flaws have been pointed out in this study (herehere).
You can add to the list of examples by leaving a comment.

Mark Follman’s quote is from an article on Mother Jones.
The Neutrinos cartoon is from xkcd

This is one of ScienceOrNot’s Science red flags. See them all here.
11 Comments
  1. Jeshyr permalink

    Ohh, I’ve seen TWO of these recently!! Firstly the Daily Mail “reports” that Global Warming reportedly stopped 16 years ago on the back of a single data set that contradicts heaps of other evidence.

    Secondly there was a recent study with a very long title Thimerosal-Derived Ethylmercury Is a Mitochondrial Toxin in Human Astrocytes: Possible Role of Fenton Chemistry in the Oxidation and Breakage of mtDNA in the Journal of Toxicology. Allll the anti-vaccination conspiracy theorists seem to be using the study as “proof” that vaccines cause autism despite the study having almost nothing to do with either topic – here’s an example:
    http://www.facebook.com/pages/Vaccine-Awareness/219712048104028 (facebook login required)

    Love your blog :)

    • Yes, good examples Jeshyr. Thanks for suggesting them. I’m thinking of updating this post every time a new one comes along, but they seem to crop up so often I probably won’t be able to keep up.

Trackbacks & Pingbacks

  1. Why do liberals hate science? | Scholars and Rogues
  2. UPDATED: Why do liberals hate science? | Ars Skeptica
  3. The bad science checklist of GMO opponents
  4. A Guide to Looking Smart on the Internet: How to Find and Evaluate Online Information | Icarian Jobbernowl
  5. About Those Industry Funded GMO Studies . . . | REALFOOD.ORG
  6. About those industry funded GMO studies… « Biology Fortified, Inc.
  7. Mahalo for the response: Fully Functional Now! | Aloha Politics
  8. A Guide to Looking Smart on the Internet: How to Find and Evaluate Online Information | Skepti-Forum
  9. Richard Green on the Scientific Consensus and GMOs | Skepti-Forum

Be part of ScienceOrNot? Write a comment, make a suggestion or add an example!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,194 other followers

%d bloggers like this: