How not to write a science story
Here’s a great example of how science should not be reported. The Fairfax Good Weekend magazine (paywall) of 17 March 2012 has a story by Janet Hawley about Barbara Arrowsmith-Young and her program for children with learning difficulties. The program is big in Arrowsmith-Young’s native Canada, and is very expensive. Now it seems the invasion of Australia is about to begin.
Hawley writes an article consisting almost entirely of anecdotes – some from Arrowsmith-Young, others from parents and children who have been through the program. Nowhere is there any attempt to find out whether there are any peer-reviewed studies on the program. As far as I can discover, there are none, but there is skepticism about the program. If the anecdotes are correct, where are the studies to back them up? How do we know this program is any more effective than less expensive existing programs? Arrowsmith-Young’s website contains no references to independent, peer-reviewed studies. Why wasn’t this mentioned in the story? How much research did Hawley do to check on the claims?
There are lots of references to ‘brain plasticity, and totally woo statements such as “It makes sense when you think of it as physiotherapy or a Pilates workout for the brain, strengthening weak areas through repetitive exercise, massaging stiff lesions and increasing flexibility so messages can travel through.” Makes sense to whom?
And there are the implications of the maverick who bucks the system, but in the end is shown to be right: “As with all paradigm shifts, education authorities have been cautiously slow in more widely adopting Arrowsmith techniques, which would mean a radical change in entrenched teaching methods for learning-disabled students.” Yes, Janet, and extraordinary claims require extraordinary evidence. Where is it? If they are going to commit to this program, education authorities should be looking for randomized controlled trials to show it’s effective. There are none.
The Arrowsmith program may turn out to be effective, but until we have reliable evidence from the real world to support this, we are not warranted in adopting it. If only Janet Hawley could have made this clear in her story.
Update: Would you believe that this story has been picked up and republished by The Week, which claims to select “the best of the Australasian and International media”? What does that say about the scientific literacy of our print media?
Science or not? 