Quacks love anecdotes. They seem to love them even more when they’re not true. On the web I’ve seen, for example, homeopaths claiming to have successfully cured cancer and kidney disease to the rapturous applause of their imbecilic devotees. I recall one even claiming to be able to treat third-degree burns better than conventional medicine and encouraging his sceptical critics to go burn themselves badly so he could prove it. Ethics aren’t something common or garden quacks like to bother their pretty little heads about, evidently.
They don’t like anecdotes so much if they show alternative therapies in a poor light. When people start claiming that altmed is totally safe, I frequently respond by posting a link to the what’s the harm? site. This is invariably greeted with snorts of derision, often by the same people who five minutes earlier were citing cases of their own as evidence of their miraculous healing power using one or other quack therapies. They don’t like that site because the stories they contain are “just anecdotes”. Seriously, that’s what they say.
I’ve had it said to me twice in the last couple of weeks, in both cases by chiropractors — or ‘spine wizards’ as Robin Ince calls them — who are reeling from the effects of the backlash against Justice Eady’s ruling against Simon Singh. Hundreds of complaints have been submitted to their regulatory body pointing out that, in contravention of their own Code of Practice, they are claiming on their websites to be able to treat all manner of ailments regardless of a dearth of evidence that chiropractic works for any of them. Whether those who disdain the many tragic stories about quackery going wrong are feigning obtuseness or really are a bit thick is anyone’s guess but for the benefit of anyone who can’t see the difference between a postive anecdote and a negative one, I’ll explain:
A few days ago I woke up with a horrible ache in my neck and shoulder, which bugged me all morning. I have always wanted to try a reiki massage and thought this would be a good time to get one. By the evening, the pain had gone — but no thanks to reiki. I hadn’t been able to find local practitioner who was free but, happily, the ache went away of its own accord. If I’d seen a reiki masseur (or, for that matter, a spine wizard, a physiotherapist or anyone else), would I have attributed my recovery to the therapy? Unlikely, though I might have done once upon a time and I can see why others still might.
If a therapist treats someone for, let’s say, a headache, and the headache gets better, does this prove that the treatment made the headache better? No, it doesn’t. The treatment may have made the headache better. Or the headache may have got better for some other reason. Until the treatment is tested in sizeable, well-designed clinical studies with controls and any positive results have been replicated, one cannot assume that the treatment made the headache better. And vice versa. A friend of mine told me she had binned a homeopathic remedy she’d bought off the shelf of a high street chemist because whenever she used it she caught a cold! She seriously thought there was a causal link.
Anecdotes are fine as long as they are not used as a basis on which to categorically claim that a treatment is effective. This goes to the heart of what the skeptic response to altmed — including all those complaints against the chiropractors — is about. I’m sure none of us have any problem with someone saying, “This treatment seemed to work for me, why not try it and see if it works for you?” and I have personally tried several alternative therapies through this kind of recommendation. (And none of them worked, probably because they required a faith in their efficacy I simply couldn’t muster).
But what if someone tries an unproven treatment, suffers ill-effects, and seeks medical assistance? In such a case a conclusion on the cause of the ill-effects will be drawn according to the evidence available to the medical practitioners treating the patient (or the coroner, if the patient died). To avoid committing the same post hoc ergo propter hoc fallacy as those who cite anecdotes in support of various quack therapies, there must be good reason for concluding that the treatment caused the ill-effect.
This article is based on the autopsy report for 24-year-old Kristi Bedenbaugh who saw her chiropractor for sinus headaches. During a neck manipulation she suffered a brain stem stroke and she died three days later.
Autopsy revealed that the manipulation had split the inside walls of both of her vertebral arteries, causing the walls to balloon and block the blood supply to the lower part of her brain. Additional studies concluded that blood clots had formed on the days the manipulation took place.
What makes the story about Kristi different from other anecdotes is that it includes a sound physiological explanation of cause and effect by somebody qualified to give one. No nonsense about removing the “spinal interference that keeps your nervous system from functioning as it should” or treating like with very small doses of like to “trigger the body to heal itself” and all the other guff they come out with.
By the way, time and again, I see quacks criticising skeptics for attacking therapies we haven’t tried but who, in their right mind, would try let a spine wizard touch them after reading this?
I’ve seen more cases of vascular injury following chiropractic manipulations than just about anybody, and these people’s lives are ruined. I’ve known of seven or eight people who died right on the table or shortly thereafter.
These are the words of Alan Bragman, an Atlanta chiropractor who has served as an expert witness in some 900 chiropractic cases in the United States, Canada and Puerto Rico. But of course, they’re just anecdotes.
The ‘diktats of evidence-based medicine’
This is the phrase one commenter on this blog, Neil Menzies, used and the suggestion of narrow-minded authoritarianism is taken a step further by a commenter on Zeno’s blog called BCA Chiro, who refers to “evidence-based fascism” which, he argues, “gets in the way of good clinical patient care”.
Let’s just be clear what the term ‘evidence-based medicine’ means. It doesn’t just mean treatments that have been proven to work by randomised controlled trials (RCTs). It means using what is the current best evidence in making decisions about the care of individual patients. Trica Greenhalgh, in her book How to read a paper; the basics of evidence-based medicine, points out that not all mainstream medicine is evidence-based: the best current evidence might not be used because practitioners don’t know about it, or they do but can’t access it or prefer to ignore it, so they rely instead on their own clinical experience when making decisions instead. Says Greenhalgh
Clinicians glean crucial information from patients’ illness narratives – most crucially, perhaps, what being ill means to the patient.
Thus, clinical experience is extremely important but it has its limitations and Greenhalgh points out that those who rely entirely on clincial experience are excluding the knowledge of countless other cases that could inform and improve their decisions.
As I said earlier, a major problem skeptics have with the promotion of altmed is that practitioners tend to rely heavily on their own experience when claiming that a given therapy is safe and effective. Then again, some do like to back up their anecdotes by digging up any RCTs they think show even the slightest suggestion that there may be something in their therapy of choice and quote it at every opportunity. Another commenter on this blog, AJP, who claimed to be a homeopath, was so ignorant and abusive to other commenters, I began to think he was a just a troll trying to give homeopaths a bad name. I only accepted he was for real when he cited the 1997 metanalysis by Linde et al as an example of positive evidence for homeopathy. This is a favourite tactic of homeopushers. What they omit to mention is Linde’s follow-up paper in 1999, which concluded that
in the study set investigated, there was clear evidence that sudies with better methodological quality tended to yield less positive results…it seems, therefore, likely that our meta-analysis at least over-estimated the effects of homeopathic treatments.
Other favourite papers include the 1991 metanalysis by Kleijnenetal which homeopushers again hold up as proof positive for homeopathy, even though Kleijnen said,
the evidence of clinical trials is positive but not sufficient to draw any definite conclusions.
Then there is Cucherat et al, another one one the list of supposedly positive results for homeopathy. What does Cucherat actually say? Let’s take a look:
There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies. Further high quality studies are needed to confirm these results.
There is a pattern here and the words “clutching at straws” spring to mind. Similarly, the BCA said there was a “plethora of evidence” for their claims that chiropractic works for a variety of ailments but they took 15 months after suing Simon Singh to produce it to a chorus of derision that reverberated around the science-blogosphere. See, for example, here, here and here.
When it comes to RCTs, quacks cannot win so they should give up on that line of defence and most have, preferring instead to argue that the RCT isn’t a useful way to test alternative medicines. Some even go so far as AJP does and say,
I like real world observational studies. They reflect what the patient feels and homeopathy has INCREDIBLE results. It is through observational studies that the overwhelmingly positive nature of homeopathy can be perceived clearest. A 70-80% favourable outcome has been experienced by thousands of homeopathic patients as detailed in multiple peer reviewed observational studies in real world situations.
There you go. Ask people how they feel and if they feel better — eureka! Homeopathy — or chiropractic or whatever it is — works. But if they don’t feel better — or if they die — that’s just an “anecdote”. Whatever.
Absence of evidence isn’t evidence of absence
This is another thing that gets quoted at me frequently as if it’s a piece of inscrutable wisdom carved in an ancient tablet of stone, though it was originally said by Carl Sagan some time in the last half century and I’m sure its appropriation by quacks and woo merchants would cause him much mirth. What do they actually mean by it? Presumably that just because there isn’t a jot of evidence that it works doesn’t mean it doesn’t work. I wouldn’t argue with that. Just stop claiming there is evidence when there isn’t and you won’t find yourself the subject of a complaint to a regulatory body.
Got it now?
This short tv broadcast from Australia seems a good one to finish on. In it we hear Bruce Walker of the Murdoch University School of Chiropractic describe two randomised controlled trials investigating whether chiropractic is a useful treatment for colic. The first RCT showed a positive effect, the second showed no effect. The difference between the two was that in the second trial the parents were blinded i.e they handed their babies over and didn’t know whether the babies were subsequently given chiropractic or not. We then hear Professor Chris Maher of the University of Sydney describe one controlled trial that singularly failed to show any difference in the rate of recovery for those who had spinal manipultion for muscular skeletal pain and those who just had ‘basic care’. We also hear disturbing “anecdotes” about deaths and strokes caused by chiropractics manipulating people’s necks.
A very useful post on zygoma’s blog looks at how a recent Daily Mail online article tries to mislead readers into believing homeopathy works.