Falsely balanced or balanced by falsity?

This is a guest post from vet, Simon Baker. It is timely for me as last week I turned down the offer of being interviewed by the BBC on a subject concerning quackery. I knew who I was ‘up against’ and struggled to see how such a report could ever be accurate and appropriately balanced. Simon’s post explores the problems of balance in the media when discussing fringe ideas.

Prologue

I am grateful to Le Canard Noir for allowing space here for this Little Black Duckling to highlight a recent radio report on homeopathy. It raises a number of interesting issues.

First let me point out that inevitably, much of what I say here is my opinion. There is a tedious tendency in pieces such as this to keep inserting “I think” or “in my opinion” not really because the author really wants to do so, but because the piece has been lawyered to make it libel-proof. Mark Crislip has made the comment;

“I hate saying in my opinion out of paranoia, it is repetitive and ruins the flow of the prose.”

The point of logical argument and the marshalling of evidence is to establish conclusions that are as secure as facts. Nonetheless, in what follows, feel free to insert the words “in my opinion” if you feel the need. Thus, if I state that, as a fact, homeopathy is bollocks, it’s only my opinion. And that’s a fact.

A bit of background

On 7th January I received an e-mail from scientist, journalist, broadcaster and sceptic Simon Singh asking whether I would be willing to take part in a debate as part of the Breakfast programme on the BBC Somerset local radio station. It was being set up on the basis of impressions of increasing use of homeopathy by farmers and the idea was that they had a farmer and a homeopathic vet lined up and wanted someone from the sceptical side. I said I was quite keen and so was put in touch with the programme’s producer. In conversation with her I confirmed my willingness to take part, but I made it clear that there is a well-recognised problem of false balance being created when fringe beliefs are set against the mainstream. The risk is that both parties are given equal time and equal prominence. This has also been more colourfully called the Hydrostatic Paradox after a comment by Oliver Wendell;

“… the hydrostatic paradox of controversy. Don’t you know what that means? Well, I will tell you. You know that, if you had a bent tube, one arm of which was of the size of a pipe-stem, and the other big enough to hold the ocean, water would stand at the same height in one as in the other. Controversy equalizes fools and wise men in the same way. And the fools know it.”

Another serious problem is that the fringe view is often dealt with from an “in Universe” perspective, taking its claims at face value rather than making any attempt to analyse them critically from an outsider’s viewpoint.

What follows is an exact transcript of the interview with the homeopath and vet, Geoff Johnson. I have annotated it in painfully pedantic detail. It is this detail that shows how careful one needs to be when listening to people making claims in support of homeopathy.

BBC Somerset:  Interview with a Farmer (MP3)

BBC Somerset: Interview with Simon Baker (MP3)

BBC Somerset: Interview with Geoff Johnson (MP3)

Interview with the Homeopath

Matt Faulkner (Presenter): Now, have you ever tried natural remedies, things like Arnica to treat bruising. It’s either something you really believe or, I guess, you’re really cynical about.

[I started by being merely sceptical. It’s only years of contact with homeopaths and my ‘shruggie”’ professional colleagues that has made me cynical, but I’ll let Matt get away with this one; I am cynical now.]

Well, BBC Somerset’s been told more and more farmers in the county are using homeopathic treatments on their animals. Jill Tapp who farms at Washford on Exmoor has been using these treatments for 12 years, including a tablet which she believes calms her cattle before TB testing.

Jill Tapp (Farmer): It does calm them. They don’t worry so much about being handled. And it’s obviously being actually physically being handled and pushed through the crush and strange people around. It’s going to stress them out and it’s out of routine.

MF: These types of remedies don’t come without critics. Simon Baker is a vet who speaks out quite publicly about his dislike for homeopathy.

[Quite publicly? It’s a scandal. Lock up your daughters. No decent people should be forced to hear his sceptical filth.]

Simon Baker (Sceptic, Cynic and Hero): Homeopathic treatments provided to animals by vets are supervised by those vets. My problem with those vets is that they shouldn’t be doing that. There’s an awful lot of self-prescribing and self-use of homeopathy as well and the British Veterinary Association some years ago described homeopathy as an offence against animal welfare and I would fully support that. The provision of homeopathic remedies is simply doing nothing.

MF: Geoff Johnson is a homeopathic vet from Wiveliscombe. He actually trained at vet school with Simon Baker and he told me why he believes so strongly in these remedies.

Geoff Johnson (Vet and Homeopath): Well, just because of the evidence I’ve seen. I’ve been using homeopathy now for 18 years as well as conventional medicine I’ve been qualified for 25 years from Cambridge(1) and I can do things with my homeopathy that I was unable to do with my conventional medicine. And I use it every day. I see the results.(2) And I’ve been involved in some of the trials, which have been done by the British homeopathic vets over the last 5 years, which have been exceedingly positive.(3) And I do it primarily, actually, for the benefit of my patients.

1. That thudding sound was that of a name being dropped heavily. Homeopathy, in common with all SCAM therapies, is heavily dependent on the perceived authority of its leading figures and this is used to give weight to their views. Sceptics prefer evidence rather than depending on the word of authority figures. As it happens, of course, I also qualified from that august institution, but have come to a rather different view on the subject of homeopathy. It really would be an impossible conundrum to resolve if authority was all you could go on. Fortunately we have more than that.

2. What’s this appearing over our horizon? Of course, it’s argument from personal experience, that well-known and reliable guide to what works in medicine, which is why we happily live in a world free of all laws regulating the sale of drugs because we are happy that the individual experience of patients tells us all we need to know. Not.
Sceptics ask for objective evidence where all the scope for personal biases, faulty recollection and the natural history of the disease are removed from the equation. This topic has been examined many, many times, but I particularly like the way that Richard Feynman phrased it;

The first principle is that you must not fool yourself—and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.

3. It would be nice to know more about these “exceedingly positive” trials. I don’t know of any. I do know of a set of rather ropy trials whose results were spun to the maximum by the sugar-retailing community. This is my favourite. It reports 20 allergically itchy dogs being given homeopathy. It was a strange experimental set up, but there is not the space to go into that now. The main point is that they started with 20 dogs and put only 5 into a blinded phase, so to draw any conclusions from it would be silly, though I still wonder whether Geoff does count it as one of those “exceedingly positive” results.

Homeopaths often seem easily impressed by things get they get into public print even if it’s their own house rag Homeopathy. Here’s one from 2007, “Outcomes from homeopathic prescribing in veterinary practice: a prospective, research-targeted, pilot study ” R.T. Mathie, , L. Hansen, M.F. Elliott, J. Hoare. The authors breathlessly conclude,

“Systematic recording of data by veterinarians in clinical practice is feasible and capable of informing future research in veterinary homeopathy.”

Homeopaths can use spreadsheets! It’s like we’re all living in the future. Amazing. But what did the Press Release issued by the Faculty of Homeopathy say?

Pilot study suggests homeopathy is effective in treating pets with common diseases
A clinical outcomes pilot study published in Homeopathy journal found that over 79% of animals prescribed homeopathic treatment by eight participating vets over a six-month period showed improvement in condition, as assessed by their owners.

“Clinical outcomes” study simply means you watch what happens to the patients. No controls. No randomisation. No basis for comparison. So, where did all those assertions of therapeutic efficacy come from? I really do not know. Oh, I think I do, but even post- BCA v. Singh I’ll leave that hanging.

Anyhow, back to our itchy dogs…the interesting thing again is in the details of how it was reported. Right from the get-go, the language used by the authors was incautious. The third line of the abstract starts with “The response to treatment was assessed…”. I’m sorry, but the point of blinded trials is to say whether there was a response to treatment. You don’t get to assert the existence of it. The study was also notable for the first mention that I am aware of in the veterinary scientific literature of a clairvoyant dog. Well, quite.

These details of language may seem pedantically subtle, but it is with the steady accumulation of these little incremental exaggerations that a greater festering heap of bullshit is built. There is a cumulative effect on the public narrative in support of homeopathy and of SCAMs more widely; tiny hints of effects are amplified and the stories successively grow in the telling to the point where a homeopath can appear on radio and tell us that recent British veterinary trials of homeopathy have been “exceedingly positive” and I can find nothing underneath that bold claim to sustain it. Perhaps “exceedingly positive” has a special meaning in homeopathy to which the uninitiated are not privy.

MF: Is this, maybe, a case where some things work for some people but they don’t work for others or do you genuinely believe it works for everyone?

GJ: No, it works for everyone.(4) A lot of the people who criticise homeopathy, in fact all of them, have never actually practised it or studied it.(5) And admittedly homeopathy is odd.(6) It works in a way which is not one of the ways we were taught when we were studying at vet school. Um, but just because it works in a different way to how, ah, we were taught, it doesn’t mean it can’t work.(7) And, ah, that’s the problem really. The people who criticise it generally know very little about it, instinctively, um, go against it primarily from a lack of knowledge.(8)

4. Everyone? Well, that’s brilliant. How come it does so poorly when trialled against adequate controls?
5. Geoffrey, please meet Prof Edzard Ernst. One might have hoped you would have heard of him, but maybe news travels slowly in Somerset. But regardless of that, here we are again as so often with homeopaths trying to play the man not the ball. The arguments posed by sceptics to homeopaths do not require that we have been indoctrinated into the faith. I don’t need to get into a detailed argument about the rules of Monopoly to be able to tell you that they are not an accurate description of the real, actual city of London.

6. At last, something we can agree on.

7. Sorta, maybe, kinda, but once we reach homeopathy the simple ridiculousness of its proposed mechanisms do put it beyond the pale. Homeopaths like to tell us about how they serially dilute their remedies to make them more powerful. The rest of us call it rinsing. If what they said was true then this morning’s cup of tea would give you a massively magnified hit from the caffeine in the coffee that was in the same cup last week. It would be more hilarious if there were not actual adult people walking around out there who believe in this. It’s not like they wear badges or anything, so you really can’t tell by looking at them.

8. I think people like myself, Andy Lewis, Alan Henness, Simon Singh, David Colquhoun, Edzard Ernst, and a host of well-informed anonymous and pseudonymous interweb commentators have studied the nuances of homeopathy for a long time and often show themselves to be more able to judge homeopathy against external objective standards but also to highlight its internal contradictions and idiocies.

MF: Now, Simon described it as strongly as being an animal welfare issue. What do you say to that?

GJ: Well, ah, interesting one. I didn’t know he’d say that.(9) There was a study done in Danish organic dairy herds, actually, about 8 years ago(10), looking at what happened when herds became organic, began to use more homeopathy and, in fact, they found that the welfare standards on the farms increased, so there’s absolutely no evidence for that whatsoever.(11) Um, if you’re interested that was in Livestock Production Science, which is a, ah, peer-reviewed magazine. Came out in 2003, actually.

9. It really is a bugger when the psychic powers fail.

10. Well, I’ve looked for it and, with some help [thanks, Mojo], have turned up this paper, which seems to be the only one that fits the bill. “Eleven years of organic dairy production in Denmark: herd health and production related to time of conversion and compared to conventional production” T.W Bennedsgaard, S.M Thamsborg, M Vaarst, C Enevoldsen. No mention of homeopathy in the title, or the abstract and the rest is behind a paywall. I presume Geoff must have read the whole thing before he cited it. But, do you know what? I suspect if homeopathy is mentioned at all it is highly tangential and the paper really does not tell us how homeopathy, aside from a whole host of other changes occurring on farms that change to organic systems, influenced the welfare of the animals. Perhaps someone familiar with its detailed content can comment on this.

It does, perhaps, bear mentioning at this point that I was recently told by the tutor of a vet student who was working on placement on an organic farm that the student had come back horrified by the use on that farm of homeopathy and, as policy, cows with mastitis who failed to get better after being given homeopathic sugar were culled from the herd rather than being given conventional treatment. Of course, it’s hearsay, so wouldn’t stand up in court, but the problem is that with so little veterinary supervision of these activities a great deal could go on unseen. We just don’t know.

11. Well, technically, speaking citing one paper that you say shows no problem does not allow us to draw a logical conclusion that there is no problem (even if the paper actually says what you say it says and I suspect it does not).

But, the problem here is, how would we find that evidence? All this goes on behind closed doors and in the face of convinced Tru Bleevers. It has been well said before that homeopathy is not a system of medicine, but a set of excuses. Any outcome can be justified and explained within the narrative of homeopathy and without reference to objective external measures. But, how many Penelope Dingles , Gloria Sams and less severe examples are there out there going unreported? The veterinary world is more loosely regulated than that of human medicine, so we really have no basis for assessing the potential for harm. The world of alternative medicine closets itself from outside judgements and its advocates seem to spend more of their time trying to excuse their therapies from having the normal standards of efficacy applied to them than seeking to obtain the necessary evidence.

The item within this broadcast that was with a farmer had her describing how she would switch to conventional antibiotics if she thought homeopathy was failing. By definition, therefore, homeopathy is given its chance to fail before effective treatment is instituted. There is a confusion, sometimes, between hazards and harms. We can define hazards based on an understanding of the situation. Whether harm actually occurs is a separate issue. An absence of reported harm does not mean that the hazard does not exist and we have no measure of the unreported harms.

MF: OK. The treatments that you use though, they’re very diluted, they’re very weak. Can they really work?

GJ: Well, they’re not very weak at all. They’re very strong.(12) Um, that’s why I use them, cos they’re so effective. Ah, but they are diluted but they’re not just diluted, you see, they’re diluted and then they are banged very, very vigorously.(13) It’s a process we call succussion. That’s just a long name.(14) It means vigorous banging. If you just diluted a homeopathic remedy, it would cease to work(15) as we in fact discovered when we were treating large herds of pigs with very big header tanks and you found that in fact the remedies didn’t work there unless you put more in.(16) It’s the process of sequential dilution and then succussion, this banging(17), and then they become stronger and stronger.

12. I think we’ve covered this already.

13. Oh, it’s not the dilution, it’s the banging as well that makes them so strong. I don’t see how that really rescues homeopathy, but homeopaths seem to believe this. Presumably they see plenty of evidence that vigorous banging is excellent for imparting order into physical structures. But, joking aside, if it’s the vigorous banging that fundamentally alters the properties of solutions as they are diluted, then biochemistry labs around the world would be in a daily state of confusion as their diluted and vortexed solutions generated all kinds of weird dose-response effects. If you’ve never worked in a biochem lab and don’t know what vortexing means in this context then this little video will tell you. Homeopaths require their clever little water molecules to know they’re in this machine not a vortexer and utterly (and permanently) alter their physicochemical properties to suit the whim of the homeopath.

Do homeopathic remedies really need all this banging? What about remedy ‘grafting’ and paper remedies? ‘Grafting’ means putting one magic sugar tablet in a bottle of fresh tablets to turn all of them into magic medicine. Don’t believe me? Read this, but there are references all over the place from homeopaths advocating the same thing. Even better are ‘paper remedies’. Here you just write the name of the remedy on a piece of paper and put it under the bottle of pills or even pin it to the patient/victim. Some homeopaths balk at this, but what can they do about it, all homeopaths depend on personal anecdote and criticism of one is criticism of all. All these more extreme ideas do is to make explicit the inherently magical nature of belief in homeopathy.

14. Well, count me impressed.
15. [Citation needed]

16. Ooh, goody an anecdote instead. I’ll skip the need to point out the lack of controls etc., instead I’ll ask why this last step of dilution should make any difference. These milligram-sized tablets are given to huge great cows and little tiny babies. Either it shouldn’t work on the cows or it should make the babies explode with all that pent up magic potentisation. Don’t ask me. I only ask for internal consistency in how I think medicine and chemistry work and we’ve already been told we’ve not been educated enough to understand homeopathy.

“Here Vroomfondel, why do we never think of things like that?” “Dunno,” said Vroomfondel in an awed whisper, “think our brains must be too highly trained Majikthise.” Adams, D.

17. Enough with the banging. I have a headache.

MF: Why do you think more farmers are trying these sorts of treatments because I’m presuming they’re not cheap.

GJ: No, they’re very, very cheap that’s one of the reasons…

MF: Oh, OK.

GJ: …they’re probably using them. I, ah, have treated on a number of occasions treated an entire herd of cows. The drug cost was about 12 pence.(18)

18. Cost-benefit ratio? Still asymptotically approaching infinity.

MF: Really?

GJ: This is one of the things about homeopathy. It is so vastly cheaper than, um, conventional medicine that people in certain areas aren’t very pleased about that because the profit you can make from homeopathy is much smaller. It’s interestingly, the Swiss government(19), their report into homeopathy was translated just last year and they investigated very, very, very thoroughly over 300 pages of [sic]…and they have now elected to include it in their equivalent of the National Health Service. One, because of its effectiveness but, two, cos they saw that it was considerably cheaper than conventional medicine.(20)

19. What? The Swiss Government, the actual Swiss Government? Themselves and not a bunch of homeopaths who took it upon themselves to do it?
20. I don’t think so, but I know some people who know all about it. I think we should let Sven Rudloff and Zeno have the floor for this. That ‘neutral’ Swiss homeopathy report

MF: Geoff Johnstone [sic](21), a homeopathic vet from Wiveliscombe.
21. Never mind. They called me Simon Barker at the end of my section, which given that I am a vet caused great hilarity in the studio. Laugh? I nearly did.

Epilogue

Following my initial agreement to appear on the programme, things went a bit quiet and I wondered what was happening. Someone at BBC Somerset then told me that the farmer they had lined up to take part had now chosen to back out because they were concerned about that appearing on the radio in support of homeopathy might damage their relationship with their conventional vet, because the conventional vet was not fully aware of the use of homeopathy on the farm. This has a set of rather concerning implications. It means that the conventional vet is not being made properly aware of management practices on the farm and a certain amount of embarrassed concealment must be occurring. Vets themselves are supposed to maintain professional contact with each other if they are caring for the same patient/s. How, one must ask, does this occur when the use of homeopathy is being kept hidden from the conventional vet? If a proper exchange of clinical information is not occurring as cases drift from homeopathy to conventional medicine then how do homeopaths really have any idea how well they their patient population is faring? I have often wondered with homeopaths whether their perception of their “success” rate involves an awful lot of patients who are simply ‘lost to follow-up’, which means that the cases with whom they maintain contact are simply the ones that, by random chance plus a host of biases, have improved during their period of treatment. Derren Brown exploited this bias in his TV programme called “ The System ”. More formally the phenomenon is described as “ Survivorship Bias . If you surround yourself only with those who were lucky enough to improve after your treatment it’s hard to avoid thinking you’re doing good. That’s why we need objective evidence.

Cases may also be reported to have improved even if by objective measures they have not. This was shown recently in an acupuncture trial. More personally, I first became acutely sensitised to the problem of homeopathy by a Radio 4 programme some years ago where a homeopath was describing a case of hyperthyroidism in a cat. He was showing to the show’s presenter the cat’s full range of persistent clinical signs (rapid heart rate, thin body condition, greasy coat) to illustrate the nature of the disease while at the same time the owner was thanking him for the success of the homeopathic treatment. It literally stopped me in my tracks. Homeopathy can be used to fiddle about in the margins of problems where as a minimum it does no harm, but it can create a justification for delaying the presentation of more significant problems to be dealt with using soundly established and effective treatments.

And finally…

On the day I started preparing this blog, I was feeling quite unwell. Muscles aching. Shivering. Could barely stay awake. Probably coming down with a cold, or even man ‘flu. I was looking forward to a weekend on duty under the miserable influence of a typical winter lurgy. The following morning I woke up feeling fine. If I had been so foolish as to take a homeopathic remedy on that first day, I could confidently have given the credit to it for my miraculous recovery. It seems so often to me that the claims for benefits from SCAM therapies are dependent on the patient defeating a prognosis that can never be asserted with total confidence. Double your woo, double your fun. SCAM therapies depend for their reported success on defeating the prognosis that the psychic power of their users had provided. “I was coming down with a cold, so I was going to be sick for a week. But I swallowed some 30C Bos Faecalis and, hey presto, I was better overnight”. I’m rather more inclined to doubt the efficiency of these psychic predictions rather than the idea of magically activated sugar pills.

As I have pointed out, these are all my opinions and they can be altered by good evidence so I would love some homeopaths to turn up and comment. Let’s see if any do.

 

[Edit: One of the YouTube linked videos wasn’t linked properly. It is now]