It seems like pretty much every day there is some new drug that’s supposed to work against COVID-19, e.g. hydroxychloroquine, ibuprofen, nicotine, remdesivir… Then the next day, they don’t work… I wanted to talk a bit about when we could get excited by one of these, going via our understanding of ‘the truth’ and a popular board game.
No pressure… So, when you read an article in the news about one of these drugs, there is normally some sort of sentence near the end along the lines of:
“Not tested in a randomised clinical trial. May not actually work. May even make COVID-19 worse. May just kill you regardless of COVID-19…”
And how does that help us?…
Well firstly, in these confusing times, I think it’s important to remember there are actually such things as facts and the truth… When a politician lies to us, they are not actually changing the truth – they are trying to change our opinion. Anyone can have an opinion about anything, however the funny thing about the truth is that it really doesn’t care what our opinion of it is. The opinion is either right or wrong, but the truth is what it is… Someone could have the opinion that gravity is a “made-up invisible force”, but if they jump off a building their opinion isn’t going to make them fly.
When it comes to a drug making a patient better, the truth of the matter is:
Either it works.
Or it doesn’t.
No amount of hypotheses, animal testing, or “I heard people are saying good things…” can actually change the fact that if a drug doesn’t work in a patient, it doesn’t work. However, proving this is actually very challenging – it is not as straightforward as just giving someone the drug and seeing if they get better… And here’s my best shot at explaining why:
One thing I love about science is that you can sometimes feel like a detective trying to figure out the truth. So, to use a ‘detective’ analogy, if you think of the board game Cluedo, your aim is to find out the murderer, the murder weapon, and the murder location. The way you do this is to discount all the other potential options, until what you are left with is the truth:
We have to apply similar logic when testing a drug in patients (a ‘clinical trial‘) because there are many different factors that can affect if a patient gets better. With COVID-19, we know age and weight affect recovery. In addition the ‘placebo effect’ (where someone thinks they are receiving a medicine so actually get better anyway) is incredibly powerful. So in a clinical trial, to really prove it is the drug that is making patients better, we need to discount age, weight, etc. and placebo effects.
To do this we take pairs of patients who have similar age, weight, etc. and randomly split them into two groups: one that gets the drug, and one that gets a placebo. Often, neither the patients nor the doctors know which group is which (called ‘blinding’). After treatment the two groups are assessed to see if the group who received the drug recovered faster:
This is what a randomised trial is, and this gives the most clear-cut answer to if it’s actually the drug that is making the patients better.
So you may be wondering what happens in a clinical trial that isn’t randomised? Well, as an example, maybe there is one group of patients who all get the drug, and they are compared to some other group of patients who didn’t get the drug, for example a group of patients at a different hospital:
The problem here is what if there is some difference in the quality of care between the hospitals? Or to take a COVID-19 specific example – what if the person running the COVID-19 tests in one hospital doesn’t know what they’re doing? Suddenly more patients in that hospital would appear to have got better because they don’t test positive for COVID, but actually this has nothing to do with the drug….
The original hydroxychloroquine study falls into this second category of not being randomised, and is what is referred to as ‘anecdotal evidence’. Basically it is an interesting observation, but it doesn’t give us a clear-cut shot at the truth:
It doesn’t tell us hydroxychloroquine works…
It also doesn’t tell us hydroxychloroquine doesn’t work…
The only thing it tells us very clearly:
We need to test hydroxychloroquine in a randomised trial ASAP to find out the truth.
And guess what?… These randomised trials are going on at the moment (e.g. here), so hopefully we’ll get a clearer answer soon. No amount of opinions will change if the drug actually works or not. And what is the most important thing in all of this that doesn’t give a damn about our opinion of a drug?… COVID-19… So we’d better make sure we’ve got the truth on this.
I guess as a rule-of-thumb – when we see a new drug in the news for COVID-19 (or any other disease for that matter), look to see if it was tested in a randomised trial:
If it was, maybe we can start to get excited…
If it wasn’t, wait for the sequel…
That’s not to say we won’t ever need to reassess a randomised trial (for example if we identify new factors that affect recovery), or that side effects won’t stop us from using a drug in future, but it is as close as we can get to the truth of if a drug works…
Next time, the scientific reasons why bleach can never work as a treatment for COVID-19… God, what a time to be a scientist…