Okey dokey, I’m gonna talk a bit about statistics (yuck). I know, I bloody hate them too. But knowing a little bit about how to critically think about statistics that get presented will make it easier to sift the bullshit from the gold in terms of science.
A good example is this often repeated statistic, “The most common cause of food poisoning is not reheating rice properly!”
Now, you’d think food poisoning would mostly be caused by chicken or mussels or raw fish or something, but no, rice! But here we have a deceptive statistic. Rice is commonly eaten in third-world countries, who have poorer sanitation and hygiene, therefore they are more likely to get a food-borne infection. It’s hard to get a food-borne infection from chicken if you don’t eat chicken. Ya get me? So the deceit in this statistic lies in the “worldwide” part of it. For example, this is a cool map of the world that is coloured according to the number of deaths that are caused by diarrheal disease, which is often the outcome of food poisoning.
The ones that are closer to being red are the ones which have high rates of death due to diarrheal disease. Notice how they’re all clustered in Sub-Saharan Africa? Yeah, they basically throw out all the statistics for the rest of the world.
So lets talk about scientific studies, and the data that is generated from them. I’m gonna use the example of a study that I was a part of in 2016 which was a new medication to help Type 1 diabetics have better blood sugars. If we’re talking human disease studies, the gold standard is what we call a double-blind randomised control trial. In English: Double blind means neither the researchers nor the patient know what they are getting. The aim of this is two fold. The first aim is to look at the placebo effect in the patients. It’s been proven that if you think you’re getting treatment, you’ll see a positive effect even if you aren’t. The second aim is so that the researchers or doctors give exactly the same treatment to all patients regardless of whether they’re on the drug or not. In my trial, a computer generated a random number which was assigned to a bottle of pills; they were mine. They could have been placebo pills (aka have no active ingredient) or the pills with the potential medication in them. Neither I nor my doctor knew what was in the pill. The allocation to the group was random (which fulfills the “randomised” part of the type of study) and the control part means that for every person who was on the active pill, there was another person who was on the placebo drug.
Double blind randomised control trials are the best, but sometimes it’s impossible to do them. For example, a trial where one group of patients were given a jacket and the other (control) group were not; we know who got a jacket cos we can see it for ourselves. Sometimes it can’t be controlled; there are some ethical limitations on what you’re allowed to do. If you have evidence that your drug can miraculously cure cancer, you ethically are not allowed to deprive one group from the drug and give them a placebo instead.
If a scientific study is being undertaken, there are a few things you need to know to be able to critically acknowledge whether the study is good or not.
- Size – that “vaccines cause autism” claim? Came from a study of 12 children, 11 of whom were boys. We would say this is a small (tiny) sample size, and it doesn’t reflect the population accurately.
- Type of study – a study that looks back retrospectively at data isn’t as good as a study of a cohort of people that is ongoing isn’t as good as a randomised control trial.
- Statistical significance – this is a way of measuring how likely it is that a result is due to chance. The more significant the result is, the more likely it is to be a true result.
- Length of study – some things have short term effects, some things have long term effects. If you look at a group of people who have been smoking for one year, by the end of that year not many of them will have lung cancer. But if you look at that same group of people twenty years in the future compared to a control group, you might see an association. Not everything is instant.
- Variables measured – people are difficult and complex. Often there are lots of factors that go into causing disease. For example, obesity increases your risk of heart disease. But also, a fatty diet, diabetes, smoking, lack of exercise and genetic factors also increase your risk of heart disease. These are called confounding variables, because they confuse the message of obesity = heart disease. You have to control for these confounding variables. You do this by making sure they are all the same. Everything is the same except the thing that is different. You get me?
Here’s a thing that you might hear people say in the science community; correlation does not equal causation. Since the 1950’s carbon dioxide in the atmosphere has been increasing. Also since the 1950’s, the divorce rate has been increasing. Therefore, carbon dioxide causes divorce? Nah. Just because two things are correlated, does not mean one causes the other.
So there’s some wee tidbits to hopefully help you out next time you read a clickbait article. Bullshit detectors on! Thanks for reading friends, GET LEARNED!