Last week I wrote about conspiracy theories and the crazy stuff that people believe (link) but this week I am following that up with a guide to trying to figure out if a source is reliable. As I have stated before, the internet is a wondrous and horrible thing all at the same time. Never before in history has there been SO much information right at your fingertips; taking into consideration your available access to a computer. Unfortunately, anyone can make a website and put almost any nonsense on said website and a lot of people believe automatically what they read when it comes to the internet. The following are a few things to consider when you are reading something about the latest “scientific study”. (I use the quotation marks in this instance because although there are many, many legitimate studies, there are a lot of illegitimate ones as well.)
Dramatic headlines that mislead. A headline is supposed to grab a person’s attention with promises of something sensational. It is used as click-bait. (I had personally used click-bait when I wrote a blog about bacon. It was one of my top read blogs. What can I say? People love bacon) It will either over simplify the results of a study or even misrepresent them.
Consider where the research was published. Reputable scientists publish their research findings in reputable publications. These publications have expert peer review of the papers submitted to maintain their commitment to the highest quality. In recent years, many bogus scientific open access magazines have sprouted up and seem to be reproducing exponentially. They take money for publication, have no peer review, and often have names that are similar to their legitimate counterparts. So, if you are reading an article from the New English Journal of Medicine, chances are it is not a trustworthy journal.
Who are the entities that sponsored the study? Are they only looking after their best interest? If some huge company that has been known to pollute the water supply with mercury has funded a study that says that mercury is a cure-all for everything, you might want to reconsider this information. This is a gross exaggeration as an example but I use it to show that if something smells fishy, there may be a fish around. It is not to say that any research sponsored by any big multinational is not accurate; you just need to be aware of where this data comes from. Objectivity is paramount to any scientific study.
Beware of the difference between correlation and causation. This is one of my favourites on this list because it can be applied to many of people’s weird beliefs and goes back to the fact that people love to connect things. Correlation is the mutual relationship or connection between 2 or more things. Causation is the action of causing something. This means that correlation does not imply causation. Dizzy yet? Take this as an example: Passenger pigeons went extinct around the same time that Queen Victoria died but that does not mean that she was a passenger pigeon. Although, a bird monarch might be the answer to the world’s woes; we have not had one yet and the world has so many problems. If we had a bird monarchy, all these problems would go away. No? These were just two examples of this principle for you. If you want to read more correlations that are not causations, take a look here: https://science.howstuffworks.com/innovation/science-questions/10-correlations-that-are-not-causations1.htm
Sketchy testing. I use this heading to cover a few different items. This is a harder thing to consider if you are reading something on the internet because not many of us would go to the actual paper and look into these things but studies are often reported out of context (and it is good to know). Consider the scientific procedure behind the testing. Consider the study for a moment. Was there a control group? There should be a group involved, especially in clinical trials, that is exempt from the testing (such as a placebo group where this group would not be given the substance the rest of the group is receiving). Did the researchers have large enough group of test subjects? In science, normally the smaller the sample size, the lower the confidence scientists have in the results seen. In human trials, were there a larger representation of the general population used or was it only white males aged 18-35? The whole world is not made up of only white males between 18 and 35 and testing needs to represent that.
Data used in studies must show all results, not only the results that support the hypotheses. A researcher should not pick and choose the results that support their research while ignoring the results that do not.
Many of these items are most applicable in the realm of medical science but the principles behind them are transferrable. The foundations of scientific research should not be forgotten. Observation: Research has its start in the observable. This is where a scientist formulates his/her hypothesis and observation is also the place where data is collected. Prediction: The researcher will make a prediction as what the outcome to this hypothesis will be. Data: This is where the observable are tested against the predictions. If the observations match the theory, the theory is strengthened; if they do not, the theory needs to be changed. This is a cycle of theorize, test, prove/disprove, start again.
Although we are not necessarily scientists and/or researchers, everyone can benefit from these principles. A lot of people want to think that science is an absolute however, it changes and shifts as we begin to learn and understand more about this world around us but that does not mean science is not trustworthy. Science is not scary or beyond understanding (although quantum physics is both to me). Remember that you use science every day when you look at the world. Take that type of reasoning and use it when you are reading about the latest thing that causes cancer (as an example) and see what is the proof and where did it come from. Before acting on something you read, make sure that you are an informed reader.
The link below is a short video about bad science and how to recognize it when you see it. https://www.ted.com/talks/ben_goldacre_battling_bad_science
(Ben Goldacre is a British physician, epidemiologist, academic, and writer of a number of books about bad science, alternative medicine, and big pharma.)
–Janice Willson
