In the modern world we have access to an almost unlimited supply of information.
Social media can rapidly spread misinformation and people often quote “evidence” without understanding where it came from, or what it actually means. It is all too easy to read information and come to the wrong conclusions.
Not all “evidence” is equal. Before coming to a conclusion after looking at “evidence” you always need to consider the:
- Type of evidence
- Strength of evidence
- Source of evidence
- Potential conflicts of interest
- Bias
- Confounding factors
- Correlation and association doesn’t mean causation
Why is the source of evidence important?
The source of the information is important to consider. Did you read / find it in a peer-reviewed scientific paper, where it can be critiqued by international specialists in their field? Or did you find it on social media?
What about conflicts of interest?
When looking at scientific papers, you need to see who funded or sponsored the study.
For example: imagine you were looking for evidence that a certain type of food was safe to give your pet. If all the studies were sponsored or paid for by the food company, there would clearly be a conflict of interest. The company is more likely to present the information in such a way to favour the outcome they wanted and they may downplay any negatives.
What is bias?
Bias is when a study has systematic errors which deviate the outcome. For example: imagine doing a study to find out the average height of a dog. If you only included Dachshunds your results would be severely biased!
Correlation doesn't mean causation!
When two factors always seem to be connected (correlated) it doesn’t mean one CAUSES the other.
For example: Imagine you did a study looking at how many dog leads you spot on a walk. For every dog lead, there is likely to be a dog attached or nearby! The dog lead isn’t the CAUSE of the dogs being taken on a walk, as it is their owner who has decided to take them out. However, the two will likely be present together when on a walk, and hence be CORRELATED or ASSOCIATED.
What are confounding factors?
Confounding factors have an influence of both variables of a study. For example: you collect information about a dog’s water consumption and panting. A confounding factor would be heat, as this will cause both more panting and more drinking.
PYRAMID OF EVIDENCE
An evidence pyramid visually represents the different types of evidence and their relative strength or weakness.
At the bottom of the pyramid we have opinions and anecdotes. Unfortunately, these are often the most plentiful but unreliable and weakest form of evidence. For example: your dog finds a piece of carrot and eats it. It would be unreliable to see this ONE dog each a carrot ONCE and then come to a conclusion that ALL dogs will ALWAYS eat carrots when offered! It is a small piece of evidence, but is weak by itself.
At the top of the pyramid we have systematic reviews and metanalysis. This type of evidence takes the most time and resources to study and collate. As a result they are fewer in number but strongest and most reliable. For example: a team of specialist dog feeders come together to find every study ever done on feeding carrots to dogs. They look at every study ever done, on every breed, at every age of their life and in every country. They then critically evaluate every study using strict criteria, such as bias and confounding factors. At the end of this they reach a strong and reliable (but not infallible) conclusion.