Part 5: Evaluating Evidence
In the last essay, we examined the different types of evidence, and the relative weight that each type carries in a debate. Now, we will turn to the question of how we shall evaluate these types of evidence for validity and accuracy. In critical thinking, there are three important ways in which we must examine evidence. First, we must test our own evidence as objectively as possible. The desired outcome, after all, is truth. In trying to answer any question, we should divorce ourselves from the notion that winning means anything. Truth means everything, and winning is only good if your proposition happens to be the correct one!
Second, we must apply the exact same standards to the evidence of our opponent, or in the case of an internal debate, the belief that we do not hold. I mention the need for examining both sides equally not to be overly pedantic, but because this is the cause of the second most errors in critical thinking that I have witnessed in my life, logical errors being the most common (Fallacies and logical mistakes come later). Let me emphasize this again. In order to be intellectually honest, and have the best chance of reaching the correct conclusion, or to have the best chance of winning an actual debate, YOU MUST EXAMINE BOTH SIDES CRITICALLY AND COMPLETELY. This point is so important, I'm going to say it yet another way. Unless you know at least as much about your opponent or the opposite idea, you have not examined the evidence sufficiently, and you are in grave danger of making a poor decision or losing the debate!.
Thirdly, we must sometimes render judgment between two debating parties or ideas. Often in life, we are not direct participants. When we watch a political debate, we must decide who the winner is. In the rare cases when facts and accurate information are actually presented, we must have the tools to accurately assess whose facts and evidence are more compelling. This can also extend to very mundane situations, such as resolving conflict at work, or in the family. Finally, in this series of essays, we are getting to some real meat. This is where critical thinking can make your life better! If you master the techniques of critical thinking, and learn to remove your ego from decisions, relying instead on facts, you will find that you make better decisions, and that as a direct consequence, your life will improve.
So, enough of the motivational seminar. On to the meat.
Tests of Logical Adequacy
The first question we must ask is very simple. Is there enough evidence. How much is enough? To be absolutely precise, "enough" is whatever amount is necessary to sway an objective audience. This may not seem helpful at first, but it's worded carefully, and precise reading will be illuminating. Notice that I did not say "enough evidence to sway your opponent." In an actual debate, your opponent will very seldom be convinced of the truth of your position. For debate, the audience is the target. How much evidence is enough for an internal debate? If you're trying to make an important decision, you have enough evidence when you can reasonably say that you have researched BOTH positions thoroughly enough to feel well versed in them.
Once we have decided if there is enough evidence to make a decision, the next question is, "Is the evidence clear?" Ambiguity is the enemy of reason. In order to make an informed decision, we must be certain that not only is there abundant evidence, but that the evidence is presented with clear, unambiguous definitions, and that we have a clear and unambiguous understanding of these definitions.
If the evidence is sufficient in volume and clear in content, we must ask if it is internally and externally consistent. Does it agree with known fact? Is it equally or more credible than any other source of evidence available? More importantly, is the evidence consistent with itself? Internal contradiction is the cardinal sin in critical thinking. The only thing we can know for certain about internally contradictory evidence is that something about it is definitely wrong. Let me say that again. If there is an internal contradiction, then we know with certainty that some element of the evidence is undeniably wrong!
It's important to recognize that consistency with "known fact" is not always a slam dunk. For many years, it was a known fact that communism was the biggest threat that America had ever faced. Once critical thinking skills finally prevailed, it became apparent that a U.S. Senator with an axe to grind and witches to hunt had been a much bigger threat, and that much of the Red Scare had been fabricated or exaggerated.
Once we have determined volume, clarity, and consistency, we must examine verifiability. Substantiation is crucial to good decision making and good debating. Conversely, unsubstantiated evidence is virtually worthless. Part of the process of substantiation involves examining the source. Is the source reliable? Is the source real evidence or personal evidence? Is it primary or secondary? Does it have a history of reliability? Perhaps most importantly, can the source be considered unbiased? An unnecessarily biased source is almost as worthless as an unsubstantiated source. One warning concerning bias, though: Sometimes bias is unavoidable. A biologist who has witnessed the mutation of a virus over several generations is unswervingly biased towards the opinion that viruses mutate. Bias is not inherently bad, but uninformed bias is. A layperson who knows nothing of biology, but believed her grandmother when she said that germs don't change because god created everything just as it is, is uninformed and biased, and her opinion is not useful in any way to a good decision.
Now, for a crucial question. Is the evidence actually relevant to the question? Suppose we are debating whether or not it will be profitable to put a gas station on a certain corner. As part of the evidence, we are presented with a questionnaire in which local respondents all expressed a great love for the Home Cooking Restaurant that occupied the spot until last year. This evidence might be presented as support for the contention that there are many people who will come to that location. In fact, it is not relevant. The restaurant was not a gas station, and the questionnaire did not address any questions about the desirability of a gas station in the same place.
If the evidence is scientific, there are many questions to ask. Is the evidence statistically sound? Are the statistics an accurate account of what the study is trying to demonstrate? Were the statistics collected properly? Have they been categorized accurately? If statistics are taken from a sample pool, was the sampling uniform and unbiased? Are all of the scientific terms accurately defined? Is the data statistically significant? Was there adequate control sampling? Is the study free of confounding variables? Are the claims based on the statistics based on reasonable precision? Are the statistics recent? Do they represent the accepted scientific explanations and theories?
Another important question to ask with regard to our evidence is best asked after we have addressed the previous questions. Is there cumulative evidence? In other words, does each piece of evidence fit together with the others like puzzle pieces, each bolstering and building up the credibility of the evidence surrounding it, or is it piecemeal and filled with gaps? In the case of large numbers of people, are there multiple examples of what the evidence is portraying? Remember, credibility and cumulative data are intrinsically linked.
Once we are reasonably convinced that we have good, compelling evidence, we get to the most tricky part of critical thinking. In the next essay, which may or may not grow into several, we will examine reasoning itself, and discover how to progress from indecision to decision through the use of valid, unbiased reasoning applied to the evidence that we have evaluated.