As a prosecutor, I was concerned about the accuracy of decisions in the criminal justice system. I did not want to charge, much less convict, a person who was innocent of the identified crime. Further, I wanted to find and convict those that had actually committed the crime. One of the areas that I identified for professional development for those in law enforcement was critical thinking, decision making, and judgment.
After becoming a judge, I thought it would be wise to further my reading about critical thinking, decision making, and judgment. I wanted to attempt to at least be aware of the situations where one is at a heightened risk of making an erroneous decision. Much empirical research on these topics also exist. In the next several blog entries, I will address some of this research.
The first topic I want to discuss is one that is common in the criminal justice system-confirmatory bias. Confirmatory bias is the tendency to selectively collect and interpret information to confirm beliefs and to avoid information that may disagree with prior beliefs.[i]
Confirmaatory bias can occur under three different scenarios.[ii] The first involves situations where people need to interpret ambiguous evidence. The research shows that a strong tendency exists that people will interpret ambiguous evidence based on their initial beliefs. This type of bias includes “a propensity to remember the strengths of confirming evidence but the weaknesses of the disconfirming evidence, to judge confirming evidence as relevant and reliable but disconfirming evidence as irrelevant and unreliable, and to accept confirming evidence while scrutinizing disconfirming evidence.”[iii]
I asked law enforcement officers to assume that a suspect is innocent, and then consider the evidence. Does the presence of an item of evidence make you change your conclusion that the suspect is innocent? Can the evidence be interpreted in favor of the suspect? Much evidence can be interpreted as either supporting or refuting a claim. Anyone who has watched a trial understands that.
Research has shown that different areas of the brain are used when people consider information that does or does not favor a predetermined conclusion. “Neural information processing related to motivated reasoning appears to be qualitatively different from reasoning in the absence of a strong emotional stake in the conclusions reached.”[iv] One only need to observe the stark differences in the interpretation of the same facts by strong political partisans of different persuasions.
A second situation that involves confirmatory bias occurs when people find a correlation between phenomena separated by time when one doesn’t exist, or when people fail to recognize a correlation when one exists. People have a difficult time interpreting information regarding correlations between two things. This bias has been identified as one of the weakest part of human reasoning.[v]
This bias underlies the power of using statistics rather than just human judgment to determine the efficacy of treatment modalities in the criminal justice system. If one hears someone extol the efficacy of anything, one should want to see the statistical evidence. Most of the time, the statistical evidence doesn’t exist at all, or it is so blatantly biased, in collection and interpretation, that it is useless.
The third situation that involves confirmatory bias is a tendency to selectively collect only that evidence that confirms prior held beliefs. This can be a big problem, as many times, only a short window of opportunity exists to collect evidence of a crime. If confirmatory bias prevents the collection of other evidence, then that evidence can be lost.
Here, one should attempt to ask a question to disconfirm what one believes. For example, if one believes Joe committed the crime, one should not merely ask what would prove that Joe committed the crime, but should ask what evidence would prove that Joe did not commit the crime. Does Joe have an alibi that made it impossible for him to commit the crime? Is Joe’s explanation for what happened physically possible? Of course, one must be very careful in how this evidence is collected, so that confirmatory bias itself does not result in a misinterpretation of ambiguous evidence.
Confirmatory bias can feed itself. If one starts with the belief a suspect is guilty, and then one erroneously interprets ambiguous evidence in a way that supports that belief, that can lead to a stronger belief in the defendant’s guilt, resulting in a greater tendency to engage in confirmatory bias. Confirmatory bias is a danger to truth seeking in the criminal justice system. We should be aware it when making important decisions.
[i] Budowle, Bruce et al, 2009, A Perspective on Errors, Bias, and Interpretation in the Forensic Sciences and Direction for Continuing Advancement, J. Forensic Sci.., July 2009 Vol. 54. No. 4 pp.798-809, 803.
[ii] Rabin, M and J.L. Schrag, 1999, First Impressions Matter: A Model of Confirmation Bias, Quarterly Journal of Economics, 114(1) Feb. 1999, pp. 37-82.
[iii] Lord, C.G., L. Ross and M.R. Lepper, “Biased Assimilation and Attitude Polarization: The Effect of Prior Theories on Subsequently Considered Evidence”, Journal of Personality and Social Psychology, XXXVII (1979), 2098-2109.
[iv] Westen, Drew, et al, 2006, Neural Bases of Motivated Reasoning: An fMRI Study of Emotional Constraints on Partisan Political Judgment in the 2004 U.S. Presidential Election, Journal of Cognitive Neuroscience 18:11, pp. 1947–1958.
[v] Nisbett, R.E. and L. Ross, Humans Inference: Strategies and Shortcomings of Social Judgment, 1980, Prentice Hall.
This is a very helpful post. As an attorney focusing my practice on criminal appeals for the past 23-24 years, I see this problem most often in assessment of resulting prejudice or harmless error. I see clients who insist that every little mistake made all the difference.
ReplyDeleteJust as often, and just as wrong, I see courts begin with the assumption that the original conviction was accurate and then undervalue the impact of errors or new evidence controverting that assumption. From my experience, more injustice results from confirmation bias in the courts than just about anything else.
The issue of confirmatory bias is an important one for every attorney and every judge. As someone who has focused my practice on criminal appeals for the past 23-24 years, I have seen it in action on numerous occasions, but most often on issues of resulting prejudice or harmless error.
ReplyDeleteOf course, I quite often have clients who desperately want some way out of a bad situation and therefore see any error, no matter how small and insignificant, as critical to their conviction or sentence.
Just as often, however, I see courts doing exactly the same thing in reverse, beginning with the assumption that the defendant must be guilty because he or she was convicted and therefore viewing even the most significant errors or newly discovered evidence as having little if any impact on the verdict. From my experience, confirmation bias in issues of harmless error and resulting prejudice probably has resulted in more injustice than just about any other cause.
I certainly am not suggesting that courts do this on purpose, but the bias definitely is there and, if they would take it into account and guard against it, the criminal justice system would be much more just.