I’m proud of this article because I think it’s one of the most important I’ve done, at least in terms of significance to real people. In the course of writing the paper, I became increasingly appalled by the unscientific and haphazard nature of forensic science. Even DNA profiling, the gold standard of forensic techniques, is not as ironclad as you might think. Many other techniques verge on alchemy. But judges and juries in our Law & Order/CSI/Bones-immersed culture tend not to question the reliability of forensic conclusions and the trustworthiness of forensic analysts. The bottom line? A lot of innocent people are probably getting their lives ruined by the legal system.
The focus of the paper is narrower; it identifies one specific source of forensic bias. Here’s the abstract:
The current organization of forensic science induces biases in the conduct of forensic science even if forensic scientists are perfectly rational. Assuming forensic examiners are flawless Bayesian statisticians helps us to identify structural sources of error that we might otherwise have undervalued or missed altogether. Specifically, forensic examiners’ conclusions are affected not just by objective test results but also by two subjective factors: their prior beliefs about a suspect's likely guilt or innocence and the relative importance they attach to convicting the guilty rather than the innocent. The authorities—police and prosecutors—implicitly convey information to forensic examiners by their very decision to submit samples for testing. This information induces the examiners to update their prior beliefs in a manner that results in a greater tendency to provide testimony that incriminates the defendant. Forensic results are in a sense ‘contaminated’ by the prosecution and thus do not provide jurors with an independent source of information. Structural reforms to address such problems of rational bias include independence from law enforcement, blind proficiency testing and separation of test from interpretation.I think that’s pretty self-explanatory, but here’s an even simpler explanation. Say you’re a forensic examiner. For the most part, your test samples are provided by the authorities (police or prosecutors). And the authorities generally don’t bother giving you a sample unless they already suspect there’s reason to expect a match. So if you’re a rational forensic examiner, you may infer a somewhat higher probability of guilt by the mere fact that you’re doing a test at all. And that fact will affect your testing process, because there is no such thing as a purely objective test; your subjective beliefs about the likelihood of guilt necessarily affect your interpretation of the evidence.
4 comments:
This bias is a problem in all experimentation and, in fact, all verification of any hypothesis. In the academic setting we submit our results to a jury of our peers for a rigorous round of disputation and hope that any bias comes out in the wash. In the case of forensics it seems that there is a large cost involved in subjecting testing to the same process. Is it right to assume that the bias against testing unless there is other evidence to suggest guilt means that innocent people are put away. It seems we could make an argument that it means that guilty people are left to roam free if the only evidence for the guilt would come from a forensic test.
Dirmeyer -- As you say, in all scientific settings we need some kind of verification. Sadly, we don't have it in forensic science. The state forensic examiner is often the final arbiter on forensic evidence, with no one double checking.
As an alternative, we could do things like occasionally submit "dummy" samples that have nothing to do with actual cases, so that forensic examiners don't automatically assume the samples are from suspects.
"It seems we could make an argument that it means that guilty people are left to roam free if the only evidence for the guilt would come from a forensic test."
There's always a trade-off between Type I and Type II errors. By putting more people behind bars, we reduce the number of guilty people set free -- but we also increase the number of innocents in jail.
But the main point is that the juries no longer get "independent" sources of information. The forensic exam is contaminated by the other (non-forensic) evidence. This reduces the jury's ability to assess guilt or innocence on the basis of reliable information.
Yes, I agree. I was only commenting that there seems to be a high opportunity cost (if the tales of long waits and under funded forensic departments are true, that is) associated with the types of verification measures that we usually use. Of course, that does not mean that there are some that would be cost effective.
I realize that pointing out that a system that tends to bias the results so that innocent people are imprisoned because of the presence of circumstantial evidence also biases the result so that guilty people go free because of a lack of circumstantial evidence is hardly comforting or counter to your original point. Its probably more support for it.
Jennifer Dirmeyer (BTW)
"Created bias" as the result of sample selection by law enforcers is indeed a problem for prosecution forensic scientists. However, this can be addressed by forensic scientists attending crime scenes and making their own sample choices and also by independent forensic scientists coming in at a later date to see exactly what has been done (which is a common occurrence in the jurisdictions in which I work, being New Zealand, England, Wales, Scotland and Northern Ireland).
The independent forensic scientist has the least bias because they are never (or very, very rarely) directly involved with anyone in a case except the defence lawyer and it is relatively easy to ignore the bias of the defence lawyer and look just at the science itself(I am an independent forensic scientist and I can count on the fingers of one hand the number of Defendants, victims and people with emotive connections to specific cases I've met in 12 years of this job - compare this with prosecution scientists who are constantly directly involved with Police, prosecutors, victims and suspects).
Even if we go on the basis that independent forensic scientists are biased by the defence (which I suggest is not generally the case for good Experts, but that's just my opinion), as long as all the information is presented in court then it's for the jury to decide what they find most credible and likely.
I think we are never going to achieve a truly unbiased court process as long as, a) we work in an adversarial system, and b) people are involved.
I agree that the academic review process is vital for research - review of forensic science by independent experts is the forensic equivalent and it's the best we have at the moment. If this were to be extended to the academic process by having two (or more) separate independent forensic scientists reviewing prosecution scientific findings in criminal cases, the system would baulk at the additional cost. Should we put a price on justice? In theory, no, but that's what happens in reality.
Post a Comment