I guess I should be really bothered by the current controversy over the questionable forensic science underlying alleged "shaken baby" cases. Honestly, though, as these re-evaluations and backtrackings start happening with increasingly depressing regularity, it's getting harder and harder to generate the requisite outrage. Another supposedly solid forensic science that's led to countless convictions turns out to be not that reliable after all? As Supreme Court Justice Elena Kagan said in a related context: "Been there, done that."
As with the similar revelation earlier this year that the FBI's supposedly "elite" microscopic hair comparison unit wasn't actually very "elite"--by which I mean that it appears to have messed up royally in the vast majority of cases that it handled--this isn't remotely surprising to anyone familiar with the way forensic evidence is developed, presented and used in criminal cases. At this point, it's not as if we don't know about these problems; virtually every systematic analysis of the system, such as the review culminating in the National Research Council's 2009 report, gives cause for grave concern. But these issues keep coming up, and no one seems to be able to do anything meaningful about them.
The problems are at every level, starting with the supposed scientists and analysts themselves. Before becoming a lawyer, I spent a few years in a biochemistry graduate program. I was always impressed by the intelligence and thoughtfulness of the scientists I studied under and listened to. So when I started handling criminal cases involving scientific evidence, I looked forward to the experience, assuming I'd see some of the same quality of work and thinking that I'd been exposed to in graduate school.
Nope. Among the purported forensic experts I've seen, there have definitely been some good ones who did high-quality science and explained it accurately, staying within the limits of what their fields' work could legitimately support. But there have been some truly awful ones as well. The underlying "science" is often very poorly conducted, and wouldn't pass muster if it had to go through the genuine peer review and criticism process the academic world demands. And the supposed conclusions in a given case are often highly subjective, arrived at by analysts whose employers are clearly hoping for certain defined results--obviously a deeply troubling combination.
If the evidence was presented along with concessions regarding its limitations, that would at least help. Often, though, it's not--and that's where one of the most striking differences between "real" science and forensic science rears its head. The genuine scientists I listened to at conferences and seminars used appropriately guarded language in describing their work and the conclusions that could be drawn from it--we have this data, and we believe this is the explanation; there are other possible explanations, and these are the experiments we want to do to see if we can narrow down the possibilities.
But in court, you're much more likely to hear a prosecution expert (or, to be fair, a defense expert) talk in absolute or near-absolute terms, with a level of confidence entirely unmoored from what the data supports. If that was because these experts were really that much smarter than their academic peers, or their work was that much better, that might make sense. In fact, though, the truth is often precisely the opposite. In short, compared to the non-forensic setting, in a criminal case you'll too often see less-qualified scientists, with lower-quality data, expressing conclusions in significantly more confident terms. Not exactly comforting.
Part of what gets exaggerated is the state of the field itself. For example, in the shaken-baby context, an expert witness from a high-profile 1997 trial now apparently claims that he's "rethought his testimony" and that "the [shaken baby] diagnosis has been invoked too readily in criminal cases and that other causes might explain" the supposed indicators.
Great--but where was this humility back in 1997, when by his own admission he was "adamant that it had to be child abuse, shaken baby syndrome"? It's not enough to say that in 1997 we didn't know about the defects in the science. That means there wasn't enough data to know how reliable that science was, so these experts shouldn't have been coming into court "adamant" about anything. A serious criminal case isn't supposed to be a testing ground for emerging scientific fields.
The "experts" themselves are only part of the problem. Prosecutors are far too willing not only to enthusiastically rely on these questionable scientists' conclusions, but also to help shape those conclusions in the service of convictions. Judges, who are supposed to act as "gatekeepers" to ensure that only reliable scientific evidence gets into a courtroom, are often far too deferential to prosecutors and government-sponsored scientists. It's well-established that jurors tend to be especially influenced by supposed expert testimony, even if the quality of that testimony is suspect. Finally, many defense attorneys are uncomfortable crossing swords with supposedly qualified technical experts, with the result that judges and jurors aren't always given opportunities to evaluate the weaknesses in those experts' testimony.
There aren't easy solutions to these problems. Every once in a while there's a call for judges to scrutinize scientific evidence more carefully before letting juries hear it, but I'm not optimistic that this is a useful solution. The reality of the criminal justice system is that many judges are inevitably going to defer excessively to prosecutors and their experts, no matter what the theoretical legal standards are.
I'd be more inclined to support reforms that make it more likely that jurors in criminal cases will learn everything that might help them to critically evaluate the technical evidence in their cases. Defense attorneys across the country should have access to intensive training in evaluating and working with scientific evidence--this recently developed program instituted by the Innocence Project is a great example. Judges should give opposing attorneys wide latitude in challenging the experts that are allowed to testify. For example, in appropriate cases jurors should hear about not only the particular types of evidence in the cases before them, but also other cases in which scientific evidence confidently presented by prosecutors turned out long after the fact to have been of dubious quality.
And finally, all of us--whether as jurors, or even just as observers of the court system--should keep ever-vigilant eyes on the quality of the evidence used to convict our fellow citizens of crimes. It's long past time to start applying this type of scrutiny, and we can't afford to wait until the next time a group of forensic scientists shuffles up to a podium to apologize for the faulty testimony they've given in the past.