Scholar raises red flags over expert forensic evidence

July 26, 2016

There’s a problem lurking in the modern courtroom. From Canada and the US, to Australia and countries the world over, judges and juries are relying on forensic evidence that one Peter Wall Visiting Research Scholar says hasn’t undergone enough formal evaluation and testing of the kind seen in the mainstream science disciplines.

“Many of the techniques– and here I’m talking about things as diversified as ballistics, voice comparison, or the use of images to identify people in robberies, for example– these [techniques] are used routinely to identify people, and yet many of them haven’t been subject to serious formal evaluation to see whether they work,” says Dr. Gary Edmond, a 2013-2014 Peter Wall Institute International Visiting Research Scholar.

Historically, scientific technologies and techniques, such as the long-studied field of DNA analysis, have used validation experiments in order to measure the limits of these approaches by gauging levels of uncertainty, accuracy and error.

According to Dr. Edmond, a professor from Australia’s University of South Wales School of Law, that kind of evidence validation just isn’t happening in many of today’s forensic sciences.

He is quick to assert that many of the techniques used in forensic investigation are extremely helpful, but what is lacking is formal understanding of the limitations of these approaches.

“We just don’t know how well they work,” says Dr. Edmond. “We don’t know how good the people using them are, and I think that’s a problem because many of these issues haven’t emerged in court.”

With the exception of nuclear DNA analysis, no forensic method has been shown able to consistently, and with a high measure of certainty, show a connection between physical evidence and the identification of an individual– National Academy of Sciences, 2009.

Concern over the reliability of expert witness evidence and the forensic sciences is surfacing in more large mainstream scientific bodies, according to Dr. Edmond.

In fact, a 2009 report from the US-based National Academy of Sciences called the forensic science system “fragmented”, and in dire need of major reforms and new research. It’s an especially crucial problem within identification forensics– methods like fingerprint or image analysis– used to match suspects to evidence involved in a criminal case.

According to the report, with the exception of nuclear DNA analysis, no forensic method has been shown able to consistently, and with a high measure of certainty, show a connection between physical evidence and the identification of an individual.

Dr. Edmond recently visited UBC as part of the Institute’s effort to bring world-class academics to the university to collaborate with UBC Faculty.

“It’s a great resource and Institution, and I’m really thankful that I had the opportunity to be there”, he says. 

At UBC, Dr. Edmond teamed up with Dr. Emma Cunliffe, a member of UBC’s Faculty of Law, to continue research into expert forensic evidence, in particular gait analysis. Gait analysis uses an individual’s movements to establish identity or similarity.

Dr. Edmond says many factors affect the reliability of this identification method, including the low quality of images, the effect different shoes and clothing have on movement and the fact that there is currently no scientific sense of how common movement gestures are in the general population.

“Even when there’s good cross-examination and skillful lawyers,” he says, “there’s still the question about [if] the jury appreciates the severe limitations here.”

There is a difference, says Dr. Edmond, between forensic teqniques that evolved from mainstream sciences– such as chemical analysis used to identify drugs and DNA testing– and other forensic methods like ballistic analysis, document comparison and even finger printing.

“Most of those techniques are quite robust and they are used by formally trained scientists,” Dr. Edmond says of the former, explaining that as a result these tests are generally more reliable.

“Then you have many other practices that came out of the techniques that police and other investigators, who often didn’t have any formal qualifications, just developed in the field or in the police laboratories.”

It’s become an issue, Dr. Edmond says, largely because courts have been focusing on the merits of evidence in isolation, on a case-by-case basis. He says in general there hasn’t been much systematic learning within many court systems about the scientific reliability of current forensic approaches.

Furthermore, he says law is largely autonomous, lacking significant interaction with the organized sciences that know the need for validation of evidence well.

“Scientists who aren’t forensic scientists… often are very surprised by what forensic scientists do, and not in a positive way,” Dr. Edmond explains. “I think at times it might be interesting if judges were to see the reactions of mainstream and eminent scientists to some of the practices going on in courts. It might be kind of therapeutic.”

In addition to supporting independent, multidisciplinary scientific oversight for forensic sciences, Dr. Edmond says that ultimately forensic analysts need to undertake validation studies, define error rates and shield themselves from irrelevant case information that has the potential to introduce cognitive contamination and bias.

Dr. Edmond, who held a public lecture titled Ordeal by Trial: Expert Evidence and the Limits of Legal Practice, during his UBC visit, plans to continue analyzing different forms of evidence, how they are used and assessed, alongside courtroom work on some appeals in an effort to integrate these concerns into legal practice.

“Forensic sciences have great potential and the legal system can do much better, it’s just a matter of generating awareness and starting to facilitate change.”