• About the Institute
• Research
• CARA
• People
• Advisory Board
• News & Events
• Contact

Francine and Michael Saferstein Memorial Lectures in Forensic Chemistry

Scientific Evidence: Developments in Forensic Science 
September 24, 2009
 
Paul Giannelli is the Weatherhead Professor of Law at Case Western University and author of the textbook Scientific Evidence, now in its fourth edition. In this 2009 Saferstein lecture, Dr. Giannelli discussed recent challenges to a hundred years of precedents in admitting the testimony of expert witnesses in techniques such as fingerprints, handwriting, or hair analysis which can be used to connect physical evidence to specific people. It is questioned whether such testimony can claim the same mantle of scientific rigor as DNA or chemical lab tests. Examples of the language that was admitted make a compelling case that more quantitative studies should be performed, studies which would be meaningful to juries, so they can evaluate of how such evidence should be weighed.

When the National Academy of Sciences reviewed forensic procedures for connecting individual people with objects or crime scenes, they concluded "only nuclear DNA analysis has been rigorously shown to have the capacity to [identify individuals] consistently and with a high degree of certainty". Ironically, it was the challenges to the admissibility of DNA evidence (People v. Castro, 1989) that brought an awareness into jurisprudence of the distinction between objective scientific method and adversarial pseudoscience. "forensic scientists, like scientists in all other fields, should subject their claims to methodologically rigorous empirical tests. The results of these tests should be published and debated" (1). Also, about this time, this historical precedence for admissibility of evidence (Frye) was replaced by Daubert (2) -- the Supreme Court's " junk science" decision in 1993.

The rigorous validation of DNA techniques provided a quantitative standard to compare with other forms of evidence that have been used for the past century. Of cases where defendants found guilty were later exonerated by DNA evidence, it was found the erroneous convictions were based on poor science in a third of cases (with mistaken eyewitnesses in 85% of the cases). 

Daubert invited “reexamination even of ‘generally accepted’ venerable, technical fields.” (3) (e.g. handwriting comparison), so “courts are now confronting challenges to testimony, as here, whose admissibility had long been settled.” (4). Although the “Daubert decision did not impact on the admission rates of expert testimony at either the trial or appellate court levels.” (5), it " requires the forensic scientists to prove that the evidence is fundamentally scientifically reliable, not just generally accepted by his/her peers in the discipline.” (6).so that “since Daubert, judges have examined the reliability of expert evidence more closely and have found more evidence unreliable as a result.” (7)

Don't Believe Everything You See on CSI

Polygraph Testing ("Lie Detectors") had long been contentious -- not admissible in most states; left to the discretion of the judge in a few.  Similarly, of handwriting comparisons, one court has ruled "a document examiner will not be permitted to testify that the maker of a known document is the maker of the questioned document.” (10), although most courts still admit it. A gunshot residue test, used for 30 yr, was found to have false positives. The NAS report itself disparaged voice prints. 
 
Of Microscopic Hair Comparisons it was found “This court has been unsuccessful in its attempts to locate any indication that expert hair comparison testimony meets any of the requirements of Daubert.” (8)).When compared to DNA evidence, microscopic hair analysis was wrong 12% of time (9). Although hair analysis is still generally admitted, the error rate tempers the "scientific certainty" the expert testimony can claim. 

Dental forensics (identification from a full set of teeth) is rigorous; however, "the fundamental scientific analysis for bitemark analysis has never been established" (11). In one case an expert concluded “that Burke's teeth matched the bite mark on the victim's left breast to a ‘reasonable degree of scientific certainty.’ [but] DNA analysis showed that Burke was excluded as the source of male DNA found in the bite mark on the victim's left breast.” (12).

-Firearms Analysis was established in the famous Sacco and Vanzetti trial of 1921; after Capone's St. Valentine's day massacre, a grand jury was so impressed with it that the money was donated to establish one of the first crime labs (Chicago, 1931). In most cases, the marks left on a bullet by machining variances of a firearm are distinguishable -- even two consecutive barrels from the same production line. But it has been found inadmissible to say "to the exclusion of every other firearm in the world". For Cartridge Case Identification the expert was permitted to testify only that it was “more likely than not” that recovered bullets and cartridge cases came from a particular weapon. (13) . 
   Chemical analysis of bullet lead composition has been used to argue that a bullet in evidence was chemically indistinguishable and therefore came from a supply in the defendant's possession. But after an investigation of the industry showed a single batch of lead can produce "from the equivalent of as few as 12,000 to as many as 35 million .. bullets" (National Research Council, 2004), the expert was permitted only to say that it "could have come from the same box".

Fingerprint Analysis: Some experts claim it has a zero error rate; however, fingerprint analysis has been overturned by DNA. Although record prints are good, latent prints recovered from crime scenes can vary (see below).

Distinguishing Good Science

Much of the tainted science in the exonerated cases lacks the methods of good science -- written records, measured error rates, accredited facilities. The hallmark of good science is objectivity, however, experts in court are generally not unbiased, but are hired by one side to support one view. Without going as far as saying forensic evidence is not uniquely immune from the risk of manipulation, studies have documented "confirmation bias” when a particular outcome is expected. It has been suggested that "verifiers should be given challenging exclusions during blind proficiency tests …” (14)

When 5 fingerprint examiners were presented with prints from their prior cases, but were told that the prints were the Mayfield prints: 3 directly contradicted their prior identifications; 1 concluded insufficient data. ("Contextual Information Renders Experts Vulnerable to Making Erroneous Identifications 156 Forensic Sci. Int’l 74 (2006).)

Two red flags are: unrealistically low error rates of a method, and reliance on an expert over presenting data supporting the conclusion. The NAS report is highly critical of claims of perfect accuracy, and of exxagerated testimony.

Many good fingerprint experts show two prints, highlight and tally the similarities for the jury, citing factual statistics on the frequency of certain features or combinations. But leave the final opinion to the jury. For the expert to claim positive identification based primarily on expertise invites compromising cross examination: "Is the lab accredited? Are you certified? By what board - with what authority and objectivity? "

Data is More Precise than Wording

The examples of admitted language ("more probable than not", "could have come from the same box") illustrate how judges can admit expert testimony as technical evidence, rather than scientific evidence. But splitting hairs over wording is fuzzy at best: confidence should be expressed as probability -- DNA analysis can give the exact odds of a match or mismatch. The specific cases above were relatively rare, intelligent challenges to traditional experts. But given the volume of cases in which these kinds of evidence are presented, there is a compelling case to create objective, publically available data that could be provided with the evidence to juries, to enable them to make intelligent decisions. For example, do professional document examiners perform better than laymen? How much? What are favorable and unfavorable cases?

Recommendations

"I want university scientists, with nothing to do with forensics, to come sit on these comittees and decide how it should be done. "

The nation should form a National Institute of Forensic Science, which could fund research through universities the way NIH and NSF do for medical and basic science research.

Crime Labs should be made independent. Although police like having the lab working for them, laboratories could practice standards for objective procedures to become accredited, but still have a close relationship with the police, arguing "we have to do the tests this way to protect the workers and to be more credible".

Drs. Saferstein, Giannelli, and Karger