Forensic science is capable of much, but absolute certainty can rarely be offered. Not even in court. So, how can you still make objective and useful statements? There is now an International guidance for that.
IN THE DUTCH NEWSPAPER 'TROUW,' JOOST VAN EGMOND INTERVIEWED DIDIER MEUWLY SCIENTIST AT THE NETHERLANDS FORENSIC INSTITUTE (NFI) AND A PROFESSOR AT THE UNIVERSITY OF TWENTE.
Above all, lawyers want to know one thing: what is the probability that the suspect did it? Charles Berger, principal scientist at the Netherlands Forensic Institute, has had to disappoint them many times in his career. Absolutely not out of reluctance, but that's not how his profession works. He explains it this way: "There are limits that logic imposes on statements you can make based on what you observe."
A seemingly innocuous question like "Does this trace come from the suspect?" for example, raises a load of problems. It is, Berger explains, the wrong question to ask the expert. "Firstly, you don't get certainty. And secondly, you also don't get a chance that the trail came from the suspect. Logic doesn't allow for that."
The why is rather technical, but no less fascinating for that. After all, it is impossible to say with certainty that a hypothesis is true if there are other possibilities that cannot be ruled out. Moreover, a piece of evidence cannot give an absolute probability that the hypothesis is true either. That probability involves many more factors, which the forensic expert does not weigh herself because they are outside her expertise. The expert can only weigh the findings against a clearly defined alternative.
Probability is relative
Sound too theoretical? Berger and his colleague Didier Meuwly deal with it on a daily basis and can give plenty of examples. "Have you ever noticed how a Dutchman writes the number 8?" asks Meuwly, a Swiss by birth. "You all make that same circle. But elsewhere, nobody writes an 8 like that. If you asked a Swiss forensic expert to compare an 8 on a piece of evidence with a suspect's 8, and they both have that same kringle, that expert would see that as a very strong indication that it came from the same person. Until you know it's from a Dutchman. Here, that kringle is not a special feature. Everyone makes it that way."
"What you always want to know in such comparative research," Berger adds, "is how particular your perception is when you assume that the accusers' story is true or when, on the contrary, the alternative is true. What matches, and how well does it match, and most importantly, what is the probability of it matching that way if it does indeed come from that person, and what is the probability of that matching if it comes from someone else? If someone writes a D as half an oval with a line next to it, everyone does that."
This problem played out, for example, in the Dreyfus affair, a court case held France around the 1900s in a tight grip. Officer Alfred Dreyfus was convicted of spying for Germany, largely on the basis of a note he allegedly wrote. A forensic expert who did not have his statistics in order saw similarities with Dreyfus' handwriting and concluded that he must be the author. Based on those observations, he made a statement about the probability of Dreyfus being the writer. The mathematician Henri Poincaré had to step in to reprimand him.
What Poincaré observed, Berger says, is solid as a rock. It is at all impossible, based on that observation, to determine the probability of the proposition that that person wrote that text. "He almost literally wrote that down like that in 1906," Berger says smugly. "So unfortunately, for a century, that had no effect whatsoever on forensic science."
Now, to their no less pride, that principle has been enshrined in a standard by the International Standards Organisation ISO. That standard is much broader than just this point and sets quality standards for the entire practice of forensic investigation. Around 15 countries participate in it, including the Netherlands. Meuwly and Berger had an important role in establishing the standards for analysis and interpretation. "There was a great need for this standard," says Berger. "It limits what experts can say. For good reasons."
How do you calculate evidential value?
What you can do, the standard prescribes, is report the 'probative value' of such observations. "You then reason back, from one observation to two or more hypotheses," Berger explains. "What is the probability of me making this observation for each of the hypotheses?" For example, a finger trace on the door handle to the crime scene. Never call that a fingerprint, stresses expert forensic biometrics Meuwly, because the trace a finger leaves is never a perfect fingerprint as you make them on the scanner. You compare that finger trace with the suspect's fingerprint.
You then have the hypothesis that the accused's finger made this trace, and an alternative, which you explicitly name. Often this is the hypothesis that some random other person made the trace, but it may also be that the defence puts forward a concrete alternative. There is a chance for all hypotheses, but it depends on many more factors, and in any case it is not up to the forensic expert to make a judgement on that. That expert is limited to an 'update' of that odds ratio, based on the comparison of the fingerprint with the suspect's fingerprint.
That odds ratio could be, say, ten to one, or a thousand to one. That is the factor by which the hypothesis 'the suspect grabbed this door' is strengthened by the observation of the fingerprint, known as probative force. That forensic probative force can be taken by the judge factor into the overall consideration of whether the accused is guilty of the crime, looking at all the evidence in context.
It is harder to understand, but the right method, Berger says. "The strength of evidence indicates how you should rationally adjust your beliefs based on the new information."
It has four legs, yet it is not a cow
The NFI itself made this approach standard some 15 years ago. A tough project, especially since the forensic expert does not do the interpretation for himself, he also has to explain it in court, where misunderstandings lurk. It was therefore not, in Berger's words, 'just a note in the Christmas newsletter'. Berger wrote a lot about it and regularly stands before rooms full of lawyers to explain these principles.
A common pitfall is the confusion of terms, Berger observes. "So, as an expert, you say something about the probability of an observation, given the hypothesis. But everyone hopes precisely that you are going to say something about the probability of the hypothesis given the observation."
Many want that so badly that they automatically start hearing it. There is an example Berger often cites. For example, the expert concludes, "the probability of a match is 1,000 times higher when the suspect left the trail than when any other person did. Compare that with the statement 'when you assume an animal is a cow, what is the probability that it has four legs'.
It is frequently misinterpreted as, 'the probability that the suspect left the trail is 1,000 times higher than that any other person did, when there is a match'. Those who do that confuse observation and hypothesis. In our equation, it becomes: 'if an animal has four legs, what is the probability that it is a cow?' A completely different statement.
For NFI scientists, it is a matter of explaining and repeating. "Lawyers by nature will not be the biggest science enthusiasts," says Berger, "but they do see the importance of the ingredients that science gives to reach a conviction or not. The Netherlands is really progressive about that. And they understand the limitations of what is logically possible and what is not. Over the years, I have had a lot of people in the room like this. You do get a long way that way."
When it's impossible to put and exact figure on it.
The logic behind the method is universal, although it is easier to apply when there are clear figures, Meuwly and Berger note. "Quantitative analysis is often about DNA testing," says Berger. "Based on the rarity of a DNA profile, there are good calculations to do, in all other cases it is much more difficult. So how do you arrive at the probability of a sighting?"
Right now, that is still mostly a human assessment, which you do as an expert. Technical progress helps a lot in this area, more and better databases are becoming available against which you can compare an observation - say, a footprint or a textile fluff - but there are usually no such hard figures as with DNA analysis yet.
"We do want to move more and more towards that quantitative analysis," Berger says, "but in the vast majority of cases that is not yet the case. Then you have to make do with qualitative terms. Those indicate 'this is not a quantitative calculation. We made a list for that."
How much is 'extremely more likely'?
Berger picks it up, they are familiar terms that regularly drop in court: 'about as likely', 'slightly more likely', and so on up to 'extremely much more likely'. To ensure that experts have roughly the same thing in mind when using those terms, an order of magnitude has been added. For example, experts are supposed to speak of 'extremely more likely' if they rate the probative value higher than a million.
Incidentally, the question is whether everyone is waiting for this method of quantification. Berger chuckling recalls that, when the terms were introduced, he braced himself for questions from the court as to what a term like "extremely more likely" actually meant. Those questions didn't come up. "We then just put those definitions on the website ourselves, because we ourselves think it is important to define that, but that question never reached me from the courtroom."
What that means Berger is happy to leave unanswered. What matters is the underlying logic: "People hear statistics and then think they have to calculate. No, it's about the type of statements you make. That you ask yourself: suppose the accuser's story is true, do I expect to see what I see here? And you do the same for the defence's story. From the ratio of those two probabilities, you figure out how that observation should contribute to your degree of conviction on those two propositions. There is no number involved in that. So it can be quantitative or qualitative, but it's the same logic you apply."
Educating a new generation
In the Netherlands and some other countries, this realisation has now dawned, say the NFI scientists. Internationally, it is still quite a task. The new standard is a big step in that, although they are under no illusions that it is finished with that. "This has to translate into practice," says Meuwly. The standard has to become established. Forensic experts will start calling each other on it, and education for a new generation will be adapted to it. "That will take 10, 15 years," he estimates.
Resistance will certainly not disappear overnight, Berger believes. He noticed the same during consultations on the ISO standard. "Some still want to be able to say just on the basis of comparison alone 'there's a very good chance he wrote that'. But so you can't. Poincaré explained that in 1906, in such an ISO context you do that again. At some point, hopefully, people will see it. And now we have really established that we are not going to do that anymore."
FOR MORE INFORMATION:
