Devlin's Angle

July-August 2007

The Professor, the Prosecutor, and the Blonde With the Ponytail

Just before noon on the 18th of June, 1964, in Los Angeles, an elderly lady by the name of Juanita Brooks was walking home from grocery shopping. As she made her way down an alley, she stooped to pick up an empty carton, at which point she suddenly felt herself being pushed to the ground. When she looked up, she saw a young woman with a blond ponytail running away down the alley with her purse.

Near the end of the alley, a man named John Bass saw a woman run out of the alley and jump into a yellow car. The car took off and passed close by him. Bass subsequently described the driver as black, with a beard and a mustache. He described the young woman as Caucasian, slightly over five feet tall, with dark blond hair in a ponytail.

Several days later, the LA Police arrested Janet Louise Collins and her husband Malcolm Ricardo Collins and charged them with the crime. Unfortunately for the prosecutor, neither Mrs. Brooks nor Mr. Bass could make a positive identification of either of the defendants. Instead, the prosecutor called as an expert witness a mathematics instructor at a nearby state college to testify on the probabilities that, he claimed, were relevant to the case.

The prosecutor asked the mathematician to consider 6 features pertaining to the two perpetrators of the robbery:

Black man with a beard

Man with a mustache

White woman with blonde hair

Woman with a ponytail

Interracial couple in a car

Yellow car

Next the prosecutor gave the mathematician some numbers to assume as the probabilities that a randomly selected (innocent) couple would satisfy each of those descriptive elements. For example, he instructed the mathematician to assume that the male partner in a couple is a "black man with a beard" in 1 out of 10 cases, and that the probability of a man having a mustache (in 1964) is 1 out of 4. He then asked the expert to explain how to calculate the probability that the male partner in a couple meets both requirements, "black man with a beard" and "man with a mustache". The mathematician described the product rule for independent events, which says that, if two events are independent, then the probability that both events occur together is obtained by multiplying their individual probabilities.

According to this rule, the witness testified,

P(black man with a beard AND has a mustache) = P(black man with a beard) x P(has a mustache) = 1/10 x 1/4 = 1/(10 x 4) = 1/40
The complete list of probabilities the prosecutor asked the mathematician to assume was:
Black man with a beard: 1 out of 10

Man with mustache: 1 out of 4

White woman with blonde hair: 1 out of 3

Woman with a ponytail: 1 out of 10

Interracial couple in car: 1 out of 1000

Yellow car : 1 out of 10

Based on these figures, the mathematician then used the product rule to calculate the overall probability that a random couple would satisfy all of the above criteria, which he worked out to be 1 in 12 million.

Impressed by those long odds, the jury found Mr. and Mrs. Collins guilty as charged. But did they make the right decision? Was the mathematician's calculation correct? Malcolm Collins said it was not, and appealed his conviction.

In 1968, the Supreme Court of the State of California handed down a 6-to-1 decision, and their written opinion has become a classic in the study of legal evidence. Generations of law students have studied the case as an example of the use (and misuse) of mathematics in the courtroom. (As you will see, it's worrying that one supreme court justice reached a different conclusion, but at least the system worked.)

The justices said, among other things:

"We deal here with the novel question whether evidence of mathematical probability has been properly introduced and used by the prosecution in a criminal case. ... Mathematics, a veritable sorcerer in our computerized society, while assisting the trier of fact in the search for truth, must not cast a spell over him. We conclude that on the record before us defendant should not have had his guilt determined by the odds and that he is entitled to a new trial. We reverse the judgment. ..."

The Supreme Court's devastating deconstruction of the prosecution's "trial by mathematics" had three major elements:

  • Proper use of "math as evidence" versus improper use ("math as sorcery").

  • Failure to prove that the mathematical argument used actually applies to the case at hand.

  • A major logical fallacy in the prosecutor's claim about the extremely low chance of the defendants being innocent.

    • Math as evidence

    The law recognizes two principal ways in which an expert's testimony can provide admissible evidence. The expert can testify as to his or her own knowledge of relevant facts, or he or she can respond to hypothetical questions based on valid data that has already been presented in evidence. What is not allowed is for the expert to testify how to calculate an aggregate probability based on initial estimates that are not supported by statistical evidence, which is what happened in the Collins case.

    The Supreme Court believed that the appeal of the "mathematical conclusion" of odds of 1 in 12 million was likely to be too dazzling in its apparent "scientific accuracy" to be discounted appropriately in the usual weighing of the reliability of the evidence. This, they opined, was mathematics as sorcery.

    Was the trial court's math correct?

    A second issue was whether the math itself was correct. Even if the prosecution's choice of numbers for the probabilities of individual features - black man with a beard, and so on - were supported by actual evidence and were 100% accurate, the calculation that the prosecutor asked the mathematician to do depends on a crucial assumption: that in the general population these features occur independently. If this assumption is true, then it is mathematically valid to use the product rule to calculate the probability that the couple who committed the crime, if they were not Mr. and Mrs. Collins, would by sheer chance happen to match the Collins couple in all of these factors.

    But I doubt that any regular reader of this column would be so foolish as to assume that the six features the prosecutor gave the expert witness came even close to being independent.

    Was the prosecutor's claim valid?

    The most devastating blow that the Supreme Court struck in its reversal of Mr. Collins' conviction, however, concerned a mistake that (like the unjustified assumption of independence) occurs frequently in the application of probability and statistics to criminal trials. That mistake is usually called "the prosecutor's fallacy," and is a a sort of bait and switch by the prosecution, sometimes unintentional.

    On the one hand, there is the prosecution's calculation, which in spite of its lack of justification, attempts to determine

    P(match) = the probability that a random couple would possess the distinctive features in question (bearded black man, with a mustache, etc.)
    Ignoring the defects of the calculation, and assuming for the same of argument that P(match) truly is equal to 1 in 12 million, there is nevertheless a profound difference between P(match) and a second figure, namely
    P(innocence) = the probability that Mr. and Mrs. Collins are innocent.
    As the Supreme Court noted, the prosecutor in the Collins case argued to the jury that the 1 in 12 million calculation applied to P(innocence). He suggested that "there could be but one chance in 12 million that the defendants were innocent and that another equally distinctive couple actually committed the robbery."

    As the justices explained in their opinion, a "probability of innocence" calculation (even if one could presume to actually calculate such a thing) has to take into account how many other couples in the Los Angeles area also have those six characteristics. The court said, "Of the admittedly few such couples, which one, if any, was guilty of committing the robbery?"

    The court went on to perform another calculation: Assuming the prosecution's 1 in 12 million result, what is the probability that somewhere in the Los Angeles area there are at least two couples that have the six characteristics as the witnesses described for the robbers? The justices calculated that probability to be over 40 percent. Hence, it was not at all reasonable, they opined, to conclude that the defendants must be guilty simply because they have the six characteristics in the witnesses' descriptions.

    More like this - and other juicy stuff

    The above account is abridged (heavily) from the forthcoming book THE NUMBERS BEHIND NUMB3RS: Solving Crimes with Mathematics, that I have just co-written with Gary Lorden, the professor of mathematics at Caltech who is the principal mathematics advisor to the hit CBS television crime series NUMB3RS. Scheduled for publication on September 1, the book looks at some of the real-life applications of mathematics in solving and prosecuting crimes that inspired the creation of the television series. As this particular example shows, in writing the book we did not restrict ourselves to the successful applications of math in fighting crime (which is what the TV series does), though many of the examples we present are of that nature. Rather, we look at the whole range of circumstances where law enforcement agents of various kinds make use of mathematics. Some chapters focus on cases depicted in episodes of NUMB3RS, others (like the Collins case just described) look at other examples of the use of math in law enforcement.

    We hope you enjoy the book.

    Devlin's Angle is updated at the beginning of each month.
    Mathematician Keith Devlin (email: [email protected]) is the Executive Director of the Center for the Study of Language and Information at Stanford University and The Math Guy on NPR's Weekend Edition. Devlin's most recent book, THE MATH INSTINCT: Why You're a Mathematical Genius (along with Lobsters, Birds, Cats, and Dogs) was published in 2005 by Thunder's Mouth Press.