# Daily Archives: November 4, 2011

## The Likelihood Ratio

In a previous post I explained what it means when we say some fact or claim “E” is evidence for some hypothesis “H”. That is, P(H|E) > P(H), or, the probability of some hypothesis H given some evidence E is greater than the probability of some hypothesis H. If P(H|E) = P(H) then E cannot be used as evidence for H; when this happens, by definition, E exist independently of H. This is the failure of probability that most religious people unsuspectingly apply when appealing to evidence for their god (this is different than P(H|E) < P(H) since this means that E is evidence against H but is not independent of H).

The simpler form of Bayes Theorem is P(H|E) = P(H)*P(E|H) / P(E). Looking at this equation, if P(H|E) = P(H), then this necessitates that P(E|H) = P(E). That's because if we have some equation that is 10 = 10 * x / y, the two unknown terms together must equal 1. Invariably, this means that the two unknown terms are also equal. Reciprocally, if P(E|H) = P(E) then this means that P(H|E) = P(H) and again, E is independent of H. This means that, at a glance, if P(E|H) / P(E) > 1, then it means that this is evidence in favor of H. If P(E|H) / P(E) is less than 1, then this means that some alternative hypothesis accounts for the evidence better than H.

Here we would have to use Bayes with two or more hypotheses (the longer form I've been using throughout all of the previous posts); if there are two hypotheses it should be that P(E|H) > P(E|~H) if E indeed is evidence for H. In other words, that the evidence is more likely under the hypothesis than under the alternative. Figuring that out is the essense of Bayes' Theorem. This weighing of the evidence is sometimes expressed by the Likelihood Ratio P(E|H) / P(E|~H).

In Bayes' Theorem, the Likelihood Ratio is how likely your hypothesis is in relation to all other hypotheses posited, and also shows how strongly the evidence favors, or disfavors, your hypothesis. This is simply dividing P(E|H) by P(E|~H). Let's say we're presented with the following scenario:
Two people have left traces of their own blood at the scene of a crime.  A suspect, Oliver, is tested and found to have type O blood.  The blood groups of the two traces are found to be of type O (a common type in the local population, having frequency 60%) and of type AB (a rare type, with frequency 1%).  Do these data (the blood types found at the scene) give evidence in favour of the proposition that Oliver was one of the two people whose blood was found at the scene?
So in this scenario, if Oliver accounts for the O type blood, then one unknown person accounts for the the AB. This gives us the P(E|H) = 1% or 0.01. You may say, at this point, that it fits the hypothesis of Oliver being there, since if Oliver was there, and he left blood, then at least one of the samples of blood would be type O.

On the other hand, if Oliver is not guilty, ~H, then this gives us P(E|~H). Or that two unknown people left blood at the scene. What is the probability of finding the evidence at hand if Oliver is not guilty? This means we have two people at random who at each random selection has to account for either the 1% or the 60%. This becomes 2 * 0.01 * .6, which is 1.2% or 0.012.

What happens when we find out the Likelihood ratio? This is 0.01 / 0.012, which equals ~0.8334. What does this mean?

 K dB bits Strength of evidence < 1:1 < 0 Negative (supports ~H) 1:1 to 3:1 0 to 5 0 to 1.6 Barely worth mentioning 3:1 to 10:1 5 to 10 1.6 to 3.3 Substantial 10:1 to 30:1 10 to 15 3.3 to 5.0 Strong 30:1 to 100:1 15 to 20 5.0 to 6.6 Very strong >100:1 >20 >6.6 Decisive
As you can see from this chart, a Likelihood Ratio that is lower than 1 means that it slightly supports the hypothesis that Oliver is not guilty! Or in other words, there's a higher probability of finding the evidence that we have if Oliver were indeed not at the scene of the crime and two other random people committed the murder. Going back to the simpler form of Bayes Theorem, P(E|H) would be the 0.01 and that is denominated by P(E). P(E) in this case would be the probability of finding the evidence at hand period, which is 2 * 0.01 * .6, which is 1.2% or 0.012. Again, P(E|H) < P(E). So even though the evidence intuitively fits the hypothesis that Oliver is guilty, it is more likely, due to the math implicit in the evidence, that Oliver is not guilty.

So much for intuition.

But, what if, instead of Oliver having type O blood, Oliver had type AB blood? The likelihood ratio then becomes .6 / 0.012, which equals 50. As you can see from the Wikipedia chart, this is highly decisive evidence that Oliver is guilty. If we think about it, say there are only 1000 people in the town. This means that there are 600 people with type O and 10 people with type AB. If Oliver has type O, this means that there are 599 other people that the prosecution has to eliminate to claim that Oliver was at the scene. A lot more legwork. We would actually have better luck grabbing two people at random and claiming they were at the scene. But if Oliver has type AB, then there are only 9 other people that the prosecution has to eliminate to put Oliver at the scene. A significantly higher probability that Oliver was there.

We might have control over the numerator, but the denominator in the Likelihood Ratio — the probability of seeing the evidence if our hypothesis is false (or in the case of the simpler Bayes Theorem, the probability of the evidence as it is given) — is out of our control; and this is the true deciding factor.

In my previous post on the likelihood of Romans 1.2-6 being an interpolation, all of the Likelihood Ratios are “barely worth mentioning”. Indeed, many scholars do in fact claim that those issues I presented are barely worth a mention. Yet, as can be seen, the accumulation of multiple slices of evidence that are barely worth mentioning accrete to a total hypothesis that is worth mentioning. And like I wrote in that previous post on Romans 1.2-6, Creationists might dismiss all of the evidence for evolution as “barely worth mentioning”, but their accumulation and conscilience is the true strength of the theory of evolution.

A lot like Voltron. Misuse of the Likelihood Ratio

In a recent debate between William Lane Craig and Lawrence Krauss, Craig seems to commit the Prosecutor's Fallacy. He makes the argument that the existence of contingent beings is evidence for the existence of a necessary being, thus god. He even explains this in Bayesian terms: the probability of the existence of contingent beings is higher given the existence of god. That is P(E | G) > P(E). Even better, at the beginning of the debate he explains the meaning of evidence much in the same way I described it above.

But wait a minute. What is P(E), the Total Probability? Or what is P(E | ~G)? While P(E | G) might be .99, who is to say that P(E) isn't also .99? As I wrote above, if this was the case then P(E | G) = P(E). This means that G is independent of E; that is, in Craig's formulation of probability theory, god exists independently of the existence of contingent beings. Remember, the denominator should ideally not be in our control. If it is, then we can win any debate we want by manipulating the numbers in our favor. Since Craig does not have the denominator, he seems to have confused P(E | G) [the probability of the existence of contingent beings given the existence of a necessary god] with P(G | E) [the probability of a necessary god given the existence of contingent beings], what he was initially trying to argue.

Craig seems to have misused the Likelihood Ratio, and misunderstood Bayes Theorem (or at least used it incompletely); the Likelihood Ratio is to be evaluated using the conditional probabilities, not the posterior.

Look at it this way. What is the probability of winning the lottery, P(E)? Probably something ridiculous like 1 out of a million (.0000001). What is the probability of winning the lottery given that a person cheats, P(E | H)? Probabily something much higher, maybe .99. Does this mean that the person cheated, i.e. P(H | E) is high? No. Even though the Likelihood Ratio is a whopping 990,000 decibles we have no idea what P(H) is. To conclude that the person cheated given a theoretically high Likelihood Ratio is an incomplete use of Bayes Theorem. It is actually an example of the Prosecutor's Fallacy: confusing P(E | H) with P(H | E). Your end goal is to find the posterior probability of your hypothesis given the evidence, not to find the posterior probability of the evidence assuming your hypothesis is true (a species of begging the question).

So let's run with these lottery numbers. Our Likelihood Ratio is 990,000 decibles. What if our prior probability for having cheated is extremely unlikely, like say .000001 (i.e. how many people cheat the lottery)? This gives us a posterior probability of .49, which is barely below random chance. The real prior is probably a lot lower than .000001 since I don't know how many people have cheated to win the lottery.

All the Likelihood Ratio does is show you how strong the evidence weighs in favor of your hypothesis. If your initial hypothesis itself is highly unlikely, a high Likelihood Ratio doesn't help.

Comments Off on The Likelihood Ratio

Posted by on November 4, 2011 in Bayes, William Lane Craig

NeuroLogica Blog

Slate Star Codex

SELF-RECOMMENDING!

Κέλσος

Matthew Ferguson Blogs

The Wandering Scientist

What a lovely world it is

NT Blog

PsyBlog

Understand your mind with the science of psychology -

Vridar

Musings on biblical studies, politics, religion, ethics, human nature, tidbits from science

Maximum Entropy

Skepticism, Properly Applied

Criticism is not uncivil

Say..

Research Digest

Disrupting Dinner Parties

Feminism is for everyone!