## The Phrase "Absence of Evidence Isn’t Evidence of Absence" is a Logical Fallacy

29 Jun

Dutch Schultz

So I’ve implied in all of my posts about Bayes Theorem that probabilities are like energy; they have to be conserved. In other words, when everything is added up it has to add up to 1.00:

P(H) + P(~H) = 1.00
P(E | H) + P(~E | H) = 1.00
P(E) + P(~E) = 1.00
P(H | E) + P(~H | E) = 1.00
etc.

Why is that? If these axioms are ignored, you could end up getting Dutch Booked:

Horse number Offered odds Implied
probability
Bet Price Bookie Pays
if Horse Wins
1 Even $\frac{1}{1+1} = 0.5$ $100$100 stake + $100 2 3 to 1 against $\frac{1}{3+1} = 0.25$$50 $50 stake +$150
3 4 to 1 against $\frac{1}{4+1} = 0.2$ $40$40 stake + $160 4 9 to 1 against $\frac{1}{9+1} = 0.1$$20 $20 stake +$180
Total: 1.05 Total: $210 Always:$200

[…]

In Bayesian probability, Frank P. Ramsey and Bruno de Finetti required personal degrees of belief to be coherent so that a Dutch book could not be made against them, whichever way bets were made. Necessary and sufficient conditions for this are that their degrees of belief satisfy the axioms of probability.

Why is this relevant? Well, there’s the common refrain that “absence of evidence isn’t evidence of absence” but if that were true, then your probabilities won’t be coherent and you’ll Dutch Book yourself above; your terms won’t equal 1.00.

In reality, the phrase “absence of evidence is not evidence of absence” is itself a logical fallacy. Or rather a probabilistic fallacy, since the person asserting it is trying to Dutch Book you. Let’s go over what would happen if absence of evidence indeed were not evidence of absence.

Say you have some hypothesis H that is a coin flip; you are agnostic about whether it’s true or false. A friend predicts that some evidence E increases the probability of H. You then rejoin that E is not found, so this must decrease the probability of H, but your friend asserts “absence of evidence isn’t evidence of absence!”. What is he actually saying by that? He’s saying that the absence of E doesn’t move H so it stays at 50%.

So the simple form of Bayes’ is P(H | E) = P(E | H) * P(H) / P(E).

In formulaic terms, your friend is saying P(H) = .5. P(H | E) > .5, yet P(H | ~E) = .5. We can model this by throwing in some values in Bayes’ Theorem. P(H) is already .5, so let’s say that P(E) is also .5 and P(E | H) is .6. This becomes:

Evidence is found:
P(H | E) = .6 * .5 / .5
P(H | E) = .6

Let’s try it for the absence of evidence. This would be P(H | ~E) = P(~E | H) * P(H) / P(~E). Again, P(H) is already .5, and to keep it simple let’s again make P(~E) equal .5. In order to make P(H) = P(H | ~E), the conditional probability P(~E | H) should also be .5:

Absence of evidence isn’t evidence of absence:
P(H | ~E) = .5 * .5 / .5
P(H | ~E) = .5

But wait, remember that P(E) + P(~E) = 1.00. This means that P(~E) is also necessarily .5. What’s the other axiom? P(E | H) + P(~E | H) should also equal 1.00. But in order to make sure that absence of evidence isn’t evidence of absence, we made P(E | H) = .6, and P(~E | H) = .5. That totals to 1.10: DUTCH BOOKED!!11!!

In reality, if P(E | H) is .6 then P(~E | H) is .4. Let’s run through the so-called “absence of evidence isn’t evidence of absence” part again:

P(H | ~E) = .4 * .5 / .5
P(H | ~E) = .4

As you can see, P(H) was .5 and the absence of the evidence that would have increased it to .6 probability, with its absence, decreases the probability to .4. This will always be easy to test. Since Bayes Theorem works for updating the probability of some hypothesis when encountering new evidence, Bayes Theorem also works for determining how the absence of said evidence would have affected the hypothesis due to the compliments rule above.

Bayes Theorem for updating upon new evidence: P(H | E) = P(E | H) * P(H) / P(E)
The compliment Bayes Theorem for absence of evidence: P(H | ~E) = P(~E | H) * P(H) / P(~E).

There’s another way of looking at it. Remember the Monty Hall problem? It’s an example of statistical independence. Independence is what happens when P(H) = P(H | E). By saying that absence of evidence isn’t evidence of absence, a person is implying that the evidence exists independently of the hypothesis. Again, if the absence of evidence doesn’t have any effect on the hypothesis, then finding the evidence would have the same effect unless they are trying to Dutch Book you.

So make no mistake. Anyone who asserts that absence of evidence isn’t evidence of absence is trying to Dutch Book you.

Comments Off on The Phrase "Absence of Evidence Isn’t Evidence of Absence" is a Logical Fallacy

Posted by on June 29, 2012 in Bayes

NeuroLogica Blog

Slate Star Codex

SELF-RECOMMENDING!

Κέλσος

Matthew Ferguson Blogs

The Wandering Scientist

What a lovely world it is

NT Blog

PsyBlog

Understand your mind with the science of psychology -

Vridar

Musings on biblical studies, politics, religion, ethics, human nature, tidbits from science

Maximum Entropy

Skepticism, Properly Applied

Criticism is not uncivil

Say..

Research Digest

Disrupting Dinner Parties

Feminism is for everyone!