## Absence of Evidence is Evidence of Absence

07 Jul
This is a quote from the blog of awesome Less Wrong:
Absence of proof is not proof of absence. But absence of evidence is always evidence of absence. According to the probability calculus, if P(H|E) > P(H) (observing E would be evidence for hypothesis H), then P(H|~E) < P(H) (absence of E is evidence against H). The absence of an observation may be strong evidence or very weak evidence of absence, but it is always evidence.
Αnd I think I'll apply this to the current tide rising in the Historical/Mythical Jesus debate.

To pick the most obvious example, the current JM hypothesis posits that Paul and other early epistle writers would have quoted a teaching of Jesus if they viewed Jesus as a preacher instead of a mythical/legendary/cosmological savior figure. Since they don't do this, then it fits their hypothesis. On the other hand, the HJ hypothesis posited by most scholars is a sort of floating ad-hoc explanation for this. That is, if the early epistle writers did quote a teaching of Jesus, this would be evidence for their hypothesis (a preaching Jesus). But, if the early epistle writers did not quote a teaching of Jesus… well, this is still evidence for their hypothesis (a preaching Jesus).

This is why Bayes theorem is useful. It points out glaring ad hoc hypotheses like these. It's a heads I win, tails you lose set up by scholars.

Let H be the hypothesis that a preaching Jesus existed. E is early epistle writers quoting from the preaching Jesus. Let's say that the probability of the preaching-historical or mythical Jesus is equally 50% (i.e. P(H) = 50%) just for pedagogical value. In the mythical Jesus model, P(H|E) > P(H); the probability of Jesus existing given the evidence of epistle writers quoting him is higher than the bare bones probability of Jesus existing. That is, the probability of Jesus existing is nudged higher than 50% if a letter writer like Paul quotes from Jesus. On the flip side, the P(H|~E) < P(H); the probability of Jesus existing given the lack of evidence of epistle writers quoting him is lower than the bare bones probability of Jesus existing. That is, the probability of Jesus existing is nudged lower than 50% if a letter writer like Paul does not quote from Jesus.

In the historical Jesus model, both P(H | E) AND the P(H | ~E) are the same. When this happens, it basically means that the probability stays at 50%. This violates the probability calculus inherent in Bayes Theorem. Since Bayes is formally/logically valid, this means that the set up that mainstream scholars have erected is logically invalid. E and ~E can't both be evidence for the same thing. The only time that happens is when you're dealing with ad hoc hypotheses: “A hypothesis that forbids nothing, permits everything, and thereby fails to constrain anticipation. Your strength as a rationalist is your ability to be more confused by fiction than by reality. If you are equally good at explaining any outcome (my emphasis), you have zero knowledge“.

Take a look at the likelihood ratio (from E. Yudkowsky's Intro to Bayes Theorem):
The likelihood ratio for X, p(X|A)/p(X|~A), determines how much observing X slides the probability for A; the likelihood ratio is what says how strong X is as evidence.  Well, in your theory A, you can predict X with probability 1, if you like; but you can't control the denominator of the likelihood ratio, p(X|~A) – there will always be some alternative theories that also predict X, and while we go with the simplest theory that fits the current evidence, you may someday encounter some evidence that an alternative theory predicts but your theory does not.  That's the hidden gotcha that toppled Newton's theory of gravity.  So there's a limit on how much mileage you can get from successful predictions; there's a limit on how high the likelihood ratio goes for confirmatory evidence.

On the other hand, if you encounter some piece of evidence Y that is definitely not predicted by your theory, this is enormously strong evidence against your theory.  If p(Y|A) is infinitesimal, then the likelihood ratio will also be infinitesimal.  For example, if p(Y|A) is 0.0001%, and p(Y|~A) is 1%, then the likelihood ratio p(Y|A)/p(Y|~A) will be 1:10000.  -40 decibels of evidence!  Or flipping the likelihood ratio, if p(Y|A) is very small, then p(Y|~A)/p(Y|A) will be very large, meaning that observing Y greatly favors ~A over A.  Falsification is much stronger than confirmation.  This is a consequence of the earlier point that very strong evidence is not the product of a very high probability that A leads to X, but the product of a very low probability that not-A could have led to X.  This is the precise Bayesian rule that underlies the heuristic value of Popper's falsificationism.
Now, what if you posit a Jesus that wasn't a preacher? This would also make sense of the evidence; P(non-preaching Jesus | ~E) > P (non-preaching Jesus). That is, we would expect the early epistle writers to not quote Jesus because they didn't view him as a preacher.

As for ad hocness, take one of the comments over at Vridar:

Cross-posted from comments on Exploring Our Matrix:

So, according to mainstream scholarship…

Accurate geographical details = Evidence for historicity.

Absence of geographical details (e.g. Sepphoris) = more evidence of historicity, and it even tells us new facts about Jesus, e.g., that he deliberately avoided big cities.*
Inaccurate historical details = no problem for historicity/no effect on the historicist model.

You can see where this is going. Accurate and absence should be on either side of the probability formula. This time E would be accurate historical details; in the HJ hypothesis P(H | E) is still equivalent to P(H | ~E).

The problem here is with constraints. I hate to say it, but most scholars work under the assumption of a wandering, preaching Jesus without any constraints on what a wandering, preaching Jesus would entail. So they look at all of the evidence through the lens of a wandering, preaching Jesus. As the quote I posted above says, we should be looking for evidence that disconfirms our pet theories and not look for confirmation of our theories. Because the disconfirmation, as the blog of awesome that I quote from is named, helps us to become less wrong. And becoming less wrong is a lot more powerful than being “right”; implicit in the phrase “less wrong” is admitting to our own fallibility and trying to mitigate it. On the other hand, “confirmation” of our theories only feeds our ego (because it's the default human cognitive bias).

On a higher level, this lack of constraints/making beliefs pay rent seems to also apply to the Problem of Evil. We have this world that seems to be indifferent to our suffering. It would make sense that the ultimate reality (god, the matrix, etc.) actually is indifferent to our suffering. But theists posit that their god loves us and has some sort of plan for all of it or is so in love with Free Will that he will not abrogate it to ease our suffering. The problem is that this god would also explain a world that is 100% free of all suffering (heaven), and this god could also be used to explain a world that is 0% free of suffering… and every single percentage point in between the two extremes. While an indifferent god or no god at all would not explain either of those last two options.

Which worldview is operating on constraints, and which one is not? Which one is making their belief(s) pay rent and which one is letting their belief(s) squat? A theory/worldview that can be used to explain every single possible outcome is really no explanation at all.

Comments Off on Absence of Evidence is Evidence of Absence

Posted by on July 7, 2011 in Bayes, scientific method, skepticism

NeuroLogica Blog

Slate Star Codex

NꙮW WITH MꙮRE MULTIꙮCULAR ꙮ

Κέλσος

Matthew Ferguson Blogs

The Wandering Scientist

Just another WordPress.com site

NT Blog

Euangelion Kata Markon

A blog dedicated to the academic study of the "Gospel According to Mark"

PsyPost

Behavior, cognition and society

PsyBlog

Understand your mind with the science of psychology -

Vridar

Musings on biblical studies, politics, religion, ethics, human nature, tidbits from science

Maximum Entropy

atheist, polyamorous skeptics

Criticism is not uncivil

Say..

Research Digest

Disrupting Dinner Parties

Feminism is for everyone!

The New Oxonian

Religion and Culture for the Intellectually Impatient

The Musings of Thomas Verenna

A Biblioblog about imitation, the Biblical Narratives, and the figure of Jesus