Extraordinary Claims Require Extraordinary Evidence

14 Nov

The first step towards wisdom is not to know that you know nothing. The first step towards wisdom is to know that, placing yourself in the totality of human history, any proposition you currently hold is much more likely to be wrong than it is to be correct; especially any unexamined proposition.

The article I read that made me crystallize that axiom of mine is this article by Isaac Asimov: The Relativity of Wrong. This article implies a philosophy of “less wrong” throughout, which has sort of become my guiding catchphrase: to be “less wrong” about everything I engage in. I'm guessing that this is where the Less Wrong blog got its title (of course, I only became aware of the Less Wrong blog courtesy of Luke at Common Sense Atheism's “Reading Yudkowsky” series, which he started posting November of last year; a number of years after I read that Asimov article).

Here is a snippet from Asimov's article (the article isn't very long, but…):

My answer to him was, “John, when people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.” The basic trouble, you see, is that people think that “right” and “wrong” are absolute; that everything that isn't perfectly and completely right is totally and equally wrong.  However, I don't think that's so. It seems to me that right and wrong are fuzzy concepts…

The Big Question

Anyway, to the point. Does a personal god or gods exist? Like I just wrote, almost every other proposition that a human selected at random from the totality of the existence of the human species was wrong. What causes rain? Why does the sun travel across the sky? What causes disease? How do plants grow? Just about any random question that you can think of had a wrong answer for a lot longer than it has a correct answer. And adhering to “less wrong” philosophy, we still do not have the “correct” answer; we only have the less wrong answer.

For the vast majority of human history, people have believed in gods. Not every single person or every single community, but the vast majority of them have posited the existence of god(s) or some supernatural being(s). Based solely on the prior exemplified in my first two sentences, it is highly unlikely that a god or supernatural beings are the correct explanation for whatever phenomena we are attributing to them.

The most convincing line of evidence that most people use to verify the existence of their god is when they have an experience they can't explain, and thus conclude that their god or the supernatural is the best explanation for said event. However, these people never take into account any competing hypothesis that might also explain the unexplainable experience. You also always have to keep in mind the prior probability for some hypothesis. Again, the probability of you having the correct explanation for something is exceedingly low because you live in the continuum of all of humanity. And the vast majority of all humans who have ever lived have had the wrong explanation.

As I wrote in a previous post, this resorting to the supernatural for an explanation is a type of Prosecutor's Fallacy: Confusing P(E | H) with P(H | E). Sure, given that supernatural beings exist it would be an explanation for your experience; P(E | H) might be relatively high. But P(H) itself is already insanely low. And what about the probability of having that experience in the first place? What about the probability of having that experience given some other competing hypothesis? Some other competing hypothesis that has a higher prior probability that can also explain the experience? Worse yet, if P(E | ~H) is higher than P(E | H), then that experience is actually evidence against your hypothesis.

What if P(~H) is something like a physiological hiccup? I would say that the probability of a physiological hiccup is much higher than the probability of a god or supernatural being existing. Especially given that the vast majority of humans are imperfect and thus physiological hiccups would be the (relative) norm. So I would think that a physiological hiccup might explain the unexplainable experience better than the existence of the supernatural. Of course, this is a very specific claim; it doesn't mean that the supernatural doesn't exist, it just means that the supernatural isn't a better explanation for your particular experience than a physiological hiccup.

Why Apply Bayes (Math) To Religion And History?

Bayes is all about making your assumptions explicit. It's also about taking into account other competing hypotheses that can explain the same evidence or event. But what's the best thing about Bayes, that is somewhat implicit? Each “test” that we have, with us being imperfect beings, we know can't be 100% accurate. There will be false positives and false negatives. The rate of these false positives/negatives is something we need to be acutely aware of when making decisions based on new evidence.

What happens when our false positives rate is too high? The “test” becomes worthless, or in some cases the test might be actually more useful for refuting a hypothesis – as in the case with Oliver's blood. In the case with some religious experience being evidence for the hypothesis that god(s) or the supernatural exists, we should pay extra attention to our rate false positives. But I don't know any religious people who take into account “false positives” of religious experiences. They seem to follow a methodology of counting the hits and ignoring the misses; a sort of selection bias. Their only corroboration of their religious experience is their own unreliable feeling of certainty. Again, what is the success rate of your feeling of certainty? Have you documented when your feeling of certainty was wrong; as in, what's your rate of false positives?

Many people, however, are intimidated by Bayes because it's math, and math can get confusing very quickly. But as I showed in the Monty Hall problem, you don't really need to do any complicated math. If you see that the denominator in the Likelihood Ratio is larger than the numerator, then what you're looking at probably isn't evidence for your hypothesis. Or, if you know that the probability of seeing the evidence at all, P(E), is equal to the probability of seeing the evidence assuming your hypothesis is true, P(E | H), then you know that the evidence really has no affect on your initial probability; the evidence actually exists independently of your hypothesis. And then, even if you think that the evidence at hand is highly likely assuming your hypothesis is true, it doesn't mean that your hypothesis actually is true because the prior probability of your initial hypothesis might be insanely unlikely to begin with. Assuming otherwise is the Prosecutor's Fallacy (i.e. confusing P(E | H) with P(H | E); never assume your hypothesis is true and then conclude that your hypothesis is true).

Carl Sagan's saying “extraordinary claims require extraordinary evidence” is a Bayesian saying. An extraordinary claim, as in a highly unlikely P(H), requires extraordinary evidence, that is, a highly unlikely P(E). Moreover, H has to be extremely well connected to E. That is, there has to be an extremely low rate of false positives – a high P(E | H). If P(H) is low, and P(E) is low, then P(E | H) will have to be HUGE in order to make a dent in P(H | E). And that's only if E actually occurred. If E places some weight on H and E is absent, then this absence is evidence against H. If not, then you have to admit that E has no affect on H at all and is independent of H. Ironically, religious experiences are actually pretty common, so this makes P(E) a relatively large number in the case of E being a religious experience (like seeing a frozen waterfall). Which in turn would make no dent in your prior: P(H), the prior probability of the existence of god or the supernatural will be damn close to P(H | E), the hypothesis that god and/or the supernatural exists given a religious experience. Again, to explain the logic and math behind this, let's go over the Prosecutor's Fallacy again.

The Prosecutor's Fallacy (Again)

Say someone wins the lottery. This would be our evidence or event, P(E). Someone comes up to the lotto winner and says “Aha! You cheated! The probability of winning the lotto is low (true) so that means that you must have cheated!”. The person is claiming that P(E | H) is high. Why is this a fallacy to conclude that the person cheated? Because the prior probability of cheating period is also extremely low. Our P(E) as I said, is the probability of the event happening, in this case it's winning the lottery. Our prosecutor might be right, that given that you cheated, the probability of winning the lottery is high; P(E | H) might be really high. But we can't assume our hypothesis is true and then conclude that it's true; we need to know what P(H) is as well.

So in the case of the Prosecutor's Fallacy, our Bayes formula is this:

P(H | E) = P(E | H) * P(H) / P(E)
P(H | E) = [probability of winning the lottery given that you cheated] * [prior probability of cheating period] / [probability of winning the lottery]

So if we look at Bayes, we can see that we have high * low / low. In this case, if P(E) was equal to P(H), then these two cancel each other out and indeed our P(H | E) is equal to P(E | H). That would be no Prosecutor's Fallacy. But let's think about this: I read stories of people winning the lottery at least once a year. I've never read any stories of anyone winning the lotto because they cheated, nor have I read any stories of people even attempting to cheat. So even if P(E), the event of winning the lottery, is like 1 in a million, P(H) has to be lower than that. With P(H) being lower, this will bring down P(E | H) even if it's like .99. Dividing this by P(E) will only make a relatively small dent in P(H | E); it won't update our prior probability P(H) enough to make it significant.

So Bayes in this case would generally look like this:

P(H | E) = high * lower / low

The two “lows” are close to canceling each other out, but not enough to make the jump from prior probability (lower) to posterior probability that huge, i.e. not enough to make P(E | H) equal to P(H | E). Which is a jump we need it to do to make it a compelling argument.

Again, there's no complicated math going on here. It's just the concept of dividing big numbers by small numbers to get a bigger number, dividing a small number by a big number to get a smaller number (as in the case of Oliver's blood), or dividing numbers that are pretty close to each other to make a not-so-big number (as in the Monty Hall problem).

This has a real world applicability, too. Don't trust your doctor's diagnosis? Ask them what the success rate, P(E | H), and false positives rate, P(E | ~H), is for some test or symptom. Then find out what the prior probability, P(H), is for that disease (that is, the number of people in the total population who have that disease). Then do the math yourself; a lot of crappy doctor diagnoses have elements of the Prosecutor's Fallacy in them, and waste a lot of the patient's money. If a breast cancer test has an 80% success rate, this does not mean that you have an 80% chance of having breast cancer if you get a positive result. As I keep saying, this is P(E | H) and not P(H | E), equivocating between the two is the Prosecutor's Fallacy. You always want to find P(H | E). P(E | H) is one piece of the puzzle that gives you P(H | E).

The Resurrection of Jesus: Ordinary or Extraordinary?

When we talk about the resurrection of Jesus, is this an ordinary claim or an extraordinary claim? There are actually two issues at hand when we talk about this. One, is whether Jesus actually came back from the dead. The other, is whether we would have stories of Jesus coming back from the dead. Jesus actually rising from the dead is an extraordinary claim, however having stories about someone rising from the dead are comparatively mundane. We know they are relatively mundane because we have many other stories in antiquity of other pagan gods and demi-gods coming back from the dead. One of the most famous cities in Western civilization was supposedly founded and named after a guy who was born from a virgin, ascended to heaven, and resurrected from the dead: Romulus, who supposedly founded Rome. There were other pagan gods like Adonis, Hercules, Asclepius, Osiris, Bacchus, Inanna, and Zalmoxis who all came back from the dead in some fashion (i.e. some form of “divine re-embodiment”, 1 Cor 15.35-54). So P(E) in this case would be a lot higher than in the lottery example.

So if we were to formulate Jesus' resurrection in Bayesian terms, our P(H) is the hypothesis that Jesus rose from the dead. Our P(E) is the event/evidence of having stories of Jesus rising from the dead, like the Gospels. Our P(E | H) would be the probability of having stories about Jesus rising from the dead given that Jesus rose from the dead. Our formula might look like this:

P(H | E) = high * low / high

In this case, we have something different than the lottery example. Instead of the prior probability, P(H), and the probability of the event, P(E) being close to each other it turns out that P(E) is closer to P(E | H) so these two terms are closer to canceling each other out. Which means that our prior probability's move to posterior probability will be a very small jump. P(H) is pretty close to P(H | E). The extraordinary claim of Jesus' resurrection only has a relatively ordinary claim of stories about Jesus' resurrection. We would need something more extraordinary to support the claims of Christians. As it happens, there's probably another reason why we would have stories about Jesus' resurrection that in themselves would have a higher prior probability and an equivalent P(E | H).

As I explained above, religious experiences are also pretty mundane due to human imperfection. So our Bayes formula is still denominated by a relatively high P(E). Again, our posterior probability only slides a couple of percentage points in favor. But not enough to make a good argument; not enough to argue from a high probability.

At this point, some apologists might claim that the success of Christianity in the Roman Empire is an extraordinary event or evidence. I actually don't think this is so; Christianity's growth rate is almost exactly the same as the growth rate of Mormonism. Mormon's experienced persecution just like the early Christians did. So if Christianity is true due to its growth rate, then so is Mormonism. However, even Christianity's success can be explained in more pragmatic terms: Christianity succeeded – in a pagan environment – because it successfully emulated pagan ideals. Even though the Gospels superficially quote from the LXX, even though the body of Christianity is cloaked in Judaism, the soul of Christianity is Greek tragedy. A quote that Neil Godfrey has in that post:

What one learned from the classical tradition was what it meant to be a respected person in the larger context of the Greek cosmos, a world controlled by the jealousy of the gods and the vicissitudes of Fate. One learned piety towards the gods, to respect the rights of others, especially the unfortunate, the suppliant, and the stranger. But what one learned above all was how to face the ultimate test, unjust suffering, the inevitable suffering unto death, with courage and integrity. The texts display a remarkable sophistication on this point

This sounds an awful lot like Christianity's ideals and worldview. Substitute the jealousy of the gods with the jealousy of Satan and there seems to be a match, it would seem as though this was the mentality behind the early Christian martyrs, and why their deaths were compelling to interested pagans. As I explained in a previous post arguing for Christians borrowing the Eucharist ceremony from Mithraists, this might just be one more thing that Christians inherited from their cultural matrix.

This is one of the reasons why people claim that the U.S. was founded on Christian ideals. They say this because the U.S. was founded on pagan Greek ideals (like rationalism, equality, and democracy) and confuse this with Christianity, which itself was ensouled with pagan Greek ideals.

So again, the success of Christianity would also be a relatively high P(E), a relatively ordinary claim. What would work for Christianity would be a truly extraordinary claim, a low P(E) coupled with a high P(E | H); a high likelihood ratio. Moreover, this P(E | H) would have to be higher than P(E | ~H) or the probability of seeing the evidence given some other competing hypothesis. In all of these cases, I think that P(E | ~H) is pretty close to P(E | H) but that's just a hunch.

Religious claims don't seem to have the extraordinary evidence that Bayes requires for their extraordinary claims. And if you agree that extraordinary claims require extraordinary evidence, then this lack of evidence is indeed evidence of absence.


Posted by on November 14, 2011 in Bayes


4 responses to “Extraordinary Claims Require Extraordinary Evidence

  1. thefloatinglantern

    November 14, 2011 at 9:57 pm

    Reminds me of the same thing Hume said about miracles, only with less math:

    “…no testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous, than the fact, which it endeavours to establish.”

    What are the chances, given what we know about every person ever who has died, that one person could have come back from the dead? The chances are much better that some people are attempting to deceive us, or have deceived themselves in thinking that a dead man has come back to life.


  2. J. Quinton

    November 21, 2011 at 2:31 am

    Well, it's always good to know that an argument is mathematically valid! Not that there was a huge amount of doubt about the logical validity of Hume's argument in the first place, but religionists can be pretty obdurate…

  3. Christopher Harris

    April 3, 2012 at 5:48 am

    I love your used of Bayes' Theorem for so many subjects. Keep up the good work!

  4. J. Quinton

    April 4, 2012 at 12:01 am

    Thanks! This keeps me motivated lol

NeuroLogica Blog

My ὑπομνήματα about religion

Slate Star Codex

The Schelling Point for being on the #slatestarcodex IRC channel (see sidebar) is Wednesdays at 10 PM EST


Matthew Ferguson Blogs

The Wandering Scientist

Just another site

NT Blog

My ὑπομνήματα about religion

Euangelion Kata Markon

A blog dedicated to the academic study of the "Gospel According to Mark"


My ὑπομνήματα about religion


Understand your mind with the science of psychology -


Musings on biblical studies, politics, religion, ethics, human nature, tidbits from science

Maximum Entropy

My ὑπομνήματα about religion

My ὑπομνήματα about religion

My ὑπομνήματα about religion

atheist, polyamorous skeptics

Criticism is not uncivil


My ὑπομνήματα about religion

Research Digest

My ὑπομνήματα about religion

Disrupting Dinner Parties

Feminism is for everyone!

My ὑπομνήματα about religion

The New Oxonian

Religion and Culture for the Intellectually Impatient

The Musings of Thomas Verenna

A Biblioblog about imitation, the Biblical Narratives, and the figure of Jesus

%d bloggers like this: