# Daily Archives: February 2, 2011

## History and Bayes’ Theorem

Tom Verenna has posted a pdf from Richard Carrier wherein Dr. Carrier describes how Bayes' Theorem can be applied to historical inquiry. I've often wondered how we can claim some things occured in history without any sort of metric for determinig its historicity. Where's the dividing line between historical and unhistorical? Usually the historicity of some event is described in subjective terms like “very likely”, “not likely”, “probably”, etc. Even though history is not a science, there should be criteria for discerning what is historical that is more rigid than the aforementioned subjective terminology.

Dr. Carrier presents Bayes' Theorem with the following form:

P(h | e.b) = P(h | b) x P(e | h.b) / [P(h | b) x P(e | h.b)] + [ P(~h | b) x P(e |~h.b) ]

Where:

P = Probability (epistemic probability = the probability that something stated is true)

h = hypothesis being tested

~h = all other hypotheses that could explain the same evidence (if h is false)

e = all the evidence directly relevant to the truth of h (e includes both what is observed
and what is not observed)

b = total background knowledge (all available personal and human knowledge about
anything and everything, from physics to history)

P(h|e.b) = the probability that a hypothesis (h) is true given all the available evidence (e)
and all our background knowledge (b)

Looks awesome enough. To illustrate the problem with the logic of how most historical arguments are formed, Dr. Carrier begins by introducing syllogistic argument for historicity:

Major Premise 1: All major cities in antiquity had public libraries.
Minor Premise 1: Jerusalem was a major city in antiquity.
Conclusion: Therefore, Jerusalem had a public library.

But Major Premise 1 is a bit unreasonable to assert, since we don't have the type of knowledge that would allow us to say that “[a]ll major cities in antiquity had public libraries”. The most we could say is that most major cities in antiquity had public libraries. But this weakens the deductive strength of the conclusion, i.e. we would have to temper it with “Therefore, Jerusalem more than likely had a public library”. But this is the subjective language I was talking about. Complicating things, we don't have the historical strength to assert with 100% confidence that Jerusalem was a major city in antiquity. There's a “high probability” that it was, but this is again subjective language.

So we would begin with assigning general probabilities to these premises. But once we do that, we would need a method that accurately slides the probabilities in one direction or another based on the weight we give each type of evidence. This is where Bayes' Theorem comes into play. Say we explicate the above syllogism like so:

Major Premise 1: There's a 60% chance that any major city in antiquity had a public library
Minor Premise 1: There's a 90% chance that Jerusalem was a major city
Conclusion: 60% * 90% probability that Jerusalem had a public library

This results in a 54% chance that Jerusalem had a public library. But how would we factor in more evidence, say a manuscript from antiquity asserting that Jerusalem had a public library? We would need some way of adding this additional evidence that doesn't rely on straightforward multiplication, since this would lead to diminishing probabilities as more evidence is gathered!

(The following I numbered for clarity)

If we use Bayes’ Theorem to determine the likelihood that Jerusalem had a public library, and if the following data is the same (note that this model of the problem and these percentages are deliberately unrealistic—realistically they should all be much higher, and the evidence is more complicated):

60% chance that any major city in antiquity had a public library [this would be the major premise]

90% chance that Jerusalem was a major city [this would be the minor premise]
60% chance that there was a library if there is a document attesting to a library [this is additional evidence]
90% chance that we have a document attesting to a public library in Jerusalem [also additional evidence]

1. P(h|b) = (0.6 x 0.9) + x = 0.54 + x = The prior probability that Jerusalem was a major city and (as such) would have a public library [x is negligible, but inserted into the equation to account for a prior probability that Jerusalem was not a major city but had a public library. For this example it is assumed to be 0%.]

[…]

2. P(~h|b) = 1 – 0.54 = 0.46 = The prior probability that Jerusalem did not have a public library = the converse of the other prior (i.e. all prior probabilities that appear in a Bayesian equation must sum to exactly 1, no more nor less, because together they must encompass all possible explanations of the evidence for Bayes’ Theorem to be valid).

3. P(e|h.b) = 1.0 = The consequent probability that we would have either some specific evidence of a public library at Jerusalem and / or no evidence against there being one.

For (3) in other words, assuming the historicity of the public library at Jerusalem and our background knowledge, finding evidence for it (like the ancient manuscript) would be no surprise. But it would also be no surprise if we found no evidence (again, like the manuscript) for this public library. So either way, the evidence that we have right now is consistent with Jerusalem having a public library.

4. P(e|~h.b) = 0.4 = The consequent probability that there would be a document attesting to a public library in Jerusalem even when there wasn’t a public library there (i.e. the converse of the 60% chance that such a library existed if we have such a document).

P(h | e.b) = 0.54 * 1.00 / [0.54 * 1.00] + [0.46 * 0.40]
P(h | e.b) = 0.54 / (0.54 + 0.184)
P(h | e.b) = 0.54 / 0.724 = 0.746 = 75%

RESULT: […W]e get the plausible result of a 75% such chance. And this was found with unrealistic percentages that biased the result against there being a library there, which means a fortiori we can be highly certain Jerusalem had a public library. This illustrates the advantage of using unrealistically hostile estimates against a hypothesis, since if the conclusion follows even then, we can have a high confidence in that conclusion.

Well isn't this just pulling percentages out of your nether regions? Not necessarily:

One common objection to using Bayes’ Theorem in history is that Bayes’ is a model of mathematical precision in a field that has nothing of the kind. This precision of the math can create the illusion of precision in the estimates and results. But as long as you do not make this mistake, it will not affect your results. The correct procedure is to choose values for the terms in the equation that are at the limit of what you can reasonably believe them to be, to reflect a wide margin of error, thus ensuring a high confidence level (producing an argument a fortiori), regardless of any inexactness in your estimations. For example, surely more than 60% of major cities in antiquity had public libraries (the evidence is compellingly in favor of a much higher percentage, provided ‘major city’ is reasonably defined). But since we don’t have exact statistics, we can say that the percentage of such cities must fall between 60% and 100% (= 80% with a margin of error +/-20%). With such a wide margin of error, our confidence level remains high (see appendix). We are in effect saying that we might not be sure it was 100% (or 90% or even 80%), even though we may believe it is, but we can be sure it was no less than 60%. Since that is the limit of what we deem reasonable, so will our conclusion be (the conclusion is only as strong as an argument’s weakest premise, and each probability assigned in a Bayesian equation is the formal equivalent of a premise).

So there you have it. This is my simple interpretation of Dr. Carrier's article, so read it on its own terms if you find mine lacking :). One of the references that Dr. Carrier makes is to Eliezer Yudkowsky's intro to Bayes' Theorem. There's a bit more I want to write about Bayes' Theorem and the ad hocness that I see in religious debates, but I'll save that for a future post.

Comments Off on History and Bayes’ Theorem

Posted by on February 2, 2011 in Bayes

NeuroLogica Blog

Slate Star Codex

SELF-RECOMMENDING!

Κέλσος

Matthew Ferguson Blogs

The Wandering Scientist

What a lovely world it is

NT Blog

PsyBlog

Understand your mind with the science of psychology -

Vridar

Musings on biblical studies, politics, religion, ethics, human nature, tidbits from science

Maximum Entropy

Skepticism, Properly Applied

Criticism is not uncivil

Say..

Research Digest

Disrupting Dinner Parties

Feminism is for everyone!