# The Bible, History, and Bayes' Theorem (part 2)

# The Bible, History, and Bayes' Theorem (part 2)

(continued from part 1)

### Recap

Abe,There are several issues you've brought up, and which I've also thought of, which remain to be addressed, so I thought I'd recap what I see as the major remaining issues here:

- How to apply Bayes' Theorem to specific problems in the Jesus Historicity debate (e.g. Paul's reference to James as 'brother of the Lord'.)
- How can we choose between different hypotheses? How can our personal assumptions (such as that it is likely that Jesus existed) bias our evaluation of alternative hypotheses? How can we test two hypotheses without succumbing to the problem of making up ad hoc 'predictions' that simply mould the hypothesis to the new evidence?
- The 'numerical' problem of Bayesian analysis. Is it not appropriate to place numbers where they are unwarranted? Why not just rank probabilities or even simply subjective judgments of confidence?
- Grounding evidence and estimates of probability ratios. How do we know our probability estimates are actually good?
- Dealing with unknown probabilities. The outcomes of our analysis may crucially depend on the probabilities we input as assumptions. But if we only have a vague idea of those probabilities in the first place, how can Bayesian theory help us narrow down the likely ranges of those vague first-estimates?
- Assessing reliability of witnesses. How should we treat statements of fact from possibly (probably) unreliable witnesses? How do we incorporate ideas about what people 'believe' vs. what they actually 'know'?
- Importance of prior information. How sensitive is Bayesian analysis to our initial estimates of probabilities, and what strategies and methods can help eliminate strong personal biases?
- How the accumulation of evidence can increase the confidence in the overall-best hypothesis.

This is only a rough list, and I've probably missed stuff, etc. But it's a decent starting point.

I will say right now that I'm not the one who's going to be able to answer such specific questions as your question about James. I don't have enough knowledge of history and existing historical methods to give a competent model for that specific of a situation.

However, I can definitely come up with analogous problems and show how, given the assumptions of a problem, Bayes' theorem can be applied very flexibly, like an all-purpose tool, to work on whatever problem you can specify. I just don't have the right background knowledge to properly specify the problem of James' relationship to a historical Jesus. But if some historian can specify that problem well-enough, then Bayes' theorem will apply to it.

Given my personal limitations on the subject of history, I've decided to focus more on the general issue of how Bayes theorem can be applied to any subject -- history included -- and how, even more generally, learning more and understanding more about how Bayes' theorem works in a practical way, can help just about anyone to improve their skill in rational, plausible, evidence-based reasoning.

Therefore, I will focus on making Bayesian tools more understandable to the average reader, and also on how we could apply Bayesian reasoning to more general problems that may have some analogous relationship to the specific question of Jesus' historicity.

The first, most crucial topic to explore now comes clearly to the fore:

## The Importance of Prior Information

In the previous example of the birthday present from Alice, we had allowed for the assumption that the likelihoods of getting a watch or getting a keychain should be of equal weight. Thus, they should have a 1 to 1 relationship, which translates to 50% chance of a watch and 50% chance of a keychain.

But what if we knew, again from 100% reliable information from trusty Bob, that Alice happened to work for a watch factory? Furthermore, Bob tells us, Alice has a long-run history of getting people watch-related presents for their birthdays 90% of the time!

Clearly, if we were to ignore this prior information about the situation, we might end up with quite bad probability estimates. It surely no longer seems reasonable to start of with an assumption of equal likelihood, 50% watch, 50% keychain.

As a reminder, when we assumed the probability of a keychain, P(K), was initially 50%, we were able to correctly calculated the probability of a keychain after hearing the box rattle, P(K|R), which turned out to be 75%, because key chains tend to rattle more often than watches (60% vs. 20%).

But now, we should instead apply our prior information from trusty Bob that the initial estimate of P(K) should be more like 10%, since it is 90% likely that Alice would get us a watch-related present, as she usually does. So, we adjust our initial prior probabilities:

P(W) = 90% = 0.9

P(K) = 10% = 0.1

Now, let's look at how this change in prior probabilities affects the posterior probabilities of K or W, after hearing a rattle sound (R). We just apply Bayes' theorem as usual. After practicing this a few times, it will begin to become natural and obvious to us.

P(K|R) = P(K) x P(R|K) / [ P(K) x P(R|K) + P(W) x P(R|W) ]

= 0.1 x 0.6 / [ 0.1 x 0.6 + 0.9 x 0.2 ]

= 0.06 / ( 0.06 + 0.18 )

= 0.06 / 0.24

= 6/24 = 1/4 = 0.25 = 25%

So, Bayes theorem tells us that, given our assumptions, the original prior P(K) of 10% should be updated to a new P(K|R) of 25%, which incorporates the new clue of the rattle sound, which is more likely in the case of a keychain than a watch.

Notice that P(K|R) is larger than 10%, but it is not very close to 75%, which was the result we got from the first example.

So, even after rattling the box, we should still expect that there's only a 25% chance of a keychain, and it's still 75% likely that the present is a watch.

Clearly, the prior probability that we assume about the likelihood of getting each of the different kinds of presents has a large, dramatic effect on the post probabilities. The clue of the rattling sound still gives us some information, updating 10% to 25%, but it is not enough to overwhelm the initial very low prior probability to boost it above 50% or anywhere close to 75%. The rattle is a good clue, but it's not that good. There's still too much of a chance that Alice got you a watch and it just happened to be one of the 20% of watches which happen to rattle when shaken.

### The Strength of Evidence

What kind of clue could overcome such a low prior probability? The strength of evidence is linked to the conditional probabilities, or likelihoods that they predict for various outcomes. The rattling clue (R) had a likelihood of happening 60% of the time when there's a keychain, and 20% of the time when there's a watch. Presumably, if you had put 'Pebbles' on your wish list, they would have something like a 95% chance of rattling when shaken, and so rattling would have even more strongly favoured pebbles in the box than a watch.

Now, imagine you had a third friend, Cindy, who was pretty trustworthy, but not quite as trusty as Bob. She tells the truth 90% of the time, and only 10% of the time is she wrong (or lying).

Bob has let you know ahead of time that Cindy definitely does know what's in the box.

While Bob distracts Alice by pointing out the importance of prior probabilities in Bayesian calculations, you quickly whisper to Cindy, "What's in the box?"

Cindy whispers back, "It's a keychain!" When Alice turns back to you, you've already straightened your face, but perhaps are smirking a little bit.

The question now is: a) If Cindy whispered her "keychain" (let's call this 'C_{k}') before you shook the box, what is the actual probability of a keychain, given that Cindy has said it's a keychain, P(K|C_{k})? b) If Cindy had whispered "keychain" after you shook the box, what is the actual probability of a keychain, given both R and C_{k}, P(K|R and C_{k})? c) If, after hearing Cindy's answer (from part a), you then shake the box and it rattles, is P(K|C_{k} and R) the same as P(K|R and C_{k})?

### Extraordinary Claims Require Extraordinary Evidence

Let's explore this by working through the three-part question above.

Part a) is a quite straight-forward Bayesian calculation. Since Cindy tells the truth 90% of the time, then the probability that she would say "keychain", in the case when it's actually a keychain, is 90%. Likewise, the case when it's a watch, she would still say "keychain" 10% of the time, presumably because she's lying. So, P(C_{k}|K) is 90% and P(C_{k}|W) is 10%. Plug these in and we get our answer:

P(K|C_{k}) = P(K) x P(C_{k}|K) / [ P(K) x P(C_{k}|K) + P(W) x P(C_{k}|W) ]

= 0.1 x 0.9 / [ 0.1 x 0.9 + 0.9 x 0.1 ]

= 0.09 / ( 0.09 + 0.09 )

= 0.09 / 0.18

= 9/18 = 1/2 = 0.5 = 50%

Thus, before shaking the box, Cindy's answer of "keychain" improves our estimated probability of a keychain from a mere 10% to 50%. It's almost as if Cindy's 90% chance of telling the truth 'cancels out' the prior 90% chance that Alice had bought us a watch, rather than a keychain. In fact, the math is exactly like that. You need strong evidence in favour of something to overcome strong prior implausibility of some claim. This is reminiscent of Sagan's motto that "extraordinary claims require extraordinary evidence". Cindy's claim that the present is a keychain is an extraordinary claim, but her high degree of trustworthiness makes her report count as extraordinary evidence in favour of the keychain hypothesis. Not enough to tip the scales, but enough to bring it back as a valid contender against the watch hypothesis.

### Cumulative Evidence Accumulates Cumulatively

Answering part b) requires us to take the prior information/evidence of the rattling box (R) into account. Essentially, the posterior probability from one piece of evidence (R) becomes the prior probability for the next piece of evidence (C_{k}).

Again, this is actually a rather straight-forward application of Bayes' theorem. You basically just apply it twice: First for the evidence of R, and then for the evidence of C_{k}. Just take P(K|R) as your prior probability for C_{k} (rather than the usual P(K).

P(K|R and C_{k}) = P(K|R) x P(C_{k}|K and R) / [ P(K|R) x P(C_{k}|K and R) + P(W|R) x P(C_{k}|W and R) ]

This equation is asking us for P(C_{k}|K and R) and P(C_{k}|W and R), which are the probabilities of Cindy saying "keychain" given that it is a keychain (or watch) and the box rattled. But Cindy's answer doesn't depend on whether or not the box rattled. Regardless of a rattle, she will answer truthfully 90% of the time, according to our assumptions. So, P(C_{k}|K and R) is actually exactly the same as P(C_{k}|K), which is 90%. Likewise, P(C_{k}|W and R) is P(C_{k}|W), which is 10%. So, we'll simplify the equation and just plug in the numbers:

P(K|R and C_{k}) = P(K|R) x P(C_{k}|K) / [ P(K|R) x P(C_{k}|K) + P(W|R) x P(C_{k}|W) ]

= 0.25 x 0.9 / [ 0.25 x 0.9 + .75 x 0.1 ]

= 1/4 x 9/10 / [ 1/4 x 9/10 + 3/4 x 1/10 ]

= 9/40 / [ 9/40 + 3/40 ]

= 9 / ( 9 + 3)

= 9 / 12 = 3/4 = 0.75 = 75%

So, Cindy's "keychain" statement raises our estimate from 25% (itself improved from 10% after hearing the rattling) all the way up to 75%, which is respectable. Even though we initially would expect that Alice's predilection for watch-related gifts made a keychain unlikely at 10%, the combined evidence of hearing a rattle sound, and getting Cindy to confess that it's a "keychain" has boosted our confidence that it really is a keychain, and not a watch.

It could still be the case that the rattle came from a watch, and Cindy was just lying to us, but both of these combined are so unlikely as to outweigh the initial unlikeliness of getting a non-watch-related gift in the first place.

Just to confirm that Bayesian calculations don't introduce weirdness or inconsistencies, let's check what would have happened if we heard Cindy's "keychain" first, and then rattled the box second. Would the answer have been different???

Remember, that Cindy's "keychain" resulted in P(K|C_{k}) = 50%, and the proper way to combine evidence is to use the posterior probabilities from one as the prior probabilities for the next. So, the calculation uses a prior probability of 50%, which is exactly the same as the very first example in the previous post. And since the rattling evidence is independent of anything Cindy might say, we don't have to consider any dependencies between the evidence (this is not always the case in the real world, but this can be handled; it's just more complex than I want for this example).

P(K|C_{k} and R) = P(K|C_{k}) x P(R|K and C_{k}) / [ P(K|C_{k}) x P(R|K and C_{k}) + P(W|C_{k}) x P(R|W and C_{k}) ]

= P(K|C_{k}) x P(R|K) / [ P(K|C_{k}) x P(R|K) + P(W|C_{k}) x P(R|W) ]

= 0.5 x 0.6 / [ 0.5 x 0.6 + 0.5 x 0.2 ]

= 0.3 / ( 0.3 + 0.1 )

= 0.3 / 0.4

= 3/4 = 0.75 = 75%

Exactly the same as P(K|R and C_{k}), as we should expect.

If you keep things straight in the calculations, each independent piece of evidence modifies the final posterior probability independently of the other pieces, giving the same final answer regardless of the order you examine/process the evidence. The most important thing in Bayesian probability is to include all the relevant evidence and background information in the overall model of the situation. The more thorough your evidence, the more confident you can be in the answers that come out.

I'm working on developing more examples which may be more closely relevant to Jesus' historicity, but I think I'll just post this part first, since it's very important to understand how prior information works in Bayesian probability calculations.

If there's some issue you'd like a more-direct answer to, which I haven't addressed yet, please remind me, and I'll try to give a brief answer in the meantime.

Wonderist on Facebook — Support the idea of wonderism by 'liking' the Wonderism page — or join the open Wonderism group to take part in the discussion!

Gnu Atheism Facebook group — All gnu-friendly RRS members welcome (including Luminon!) — Try something gnu!

#1Another useful result of Bayes' theorem. Here God is presumed to be omnipotent

Hypothesis 1: God exists.

Hypothesis 2: God does not exist.

These form a dichotomy, so we can apply normalization to demand P(H1) + P(H2) = 1 before and after we apply any evidence.

Now I'm going to pray. What are the odds that God will fail to answer my prayer? (A)

By hypothesis 2, God does not exist and thus cannot answer my prayer: P(A|H2) = 1

By hypothesis 2, God is capable of answering my prayer and thus there must be some chance, however small, that he will: P(A|H1) < 1

Oh look, it turned out God didn't answer my prayer.

Now we apply Bayes' Theorem to see if this lack of evidence favors one of the hypotheses. Remember, we don't care what the final result is (and thus we don't care what the priors are). We only care about the

changein the probabilities assigned to each hypothesis. Does the lack of God answering prayers favor H1 or H2? For conveniency, denote the denominator (which is the same for each hypothesis) as C.P(H1|A) = P(H1)*P(A|H1)/C = P(H1)*(Something less than 1)/C < P(H1)/C

P(H2|A) = P(H1)*P(A|H1)/C = P(H1)*1/C = P(H2)/C

By normalization, 1 = P(H1|A) + P(H2|A) < P(H1)/C + P(H2)/C = 1/C So C < 1.

Since C < 1, P(H2|A) = P(H2)/C > P(H2), so the hypothesis 2 gains favor

By normalization, hypothesis 2 gaining favor implies that hypothesis 1 loses favor.

Note that everything is a variable here. It doesn't matter what your priors are. It doesn't matter if your prior probability of God existing is 99.99999999% and your probability of him ignoring my prayer is 99.9999999999%. In such a biased scenario the shift will be slight, but still present.A lack of evidence is evidence of a lack.Thisis why the burden of proof is on the theist.Questions for Theists:

http://silverskeptic.blogspot.com/2011/03/consistent-standards.html

I'm a bit of a lurker. Every now and then I will come out of my cave with a flurry of activity. Then the Ph.D. program calls and I must fall back to the shadows.

#2Zaq wrote:What if what you pray for, coincidentally does happen? Or what if you can rationalize or perceive said prayer as being answered?

My point in part one was that the flaw of the application of theorem rests in the lack of an objective value to each piece of evidence. I understand how it can be applied, but each side of the argument would weigh the evidence subjectively.

"Don't seek these laws to understand. Only the mad can comprehend..." -- George Cosbuc

#3Zaq wrote:Hi Zaq. Actually, you are presuming one or the other hypothesis to interpret whether or not 'God answered the prayer'.

This is not a violation of Bayes' theorem. If you

didknow for sure whether or not God answered a prayer, then you could use your model to modify the probabilities as you suggest.However, you

don'tknow for sure if God answered the prayer or not. All you can know in this circumstance is that the evidence conforms or does not conform to the 'prayer being answered as asked'.Imagine you ask "for a $5 bill by tomorrow at 9:00am". Time passes, no money, 9:00am rolls around. Okay: Prayer was not answered. However, what if by some coincidence: Time passes, a friend slips you $5 to pay back an old debt, 9:00am rolls around. Prayer answered? Or not? Hard to say. All you can say is "The conditions of the prayer were satisfied as specified in the prayer, whether or not a God exists, and whether or not this supposed God answered the prayer."

So, you would have to assess the probability of these prayed-for conditions to be satisfied, under each specific circumstance.

H2: God doesn't exist, so cannot answer the prayer. The conditions will be met according to random chance. Say, with probability p.

H1: God exists and adds a positive increase to the probability of whatever is prayed for, above and beyond pure random chance. Therefore, probability is p + g, where g is the additional 'god' factor.

So, whenever a coincidence occurs, you'd have to give the prayer theory at least a little tiny bit of extra boost. However, whenever the coincidence doesn't happen, the prayer theory would automatically get penalized.

Over time, as events play out basically randomly, the prayer theory would get more (and heavier) penalties than bonuses, and would creep inevitably towards disconfirmation. The bigger the hoped-for 'god factor' g, the faster the disconfirmation will happen. These days theists have to basically say that their god has a completely undetectable effect in order to protect their superstitions. Essentially, they concede that g=0.

Here's how some such calculations might play out if a theist foolishly claimed that prayed-for events would have 10% greater odds of occurring than non-prayed-for events.

Starting with 50%/50% prior probabilities (very heavily biased in favour of prayer working).

After 100 patients tested in each group (prayed for, not prayed for), with no increase for those prayed for: 47% for prayer, 53% against.

After 1000 patients: 24% for prayer, 76% against.

After 2000: 9% to 91%

After 5000: 0.3% to 99.7%

After 10,000: 0.0012% to 99.9988%

And much worse after that.

The more you test it, the worse the case for prayer would get, as the margin for error would shrink and shrink making it clearer and clearer that there's no observable advantage.

As the saying goes: Nothing fails like prayer.

Wonderist on Facebook — Support the idea of wonderism by 'liking' the Wonderism page — or join the open Wonderism group to take part in the discussion!

Gnu Atheism Facebook group — All gnu-friendly RRS members welcome (including Luminon!) — Try something gnu!

#4natural wrote:This is a fine example for prayer, or anything testable. Any apologist worth his salt would take a Calvinistic approach to this in saying god just hates you, and he's not answering your prayers for that reason. Where this does fail, is in evaluating evidence that cannot be ruled out statistically. That's why the majority of "evidence" provided by theists is so vague, or can be interpreted so many ways. I find this formula to be a good mathematical representation of how our mind works in calculating odds.

In your example in part one, with the watch and bracelet, it is obvious that one is more probable than other right off the bat. The mind evaluates and we intuitively 'know' which is more likely to occur. However, if you were taught since birth that watches rattle more than bracelets, and you were indoctrinated by the church of the Watch. You would most likely conclude that it was a watch in the box.

"Don't seek these laws to understand. Only the mad can comprehend..." -- George Cosbuc

#5I have known about Bayes' theorem for quite a while, but only in recent years have I really tried to look into it, and realized just how significant it is.

To me it goes a long way to make "induction" and reasoning with probabilities as rigorous as standard logic.

IOW, where we don't have clear yes/no, true/false data to work with.

Our brains have very limited ability to juggle probabilities, and draw accurate conclusions from fuzzy data, in all but the simplest cases.

Favorite oxymorons: Gospel Truth, Rational Supernaturalist, Business Ethics, Christian Morality

"Theology is now little more than a branch of human ignorance. Indeed, it is ignorance with wings." -Sam HarrisThe path to Truth lies via careful study of reality, not the dreams of our fallible minds- meFrom the sublime to the ridiculous: Science -> Philosophy -> Theology

#6Thanks, natural, and I apologize for getting back to you so late. Just today finished up two courses in grad school. I certainly don't deny the expansive utility of Bayes' Theorem. So, to make your thought experiment sufficiently analogous for problems of history, here is what you will need to do: integrate evidence from subjective sources. By, "subjective sources," I mean written and spoken language. The evidence of ancient history, in fact, is almost nothing but written language. So, a better thought experiment is where you have nothing but subjective evidence.

Let's say you have no idea about probabilities from shaking the box, you don't know how often anyone tells the truth, and you have these lines of evidence:

(1) The card on the note says, "I hope you enjoy wearing this to your graduation party."

(2) Cindy tells you, "Alice has worn keychains for necklaces before."

(3) Alice's little brother Gilbert says, "A dude wearing a keychain for a necklace would look dorky."

(4) Cindy says to Gilbert, "Obviously, Alice doesn't care what you think is dorky."

(5) Gilbert says, "It don't matter what I think. Dad wears watches, not keychains."

(6) Cindy says, "He wore a keychain once. Your mom told me that he told her that he did!"

Now, come up with a few probability input values for each of this evidence, and use Bayes' Theorem to estimate the odds of whether the box contains a watch or a keychain.

This problem has a small fraction of the complexity of the simplest problems of New Testament history, such as the matter of James, the brother of Jesus. If you need to, then start even smaller and cut out a few of these lines of evidence. It is my opinion that the attempted applications of Bayes' Theorem to problems of completely-subjective evidence are DOA. Richard Carrier wrote a long article encouraging Bayes' Theorem to be applied to problems of New Testament history, without ever solving a practical example problem. Richard Carrier is a hack and a fraud, in my opinion--he is a smart guy and his proposal seems damned stupid, designed merely for mythers, and he has delayed publishing his book on the topic, but you can prove me wrong by solving a problem like this, and I wish you the best of luck.

#7Thanks, Abe. That looks like an interesting challenge. I'll see what I can do with it.

However, you should know that it is not possible to draw

anyinferences ifallthe evidence is subjective (well, I guess you could draw inferences about what people subjectively believe, but that's hardly going to help in the case of Jesus historicity). We need to havesomegrounding in objective, publicly available facts, knowledge, and evidence. I think a good place to start with that would be archaeology, anthropology, linguistics, psychology, sociology, etc.All of this foundational knowledge is usually called 'background evidence' or 'background knowledge' in Bayesian inference. I left it out of my first posts because it can make things seem a bit more complicated (though it's not really that complicated in practice), but the links I gave earlier to Richard Carrier's papers on this topic deal with the issue of background evidence right from the get-go.

An example of how you would need some good background evidence is to estimate the initial (prior) probability of some average person stating something factual vs. something imagined vs. something intentionally fabricated. Basically, you would need at least a simple background hypothesis of the reliability of witnesses, given various parameters (such as the prevailing culture, the mode of speech (prophecy vs. retelling a story), etc.). I don't claim to know how to tackle that. Probably it would require evidence/theory from psychology, archaeology (to check that statements are factual or not, based on objective evidence), anthropoloy, and history, among other fields.

But my main point is that while it may be rather difficult to develop a truly thorough theory of witness reliability for any/all situations, we can start simple and build up as needed. This is one of the additional advantages of Bayesian reasoning. If new evidence/hypotheses come to light, you don't have to throw out all your previous work, you just incorporate it into a more sophisticated model and you'll get better and better predictions.

So, I'll try out your challenge, but I will necessarily have to make a few simplifying background assumptions, based on my limited knowledge of the requisite science/evidence. But if you disagree with my simplifying assumptions, just note that it won't topple the model I build on them. We would simply have to find better evidence to support or refute my assumptions, and then hook them in to the model to see how they modify the results. It may drastically change the results, of course, but that's how these models are supposed to work: If you dramatically change the underlying assumptions, you will probably get dramatically different output. The model remains valid, it's just the inputs which are refined.

(That paragraph may be unclear, so I'll try a different tack: A Bayesian model is not necessarily like a logical argument. Well, it IS like a logical argument, but more flexible. In a logical argument, the premises must be true for the argument to be sound and valid. In a Bayesian model, premises can be probabilities, not strictly true or false, and the validity of the argument rests upon the correct application of the axioms/theorems of probability, not so much whether the premises are 'true' or 'false'. I may end up proposing some weak, over-simplified premises, but it would be a mistake to focus on those as showing my overall model as invalid.)

Can't say when I'll get back to this, but it is pretty interesting to me, so I think probably within a week I imagine.

Uhhh, what do you base that on? His preliminary work on Bayes?! Can you point to any of his published scholarly work as being exemplary of hackery and fraudulence?

Seems a wee bit uncharitable, I must say, and it certainly seems to indicate a prior bias on your part.

Wonderist on Facebook — Support the idea of wonderism by 'liking' the Wonderism page — or join the open Wonderism group to take part in the discussion!

Gnu Atheism Facebook group — All gnu-friendly RRS members welcome (including Luminon!) — Try something gnu!