Bayes’s Theorem

Bayes’s theorem is a way of estimating the likelihood of some event having occurred, or some condition being true, given some evidence that is related to the event or condition. This entry describes how Bayes’s theorem is used, discusses three forms of the theorem, and provides detailed examples of the use of the theorem.

Throughout the sciences, we are faced with questions about how likely an event of interest is, given some information. For example,

• How likely is a student to achieve at least a certain level of academic performance in the future, given the student’s past performance?
• What is the probability that a patient has a certain disease, given the patient’s diagnostic test results and background health information?
• How likely is it that a defendant is guilty of a certain crime, given all available evidence?
• What is the probability that a coin would land heads at least 60 times in 100 tosses, given that the coin is fair?
• How likely is a certain hypothesis, given the observed data?

These are all questions about conditional probability. Philosophical controversies have raged for centuries about exactly how to interpret probability, but conditional probability has a simple, uncontroversial definition: the probability of an event A, given an event B (with P(B) > 0), is

$$P\left(A|B\right)=\begin{array}{c}P\left(A\text{and}B\right)\\ P\left(B\right)\end{array}$$

Swapping the roles of A and B, we have

$$P\left(A|B\right)=\begin{array}{c}P\left(A\text{and}B\right)\\ P\left(A\right)\end{array}$$

Note that P(A|B) is different from P(B|A). Confusing P(A|B) with P(B|A) is a common—and commonly devastating—blunder, sometimes called the prosecutor’s fallacy (though not all prosecutors commit this fallacy, nor is the fallacy exclusive to prosecutors).

This definition immediately yields two useful expressions for P(A and B):

$$P\left(A|B\right)P\left(B\right)=P\left(A\text{and}B\right)=P\left(B|A\right)P\left(A\right),$$

for P(B) > 0. Dividing through by P(B) gives a simple but powerful result that explains precisely how P(A|B) and P(B|A) are related. The next section gives several ways to express this relationship.