Bayes' Theorem
Bayes' theorem calculates conditional probability P(A|B) by relating it to P(B|A).
Foundation of Bayesian statistics.
The Formula
Bayes' theorem lets you update the probability of a hypothesis (A) based on new evidence (B). It "reverses" the conditioning — turning P(B|A) into P(A|B).
Variables
| Symbol | Meaning |
|---|---|
| P(A|B) | Posterior probability — probability of A given that B has occurred |
| P(B|A) | Likelihood — probability of B given that A is true |
| P(A) | Prior probability — initial probability of A before seeing evidence B |
| P(B) | Evidence — total probability of B occurring (under all scenarios) |
Expanded Form (Law of Total Probability)
This expanded form is often easier to use in practice, since P(B) is frequently not known directly.
Example 1
A medical test is 99% accurate for detecting a disease (sensitivity). The disease affects 1 in 1,000 people. If someone tests positive, what is the probability they actually have the disease?
Define: A = has disease, B = tests positive
P(A) = 0.001, P(not A) = 0.999
P(B|A) = 0.99, P(B|not A) = 0.01 (1% false positive rate)
P(A|B) = (0.99 × 0.001) / (0.99 × 0.001 + 0.01 × 0.999)
P(A|B) = 0.00099 / (0.00099 + 0.00999) = 0.00099 / 0.01098
P(A|B) ≈ 0.090 or about 9% — despite the test being 99% accurate, there is only a 9% chance the person actually has the disease
Example 2
A factory has two machines. Machine 1 produces 60% of items and has a 2% defect rate. Machine 2 produces 40% and has a 5% defect rate. A defective item is found. What is the probability it came from Machine 2?
Define: A = from Machine 2, B = defective
P(A) = 0.40, P(not A) = 0.60
P(B|A) = 0.05, P(B|not A) = 0.02
P(A|B) = (0.05 × 0.40) / (0.05 × 0.40 + 0.02 × 0.60)
P(A|B) = 0.02 / (0.02 + 0.012) = 0.02 / 0.032
P(A|B) = 0.625 or 62.5%
When to Use It
Bayes' theorem is used whenever you need to update probabilities based on new evidence.
- Medical diagnosis — interpreting test results given disease prevalence
- Spam filtering — updating the probability that an email is spam based on its content
- Machine learning and artificial intelligence — Bayesian classifiers
- Quality control — determining the source of defective products
- Legal reasoning — evaluating the weight of evidence
- Any situation where prior knowledge and new data must be combined