Bayes' theorem · Math Foundry

The idea

Bayes' theorem is the rule for reversing a conditional probability. You usually know how evidence behaves given a hypothesis — test accuracy, model recall, symptom rates. You want the reverse: how probable is the hypothesis given the evidence. They are not the same thing, and the gap between them is set by the prior:

$P(H \mid E) = \frac{P(E \mid H)\,P(H)}{P(E)}, \qquad P(E) = P(E \mid H)P(H) + P(E \mid \neg H)P(\neg H)$

Read it as: posterior $\propto$ likelihood $\times$ prior. The denominator is just the total probability of seeing the evidence at all, summed over both worlds.

The equivalent odds form is often faster mental math and chains cleanly:

\begin{aligned} \underbrace{\frac{P(H \mid E)}{P(\neg H \mid E)}}_{\text{posterior odds}} &= \underbrace{\frac{P(H)}{P(\neg H)}}_{\text{prior odds}} \\ &\quad\times \underbrace{\frac{P(E \mid H)}{P(E \mid \neg H)}}_{\text{likelihood ratio}}. \end{aligned}

Each independent piece of evidence multiplies the odds by its likelihood ratio. A signal with LR = 10 is "worth one order of magnitude of odds" no matter what the prior is.

Why interviews love this: the base rate fallacy. When the hypothesis is rare (fraud, disease, churned-whale accounts), even a strong likelihood ratio leaves the posterior small, because the enormous negative class manufactures false positives faster than the tiny positive class produces true ones. Every "the test is 95% accurate, so you are 95% likely sick" claim is this error.

Worked example

Condition prevalence 1%. Test sensitivity 95% ( $P(+ \mid \text{sick})$ ), specificity 90% (so false positive rate 10%). Take 10,000 people:

	Sick (100)	Healthy (9,900)
Test positive	95	990
Test negative	5	8,910

Positives total $95 + 990 = 1085$ , of which only 95 are real:

$P(\text{sick} \mid +) = \frac{95}{1085} \approx 8.8\%$

Same answer via odds: prior odds $1{:}99$ , likelihood ratio $0.95 / 0.10 = 9.5$ , posterior odds $9.5{:}99 \approx 0.096$ , probability $0.096 / 1.096 \approx 8.8\%$ .

The population table is the interview-safe method: pick a round number of people, fill in the four cells, and read the answer off the positive row. It is nearly impossible to get wrong under pressure, unlike formula juggling.

Cheat sheet

Bayes: $P(H \mid E) = \dfrac{P(E \mid H) P(H)}{P(E \mid H)P(H) + P(E \mid \neg H)P(\neg H)}$
Odds form: posterior odds = prior odds $\times$ LR, where $\text{LR} = \dfrac{P(E \mid H)}{P(E \mid \neg H)}$ . Independent evidence → multiply LRs.
Under pressure: build the 4-cell population table with 10,000 people; read off precision = TP / (TP + FP).

Term	Also called	Formula
Sensitivity	recall, TPR	$P(+ \mid H)$
Specificity	TNR	$P(- \mid \neg H)$
FPR	fall-out	$1 - \text{specificity}$
$P(H \mid +)$	precision, PPV	what Bayes computes

Gotchas:

$P(E \mid H) \neq P(H \mid E)$ — the direction of the bar is everything.
Rare positives → low precision even with excellent sensitivity and specificity. Precision depends on prevalence; sensitivity does not.
Multiplying likelihood ratios requires evidence conditionally independent given $H$ ; correlated signals double-count and inflate the posterior (naive Bayes tolerates this for ranking, not for calibrated probabilities).
A p-value is $P(\text{data at least this extreme} \mid H_0)$ , a likelihood-style quantity — turning it into $P(H_0 \mid \text{data})$ requires a prior, which frequentist testing never supplies.