Common distributions

The idea

Every named distribution is a compressed story about how data gets generated. Interviewers rarely ask "define the Poisson" — they describe a scenario and expect you to recognize the story. The five to know cold:

• Bernoulli$(p)$ — one trial, two outcomes. $P(X = 1) = p$. Mean $p$, variance $p(1-p)$. The atom every conversion metric is built from.

• Binomial$(n, p)$ — the sum of $n$ independent Bernoullis:

  $P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}, \qquad \mathbb{E}[X] = np, \quad \text{Var}(X) = np(1-p)$

  The story: fixed number of independent trials, same $p$ each time, count the successes. Clicks out of impressions, conversions out of visitors.

• Poisson$(\lambda)$ — counts of arrivals in a fixed window when events occur independently at constant average rate:

  $P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}, \qquad \mathbb{E}[X] = \text{Var}(X) = \lambda$

  Mean equals variance is the fingerprint. It is also the limit of Binomial$(n, p)$ as $n \to \infty$, $p \to 0$ with $np = \lambda$ fixed — the "law of rare events". Tickets per hour, defects per batch, goals per match.

• Exponential$(\lambda)$ — the continuous waiting time between Poisson arrivals: density $f(t) = \lambda e^{-\lambda t}$ for $t \geq 0$, mean $1/\lambda$, variance $1/\lambda^2$, and $P(T > t) = e^{-\lambda t}$. Its defining quirk is memorylessness: $P(T > s + t \mid T > s) = P(T > t)$.

• Normal$(\mu, \sigma^2)$ — density

  $$f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-(x-\mu)^2/(2\sigma^2)}.$$

  It appears whenever a quantity is the sum or average of many small independent contributions — which, thanks to the CLT, is why sample means, regression coefficients, and test statistics are approximately normal even when the raw data is not. Remember 68 / 95 / 99.7 within $1\sigma$ / $2\sigma$ / $3\sigma$.

The connective tissue matters more than any single pmf: Bernoulli sums to binomial; binomial with rare events limits to Poisson; Poisson gaps are exponential; and almost any average of enough of anything limits to normal.

Interact with the shapes directly — the same distribution can look sharply peaked or nearly flat depending on its parameters:

Worked example

Support tickets arrive at $\lambda = 4$ per hour.

• Count in one hour is Poisson(4). Probability of a silent hour: $P(X = 0) = e^{-4} \approx 0.018$ — about one hour in 55.
• Gap between tickets is Exponential(4/hr), mean 15 minutes. Probability the next gap exceeds 30 minutes: $P(T > 0.5) = e^{-4 \cdot 0.5} = e^{-2} \approx 0.135$.
• Now switch stories: 1,000 visitors each convert independently with $p = 0.1$. Conversions are Binomial(1000, 0.1): mean $np = 100$, standard deviation $\sqrt{np(1-p)} = \sqrt{90} \approx 9.5$. Since $np = 100$ and $n(1-p) = 900$ are both large, the normal approximation applies: a 95% range is roughly $100 \pm 1.96 \times 9.5 \approx [81, 119]$. If tomorrow you see 135 conversions, that is more than $3.5\sigma$ above the mean — investigate before celebrating.

The diagnostic habit: after picking a distribution, check its fingerprint against the data. Poisson forces variance = mean; if hourly ticket counts have mean 4 but variance 12, the constant-rate assumption is broken (overdispersion) and a negative binomial or a time-varying rate is the honest model.

Cheat sheet

Recognition rules:

• Binary outcome, one shot → Bernoulli. Count out of a fixed $n$ → binomial.
• Count with no fixed $n$ (arrivals in time/space) → Poisson.
• Continuous waiting time, constant hazard → exponential (memoryless).
• Sum or mean of many independent things → normal (CLT).

Approximation chains:

• Binomial → Poisson when $n$ large, $p$ small, $np$ moderate ($n \geq 100$, $np \leq 10$ as a rule of thumb).
• Binomial → Normal when $np \geq 10$ and $n(1-p) \geq 10$.
• Poisson$(\lambda)$ → Normal$(\lambda, \lambda)$ when $\lambda$ is large (roughly $\geq 20$).

Gotchas:

• Variance ≫ mean in count data → overdispersion; Poisson is wrong, use negative binomial or a mixed/time-varying rate.
• Exponential assumes no aging; wear-out or warm-up means use Weibull.
• Normal is a terrible model for raw heavy-tailed metrics (revenue, latency) — it earns its place through the CLT on means, not on individual observations.
• 95% of a normal is $\pm 1.96\sigma$ ("two sigma"); $\pm 3\sigma$ is 99.7%.