Poisson, geometric, and hypergeometric (and a key approximation)

Key idea

For discrete random variables, probabilities come from a PMF \(P(X=k)\). Three high-frequency models:

• Poisson \(\mathrm{Poisson}(\lambda)\): counts of events in an interval (with a constant average rate).
  • Support: \(k=0,1,2,\dots\)
  • PMF: \(P(X=k)=e^{-\lambda}\dfrac{\lambda^k}{k!}\)
  • Mean/variance: \(E[X]=\mathrm{Var}(X)=\lambda\)
  • Parameter rule: \(\lambda\ge 0\) (typically \(\lambda>0\) in practice)
• Geometric \(\mathrm{Geom}(p)\) (trials until first success):
  • Support: \(k=1,2,3,\dots\)
  • PMF: \(P(X=k)=(1-p)^{k-1}p\) for \(0<p\le 1\)
  • Mean/variance: \(E[X]=\dfrac{1}{p}\), \(\mathrm{Var}(X)=\dfrac{1-p}{p^2}\)
  • Note: \(E[X]\) is finite for every valid \(p>0\), but it can be very large if \(p\) is small.
• Hypergeometric \(\mathrm{Hypergeometric}(N,K,n)\): sampling without replacement from \(N\) total items with \(K\) “successes,” drawing \(n\) items.
  • Mean: \(E[X]=n\cdot\frac{K}{N}\)

Must-know approximation

If \(X\sim \mathrm{Binomial}(n,p)\) with large \(n\) and small \(p\) such that \(\lambda=np\) is moderate, then \[ \mathrm{Binomial}(n,p)\ \approx\ \mathrm{Poisson}(\lambda=np). \] This is one of the most common “fast approximation” facts on quizzes and exams.

Worked example

Try it

Summary

• Poisson counts: \(k=0,1,2,\dots\), \(E[X]=\mathrm{Var}(X)=\lambda\).
• Geometric (trials to first success): \(k=1,2,3,\dots\), \(P(X=k)=(1-p)^{k-1}p\).
• Hypergeometric mean: \(E[X]=n\cdot\frac{K}{N}\).
• Approximation: \(\mathrm{Bin}(n,p)\approx \mathrm{Poisson}(\lambda=np)\) for large \(n\), small \(p\).

Exponential distribution: rate, scale, CDF, and waiting times

Key idea

The exponential distribution models a waiting time until the next event when events arrive at a constant average rate. If \(X\sim\mathrm{Exponential}(\lambda)\), then:

• Support: \(x\ge 0\)
• PDF: \(f(x)=\lambda e^{-\lambda x}\) for \(x\ge 0\)
• CDF: \(F(x)=P(X\le x)=1-e^{-\lambda x}\) for \(x\ge 0\)
• Survival: \(P(X>x)=e^{-\lambda x}\)
• Mean/variance: \(E[X]=\dfrac{1}{\lambda}\), \(\mathrm{Var}(X)=\dfrac{1}{\lambda^2}\)
• Scale parameter: \(\theta=\dfrac{1}{\lambda}\)
• Parameter rule: \(\lambda>0\)

The exponential distribution is also memoryless: \[ P(X>s+t\mid X>s)=P(X>t). \]

Worked example

Try it

Summary

• Exponential CDF: \(F(x)=1-e^{-\lambda x}\) for \(x\ge 0\); in particular \(F(0)=0\).
• Rate/scale: \(\lambda>0\), scale \(=1/\lambda\), mean \(=1/\lambda\), variance \(=1/\lambda^2\).
• Waiting-time model: \(P(X>t)=e^{-\lambda t}\).

Gamma and chi-squared: degrees of freedom, shape, and key facts

Key idea

A chi-squared distribution with \(k\) degrees of freedom is written \(\chi^2_k\). It appears naturally as a sum of squares: \[ Z_1^2+\cdots+Z_k^2 \sim \chi^2_k \quad \text{when } Z_i\sim N(0,1)\text{ independently.} \]

• Support: \(x\ge 0\) (so it can never be negative)
• Shape: right-skewed for small \(k\); becomes more symmetric as \(k\) increases
• Mean/variance: \(E[\chi^2_k]=k\), \(\mathrm{Var}(\chi^2_k)=2k\)
• Parameter that controls shape: \(k\) (degrees of freedom)

Connection to the gamma distribution

Chi-squared is a special case of the gamma distribution: \[ \chi^2_k \sim \mathrm{Gamma}\!\left(\alpha=\frac{k}{2},\ \theta=2\right), \] where \(\alpha\) is the shape and \(\theta\) is the scale.

Worked example

Try it

Summary

• \(\chi^2_k\) is always \(\ge 0\) and is typically right-skewed for small \(k\).
• \(E[\chi^2_k]=k\), \(\mathrm{Var}(\chi^2_k)=2k\).
• Degrees of freedom \(k\) controls the shape.

F distribution: ratios of variances and mean existence

Key idea

The F distribution arises from a ratio of two independent chi-squared variables divided by their degrees of freedom: \[ F=\frac{(\chi^2_{d_1}/d_1)}{(\chi^2_{d_2}/d_2)} \sim F(d_1,d_2). \] It is widely used in ANOVA and in testing/estimating ratios of variances.

• Support: \(x>0\) (never negative)
• Parameters: \(d_1>0\), \(d_2>0\) (degrees of freedom)
• Mean: \(E[F]=\dfrac{d_2}{d_2-2}\) if \(d_2>2\) (otherwise the mean does not exist)

Worked example

Try it

Summary

• \(F(d_1,d_2)\) is positive: support \(x>0\).
• \(E[F]=\dfrac{d_2}{d_2-2}\) exists only when \(d_2>2\).
• F distributions appear in variance ratios and ANOVA.

Logistic variance and Cauchy “undefined mean” trap

Logistic distribution

A logistic random variable \(X\sim \mathrm{Logistic}(\mu,s)\) has a smooth S-shaped (sigmoid) CDF: \[ F(x)=\frac{1}{1+e^{-(x-\mu)/s}}, \] where \(\mu\) is a location parameter and \(s>0\) is a scale parameter.

• Mean: \(E[X]=\mu\)
• Variance: \(\mathrm{Var}(X)=\dfrac{\pi^2 s^2}{3}\)
• Why it matters: logistic CDFs show up in logistic regression and “probability as a smooth threshold” modeling.

Cauchy distribution

The Cauchy distribution is a classic heavy-tailed distribution. For \(X\sim \mathrm{Cauchy}(x_0,\gamma)\), the tails are so heavy that the mean and variance are undefined. A common special case is the standard Cauchy \(\mathrm{Cauchy}(0,1)\) with PDF: \[ f(x)=\frac{1}{\pi(1+x^2)}. \]

Worked example

Try it

Summary

• Logistic: \(E[X]=\mu\), \(\mathrm{Var}(X)=\dfrac{\pi^2 s^2}{3}\).
• Cauchy: mean and variance are undefined (do not exist).

How to choose the right distribution (fast identification)

Story → distribution shortcuts

• Counts in a time/area interval (arrivals, defects, calls, emails): Poisson\((\lambda)\).
• Waiting time until next event: Exponential\((\lambda)\).
• Trials until first success (first success on trial \(k\)): Geometric\((p)\).
• Sampling without replacement from a finite population: Hypergeometric\((N,K,n)\).
• Sum of squares of standard normals: \(\chi^2_k\).
• Ratio of variances (scaled chi-squared ratio): \(F(d_1,d_2)\).

Parameter checks (quick sanity)

• Poisson: \(\lambda\ge 0\) and outcomes are \(0,1,2,\dots\).
• Exponential: \(\lambda>0\) and outcomes satisfy \(x\ge 0\).
• Geometric: \(0<p\le 1\) and outcomes are \(1,2,3,\dots\) (trials-until-success version).
• \(\chi^2\), F: degrees of freedom are positive; values are nonnegative (and F is strictly positive).

Worked example (Poisson ↔ exponential connection)

Try it

Summary

• Exponential models waiting time; Poisson models counts.
• Always check support: Poisson is \(0,1,2,\dots\); exponential is \(x\ge 0\); \(\chi^2\) is \(x\ge 0\); F is \(x>0\).
• Parameter meaning matters: exponential \(\lambda\) is a rate; Poisson \(\lambda\) is the mean count per interval.

Why these distributions matter (and a final check)

Where Distributions II shows up

• Queueing & reliability: Poisson counts and exponential waiting times (calls, arrivals, failures).
• Quality control: defects per unit (Poisson), pass/fail trials (geometric).
• Sampling & genetics: hypergeometric sampling without replacement.
• Hypothesis testing: chi-squared tests (goodness-of-fit, independence) and F tests (variance ratios, ANOVA).
• Data modeling: logistic for S-shaped probability curves; Cauchy as a reminder that not every distribution has a mean.

Worked example: geometric “first trial success”

Try it

Final recap

• Poisson: \(P(X=k)=e^{-\lambda}\dfrac{\lambda^k}{k!}\), support \(k=0,1,2,\dots\), \(E[X]=\mathrm{Var}(X)=\lambda\).
• Geometric (trials to first success): support \(k=1,2,3,\dots\), \(P(X=k)=(1-p)^{k-1}p\), \(E[X]=1/p\).
• Hypergeometric: sampling without replacement, \(E[X]=n\cdot\frac{K}{N}\).
• Exponential: \(f(x)=\lambda e^{-\lambda x}\), \(F(x)=1-e^{-\lambda x}\) for \(x\ge 0\), mean \(1/\lambda\), variance \(1/\lambda^2\).
• \(\chi^2_k\): nonnegative, \(E[\chi^2_k]=k\), \(\mathrm{Var}(\chi^2_k)=2k\), shape controlled by \(k\).
• F(d\(_1\),d\(_2\)): positive, \(E[F]=\dfrac{d_2}{d_2-2}\) for \(d_2>2\).
• Logistic: \(\mathrm{Var}(X)=\dfrac{\pi^2 s^2}{3}\). Cauchy: mean and variance are undefined.