Confidence intervals, critical values, standard error, and margin of error

Key idea

Most confidence intervals follow the same blueprint: \[ \text{CI} = \text{estimate} \pm (\text{critical value})\times(\text{standard error}). \] The margin of error is: \[ \text{ME}=(\text{critical value})\times \mathrm{SE}. \] A higher confidence level (like 99% vs 95%) uses a larger critical value, which makes the interval wider. Larger sample size usually makes \(\mathrm{SE}\) smaller (often proportional to \(1/\sqrt{n}\)), which makes the interval narrower.

Critical values you’ll see often

• z critical value: \(z_{1-\alpha/2}\) for two-sided confidence intervals when the sampling distribution is (approximately) normal.
• t critical value: \(t_{1-\alpha/2,\;df}\) for means when \(\sigma\) is unknown (common in practice).
• One-sided bounds: for a lower bound with confidence \(1-\alpha\), the critical value is \(z_{1-\alpha}\) (or \(t_{1-\alpha,\;df}\)).

Worked example

Try it

Summary

• General CI: estimate \(\pm\) (critical value)\(\times\)(SE).
• Margin of error: \(\text{ME}=(\text{critical value})\times \mathrm{SE}\).

What changes confidence interval width? (and how to plan \(n\))

Key idea

Confidence interval width is controlled by two pieces:

• Critical value: larger confidence level \(\Rightarrow\) larger critical value \(\Rightarrow\) wider interval.
• Standard error: larger \(n\Rightarrow\) smaller \(\mathrm{SE}\Rightarrow\) narrower interval. For many estimators, \(\mathrm{SE}\propto 1/\sqrt{n}\).

A common planning formula comes from \(\text{ME}=z^\*\sigma/\sqrt{n}\) (mean with known \(\sigma\)): \[ n=\left(\frac{z^\*\sigma}{\text{ME}}\right)^2. \] This “square rule” explains why reducing margin of error can require much larger samples.

Worked example: sample size for a target margin of error

Try it

Summary

• Higher confidence \(\Rightarrow\) wider CI (bigger critical value).
• Larger sample size \(\Rightarrow\) narrower CI (smaller SE, often \(1/\sqrt{n}\)).
• For many problems, planning \(n\) comes from \(n=\left(\frac{\text{critical}\times \sigma}{\text{ME}}\right)^2\).

Confidence intervals for a mean: z vs t, and paired t ideas

Key idea

For a population mean \(\mu\), the confidence interval depends on whether \(\sigma\) is known:

• Known \(\sigma\) (z interval): \[ \bar x \pm z_{1-\alpha/2}\frac{\sigma}{\sqrt{n}}. \]
• Unknown \(\sigma\) (t interval): \[ \bar x \pm t_{1-\alpha/2,\;n-1}\frac{s}{\sqrt{n}}. \] Here \(df=n-1\). As \(df\) increases, the t distribution approaches the standard normal.

Paired t intervals treat each pair as one observation by using differences \(d_i\) (for example, “after − before”). Compute \(\bar d\) and \(s_d\), then use: \[ \bar d \pm t_{1-\alpha/2,\;n-1}\frac{s_d}{\sqrt{n}}, \] where \(n\) is the number of pairs.

Worked example (t interval)

Try it

Summary

• Known \(\sigma\): z interval and z test use \(\sigma/\sqrt{n}\).
• Unknown \(\sigma\): t interval and t test use \(s/\sqrt{n}\) with \(df=n-1\).
• Paired t focuses on differences \(d_i\) and uses \(df=n-1\).

Confidence intervals for a proportion \(\,p\) and a variance \(\,\sigma^2\)

Proportion confidence interval (large-sample)

For a one-sample proportion, let \(\hat p=\dfrac{x}{n}\), where \(x\) is the number of successes. When conditions for a normal approximation hold (a common rule of thumb is \(n\hat p\ge 10\) and \(n(1-\hat p)\ge 10\)), an approximate \(100(1-\alpha)\%\) CI is: \[ \hat p \pm z_{1-\alpha/2}\sqrt{\frac{\hat p(1-\hat p)}{n}}. \]

Variance confidence interval (chi-square)

If the population is normally distributed, then \[ \frac{(n-1)s^2}{\sigma^2}\sim \chi^2_{n-1}. \] A \(100(1-\alpha)\%\) CI for \(\sigma^2\) is: \[ \left(\frac{(n-1)s^2}{\chi^2_{1-\alpha/2,\;n-1}},\;\frac{(n-1)s^2}{\chi^2_{\alpha/2,\;n-1}}\right). \]

Worked example (proportion CI setup)

Try it

Summary

• Proportion CI uses \(\hat p=x/n\) and a z critical value (large-sample conditions).
• Variance CI uses chi-square quantiles and assumes a normal population.

Hypothesis testing: \(H_0\), \(H_1\), test statistic, p-value, and the CI connection

The standard testing workflow

• 1. State hypotheses: \(H_0:\theta=\theta_0\) vs H_1:\theta≠\theta_0 (two-sided) or \(H_1:\theta>\theta_0\), \(H_1:\theta<\theta_0\) (one-sided).
• 2. Choose significance level: \(\alpha\) (common values: 0.10, 0.05, 0.01).
• 3. Compute a test statistic: z, t, or \(\chi^2\) depending on the setting.
• 4. Compute a p-value and decide: reject \(H_0\) if p-value \(\le \alpha\); otherwise fail to reject.

Connection to confidence intervals

For a two-sided test at level \(\alpha\), there is a tight link: reject \(H_0:\theta=\theta_0\) if and only if \(\theta_0\) is outside the \(100(1-\alpha)\%\) confidence interval for \(\theta\).

One-sided critical values

A 95% lower confidence bound corresponds to \(\alpha=0.05\) one-sided and uses the critical value \(z_{0.95}\approx 1.645\) (or \(t_{0.95,df}\)). A common lower bound form for a mean with known \(\sigma\) is: \[ L=\bar x - z_{0.95}\frac{\sigma}{\sqrt{n}}. \]

Worked example (z test setup)

Try it

Summary

• Reject \(H_0\) if p-value \(\le \alpha\); otherwise fail to reject.
• Two-sided test at \(\alpha\) matches a \(100(1-\alpha)\%\) CI: \(\theta_0\) outside the CI \(\Rightarrow\) reject \(H_0\).
• One-sided 95% critical z value is \(z_{0.95}\approx 1.645\).

Which test should you use? (z, t, and chi-square)

Quick “which test?” guide

• Mean vs known value, \(\sigma\) known: one-sample z test.
• Mean vs known value, \(\sigma\) unknown: one-sample t test with \(df=n-1\).
• Paired measurements: paired t test on differences \(d_i\).
• Proportion vs known value: one-sample z test for a proportion (large-sample conditions).
• Variance vs known value: chi-square test for one variance (normal population assumption).
• Categorical counts: chi-square goodness-of-fit or chi-square test of independence.

One-sample variance test statistic

To test \(H_0:\sigma^2=\sigma_0^2\) using a normally distributed population: \[ \chi^2=\frac{(n-1)s^2}{\sigma_0^2}\sim \chi^2_{n-1}\quad \text{under } H_0. \]

Worked example (test statistic only)

Try it

Summary

• Known \(\sigma\): one-sample z test for \(\mu\).
• Unknown \(\sigma\): one-sample t test for \(\mu\) (and paired t for differences).
• Variance tests and variance CIs use the chi-square distribution under normality.
• Categorical counts often use chi-square tests (goodness-of-fit or independence).

Type I & Type II errors, power, and why sample size matters

Errors and power in one picture

• Type I error (false positive): reject a true \(H_0\). Probability \(=\alpha\).
• Type II error (false negative): fail to reject a false \(H_0\). Probability \(=\beta\).
• Power: \(1-\beta\). This is the chance you correctly detect a real effect.

How sample size affects inference

• Confidence intervals: larger \(n\Rightarrow\) smaller \(\mathrm{SE}\Rightarrow\) narrower CI (more precision).
• Hypothesis tests: larger \(n\Rightarrow\) smaller \(\mathrm{SE}\Rightarrow\) larger test statistic magnitude (for a fixed effect) and therefore higher power.

Extra note: chi-square tests and survival curves

In categorical data, you’ll often see chi-square goodness-of-fit and chi-square independence tests. In survival analysis, a common test to compare survival curves between two groups is the log-rank test, which is typically reported using a chi-square reference distribution.

Try it

Final recap

• CI blueprint: estimate \(\pm\) (critical value)\(\times\)(SE), with \(\text{ME}=(\text{critical})\times \mathrm{SE}\).
• Mean CIs: z interval uses \(\sigma\); t interval uses \(s\) with \(df=n-1\).
• Proportion CI: \(\hat p \pm z_{1-\alpha/2}\sqrt{\hat p(1-\hat p)/n}\) under large-sample conditions.
• Variance CI: chi-square quantiles, \(\left(\frac{(n-1)s^2}{\chi^2_{1-\alpha/2}},\frac{(n-1)s^2}{\chi^2_{\alpha/2}}\right)\) under normality.
• Testing: set \(H_0\) and \(H_1\), choose \(\alpha\), compute test statistic and p-value, then decide.
• Power: increasing \(n\) tends to increase power by reducing standard error.