When do you use a z-test versus a t-test?
Answer
Both tests compare an estimated effect with its standard error to judge whether a mean differs from a reference value. Use a z-test when the reference distribution can be treated as exactly normal: the population standard deviation 
is known, or the sample is large enough that the estimate of is essentially exact. Use a t-test when is estimated from a small sample: the extra estimation uncertainty fattens the tails of the test statistic, and the t-distribution with 
degrees of freedom accounts for it. As 
grows, the t-distribution converges to the normal, so beyond a few dozen observations the two tests give nearly identical p-values.
(1) Same Statistic, Different Reference: both divide an effect estimate by its standard error; they differ only in whether the reference distribution is the standard normal or the heavier-tailed t.
(2) When t Is Mandatory: small samples with unknown , where the t’s heavier tails keep the false positive rate at the promised 
. Use Welch’s variant when the two groups have unequal variances.
(3) In Practice at Scale: conversion metrics in online experiments are proportions with millions of users, so the normal (z) approximation is effectively exact; Booking.com’s experimentation write-ups fall back to the two-sample t-test when comparing means on smaller or skewed samples.
Figure 1: Why the distinction exists: the t-distribution (3 degrees of freedom shown) has visibly heavier tails than the standard normal, so its critical values sit farther out. As degrees of freedom grow, the t curve collapses onto the normal and the two tests coincide.
Mathematical Formulation:
Where:
is the sample mean, 
the hypothesized mean, and 
the sample size.- is the known population standard deviation used by the z-test;
is the sample-estimated standard deviation used by the t-test. - The t statistic follows a t-distribution with
degrees of freedom, which approaches 
as grows.
is the sample mean, 
the hypothesized mean, and 
as | Feature | Z-Test | T-Test |
|---|---|---|
| Spread Parameter | known (or huge) |  estimated from the sample |
| Reference Distribution | Standard normal | t with degrees of freedom |
| Tail Behavior | Thinner tails, tighter critical values | Heavier tails at small |
| Typical Case | Proportion tests at scale (conversion) | Small samples, heavy-tailed metrics |
| Large-n Behavior | Asymptotically valid for proportions via the CLT | Coincides with the z-test |
Leave a Reply