What are Type I and Type II errors, and how do they trade off?
Answer
A Type I error (false positive) rejects a true null hypothesis: you announce an effect that is not there. A Type II error (false negative) fails to reject a false null: a real effect goes undetected. Their rates are 
, the significance level you choose up front, and 
, with power equal to 
. For a fixed sample size the two trade off through the decision threshold: making the test stricter lowers but raises , and only more data or lower metric variance reduces both at once.
(1) Type I (): rejecting a true null, the “convicting the innocent” error. It is controlled by fixing before looking at data and by correcting when many hypotheses are tested at once.
(2) Type II (): missing a real effect. Power grows with sample size, true effect size, and lower metric variance.
(3) Costs Set the Threshold: the right balance is a business decision. Gmail blocks over 99.9% of spam while misrouting only about 0.05% of legitimate mail, because a false positive (lost mail) costs far more than a false negative (spam seen in the inbox).
Figure 1: Sampling distribution of the test statistic under 
(left) and 
(right) with one decision threshold. The right tail of is (Type I), the part of left of the threshold is (Type II), and the rest of is power.
Mathematical Formulation:
Where:
is the null hypothesis (no effect) and 
the alternative (a real effect exists).
is the significance level chosen before the test; 
is the false negative rate under a specific true effect.- Power is the probability of detecting the effect when it is real, and it rises with sample size, effect size, and reduced variance.
Figure 2: Moving the threshold at fixed sample size swaps the errors: stricter (left panel) shrinks and inflates ; looser (right panel) does the reverse. Shrinking both at once requires more data or lower variance, which pulls the two curves apart.
| Decision \ Reality | H0 True (No Effect) | H1 True (Real Effect) |
|---|---|---|
| Reject H0 | Type I error, rate | Correct detection, power |
| Fail to Reject H0 | Correct, rate  | Type II error, rate |
Leave a Reply