A study of kidney cancer rates across US counties found that the counties with the lowest cancer rates were mostly rural, sparsely populated, and in the Midwest. A plausible story: fresh air, clean water, healthy farming lifestyle. But the counties with the highest cancer rates were also rural, sparsely populated, and in the Midwest. The explanation has nothing to do with lifestyle. It's statistics.
The Framework
The "law of small numbers" — Kahneman's ironic label — is the mistaken belief that small samples are representative of the population. They aren't. Small samples produce extreme results through chance alone. A county with 100 residents might have zero cancer cases one year and three the next — swinging from "lowest rate" to "highest rate" without any change in the underlying risk. A large county with 10 million residents will always be near the national average, because the law of large numbers ensures that extreme fluctuations cancel out.
System 1 cannot process this. It sees a pattern ("rural counties have low cancer rates") and instantly constructs a causal story ("healthy rural lifestyle"). The possibility that the pattern is random noise — a statistical artifact of small sample size — never enters the picture. This is WYSIATI applied to data: the pattern is vivid, the sample size is invisible.
Where It Comes From
Chapter 10 of Thinking, Fast and Slow opens Part II by demonstrating that even professional researchers fall for the law of small numbers. Kahneman and Tversky surveyed mathematical psychologists and found that they dramatically overestimated the reliability of results from small samples — believing that a study with 20 participants would replicate findings from a study with 200. The "hot hand" in basketball (the belief that a player who has made several shots in a row is more likely to make the next one) is another manifestation: the apparent streaks are consistent with random variation.
> "We are far too willing to reject the belief that much of what we see in life is random." — Thinking, Fast and Slow, Ch 10
Cross-Library Connections
Hormozi's emphasis in $100M Leads on tracking metrics over sufficient time periods (30+ days for ad performance) is a practical correction for the law of small numbers. A campaign that performs brilliantly for 3 days may simply be small-sample noise. Scaling based on 3 days of data is the business equivalent of concluding that rural counties are healthier.
The Implementation Playbook
A/B Testing: Never draw conclusions from small sample sizes. A variant that outperforms by 50% over 100 visitors may be pure noise. Require statistical significance before acting. The law of small numbers guarantees that early results are unreliable — the signal needs time to emerge from the noise.
Hiring: A candidate who performed brilliantly in a single interview may be regressing from a peak. A candidate who stumbled may have had an off day. One data point is a very small sample. Multiple interviews, structured evaluations, and work samples increase the sample size and reduce the noise.
Performance Evaluation: A salesperson's results in a single month are a small sample. Evaluating and rewarding based on monthly results rewards luck, not skill. Quarterly or annual evaluations increase the sample size enough for the signal (actual performance) to emerge from the noise (random variation).
Market Research: Customer feedback from 10 respondents is noise, not signal. The minimum viable sample for any business decision depends on the effect size you're trying to detect, but for most purposes, you need hundreds of responses before patterns become reliable. Five-star reviews from 5 customers don't prove product quality — they prove that you asked 5 friendly customers.
Key Takeaway
The law of small numbers explains why we find patterns in random data, why early results mislead, and why anecdotes outperform statistics in human judgment. The correction is always the same: ask "how large is the sample?" before trusting any pattern. If the answer is "small," the pattern is probably noise. System 1 will protest — the pattern feels real, the story is compelling. But feelings of certainty and sample sizes of five don't mix.
Continue Exploring
[[Regression to the Mean]] — The related phenomenon: extreme results from small samples will regress
[[Representativeness Heuristic]] — Why System 1 expects small samples to be representative
[[WYSIATI]] — The mechanism: the pattern is vivid and available; the sample size is invisible
📚 From Thinking, Fast and Slow by Daniel Kahneman — Get the book