The plan had a deadline. The plan also had a planner who had never before accurately forecasted one of their own projects.
The tendency to underestimate the variability that can occur in small samples, leading to overconfident judgments based on limited data.
A Scene Worth Recognising
A product team reviews the features that made their last three launches successful and sets out to replicate every pattern they find. The review is thorough — but it only covers the launches that went well. The four features that appeared in failed launches and were quietly removed do not make it into the analysis. What drives the next roadmap is shaped by Insensitivity to sample size.
What it means and how it works
The bias stems from the representativeness heuristic: individuals judge the likelihood of an event by how closely it matches their mental prototype or expectation, neglecting the influence of sample size on sampling variability. When a small sample appears representative of a population, its size is ignored, producing overconfident inferences.
Insensitivity to sample size (also known as the 'law of small numbers' bias) describes a cognitive shortcut where people treat the outcomes of a small number of observations as if they were as reliable as those from a large sample. Consequently, they expect less fluctuation in small samples than statistics would predict, and they draw strong conclusions from limited evidence.
Why it matters
In fields such as medicine, finance, hiring, and everyday decision‑making, relying on small samples can lead to erroneous conclusions, suboptimal choices, and overestimation of one's knowledge. Recognizing the bias helps improve statistical reasoning and reduces the risk of overreacting to anecdotal or limited evidence.
The verified research on this pattern supports the following:
- Tversky and Kahneman (1974) demonstrated that participants' probability judgments were insensitive to sample size, treating small samples as if they were as informative as large ones.
- The bias arises from the representativeness heuristic, where individuals judge the likelihood of an event by how much it resembles their mental model, ignoring sample size effects.
Common misunderstandings
Misunderstanding 1: People often confuse this bias with a belief that small samples are inherently 'more accurate' or that large samples are unnecessary. In reality, the bias is about ignoring the expected increase in variability with smaller N, not about claiming small samples are better.
Sources
- Niroula, Rishab. REV 2.0 Topic Catalog. Hello to Halo.
- Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011.
- Taleb, Nassim Nicholas. The Black Swan: The Impact of the Highly Improbable. Random House, 2007.
- Tversky, Amos, and Daniel Kahneman. "Judgment Under Uncertainty: Heuristics and Biases." Science 185, no. 4157 (1974): 1124–1131.
The next time this pattern surfaces, the move is not to fight it — it is to notice it. Naming Insensitivity to sample size creates a moment of pause before the decision. That moment is usually enough.