Eighty-five percent of the cabs are green
In 1977, Amos Tversky and Daniel Kahneman presented experimental subjects with a deceptively simple problem. A taxicab was involved in a hit-and-run accident at night. Two cab companies operate in the city: the Green company, which runs 85% of the cabs, and the Blue company, which runs 15%. A witness identified the cab as Blue, and tests showed the witness could correctly identify cab colors 80% of the time under similar conditions (Tversky & Kahneman, 1977).
The question: what is the probability that the cab involved in the accident was actually Blue?
Most people answer 80% — the reliability of the witness. It feels obvious. The witness said Blue. The witness is right 80% of the time. So there is an 80% chance the cab was Blue.
The correct answer is 41%.
The math is straightforward. Out of 100 accidents in this city, 85 would involve Green cabs and 15 would involve Blue cabs. The witness would correctly identify 12 of the 15 Blue cabs as Blue (80% of 15). But the witness would also incorrectly identify 17 of the 85 Green cabs as Blue (20% of 85). So when the witness says "Blue," there are 29 total cases where that happens — 12 correct identifications and 17 incorrect ones. The probability the cab was actually Blue is 12 out of 29, which is 41%.
The witness testimony is not worthless. It moved the probability from the base rate of 15% up to 41%. But it did not get anywhere close to the 80% that most people intuitively believe. The base rate — the overwhelming preponderance of Green cabs — anchors the answer far more than the individual testimony does.
This is the base rate fallacy, also called base rate neglect. It is one of the most robustly demonstrated errors in human judgment. And it is not a laboratory curiosity. It is a perceptual distortion that operates every time you encounter a compelling story and forget to ask: but how common is this actually?
The cognitive machinery behind the error
Base rate neglect is not random. It is produced by a specific cognitive mechanism that Kahneman and Tversky identified as the representativeness heuristic — a mental shortcut in which you judge the probability of an event by how well it matches a prototype rather than by its actual statistical frequency (Kahneman & Tversky, 1973).
Here is how the mechanism works. Your brain maintains prototypes — mental summaries of what a category member "tends to be like." When a new piece of information matches the prototype closely, your mind substitutes a similarity judgment for a probability judgment. The witness said Blue. Blue cabs look Blue. The witness's testimony matches the "Blue cab" prototype perfectly. So your brain concludes: probably Blue. The statistical context — how many Blue cabs exist in the population — gets discarded because it does not enter the similarity computation at all.
This substitution is fast, automatic, and feels subjectively confident. That is what makes it dangerous. You do not experience base rate neglect as uncertainty. You experience it as clarity. The narrative — the witness saw Blue, therefore it was Blue — produces a feeling of certainty that the statistics cannot match. A number like 41% feels weak and equivocal. A witness saying "I saw Blue" feels definitive. Your perceptual system treats narrative evidence as higher quality than statistical evidence, even when the statistics are objectively more informative.
This is not a flaw in uneducated thinking. It occurs across all populations: children and adults, laypeople and experts, across cultures and experimental contexts. In one of the most cited studies in medical decision-making, Casscells, Schoenberger, and Graboys (1978) gave physicians at Harvard Medical School a base rate problem: a disease has a prevalence of 1 in 1,000, and a test has a 5% false positive rate with no false negatives. If a patient tests positive, what is the probability they have the disease? The most common answer from these physicians — doctors at one of the world's most prestigious medical schools — was 95%. The correct answer is approximately 2%. The physicians anchored on the test's accuracy and ignored the base rate of the disease almost entirely.
The recency bias you studied in L-0150 distorts your perception of what is normal by overweighting recent events. Base rate neglect distorts your perception through a different channel: by overweighting vivid, narrative-shaped evidence at the expense of statistical reality. Where recency bias warps your time horizon, base rate neglect warps your probability estimates. Both are perceptual miscalibrations. Both operate below conscious awareness. And both require deliberate correction.
Where base rate neglect costs the most
The consequences of ignoring base rates are not abstract. They cascade through every domain where humans make probability judgments under narrative pressure.
Medical diagnosis. When a disease is rare, even a highly accurate test produces more false positives than true positives. Consider breast cancer screening. The prevalence of breast cancer among women receiving mammograms is approximately 0.8%. Mammograms correctly detect cancer in about 90% of women who have it. But they also produce false positives in about 7% of women who do not. If you get a positive mammogram result, the probability you actually have breast cancer is approximately 9% — not 90% (Gigerenzer et al., 2007). Most women and many doctors dramatically overestimate this probability because the narrative — "the test detected cancer" — overrides the base rate of how rare the disease actually is in the screening population. This leads to unnecessary biopsies, anxiety, and in some cases, overtreatment of conditions that would never have progressed.
Criminal justice. Forensic evidence shares the same structure. A DNA match with a one-in-a-million false positive rate sounds definitive. But if the suspect pool is a city of ten million people, you would expect ten false matches. The individual match is consistent with guilt, but the base rate of false matches in a large population means the evidence alone is far less conclusive than it sounds. Jurors, like doctors, anchor on the test accuracy and ignore the base rate.
Business and investing. Entrepreneurs hear stories of startups that grew from a garage to a billion-dollar valuation. The narrative is vivid, detailed, and emotionally compelling. It is also survivorship bias overlaid on base rate neglect. The base rate for venture-backed startups reaching a billion-dollar valuation is approximately 1%. The base rate for startups surviving five years is approximately 50%. But the narrative infrastructure — books, podcasts, conferences — is built entirely around the 1%, creating a systematic perceptual distortion in which success looks like the default rather than the extreme exception.
Personal risk assessment. After a highly publicized shark attack, beach attendance drops. After a plane crash makes international news, people switch to driving — even though driving is statistically far more dangerous per mile traveled. The narrative of a specific, vivid death overrides the base rate probability that the drive to the airport is the most dangerous part of the trip. As Nassim Nicholas Taleb observed in The Black Swan, the narrative fallacy — our compulsion to fit events into coherent stories — makes us "prone to the error of ignoring silent evidence," the vast statistical backdrop of non-events that defines the true probability of any outcome (Taleb, 2007).
In every one of these cases, the structure is the same: a vivid individual case captures attention, the base rate sits quietly in the background, and the brain routes the judgment through the narrative system rather than the statistical system. The result is a systematic overestimation of rare events that produce good stories and a systematic underestimation of common events that do not.
Bayesian reasoning: the antidote your brain resists
The formal correction for base rate neglect is Bayesian reasoning — a mathematical framework for updating beliefs in light of new evidence, named after the Reverend Thomas Bayes. Bayes' theorem provides the exact formula for combining base rates (your prior probability) with new evidence (the likelihood) to produce an updated estimate (the posterior probability).
The structure is simple: start with the base rate. Then adjust — but only proportionally to the strength of the evidence. The witness said Blue. That is evidence. It shifts the probability from 15% toward Blue. But it does not shift it to 80%, because the base rate is so strongly against Blue that even a reliable witness can only move the needle to 41%. The base rate is the anchor. The evidence is the adjustment. The anchor always matters more than any single piece of evidence, unless the evidence is extraordinarily strong.
This is the principle that separates calibrated thinkers from uncalibrated ones. When you hear a story — a friend who got food poisoning at a restaurant, a colleague who was laid off without warning, a news report about a new disease — the calibrated response is not to dismiss the story. The story is data. But the calibrated response is to ask: what is the base rate? How common is food poisoning at restaurants? What is the actual layoff rate in my industry? What is the prevalence of this disease in my demographic? The story adjusts your estimate. The base rate determines where that estimate starts.
In Bayesian language, the base rate is your prior — your best estimate before you see any new evidence. The evidence updates the prior to produce a posterior. The mistake of base rate neglect is treating every new piece of evidence as if your prior were 50/50 — as if you had no background information at all. When a doctor hears that a patient tested positive for a rare disease and immediately thinks "the patient probably has the disease," the doctor is implicitly setting the prior to 50/50 and letting the test result dominate. But the prior is not 50/50. The prior is 0.1%, or 1%, or whatever the actual prevalence rate is. And starting from that prior, even a positive test result often means the patient probably does not have the disease.
The reason your brain resists this is not stupidity. It is architecture. Your cognitive system evolved in an environment where narrative evidence was usually the best evidence available. If a tribemember reported seeing a lion at the watering hole, the base rate of lions at watering holes was irrelevant — the cost of ignoring the report was death. In small-group, high-stakes, low-data environments, treating testimony as near-certain was adaptive. You are the descendant of people who took stories seriously. The problem is that you now live in an environment with abundant statistical data, and your perceptual system still defaults to treating stories as the most reliable source of truth.
How to make base rates intuitive: the natural frequency revolution
If base rates are so important and humans are so bad at using them, is the situation hopeless? No. Research by Gerd Gigerenzer and Ulrich Hoffrage demonstrated that the problem is partly one of representation. When base rate information is presented as conditional probabilities — the standard format in textbooks and medical literature — performance is terrible. Only about 4% of people can solve Bayesian inference problems correctly in probability format (Gigerenzer & Hoffrage, 1995).
But when the same information is presented as natural frequencies — raw counts that preserve base rate information — performance jumps dramatically. In the same problems, 24% of people solve them correctly, an odds ratio of 7.1. And with training, the improvement is even more dramatic. When twenty-four experienced physicians were given the mammography problem in probability format, only one could find the correct answer. When the same problem was rephrased in natural frequencies, sixteen of twenty-four got it right (Gigerenzer et al., 2007).
Here is the difference. Probability format: "The probability of breast cancer is 0.8%. The sensitivity of the mammogram is 90%. The false positive rate is 7%. What is the probability that a woman with a positive mammogram has cancer?" This is hard for the human brain.
Natural frequency format: "Out of every 1,000 women screened, 8 have breast cancer. Of those 8, the mammogram detects 7. Of the 992 women without cancer, the mammogram falsely flags 69. So out of 76 women with positive mammograms, 7 actually have cancer." This is dramatically easier. You can almost see the answer: 7 out of 76, or about 9%.
Natural frequencies work because they preserve the base rate inside the numbers themselves. You do not need to remember to incorporate the base rate as a separate step — it is already built into the population counts. Gigerenzer calls this "ecological rationality": the human mind is not inherently bad at probabilistic reasoning. It is bad at processing information in formats that did not exist in the environment where it evolved. Give the mind information in a format that matches its architecture — concrete counts of events in a population — and its performance improves by an order of magnitude.
The practical implication for your perceptual calibration: whenever you encounter a probability claim, translate it into natural frequencies. Do not ask "what is the probability?" Ask "out of how many people does this happen to how many?" The translation forces you to confront the base rate directly, because you cannot construct the frequency count without it.
AI and the Third Brain: base rate correction at scale
Artificial intelligence does not suffer from base rate neglect. This is one of its most important cognitive advantages — and one of the most valuable things it can do for your thinking.
Machine learning systems are, at their mathematical core, Bayesian updating engines. Every classifier, every prediction model, every recommendation system operates by combining prior distributions (base rates) with observed evidence (data) to produce posterior estimates. When a spam filter evaluates an incoming email, it does not just look at whether the email contains the word "free." It weights that evidence against the base rate of spam in the population of emails you receive. When a medical AI evaluates a diagnostic image, it incorporates the prevalence of the condition in the relevant population — the very step that human physicians routinely skip (Fortuin, 2022).
This means AI can serve as a base rate correction layer in your thinking. When you encounter a vivid story that makes you feel like something is common or dangerous or likely, you can ask an AI system: what is the actual base rate? How often does this happen in the relevant population? What does the statistical evidence say, independent of this particular narrative? The AI does not have an amygdala. It does not experience narratives as more compelling than statistics. It can give you the dry, un-vivid, emotionally unrewarding number that your perceptual system needs to hear.
But the partnership is not one-directional. AI systems have their own base rate problems. When training data is unrepresentative — when the base rates in the training set do not match the base rates in the real world — AI systems inherit and amplify those distortions. A model trained on data where 90% of CEO images are of men will "learn" a base rate that does not reflect the actual (still skewed, but less so) distribution. A model trained on medical data from one demographic will produce calibrated predictions for that demographic and miscalibrated predictions for everyone else.
Your role in the human-AI partnership is to bring the contextual judgment that asks: does the base rate this system is using match the actual population I care about? Are the priors appropriate for my specific situation? This is where the human capacity for domain knowledge and contextual reasoning complements AI's capacity for statistical computation. You catch the base rate errors that AI's training data encodes. AI catches the base rate errors that your narrative bias produces. Together, your probability estimates are more calibrated than either could achieve alone.
The base rate protocol
Here is the deliberate practice that converts this knowledge into a perceptual skill.
Step 1: Catch the narrative. Every time you encounter a vivid individual case that makes you feel like you know the probability of something — "startups fail," "this treatment works," "the market is going to crash" — notice the feeling. That feeling of certainty triggered by a story is the signal that base rate neglect may be operating.
Step 2: Ask the base rate question. Before forming any judgment, ask: what is the actual frequency of this event in the relevant population? Not "is this story true?" — the story may be perfectly true. The question is: how representative is this story of the overall pattern?
Step 3: Translate to natural frequencies. Convert the problem from probabilities to counts. Not "what is the probability of X?" but "out of 1,000 people in my situation, how many experience X?" This translation automatically forces the base rate into your calculation.
Step 4: Update proportionally. Let the new evidence move your estimate, but from the base rate starting point, not from 50/50. If the base rate of startup success is 10%, and you have evidence that your startup has a strong team and product-market fit, your updated estimate might be 25% or 30%. Not 90%. The evidence adjusts the base rate. It does not replace it.
Step 5: Record and review. Log your base rate estimates alongside actual outcomes. Over time, this creates a calibration record — an empirical measure of whether your probability judgments are accurate. This record is the raw material for the calibration work that continues through the rest of Phase 8.
The bridge to domain-specific calibration
Base rate neglect is universal — it operates in every domain where narratives compete with statistics. But the specific base rates that matter, and the specific narratives that distort them, vary dramatically by domain. The base rates that matter in medicine are different from the base rates that matter in investing, which are different from the base rates that matter in career decisions, which are different from the base rates that matter in personal relationships.
This means that correcting for base rate neglect is not a one-time fix. It is a domain-specific calibration project. You need to learn the relevant base rates for each domain where you make important decisions, and you need to practice overriding the domain-specific narratives that distort your estimates in each area.
That is exactly what L-0152 addresses. Being well-calibrated in one area does not transfer automatically to others. The physician who correctly incorporates base rates in cancer screening may completely ignore them when evaluating investment opportunities. The investor who intuitively adjusts for base rates in portfolio construction may panic when a vivid health scare overrides the epidemiological statistics. Calibration is domain-specific because the base rates are domain-specific, the narratives are domain-specific, and the emotional triggers are domain-specific.
You now understand why base rates matter more than narratives. The next lesson teaches you why that understanding must be rebuilt, deliberately, in every domain where it counts.
Sources:
- Tversky, A., & Kahneman, D. (1977). "Causal Schemas in Judgments Under Uncertainty." In M. Fishbein (Ed.), Progress in Social Psychology. Hillsdale, NJ: Erlbaum.
- Kahneman, D., & Tversky, A. (1973). "On the Psychology of Prediction." Psychological Review, 80(4), 237-251.
- Casscells, W., Schoenberger, A., & Graboys, T. B. (1978). "Interpretation by Physicians of Clinical Laboratory Results." New England Journal of Medicine, 299(18), 999-1001.
- Gigerenzer, G., & Hoffrage, U. (1995). "How to Improve Bayesian Reasoning Without Instruction: Frequency Formats." Psychological Review, 102(4), 684-704.
- Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz, L. M., & Woloshin, S. (2007). "Helping Doctors and Patients Make Sense of Health Statistics." Psychological Science in the Public Interest, 8(2), 53-96.
- Taleb, N. N. (2007). The Black Swan: The Impact of the Highly Improbable. New York: Random House.
- Bar-Hillel, M. (1980). "The Base-Rate Fallacy in Probability Judgments." Acta Psychologica, 44(3), 211-233.
- Fortuin, V. (2022). "Priors in Bayesian Deep Learning: A Review." International Statistical Review, 90(3), 563-591.