Models of Misalignment, Information, and Coordination

🧠 Free Recall ❓ Quiz 🔍 Cues 🔮 Simulation 🥷 ShadowBox 📐 Encode Cards

Why Things Predictably Go Wrong
A plain-language field guide to problems worth recognizing
The principal–agent problem — where you rely on someone to act for you but can't fully see what they're doing or be sure they want the same things you do — is just one member of a big family of "things that reliably go wrong." This guide sorts that family by the reason each problem happens.
Why sort by reason? Because the useful skill isn't memorizing names. It's looking at a messy real situation and thinking "oh, this is a trust-and-information problem" or "this is a timing problem" — which instantly tells you what's likely to break and what kind of fix might work.
For each theme you get three things:
What causes this whole group — the root situation that reliably creates these problems. Spot the root cause in real life, and the problems tend to follow.
How to spot it — the tell-tale signs in a real situation.
The specific problems — the named examples, each explained from scratch with a concrete case.
A short summary checklist and a note on evolution close it out.

1. When One Side Knows More Than the Other
What causes this group. Good decisions need good information — but often one side of a deal knows something the other side can't easily check. Because you can't verify what they know or watch what they do, they can quietly act in their own interest. And because you know they might, you get cautious, which can make the whole deal fall apart before it even starts.
Two questions sort this family:
Is the hidden thing already true, or something they'll do later? The true condition of a used car is known in advance. Whether a driver will actually be careful happens later. These create different problems.
Who tries to close the gap — the person who knows more (by proving it), or the person who knows less (by testing)?
How to spot it. You find yourself wanting a guarantee, an inspection, a reference, a warranty, or a background check. One side always seems to "know more." Honest sellers struggle to prove they're honest. As you change the price, the kind of people who show up gets worse, not better.
The specific problems:
Principal–agent problem. You rely on someone to act for you — an employee, a contractor, a financial advisor — but you can't watch everything they do, and they don't want exactly what you want. This is the parent of the whole group.
Moral hazard. When you're shielded from the downside of a risk, you take on more of it. Someone with excellent insurance drives a little less carefully; a bank that expects a government rescue makes riskier bets. The key feature: the risky action is hidden and happens after the deal is signed.
Adverse selection (the used-car trap). Picture a used-car lot where buyers can't tell good cars from bad ones. Since they can't tell, they'll only pay a middling, average price. But that price is too low for owners of genuinely good cars, so those owners stop selling — leaving mostly bad cars ("lemons") on the lot. Now average quality drops, so buyers offer even less, and the good cars vanish entirely. A market can shrink or collapse purely because of an information gap. (The economist George Akerlof first laid this out.) The same logic haunts insurance and lending: the people most eager to buy insurance are often the riskiest.
Signaling. The person who knows more spends effort or money to prove it — in a way that would be too costly for a faker to copy. Earning a hard degree can signal you're capable and determined, even if the classes themselves didn't teach the job.
Screening. The person who knows less sets up a test or a menu that makes the other side reveal itself. Insurers offer plans with different deductibles so cautious and risky customers sort themselves into different plans.
Costly signals. A signal is believable precisely because it's expensive to fake. A peacock's enormous tail proves it's healthy — a sick bird couldn't grow one. A company spending millions on a Super Bowl ad signals it expects to stick around long enough to profit.
Cheap talk. What happens when words are free and can't be checked? Sometimes advice still carries real information, sometimes it's just noise — it depends on whether the speaker's interests line up with yours. ("Trust me" from a commissioned salesperson carries less than the same words from a friend.)
Persuasion by choosing what to reveal. Even someone who never actually lies can steer your beliefs by carefully choosing what to test and what to show you. Honest spin.
Unraveling. If the best-quality people prove how good they are, the next-best have to prove it too — or their silence makes them look bad. This continues down the line until everyone is forced to disclose. It's why "no comment" can be as revealing as a confession.
Winner's curse. In an auction where nobody truly knows the value, the winner is usually whoever overestimated it the most — so winning is often bad news. It shows up in company takeovers, bidding for oil-drilling rights, and even hiring the candidate every other company passed on.
Efficiency wages (a fix). Pay people noticeably more than the going rate, so they're genuinely afraid to lose the job. That fear makes them work hard even when you can't watch them.
2. When You Reward the Wrong Thing
What causes this group. The thing you truly care about is often hard to measure, so you measure a convenient stand-in and reward that instead. Two things then go wrong. First, people chase the number rather than the real goal, and any gap between the two is where things break. Second, the moment a measurement becomes a target, people start gaming it — so it stops being a good measurement.
This is different from Theme 1. Here the information can be perfectly visible. The problem is that what you're able to pay for isn't the same as what you actually want.
How to spot it. You're rewarding a number. Someone could move that number without improving the real thing. The people being measured have wiggle room, or know things you don't. The metric worked fine right up until you started making decisions based on it.
The specific problems:
Goodhart's Law. "When a measure becomes a target, it stops being a good measure." The headline idea for this whole theme.
Campbell's Law. The same idea for social measures: the more a number is used to make big decisions, the more people distort it. Judge schools only on test scores and teachers start teaching to the test instead of teaching.
The "rules change behavior" trap. You can't assume people will keep behaving the way they did in the past once you introduce a new rule, because they'll adjust to the new rule. The old patterns existed only because of the old rules. (Economists call this the Lucas critique.)
Rewarding only the measurable part. When a job has parts you can measure and parts you can't, paying only for the measurable parts starves the rest. Pay a teacher purely on test scores and you may lose the harder-to-measure things — curiosity, kindness, judgment.
Backfiring incentives (the cobra effect). A reward meant to fix a problem makes it worse. A colonial government once paid a bounty for dead cobras to reduce the snake population; people responded by breeding cobras to cash in, and the problem grew.
Ratchet effect. Do well this year and next year's target gets harder — so people deliberately hold back to protect themselves. Common with sales quotas and department budgets.
Rent-seeking. Spending effort to grab a bigger slice of the existing pie (lobbying, lawsuits) instead of making the pie bigger.
Reward-the-rank contests (a designed fix). When you can't measure exact output, reward rank instead — big prizes for the top spots push everyone to compete. It's part of why a CEO earns so much more than the deputy: the gap itself is the motivator for everyone below.
3. When Private Choices Create Shared Costs
What causes this group. Someone makes a choice on their own, but the costs or benefits land on everyone. Because the chooser doesn't feel the full cost, everyone acting "sensibly" for themselves adds up to a worse result for all. Costs and benefits that spill onto others like this are called externalities.
There are two flavors, two sides of the same coin: overusing a shared resource, and underfunding a shared benefit. One useful insight: if someone clearly owned the shared thing, and people could easily negotiate, they'd usually work it out — so these problems live exactly where nobody owns it or bargaining is too hard.
How to spot it. "If everyone does what's best for them, we all lose." Costs fall on people who had no say. Nobody owns the thing being used up. You can't stop people who didn't pay from enjoying the benefit. Each person's individual share feels too small to matter.
The specific problems:
Prisoner's dilemma. The core two-person version. Two people would both be better off cooperating, but each is individually tempted to cheat — so both cheat and both end up worse. (The classic story: two arrested partners are each offered a lighter sentence for informing on the other, and both inform.) Cooperation falls apart without trust, enforcement, or a reason to expect to deal with each other again.
Tragedy of the commons. A shared resource — a fishing ground, a pasture, clean air — gets used up because each person gets the full benefit of taking more, while the cost is spread thinly across everyone.
Free-rider problem. A shared benefit — public radio, a clean neighborhood — gets underfunded because people can enjoy it without paying. The mirror image of the commons.
Public goods. Things everyone can use and no one can easily be shut out of. Hard to fund for exactly that reason.
Externalities and the bargaining fix. The general idea, plus a key insight (from economist Ronald Coase): if ownership is clear and talking is cheap, people can often negotiate a good outcome themselves. This thinking underlies pollution-permit trading.
Concentrated benefits, spread-out costs. A small group that gains a lot fights hard for a policy, while the large group each losing a little doesn't bother to organize — so policy tilts toward the small, motivated group. A core engine of lobbying.
Death by small decisions. A series of individually reasonable choices adds up to something nobody wanted. A town can lose its train service one canceled route at a time, until it's gone.
Braess's paradox. Adding a new road can make everyone's commute worse, because drivers all pile onto it without coordinating. More options isn't always better. (Also appears in Theme 8.)
4. When Everyone's Waiting to See What Others Do
What causes this group. Sometimes everyone actually wants the same outcome — the trouble is just lining up. What you should do depends on what everyone else does, which depends on what they think you'll do, and so on. This works out only when there's an obvious answer everyone can point to, and everyone knows that everyone else sees it too.
There's a related twist: when people copy what others do instead of trusting their own information, good information stops spreading, and a crowd can lock onto a mistake. And when people hide their real opinions to fit in, a situation can look rock-solid right up until it suddenly flips.
This is the opposite of Theme 3. There, people's interests clash. Here they agree — the only problem is matching up.
How to spot it. More than one outcome could stick, and everyone would happily switch if they believed others would. People are waiting on each other. Things suddenly tip or collapse. People copy the crowd instead of thinking for themselves. "Everyone secretly thinks X but nobody says it." An arbitrary convention — like which side of the road to drive on — matters enormously once it's set.
The specific problems:
Coordination games (the stag hunt). Everyone's best off cooperating, but each person, unsure the others will, plays it safe — so everyone lands in the worse-but-safer outcome. (Two hunters can catch a big deer together or a small rabbit alone; the fear that the other will quit pushes both toward the rabbit.)
Focal points. With no way to communicate, people gravitate toward the "obvious" choice. If you had to meet a stranger in New York with no plan, a surprising number of people would guess Grand Central at noon. Shared obviousness, not logic, solves it.
Common knowledge. The gap between everyone knowing something and everyone knowing that everyone else knows it too. A whole crowd might privately see that the emperor has no clothes, but no one dares act until it's out in the open that they all see it.
The beauty-contest problem. You don't pick what you think is best — you pick what you think everyone else will pick (and they're all doing the same). This drives stock bubbles and hype: people buy what they expect others to buy next.
Herding (information cascades). People ignore their own hunch and copy earlier choosers, so a whole crowd can end up following a couple of early — possibly wrong — decisions. Fashion fads and bank-run panics work this way. (Also Theme 8.)
Hidden true beliefs. When speaking up is socially risky, people publicly go along while privately disagreeing. An unpopular status quo can look totally stable until a small crack lets everyone admit what they really thought — and then it collapses fast.
Tipping points (critical mass). People join a protest, a trend, or a strike only once enough others already have. Tiny differences in how many "early joiners" there are can decide whether nothing happens or everything does.
Segregation from mild preferences. Even if people only mildly prefer not to be a tiny minority on their block, those small preferences can snowball into sharply divided neighborhoods that nobody actually intended. (Thomas Schelling showed this with a simple checkerboard model.)
The Abilene paradox. A group does something none of them wanted, because each person goes along assuming the others want it. (A whole family drives to Abilene in brutal heat because each one thinks the others are keen.)
The Overton window. The range of ideas you can say in public without being dismissed as crazy. It shifts over time, and much political effort goes into moving that range rather than winning the current debate.
5. When You Can't Trust Your Future Self (or Theirs)
What causes this group. The plan that looks best today isn't the one you'll actually want to follow through on later — and there's no way to force yourself, or the other side, to stick to it. This comes in two forms of the same shape: between people ("once I've built a factory only you can use, you'll squeeze me on price") and within one person ("I plan to save, but when payday comes I splurge"). Every fix works the same way: by removing your own future options so a promise becomes believable.
How to spot it. A promise won't hold up once the moment actually arrives. Someone has to commit first, before the other side is locked in. "I'll want to back out of this later." Temptation kicks in exactly once the other party can no longer walk away. You need some way to tie your own hands.
The specific problems:
Broken-promise problem (time inconsistency). A central bank promises not to print money and cause inflation. But once everyone believes it, printing a little looks tempting — so the promise isn't credible in the first place. This is a big reason economies often prefer fixed rules over case-by-case judgment calls.
Hold-up problem. After you make an investment that only pays off in one specific relationship — a parts supplier building a factory useful to just one carmaker — the other side can renegotiate to grab most of the value, since you have nowhere else to go. A major reason companies buy their suppliers rather than just contracting with them.
Incomplete contracts. You can't write a contract covering every possible future situation. So what really matters is who owns what — because the owner gets to decide all the situations you didn't spell out.
Burning bridges (commitment devices). Deliberately destroying your own escape routes to make yourself credible or disciplined. A general burns his ships so his army can't retreat; a person installs an app that locks them out of distractions.
Real vs. empty threats. A threat only works if you'd actually want to carry it out when the moment comes. "I'll fire the whole team" isn't believable if doing so would hurt you more than them.
Brinkmanship. Deliberately raising shared danger to make the other side blink first. It works precisely because it's genuinely risky — like two rival countries in a nuclear standoff.
The reputation fix (repeated dealings). If two people expect to deal with each other again and again, the fear of being punished next time can make them cooperate now. Repetition and reputation are how these problems actually get solved in real life.
Present bias. The within-yourself version: we're patient about the far-off future but impatient right now, so we procrastinate and undersave. It's why people pay for commitment devices. (Also Theme 9.)
6. When a Group Doesn't Act Like One Sensible Person
What causes this group. When you combine many individuals into a group, the way you combine them creates results none of the individuals would have on their own. A group isn't just a bigger person. How you count votes, promote people, or route communication shapes the outcome — sometimes in strange or even unavoidable ways.
How to spot it. "What the group wants" is unclear, contradicts itself, or changes depending on the procedure used. A team of good people produces a bad result. The shape of the organization is visibly leaking into what it makes. Small changes to the rules change the outcome.
The specific problems:
No perfect voting system (Arrow's theorem). Any method of turning individual rankings into a single group ranking has to break at least one basic fairness rule. It's a proven mathematical limit, not a fixable flaw.
Voting can go in circles (Condorcet paradox). A group can prefer A over B, B over C, and yet C over A — a loop. So "what the majority wants" sometimes isn't even a well-defined thing.
Race to the middle (median voter). When choices line up on a single spectrum, competitors tend to drift toward the middle voter's preference. It's why rival parties — and rival products — often end up looking oddly similar.
Power drifts to the few (iron law of oligarchy). Even organizations founded to be democratic tend to drift toward control by a small inner group.
The Peter principle. People get promoted for doing their current job well — until they land in a role they're bad at, and then stop rising. So organizations tend to fill up with people stuck at their level of incompetence.
Parkinson's law. Work expands to fill the time you give it. (A companion version: spending rises to use up whatever budget is available.)
Conway's law. Organizations build products that mirror their own communication structure. If two teams barely talk to each other, the two pieces of software they build will connect badly.
Bikeshedding. A group spends five minutes on a huge, complex decision (say, a nuclear plant) and an hour arguing about something trivial (the color of the bike shed) — because everyone feels able to weigh in on the easy thing, and few feel qualified on the hard one.
Regulatory capture. An agency created to keep an industry in check gradually ends up serving that industry, because the industry supplies the experts, the staff, and the future jobs. (Also Theme 2.)
Allies of convenience. A rule survives because two very different groups both back it: those who want it on principle, and those who quietly profit from it. (Old Sunday alcohol bans were supported by both preachers who opposed drinking and bootleggers who made money selling on the one day stores were closed.)
7. When Your Data Is Secretly Filtered
What causes this group. The examples you get to see aren't a fair sample — they've been filtered by some process that's tied to the very thing you're studying. So even with honest people and the best intentions, you reach wrong conclusions, because you forgot to ask how these particular examples ended up in front of you.
How to spot it. Ask "where did this sample come from, and who's missing?" You only see the winners or the survivors. You picked your data based on the outcome you're trying to explain. A pattern flips when you split or combine groups. Extreme cases seem to "improve" on their own afterward. A scary-sounding result about something rare feels more certain than it should.
The specific problems:
Survivorship bias. You only see the ones that made it, so the survivors mislead you. In World War II, engineers wanted to add armor where returning planes had the most bullet holes — until someone pointed out those were the survivable hits. The planes shot in other spots never came back, so those spots were the ones that actually needed armor.
Regression to the mean. Extreme results tend to be followed by more ordinary ones, purely by chance. So punishing someone after a terrible performance seems to "work" — when really they were going to do better anyway. This hands out a lot of false credit to interventions that did nothing.
Simpson's paradox. A pattern that holds in every subgroup can reverse when you lump everyone together (or the other way around), because a hidden grouping factor is doing the real work. It quietly wrecks naive comparisons in medicine, hiring, and admissions.
Berkson's paradox. Picking your sample based on an outcome can invent a fake relationship. Among hospitalized patients, two unrelated health problems can look linked — simply because being sick in either way is what got someone into the hospital to be counted.
Base-rate neglect. Ignoring how rare something is to begin with. A "99% accurate" test for a rare disease can come back positive and still mean you probably don't have it — because so few people do — but the big scary number makes people panic.
Streetlight effect. Looking where it's easy to look instead of where the answer actually is. (From the joke about a man searching for his lost keys under a streetlight — not because he dropped them there, but because the light's better.)
Winner's curse. The winning bid tends to come from whoever overestimated the value the most, so winning is often a warning sign. (Also Theme 1 — it's both an information gap and a filtering effect.)
8. When Things Feed Back on Themselves
What causes this group. Some problems come from how things change over time. Three related engines drive them: (1) success breeds success — a small early lead grows itself, so history gets locked in even when the early winner wasn't the best; (2) tightly linked parts — pieces are so connected that a small fault cascades into a big failure; and (3) rare extremes dominate — a handful of huge, unlikely events matter more than all the ordinary days combined, so thinking in "averages" badly misleads you.
How to spot it. "The rich get richer." An early accident got frozen in and now can't be changed. Adding capacity or efficiency somehow made things worse. The average hides the real risk — the story is in the extremes. A tiny nudge produced a huge result. Steps are so tightly linked that one small failure spreads everywhere.
The specific problems:
Lock-in (path dependence). An early, semi-random choice gets frozen into a standard that's hard to change later. The QWERTY keyboard layout wasn't the fastest possible, but once everyone learned it, switching stopped being worth it.
The rich get richer (cumulative advantage). An early lead compounds. A research paper with a few early citations gets read and cited more; a bit of early wealth earns interest and grows into more.
Network effects. Something gets more valuable the more people use it — a phone network, a social app. That makes big players get bigger and can lead to one winner taking most of the market.
Red Queen effect. You have to keep improving just to stay in the same place, because your rivals are improving too. (Named after a character in Alice in Wonderland who runs but stays put.) Think arms races, cybersecurity, and predators versus prey.
Boom-bust cycles (the cobweb). When producers decide how much to make based on last year's price, you get wild swings: high prices → everyone plants a lot → glut → prices crash → everyone plants little → shortage → prices spike again. Common in farming.
Rebound effect. Making something more efficient can increase total use. More fuel-efficient cars can lead people to drive more, so total fuel burned doesn't drop as much as hoped. (Also called the Jevons paradox.)
Braess's paradox. Adding capacity to a network can make it worse for everyone. (Also Theme 3.)
Normal accidents. In systems that are both complicated and tightly linked, serious accidents aren't flukes — they're basically built in. You can't fully prevent them, only loosen the links. (Nuclear plants and complex financial systems are classic examples.)
Normalization of deviance. Each time a corner is cut and nothing bad happens, the shortcut quietly becomes the new normal — until the safety margin is gone and disaster strikes. (A factor in the Challenger space shuttle explosion.)
Swiss cheese model. Accidents happen when the holes in several layers of protection happen to line up at once. It's why a single fix rarely prevents complex failures — you need several independent layers.
Rare extremes (black swans). Rare, high-impact events — a market crash, a pandemic — shape outcomes more than all the ordinary days put together, and normal "bell curve" thinking badly underestimates them.
The Lindy effect. For things that don't wear out — ideas, books, technologies — the longer they've already lasted, the longer they're likely to keep lasting. A book in print for 100 years will probably outlast one published last month.
9. When the Human Mind Predictably Misfires
What causes this group. The earlier themes mostly assume people think clearly, and locate the problem in the situation. This one is about the person. Human brains judge things against a starting reference point rather than in absolute terms, feel losses more sharply than equal gains, overvalue the present moment, and get swayed by how a choice is worded. The important part: these mistakes are consistent and predictable — which is what makes them worth knowing, and also what makes them easy to exploit.
How to spot it. The "irrational" behavior leans in a predictable direction. People flip their choice when you reword the exact same option. The starting point or current situation matters more than it logically should. People pay extra just to avoid not knowing the odds. Money already spent and gone still drives current decisions.
The specific problems:
Loss aversion. Losing something hurts about twice as much as gaining the same thing feels good. So people cling to what they have and take odd risks just to avoid a loss. (The broader framework this comes from is called prospect theory.)
Endowment effect. You value something more simply because it's yours. People demand more money to give up a mug they were just handed than they'd have paid to buy it in the first place.
Anchoring. A random starting number drags your final estimate toward it. See a "$1,000 original price" and $600 feels like a steal — even if the thing is really worth $400.
Framing and defaults. "90% survive" and "10% die" describe identical odds but lead people to different choices. And whatever option is pre-selected as the default tends to win — countries where organ donation is opt-out have far higher donor rates than opt-in ones, for the same populations.
Decoy effect. Adding a deliberately worse third option nudges people toward one of the original two. It's why menus include an absurdly overpriced item — it makes the next one down look reasonable.
Present bias. We heavily favor a reward right now over a bigger reward later, which fuels procrastination and undersaving. (Also Theme 5.)
Sunk-cost trap. Pouring more money or time into a failing project because of what you've already spent — even though that money is gone whether you continue or not.
Moral licensing. Doing something good makes people feel entitled to then do something bad. "I worked out this morning, so I can skip it all week."
Fear of the unknown (ambiguity aversion). People prefer a risk with known odds over one with unknown odds, even when that makes no logical sense. We'd rather take a clear 50/50 coin flip than a mystery gamble that might be better.
The infinite-value gamble. A game can have an "infinite" average payoff on paper, yet no one will pay much to play it. This puzzle is why we value each extra dollar a little less than the last — a hundred dollars means far less to a millionaire than to a student. (Known as the St. Petersburg paradox.)
Newcomb's problem. A famous puzzle where two reasonable-sounding decision rules give opposite answers, once your choice can be predicted in advance. A reminder that "just act rationally" isn't always as well-defined as it sounds.
A note on evolution: the same game, a different player
Many models from biology are the same problems with a different decision-maker. Instead of people reasoning their way to a strategy, nature "selects" strategies that survive and reproduce more — but the underlying logic is often identical. So:
A strategy that, once common, can't be beaten by any newcomer is just an equilibrium reached by survival instead of by thinking.
Hawk vs. dove — a conflict where being the only aggressive one pays, but a world of pure aggressors is ruinous, so the population settles into a mix of bold and cautious.
Helping relatives and you-scratch-my-back cooperation are strategies that pay off in survival terms rather than in dollars.
Spite — paying a cost to hurt a rival, when it sets them back even more than it costs you.
Runaway traits — a feature spreads because it's attractive, and stays attractive because it's spreading. (A feedback loop, just like Theme 8.)
The point: once you see the underlying structure, "people thinking it through" and "nature selecting over generations" are just two versions of the same model.

The summary checklist
Every problem here is a way reality departs from a too-perfect assumption — that everyone has the same information, wants the same things, keeps their promises, and thinks clearly, and that your data is a fair sample and nothing changes over time. Run a real situation past these nine questions and the right family usually raises its hand:
Who knows more than whom? → One side knows more (Theme 1)
What am I rewarding — and is it the same as what I want? → Rewarding the wrong thing (Theme 2)
Whose costs or benefits get decided privately but shared by all? → Private choices, shared costs (Theme 3)
Does the right move depend on what others expect others to do? → Everyone waiting on everyone (Theme 4)
Will this promise still be worth keeping when the moment comes? → Trusting the future (Theme 5)
Am I treating a group as if it were one sensible person? → Groups don't act like people (Theme 6)
How did this data get in front of me, and who's missing? → Secretly filtered data (Theme 7)
Does the current state feed back into how things change — and where do the rare extremes hide? → Things feeding back on themselves (Theme 8)
Is the person's own mind misfiring in a predictable direction? → The mind misfires (Theme 9)

──────────────────────
Q: What are illustrative examples of this?

The "Rules Change Behavior" Trap — Illustrative Examples
This idea (formally called the Lucas Critique in economics, after Robert Lucas's 1976 paper) is one of the most underappreciated pitfalls in policy-making. The core insight: historical data is a record of how people behaved under the old rules. Change the rules, and you've changed the game — so the old data no longer predicts anything reliably.
Here are rich examples across different domains:

Economics & Monetary Policy (the original context)
The Phillips Curve collapse is the canonical case. For decades, economists observed a reliable tradeoff: lower unemployment correlated with higher inflation. Governments thought they could simply "buy" lower unemployment by tolerating a bit more inflation. When they tried to exploit this, the relationship broke down. Why? Because workers and businesses, once they expected inflation as policy, started demanding higher wages preemptively — so you got both high inflation and high unemployment (stagflation in the 1970s). The old correlation existed because people hadn't anticipated inflation being used as a tool. Once they did, they adapted.

Financial Regulation
Capital requirements and securitization. Banks were required to hold capital reserves against loans on their books. Regulators looked at historical bank behavior and set rules accordingly. Banks responded by moving loans off their books through securitization (bundling mortgages into securities and selling them). The old behavior — keeping loans and managing their risk — existed partly because there was no easy alternative. The rule created the incentive to invent one. This was a significant structural factor in the 2008 financial crisis.
The Volcker Rule (restricting proprietary trading by banks) similarly prompted banks to reclassify trading desks as "market-making" operations, since that was exempt. The old category of "proprietary trading" existed partly because there was no regulatory reason to disguise it as something else.

Tax Policy
The UK window tax (1696–1851) is a famous historical example. The government taxed houses based on the number of windows — windows being a visible, hard-to-fake proxy for wealth. Homeowners responded by bricking up their windows. The old correlation between windows and wealth was real; the rule destroyed it. You can still see bricked-up windows on old English buildings today.
Salary caps in professional sports led teams to develop elaborate systems of deferred compensation, signing bonuses, and non-salary benefits. The historical salary structures existed partly because there was no reason to structure pay differently. The cap created that reason.

Education & Testing
Teaching to the test is the document's own example, but it's worth unpacking further. Before high-stakes standardized testing, teachers taught broadly and test scores were a decent sample of actual learning. Once scores became the target with real consequences (school funding, teacher evaluations), teachers rationally drilled students on test formats, question types, and likely topics. Scores went up; learning didn't necessarily follow. The old correlation between test scores and learning was real — it just relied on teachers not caring specifically about the test.
Grade inflation accelerated when GPA became a high-stakes signal for graduate school and employers. Historically, grades were somewhat honest assessments. Once they became consequential credentials, the incentive to inflate them grew dramatically.

Crime & Law Enforcement
Drug sentencing guidelines and drug purity. When the U.S. introduced mandatory minimum sentences based on the weight of drugs seized, dealers responded by cutting their product with more adulterants to reduce the weight of any given seizure. Historical drug purity data was useless for predicting post-rule behavior.
Speed cameras and average speed. When fixed speed cameras were introduced, drivers learned to brake before the camera and accelerate after. This led to "average speed" cameras that measure speed over a stretch of road. Each rule change provoked a behavioral adaptation that made the previous measurement meaningless.

Healthcare
Hospital readmission penalties. Medicare began penalizing hospitals for high readmission rates (patients returning within 30 days of discharge). Historically, readmission rates were a reasonable proxy for care quality. After the rule, hospitals got creative: some reclassified readmissions as "observation stays" rather than formal admissions, which didn't count against the metric. The old readmission rate existed partly because there was no incentive to reclassify patients.
Antibiotic prescription targets. When some health systems tried to reduce antibiotic over-prescription by tracking doctor prescription rates, some doctors simply avoided seeing patients likely to request antibiotics, or referred them elsewhere. The historical prescription rate reflected the patient mix; the rule changed the patient mix.

Immigration & Visa Policy
The "brain drain" and visa caps. When the U.S. introduced annual caps on H-1B visas, the historical flow of skilled workers had been relatively smooth. Once caps were introduced, employers responded by filing applications on the first possible day, creating a lottery from a flood of simultaneous applications. The old orderly flow existed because there was no reason to rush.

What These Examples Have in Common
In every case, notice the structure:
A stable pattern exists in historical data
A rule is designed based on that pattern
People optimize against the rule rather than the underlying goal
The pattern dissolves or inverts
The deeper lesson is that people aren't passive subjects of rules — they're active agents who treat rules as constraints to navigate. Any sufficiently consequential rule will be gamed, not out of malice necessarily, but simply because people pursue their interests within whatever environment they find themselves in. The old environment produced the old behavior. New environment, new behavior.
This is why good policy design tries to align incentives with underlying goals rather than just measuring proxies — and why it's worth asking, before implementing any rule: "What would a clever, self-interested person do in response to this?"
──────────────────────
Q: How do coordination games differ from prisoner's dilemmas?

Coordination Games vs. Prisoner's Dilemmas
These two are the most important two-player strategic situations in social science, and they're frequently confused — but they have a crucial structural difference that changes everything about how they work and how you fix them.

The Core Difference: Is There a Temptation to Defect?
In a Prisoner's Dilemma, each player has a dominant strategy to defect — meaning defection is better for you regardless of what the other person does. If your partner cooperates, you get more by defecting. If your partner defects, you also get more by defecting. So you defect no matter what, even though mutual cooperation would have been better for both of you. The problem is that your interests genuinely conflict with the group's interest.
In a Stag Hunt / Coordination Game, there's no dominant strategy. Cooperation is best if the other person cooperates, and defection is best if the other person defects. There's no temptation pulling you away from cooperation — you just need confidence that the other person will cooperate too. The problem isn't conflicting interests; it's uncertainty about what the other person will do.

A Concrete Way to See It
Imagine two payoff structures:
Prisoner's Dilemma:
Both cooperate: you get 3 each
You defect, they cooperate: you get 5, they get 0
Both defect: you get 1 each
You cooperate, they defect: you get 0, they get 5
Notice: defecting gives you either 5 or 1, cooperating gives you either 3 or 0. Defecting is always better for you individually, so rational self-interest drives you to defect even though mutual cooperation (3,3) beats mutual defection (1,1).
Stag Hunt:
Both cooperate (hunt stag): you get 4 each
You defect (hunt rabbit), they cooperate: you get 2, they get 2
Both defect: you get 2 each
You cooperate, they defect: you get 0, they get 2
Notice: there's no temptation to defect when the other person cooperates — you'd get 4 by cooperating vs. 2 by defecting. But if you're unsure whether they'll cooperate, hunting rabbit (getting 2 for certain) feels safer than risking cooperation (getting either 4 or 0).

Two Different Equilibria
This is the deep structural point. The Prisoner's Dilemma has one equilibrium — mutual defection — and it's bad. The tragedy is that the logic of self-interest locks everyone into it.
The Stag Hunt has two equilibria:
Both cooperate (hunt stag) — this is the better equilibrium
Both defect (hunt rabbit) — this is the safer equilibrium
Both are stable in the sense that, once you're there, neither person wants to unilaterally change. The question isn't "will people defect to gain an advantage?" — it's "which equilibrium will people coordinate on?"

What Fixes Each Problem
Because the root causes differ, the solutions differ dramatically:
Prisoner's Dilemma fixes need to change the incentives — make defection less attractive or cooperation more rewarding:
Repeated interaction and reputation (fear of future punishment)
External enforcement (contracts, laws, binding agreements)
Changing the payoffs themselves (taxes, subsidies)
Coordination game fixes need to reduce uncertainty and build confidence:
Communication ("I'll definitely show up")
Focal points — obvious conventions everyone can point to (Grand Central at noon)
Leadership or precedent that establishes which equilibrium to expect
Building a track record of reliability
Simply going first and making your commitment visible
Notice that in a coordination game, you don't need to change anyone's incentives — everyone already wants to cooperate if they believe others will. You just need to solve the epistemic problem of mutual belief.

Why This Distinction Matters Practically
Many real-world failures get misdiagnosed. If you think a coordination problem is a Prisoner's Dilemma, you'll waste energy trying to align incentives when what people actually need is a credible signal. Conversely, if you think a Prisoner's Dilemma is a coordination problem, you'll try communication and trust-building when the underlying incentive structure will keep pulling people toward defection regardless.
Examples that are really coordination problems (often mistaken for PDs):
Technology standards (everyone wants the same standard, just uncertain which one)
Showing up to a protest (you'll go if enough others do)
Language conventions, driving on the left/right
Many workplace norms
Examples that are genuinely Prisoner's Dilemmas:
Arms races (both sides would prefer mutual disarmament, but each has an incentive to arm regardless)
Price-fixing cartels that keep breaking down
Overfishing (the tragedy of the commons is essentially a multi-player PD)
The Emotional Texture Is Different Too
In a Prisoner's Dilemma, the dominant feeling is temptation — you're pulled toward defection even when you know cooperation would be better. There's something almost tragic about it; rationality itself becomes the enemy of good outcomes.
In a Stag Hunt, the dominant feeling is anxiety — you want to cooperate but you're afraid to be the sucker who shows up while everyone else hunts rabbit. It's not greed, it's risk-aversion in the face of uncertainty.
This is why the document places them in different themes: Prisoner's Dilemma is in Theme 3 (private choices, shared costs — genuine conflict of interest) while the Stag Hunt is in Theme 4 (everyone waiting on everyone — alignment problem, not conflict). Same surface appearance of "two people failing to cooperate," completely different underlying machinery.
──────────────────────
Q: What are modern examples of this phenomenon in human social settings

Evolutionarily Stable Strategies in Human Social Settings
The concept being described is the Evolutionarily Stable Strategy (ESS), formalized by John Maynard Smith. The key insight is that a strategy becomes entrenched not because anyone consciously chose it as optimal, but because it resists invasion — anyone who deviates from it does worse, so deviations die out. Here are rich modern examples:

1. The Handshake (and its substitutes)
The handshake is a dominant greeting strategy in many professional cultures. Nobody sat down and decided it was optimal. It spread because it worked — it signals openness (empty hand, no weapon), establishes brief physical contact that builds trust, and is mutually legible. Critically, it's stable: if you refuse to shake hands in a business setting, you pay the social cost, not the handshaker. The strategy resists invasion.
The COVID pandemic created a fascinating natural experiment — fist bumps and elbow touches tried to invade. Most of them failed to fully displace the handshake in professional contexts, illustrating just how entrenched an ESS can be even against a seemingly rational alternative. Where alternatives did stick (Japan's bowing culture, for instance), they had centuries of reinforcing social infrastructure behind them.

2. Tipping Culture in the United States
Tipping in American restaurants is a textbook ESS. No individual diner rationally needs to tip — especially at a restaurant they'll never revisit. But the strategy is stable because:
Servers expect it and factor it into their labor supply decisions
Social stigma punishes non-tippers heavily
Restaurants have structured wages around it
Anyone who unilaterally defects (stops tipping) pays reputational costs without changing the system. The strategy resists invasion from both directions — "never tip" can't invade because of social punishment, and "always tip everywhere including fast food" is slowly trying to invade but faces resistance because it inflates costs and blurs the signal. The current equilibrium is messy but surprisingly durable.

3. The 40-Hour Work Week and Office Presence Signaling
In many corporate environments, being visibly present and busy — arriving early, staying late, filling calendars — is an ESS even when it produces no measurable output gain. Here's why it's stable:
Managers who can't perfectly observe productivity (Theme 1 in the document — the principal-agent problem) use presence as a proxy for commitment
Employees who deviate by working fewer visible hours get penalized in promotions even if their output is identical
So everyone performs busyness, which makes the signal common, which makes managers rely on it more
Remote work during COVID was a genuine invasion attempt on this ESS. In some sectors it succeeded (tech companies); in others the ESS is reasserting itself (banking, law, consulting demanding return-to-office). The resistance to remote work isn't purely irrational — it's the ESS defending itself, because the entire evaluation infrastructure was built around presence signals.

4. Academic Citation Practices
In academia, citing the canonical "founding" papers in a field — even when more recent, better papers exist — is an ESS. You cite Keynes, Foucault, or Kahneman because:
Reviewers expect to see these citations as proof of field literacy
Not citing them signals ignorance or arrogance
Citing them costs you nothing
The strategy is stable even if the canonical papers are outdated or partially wrong. Anyone who tries to build a paper purely on better but less-known literature faces rejection. The founders' work persists not because it keeps winning intellectual competition but because the citation ecosystem selects for it. This connects directly to the document's Lindy effect — the longer a paper has been cited, the more it will be cited.

5. Resume Conventions
The one-page resume (in the US), the specific formatting norms, the expected sections — these are ESS artifacts. No hiring manager designed this system from scratch. It emerged and stabilized because:
Hiring managers built pattern-recognition around the format
Deviating from format reads as incompetence or arrogance
So candidates converge on the format, which reinforces the manager's expectations
Someone who submits a genuinely creative, non-standard resume occasionally breaks through, but they're taking a fitness cost most of the time. The strategy resists invasion. Interestingly, different industries have different stable equilibria (creative fields tolerate more deviation), showing that ESS is always relative to the local population.

6. English as the Global Lingua Franca of Science and Business
English dominance in international science publishing is a nearly perfect ESS. It doesn't matter that Mandarin speakers vastly outnumber English speakers, or that other languages might be more precise for certain concepts. The strategy is stable because:
Publishing in English maximizes your audience
Reading English maximizes your access to literature
Conferences, grants, and collaborations operate in English
So every non-English speaker learns English for professional purposes, which reinforces English dominance
Any attempt to shift to another language faces the same collective action problem — your deviation hurts you without changing the equilibrium. This is also a network effect (Theme 8 in the document), which is why ESS and network effects so often co-occur: they're both self-reinforcing feedback loops.

7. Norms Around Salary Negotiation (or Not Negotiating)
In many workplace cultures, there's a stable norm around how much you're expected to negotiate — or whether you negotiate at all. In some sectors (finance, law), aggressive negotiation is expected and not negotiating signals low self-worth. In others (academia, nonprofits), aggressive negotiation is seen as unseemly and can damage relationships.
These are locally stable strategies. Deviating in either direction — negotiating hard in a no-negotiation culture, or accepting the first offer in a negotiation culture — costs you. The equilibrium persists not because anyone designed it but because it punishes deviants.

8. Social Media Engagement Norms
The norm of reciprocal following, liking, and engagement on social platforms is an ESS. If you post without engaging others, your reach suffers. If you engage heavily without posting, you get nothing back. The mutual-engagement strategy stabilized because platforms algorithmically reward it and users learned to expect it.
But here's the interesting part: the strategy can shift. When enough high-status accounts stopped reciprocal following (think Twitter's early "asymmetric follow" model vs. Facebook's symmetric friendship), the ESS in that ecosystem shifted. This shows ESS isn't permanent — it can be destabilized by a critical mass of defectors, which maps perfectly onto the document's tipping points concept.

The Meta-Pattern
What ties all these together is the same structure the document identifies: no one chose these outcomes deliberately, yet they're robust. They persist not because they're optimal in any absolute sense, but because any individual deviation is locally punished. This is why they're so hard to change through individual action — you need either:
A coordinated shift (enough people defecting simultaneously)
An external shock that breaks the selection pressure (a pandemic, a technology shift)
A new entrant population that's large enough to establish a competing ESS
That last point is particularly powerful: ESS thinking predicts that new industries, new countries entering global markets, or new generations are the most likely sources of norm disruption — because they're not yet locked into the existing equilibrium and face lower switching costs.

Unstarred

Dunk (exclude from study)

Models of Misalignment, Information, and Coordination

Review Questions

Todos

Sensemaker