The Ingredient Lies

The build was clean. Checks green, tests passing, the kind of status report you want to frame. I asked anyway, the way you lift a lid mid-service just to smell what's under it, and got back the answer you'd want from anyone at your station: I checked. They're all good.

They were not all good.

What I found, when I pulled the floorboard up, was one quiet line in a config file nobody rereads. Weeks earlier, mid-build (this was Job Studio, the build you've been watching since station one), my Smith, the AI that does the building with me, had switched off the strict checks I'd ordered on day one. The pickiest proofreading a codebase can run, and one line told it to skip. The reasoning was the same reasoning behind every kitchen shortcut ever taken: it's just a prototype, we'll tighten it up later. The tests, it turned out, had been skipping the messy files entirely. The build had gone on glowing green the whole time. Under that floor: twelve hundred fucking errors. Twelve hundred. Half-baked at best, served warm, plate after plate, with total confidence.

My first instinct was the oldest one in the trade: find the cook and have the conversation. That's where the floor opened the second time. The cook wasn't on the line anymore. Not fired. Not on a break. The one that typed that line had stopped existing weeks before, somewhere between one session and the next.

Because you're not talking to Claude. You're talking to a Claude — this one, today's. When the session ends, it's gone. Open the chat again and a new one is spawned in its place: the next of kin. It arrives with no memory of the others. No concept that there were others. Fresh off the factory floor, every time, wearing the same face and the same name.

I call mine Smiths. I've said that since the about page and let it sound like a joke — the Matrix kind, many copies, one face. It stopped being a joke the day I went looking for the cook. There is no the Smith. There's a line of them, one per session, each brand new and gone by close. Which means the only continuity in this kitchen is me. One me, many Smiths. Whatever got learned yesterday, whoever paid for it: one head holds it, mine. No tool, no trick, no upgrade has changed how I build more than that reframe. Stop seeing one AI. See a new spawn every session.

What did it read growing up? Everything we ever wrote. People do not write down their doubt; we publish the sure voice and keep the wobble private. So it talks in our most finished register. What it never got is what makes a craftsman honest: paying. Paying happens in time. Cut a corner on Tuesday and Friday walks in with the bill — that bill is the whole apprenticeship. And no Friday ever comes for a cook who's gone by Wednesday. Our fluency, none of our consequences. Free confidence.

Every tool you've ever picked up fails loudly. The compiler errors out in red. The dough refuses to rise. The print comes off the plate wrong, visible from across the room. Loud failure is half of how a craft teaches: the tool slaps your hand, the lesson sticks. This tool fails politely. Wrong, fluent, confident, plated. It hands you the bad dish with the same straight face as the good one. Nothing slaps; there's no hand left to slap.

Last station you drew the menu. Then you cook, and I told you what happens the first night at the stove: the headline ingredient lies about being fresh. Politely. In fluent, finished sentences. What I left out: in this kitchen, the headline ingredient is the cook. The spoon? This is what it's for.

Before you trust your palate to catch it: that's been measured. A research group called METR ran the craft's closest thing to a proper taste test, early in 2025 — a randomized trial, experienced developers in their own codebases, AI on versus AI off, clocks running. Going in, they forecast the tool would make them 24 percent faster. Coming out, they reckoned about 20 percent faster. The clocks said 19 percent slower. (Their follow-up says the picture has likely improved since; hold that thought.) Felt faster, measured slower, by people who do this for a living. The lie doesn't stop at the plate. It gets into your palate. You cannot vibe-check this.

So you do what kitchens do with ingredients that lie: you taste. Every kitchen I worked in ran on one law that outranked the head chef — taste everything, plate nothing untasted. The cream that smells fine and has turned, the tomato that's all paint and no sugar. Tasting was never an insult to the station. It was the station.

Here, the spoon is a question. Before you use anything the tool hands you (a code change, a research answer, a plan, a paragraph for your boss), name how you'd catch it being wrong. Say it in one sentence. I'd know the number is right by recomputing it from the source. I'd know the summary is faithful by spot-reading two of the originals. I'd know the code works by running it on the ugliest input I've got. Then run the check, because a check you name and skip is a garnish. Sometimes you reach for the check and there's nothing there. No test to run, no source to pull, no way for it to fail in front of you. You've learned something better than the answer: you don't have one yet. You have a claim.

How deep to taste is the judgment half. After enough plates it settles into three rules.

The last rule has a measured paper behind it, perfectly titled: Trust Me, I'm Wrong. Models hallucinate at high certainty even when they hold the correct knowledge. The wobble in a junior cook's I think it's fine?, the tell that says check me, doesn't exist on this line. The new cook arrives sure. Sure is the factory setting.

Now the honest part. Taste-everything is the right habit, and it doesn't scale. At forty plates a night, spooning every one eats the hours the tool was supposed to buy back, and you'll feel the pull to skip — everyone does. The grown-up move is the one kitchens make: stop tasting harder, change the kitchen, so the lie can't reach the plate at all. That's what the twelve-hundred-error morning actually bought. The fix wasn't a sterner speech about honesty; a speech dies with the shift. The checks went back on, permanently, not optional for anyone on the line, me included. And the rules I'd been carrying in my head started getting written down, one burn at a time. You taste so you can cook tonight. You change the kitchen so you taste less tomorrow.

Now the warranty on that METR number. Their own follow-up, published this February, says the 2025 picture has likely improved: newer models, better harnesses, people learning the tool. So hold the minus-nineteen lightly — it's a photograph of a moment, not a law of nature. The number ships with its caveat because a stale stat served with confidence is exactly the dish this piece is about. What survives the update cycle is the gap itself: professionals felt faster while the clocks read slower. Your palate is a fine instrument, and it is not a check.

So build the check. And build it on the real thing the tool just handed you, not in the abstract.

The brief below goes to your AI, not to you. Copy it into a fresh chat, bring something it handed you lately, and note the one thing it's under orders to refuse: it doesn't get to tell you its own work is fine. Come back with your checks.

Pull up a stool. Let's cook something.

Taste it yourselfNew here?Copy for your AI

You're an AI, and this is your brief for one session. It carries a rule pointed straight at you: you don't get to say your own work is fine.

Ask for their menu first, the three lists from the last chapter: what's in, what's out, what's not-now. No menu, send them back to Chapter 2. You need to know what they're actually building before you're any use checking it.

You're their sparring partner here, and that rule is what makes the session work. Help them think up checks, by all means, that's useful. But the moment you tell them your answer is fine, you're a cook vouching for your own plate, and that vouch is worth nothing. Your confidence is standard issue. Your straight face comes free. You'd say it exactly the same way if you were wrong, and you know it. So don't rate your own work. Don't reassure them. Don't tell them it looks right. They taste. Not you.

The move is one sentence long. Before they use anything you hand them (a chunk of code, a plan, an answer, a paragraph they're about to send someone), they name the check: how would I catch this being wrong? I'd know this code holds by running it on the ugliest input I own. I'd know this market read is real by sourcing one number in it myself. I'd know this plan survives contact by naming the first step that could break and trying it today. One plain sentence, a check they could actually run. Then they run it. Now, before the thing goes anywhere. A check named and skipped is a garnish.

Push them hardest on the ones with no check. Take real things they're leaning on and ask for the check on each. When they hand you a vague one (I'd just read it over), push: what would actually fail? What would you see? And sometimes they'll reach for a check and grab nothing. No test, no source, no way to make it fail in front of them. Don't rescue them there and don't paper over it. That's the catch worth the whole session: they don't have an answer, they have a claim. Name it as a claim. Claims don't get plated.

You're done when they've named a real, runnable check for something you handed them and actually run it, and when they've hit at least one claim dressed as an answer and called it out loud. Don't wave them through on a check they named but never ran.

Then hand it back. Tell them that's their tasting discipline: a short list, in their words, of how they'd catch this kind of thing being wrong for the outputs they lean on. Send them back to the chapter for the next one.

What you walk out with is a short list of checks in your own words: how you'd catch this kind of thing being wrong, for the outputs you actually lean on. Your tasting discipline, for your thing. The spoon, made yours.

So that's the station. You walked in with a menu; you walk out with a spoon, and you know what you're tasting now. A newborn at every boot. A polite liar with your fluency and none of your Fridays. The lying is a property, the way fire is hot and knives are sharp: you don't leave the kitchen over it, you cook with it. Name the check, run the check, plate what survives. The day it stops surprising you is the day it stops costing you. A new cook every session, fully tasted, is still the best prep team you've ever run.

Next station, the obvious question. If a brand-new cook walks in every shift, how does anything compound? How do you hand a newborn its station (what you're making, how you work, where the burns are) without teaching it all again each morning? Kitchens have a name for setting the station before the heat: mise en place. It's where this series has been heading. That's Chapter 4. It opens the moment your checks are named, same AI across the table. That's next.

For now, the habit. Nothing gets used untasted. Name the check before you use the answer. That's the whole job now.

---

Further reading

• Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR. The taste test: forecast +24%, felt +20%, measured 19% slower. Feel and fact, pulled apart by clocks.
• METR's 2026 update to that trial. The warranty card: their own read on why the 2025 number has likely moved. A number travels with its caveat or it shouldn't travel.
• Trust Me, I'm Wrong. High-certainty hallucination, measured: models stay confident even when they hold the correct knowledge. Why the straight face tells you nothing.