I once stood in front of a whiteboard with an application flow laid out across seven steps. By the end of the session there were four.
Two of the deleted steps showed information you didn’t need in order to continue. The third explained how a figure had been calculated — the figure displayed two steps earlier. Completions went up afterwards, drop-offs went down. By every measure we had, it was a good day.
In my line of work the ideal is called a seamless experience. Frictionless. It’s the highest praise a digital product can receive. Nobody writes in an investor pitch that a process could have used more friction.
It took me roughly ten years to notice that something in that definition doesn’t add up.
* * *
Try something. It takes thirty seconds.
Explain how a refrigerator works. Not roughly. Not “something with gas” or “the motor makes it cold.” The mechanism. Where the cold comes from. What happens to the heat. Why the thing hums.
I’ll wait.
First comes the confidence. Of course you know how a refrigerator works — you open one every day, you have a familiar feeling about it and a steady hand on the door. Then the attempt. There’s a compressor in there somewhere, and it cools the air, or maybe a liquid, and —
And then nothing.
In 2002 the psychologists Leonid Rozenblit and Frank Keil studied exactly this gap at Yale University. They gave students a list of everyday objects — zippers, flush toilets, helicopters, sewing machines — and asked them to rate, on a scale of one to seven, how well they understood how each one worked. Participants gave themselves middling marks, just under four on average. Then they were asked to write out the mechanism step by step.
After the attempt, their ratings dropped by just under a point. Then came a diagnostic question — how do you pick a cylinder lock? — and the figure sat roughly a point and a half below where it started. “I have some idea” became “I have a vague notion, and not much more.” Rozenblit and Keil called it the illusion of explanatory depth.
The refrigerator, incidentally, wasn’t on their list. It works anyway, and that’s the point: the effect isn’t tied to the object.
I ran the experiment once at a dinner party. Eight adults, all with university degrees. I asked whether anyone could explain how a flush toilet works. First laughter, then silence, then twenty minutes of discussion that ended with someone pulling out their phone. Nobody could explain it. Everyone had assumed they could.
* * *
The striking thing about this finding isn’t the gap in knowledge. Everyone has gaps. The striking thing is the surprise.
Up until the moment you start explaining, your knowledge feels competent. It isn’t vague half-knowledge you’re papering over — it’s a quiet certainty, a soft sure, I know that. You’d cheerfully explain it to anyone who asked, without a second’s hesitation. The gap only exists once you see it.
Rozenblit and Keil worked out where it comes from. They had forty devices rated on several scales: how familiar they were, how many parts they had, how many of those parts you can see while the thing is running. Then they checked which of those measures predicted the overestimation.
Familiarity didn’t. Neither did the number of parts. It was the ratio of visible to hidden parts — a single value that accounted for nearly half the variation.
That’s more precise than the story you usually hear. It isn’t proximity to an object that fools you. What you can read off an object, you file as knowledge in your head. A zipper lies there in the open, teeth visible, slider visible; as long as you’re looking at it, you don’t have to remember anything. Take it away and you notice how little of it was ever inside you.
The illusion also isn’t everywhere. For capital cities, self-ratings barely moved. For film plots and recipes they didn’t move at all — people asked to describe how to bake chocolate chip cookies had judged themselves correctly from the start. Factual knowledge has almost no middle ground: you know the capital of Portugal or you don’t. Explanatory knowledge is graded, with layers and depths. And that’s where the trap sits. We mistake understanding a little for understanding.
You have no direct access to what you know. You only have access to what knowing something feels like.
That feeling lies.
* * *
The mechanism behind it is well described. Your brain distinguishes two kinds of remembering: recognition and recall. Recognition is cheap — you see a refrigerator and something in you says know that immediately. Recall is expensive. You have to reconstruct, articulate, run into the gaps.
For most everyday things the shortcut holds. You recognise a chair and you understand how to sit on it. For complex mechanisms it fails. You recognise the refrigerator, and that tells you nothing about its interior.
This isn’t a defect. Evolution doesn’t optimise for truth, it optimises for survival, and survival rarely depended on seeing through a zipper. Being able to use one was enough.
Rozenblit and Keil close their paper with an objection against themselves. The illusion, they write, might be necessary: it ends a search that would otherwise have no end. Anyone who followed every explanation to the bottom would never get through the day.
The objection holds. It only shifts the question. If the illusion is the dial that switches the search off, then the question that matters is who sits at that dial.
In 2002, you did.
* * *
There’s a word I’ve come to rely on for this state, because nothing more precise was available: sham understanding.
Sham understanding is the feeling of having understood something without actually having understood it. The nod in the meeting. That small internal ah, now I get it after a fluent explanation. You consider yourself informed when in fact you’ve only consumed information.
The German word for it isn’t a pretty one. Perhaps that’s what makes it useful. It carries something stiff with it, a small embarrassment, and you’re reluctant to apply it to yourself.
Sham understanding isn’t ignorance. Ignorance is honest. Sham understanding arrives with every symptom of understanding — the certainty, the satisfaction, the internal nod — and none of its substance.
For years now we’ve talked about fake news, about disinformation, about algorithms that move lies faster than truth. The problem is real, and others have written about it at length. But fake news triggers something. Eventually someone checks the fact, a newspaper runs the correction, a friend sends you a link. The mechanism isn’t perfect, but it exists: contradiction is possible because a lie is recognisable as a lie the moment someone looks closely.
Sham understanding produces no contradiction. It produces satisfaction. Nobody disputes it, because nothing about it is demonstrably false. The explanation was plausible, the words were correct. The only thing missing was your understanding — and nobody can see that missing but you.
Fake news is falsifiable. Sham understanding is private.
* * *
Rozenblit and Keil’s paper appeared in Cognitive Science in 2002. Google was four years old and Facebook didn’t exist. The things we didn’t understand had visible parts: zippers with teeth, flush toilets with levers and floats, helicopters with rotors.
If you wanted to, you could learn. Take the zipper apart, put it under a magnifying glass, examine every tooth. Your not-understanding was curable: information was missing, and information is something someone can give you.
Today the same finding describes your feed.
You interact daily with systems where that no longer holds. Language models that compose explanations for you. Recommendation algorithms. Scoring systems at your bank. With these, no information is missing. There is no step-by-step account a person could read: first A, then B, therefore C. What the system does sits in its parameters — numbers that determine how strongly each signal influences every other. They aren’t hidden. There are simply too many.
How large is GPT-4? OpenAI won’t say, and the same game repeats with each successor. What circulates comes from industry reporting: upwards of a trillion parameters. Count one every second, day and night, and you’d need more than thirty thousand years. You wouldn’t have read a single one.
The parameter count is an estimate. That nobody can check it isn’t.
These systems supply you with explanations regardless. Open your feed, look at any post, find the line that says “Why you’re seeing this” and tap it. You’ll read: “Based on posts you’ve liked.” Or: “Because you follow similar accounts.” It sounds like an explanation, it’s grammatically correct, and it uses causal language. You read it and keep scrolling.
The explanation survives no follow-up question at all. It shows you a surface designed to feel like depth.
Design decisions. Every one of them.
* * *
Now the uncomfortable part. These surfaces didn’t emerge from nowhere. They were built, and mostly by people like me.
I’ve been designing financial products for ten years. I know which option has to be preselected for it to be chosen. And I know how an explanation has to sound so that nobody asks again.
I’ve tested explanations like that myself. You put two versions against each other and see which performs better. Better means: fewer drop-offs, fewer support tickets, better scores in the satisfaction survey. Whether anyone could afterwards explain what the system does with their data — we measured that in none of those rounds. You’d have to make them explain it, and a test cycle has no room for that.
That says nothing about intentions. It says something about the direction such tests push a piece of text. Refine an explanation until nobody asks about it any more, and you’ve optimised it for the absence of friction, whether or not you meant to.
And not everyone is affected equally. Someone who can find another provider eventually does. Someone tied to this one lives with an explanation they were never able to check.
* * *
There isn’t much you can do about it. There’s one thing, and it’s uncomfortable.
Ask yourself whether you could explain it right now. Not in theory — now, out loud, without jargon, without “it’s complicated,” in a way someone with no background could follow.
The idea comes from Richard Feynman, though not the name it circulates under today. A colleague at Caltech once asked him to explain why particles with half-integer spin obey Fermi-Dirac statistics. Feynman agreed: he would prepare a freshman lecture on it. A few days later he came back. He hadn’t managed it. He couldn’t bring it down to that level — and that, he said, meant we don’t really understand it.
Feynman distinguished between knowing the name of a thing and knowing the thing itself. You can know the name of a bird in every language in the world and know nothing about the bird.
Why the question works has a name: the generation effect. When you read, recognition is doing the work — the same signal that fires instantly at the refrigerator. When you explain, your brain has to produce: select, order, connect to what you already know. That’s where the illusion tears. Not because explaining is harder, but because it’s more honest.
One objection is fair. The question is a diagnostic instrument, not a knowledge generator. If you know nothing about a subject, explaining won’t help you; you’ll just talk into the void. Read first, then explain. Not the other way round.
And it doesn’t apply everywhere. Asking yourself whether you could follow a recipe teaches you nothing. It’s a precision instrument for one kind of knowledge: causal relationships inside complex systems. Which is exactly what a machine hands you when you ask it how something works.
* * *
I didn’t leave this profession. I’m still building, and I ask different questions while I do.
No longer only: how do I make this easier? But: where in this flow should it not be?
That question is trickier than it sounds. For people with reading difficulties, with cognitive impairments, or living their daily life in a second language, smoothness isn’t a trick — it’s access, and awkwardness quickly becomes a gatekeeper. So the question isn’t whether a surface is allowed to be easy. The question is where in the flow it shouldn’t be.
And the question I ask myself doesn’t help everyone. It costs time and concentration, and both are unevenly distributed. Someone under pressure, tired, doing three things at once, doesn’t have the attention this kind of checking demands. Someone signing a mortgage whose terms nobody explained to them gets nothing from my exercise — least of all the option to say no. What decides their outcome isn’t whether they turn sceptical. It’s whether someone was required to build the product so that their not-understanding wouldn’t be a catastrophe.
That’s the asymmetry, and it’s why I don’t want this read as self-improvement advice. The question helps you. It doesn’t help the person who gets the bill. What helps them are rules that take effect before anyone has understood anything — and people who write those rules. People who have noticed in themselves how easily a fluent explanation feels like understanding.
The path from one to the other runs through you.
The gap was always there. What’s new is how easy it’s become to look past it.
* * *
About the book
This essay is a first step into a topic that has occupied me for years — and that my upcoming book «Die Illusion des Verstehens» (The Illusion of Understanding) explores in depth.
If this has interested you: subscribe to the newsletter to be the first to know when the book is published.
