I once stood in front of a whiteboard showing an application flow in seven steps. By the end of the meeting there were four.
Two of the steps we cut showed information nobody needed to move forward. The third explained how a figure shown two steps earlier had been calculated. Completions went up afterwards, drop-offs went down. By every measure we had, it was a good day.
In my profession the ideal is called a “seamless experience”. It is the highest praise a digital product can receive. Nobody writes in an investor pitch that a process could have used more friction. It took me years to notice that something about this definition doesn’t add up.
Try something. It takes thirty seconds. Explain how a fridge works: how does the cold get in, and what happens to the heat?
Most people probably won’t get far, and many would be surprised. It is exactly this surprise that the psychologists Leonid Rozenblit and Frank Keil measured at Yale University in 2002. People who had to write down how everyday devices work rated their own understanding markedly lower afterwards than before. They called it the illusion of explanatory depth. How the test works, and why visible parts in particular fool us, is in “The Fridge Test”.
Everyone has gaps. What’s odd is how quiet they stay. Our brain distinguishes two kinds of remembering: recognition and recall. Recognition is cheap – you see a fridge, and something in you says at once: I know that. Explaining requires recall, and recall is expensive. As long as nobody asks, you mistake the one for the other.
Rozenblit and Keil end their paper with an objection to themselves. The illusion, they write, might be necessary: it ends a search that would otherwise never end. Anyone who followed every explanation to the bottom would never get through the day.
The objection holds. It only moves the question. If the illusion is the dial that switches the search off, the decisive question is who sits at that dial.
So far, you do.
For some years I have needed a word for this state, because I haven’t found a more precise one: sham understanding. Sham understanding is the feeling of having understood something without actually having understood it. The nod in the meeting. That “ah, I see” after a fluent explanation. You think you’re informed, when all you have really done is consume words.
The word isn’t pretty. Perhaps that is why it works. It carries a small embarrassment, and you don’t like saying it about yourself.
Ignorance is honest. Sham understanding comes with everything that feels like understanding from the inside – the certainty, the satisfaction, the inner nod – and without its substance.
A false claim provokes contradiction. A good feeling of understanding provokes nothing. Why your own feeling won’t help you here is in “The Feeling Lies”.
Rozenblit and Keil’s paper appeared in “Cognitive Science” in 2002. Google was four years old, and Facebook didn’t exist. What we didn’t understand had visible parts: zips with teeth, toilet flushes with levers and floats, helicopters with rotors.
Anyone who wanted to could learn it. Take the zip apart, put it under a magnifying glass, examine every tooth. Your not-understanding was curable: information was missing, and someone can give you information.
Today the same finding applies in front of your feed, except that the gap can no longer be closed with information. You interact every day with systems like ChatGPT or the algorithms behind your feed. With them there is no step-by-step description a human could read. What the system does sits in its parameters – numbers that set how strongly each signal influences every other. Even if you could see them all, there would be too many to turn into a description.
How big is GPT‑4? OpenAI won’t say, and the same game repeats with every successor. What circulates comes from industry reporting: more than a trillion parameters. If you counted one per second, day and night, it would take you more than thirty thousand years. And you wouldn’t have read a single one. How they interact in detail, not even the people who built the system know.
The parameter count is an estimate. That no one can trace how the parameters interact is not.
Still, these systems give you explanations. Find an ad in your LinkedIn feed, open its menu and choose the option that explains why you’re seeing it. You’ll read that your profile and activity match the advertiser’s target audience, followed by a few reasons, such as similar skills or an employer of similar size. It is grammatically correct and uses the language of cause. You read it and keep scrolling.
At least an explanation like this names something you could follow up on – a profile field, an advertiser. The fridge never offered you that. Whoever follows up has a starting point, and the explanation itself admits there may be other reasons it doesn’t list. For everyone else, the mere presence of the explanation does the work. It shows a surface designed to feel like depth.
Behind every one of these explanations are design decisions.
The people who built these surfaces are mostly people like me.
I have tested such explanations myself, two versions against each other, until the better one was clear. Better meant whatever the dashboard showed. Whether anyone could afterwards explain what the system does with their data, we never measured. Anyone who improves an explanation until nobody asks any more has optimised it for frictionlessness, whether they intend to or not. What follows for design, and why more friction still isn’t the answer, is in “What I Learned About Good Design by Doing Bad Design”.
Not everyone is affected equally. Those who can switch will do so as soon as something bothers them. Those tied to one provider live with an explanation they could never check.
There isn’t much you can do about it. There is one thing, and it’s uncomfortable: ask yourself whether you could explain this right now. Now, out loud, so that someone with no background could follow. Where you stall, that’s the gap. Where the question comes from, why it works and where it fails is in “The Question That Breaks Sham Understanding”.
I’m still building in this profession, just with different questions.
The question I ask myself doesn’t help everyone. It costs time and concentration, and both are unevenly distributed. Someone under pressure, tired, doing three things at once, doesn’t have the attention to spare that this kind of questioning demands. Someone signing a mortgage whose terms nobody explained gets little from my exercise: it shows them the gap, not the way out. For them, what matters is whether someone had to build the product so that their not-understanding doesn’t trigger a disaster.
The gap was always there. What’s new is that today an explanation lies over it.
I have written a book about this gap, in German. The essay ends at the gap; the book begins there.
One of its figures is Thomas, a composite of many conversations. He invests part of his savings because the platform explains its recommendation so clearly. Eighteen months later he reads the same explanation again, and it no longer helps him.
