Paolo Stolfo
Die Illusion des Verstehens
The Illusion of Understanding — why we believe we understand what we’ve only read.
A short test
Ask a language model how a quantum computer works. Read the answer. Close the tab.
Now explain it. Out loud, in your own words, without the terms from the answer.
Wherever you stall, the feeling was faster than the understanding. You read words. It felt identical.
Cognitive psychologists have known the effect since 2002: the illusion of explanatory depth. We systematically overestimate how well we understand mechanisms — until we have to explain them. What’s new isn’t the illusion. What’s new is that the explanation now comes included, free, in unlimited quantity, in a tone that invites no follow-up question.
The question isn’t: is that true? It’s: do you understand it?
A word for a state that had none
Scheinverstehen — sham understanding. The feeling of having understood something without actually having understood it. That internal ah, now I get it after a fluent explanation. The conviction that you’re informed, when in fact you’ve only consumed information.
Real understanding and the feeling of it are indistinguishable from the inside. Your brain has no built-in alarm.
Unlike fake news, sham understanding provokes no contradiction. It produces satisfaction. Nobody disputes it, because nothing about it is demonstrably false.
Fake news is falsifiable. Sham understanding is private.
What the book is about
It doesn’t explain how AI works. It describes what AI explanations do to your thinking.
Why fluent answers suspend your checking. What happens to your memory when everything is retrievable at any moment. And the point at which sham understanding stops being harmless — at the loan desk, in the doctor’s office, in the boardroom.
No abstinence, no renunciation, no going back to anything. Twelve chapters on what you can test in an explanation before you believe it.
«Understanding doesn’t live in the explanation. It lives in the work you do with it.»
Paolo Stolfo
Why I’m writing this book
For over a decade I’ve been designing digital financial products. My job is to move people towards clicks. I know which option has to be preselected for it to be chosen. And I know how an explanation has to sound so that nobody asks again.
I’ve tested explanations like that. Two versions against each other, two weeks, then a number. Better meant: fewer drop-offs, fewer support tickets. Whether anyone could afterwards explain what the system does with their data — we measured that in none of those rounds.
I know these mechanisms because I built them. The book makes them visible — the introduction and one chapter are open to read, in German.