The journalist Karen Hao calls the big AI companies “empires”. In her reading, they lay claim to resources that don’t belong to them – data, the work of writers and artists, land, energy and water – and take them from the communities they come from. You don’t have to follow her on scale to take on the core: the logic behind it reaches beyond AI. The same mechanics run, far more quietly, in the ordinary digital product.
The logic is called extraction. Value is drawn from the user without anything in return and without consent: attention, data, behaviour. That takes no data centres; the pattern sits in the detail: engagement placed above the user’s interest, defaults that work for the provider and pass as convenience, nudges without an exit, friction between the user and cancellation, cross-selling that presents itself as a recommendation. Each pattern looks harmless on its own. Together they follow the same logic as the empires, only on a smaller scale.
Most of these patterns are a by-product of a single, barely questioned optimisation for the next step in the funnel, rarely of intent. That explains why they persist.
Hao calls for regulation, and rightly so. More interesting is where she goes beyond it herself. In an interview in July she compared the situation with the fashion industry: when it became known under what conditions clothes are made, nobody stopped wearing clothes. Pressure, organising and regulation reined the industry in, and only once alternatives existed could consumers vote with their wallets. In AI, Hao says, those alternatives are still missing. Europe has the chance to show a third way, but only if it knows what for.
Resisting is not yet design. Whoever only regulates draws the limits of what is allowed and leaves the building to those who are building anyway. Bans produce at most a tamer variant of the same system. The more productive question is therefore what to put in place of extraction.
The most common objection to ethical design is: less steering, weaker conversion. Its first half rests on a mistake. Choice architecture starts from an insight within the discipline itself: there is no neutral option. Every screen, every default, every ordering decides on the user’s behalf before they decide themselves. Steering happens anyway; the only open question is where to and in whose favour. Ethical design therefore does not give up steering but makes it transparent and reversible. It becomes concrete in defaults, in the way out and in the question of who keeps the value. For the first two, the EU caught up in 2026 for financial services.
The default is among the most effective decisions in a product, because very few users actively change it. It becomes extractive when it works for the provider and passes as convenience: the preselected insurance, the silent opt-in. The test is simple. Would the user choose this default if they saw the reasoning behind it? Where the answer is no, the product works against them. This can be measured straight away, through the correction rate: how many change the default once they understand it. It shows within a single session.
How seriously defaults are now taken is shown by the settlement Meta reached with US states in August. It prescribes default settings: a daily limit for minors that only parents can lift, a night-time block, silent notifications during school hours. Something treated as a product decision for years is now in a court document. For financial services, the EU goes a step further: since it has been prohibited there to give certain options more prominence in a choice when this impairs free decision-making.
Just as revealing is the asymmetry of exit costs. One click to sign up, five screens to cancel. Ethical architecture treats both alike: the way out is as clear as the way in. That sounds like giving away retention, but it keeps the more valuable kind of loyalty. Keeping users only because they can’t get away means holding on to dead conversion. This too can be checked today, with an audit of click paths: steps to sign up against steps to cancel.
In the EU, this principle has been law since , in two stages. Anyone offering contracts via a website or app must provide a clearly visible withdrawal function, and withdrawing must not be more cumbersome than signing up. The rule came, of all places, with the directive on financial services contracts concluded at a distance, but it applies to banks, insurers and lenders just as it does to any online shop. It covers withdrawal within the statutory period. For financial services, the same directive goes further and prohibits making cancellation harder than signing up; for other sectors, the announced Digital Fairness Act is expected to address this. Switzerland has no comparable obligation. The standard is set nonetheless: anyone in Switzerland who specifically targets customers in the EU is generally subject to their consumer law anyway, and customers will hold every other Swiss app to it.
The heaviest question is who keeps the value in the end, and here the law helps only at the margins. The same nudge serves both directions. A nudge to save creates value the user keeps; a nudge towards an overdraft creates value the bank skims off. Same mechanics, opposite ethics. The deciding factor is who keeps the gain. Ethical design chooses the variant in which part of it stays with the user. The criterion is not new. Price and Jaffe’s value–irritant matrix pins down value-adding contact precisely where bank and customer benefit at the same time. It is usually read as a cost question, rarely as an ethical one. That is where the gap lies, and it stays a gap because no regulation can prescribe who should earn from a nudge.
That leaves the question of who defines the user’s interest. If the bank does, that is just the next presumption, called care. The answer lies in the first principle. The yardstick is not what the bank considers right but what the user would confirm if they saw the reasoning. Transparency and an open exit give them the means to overrule any steering. An architecture that comes with its own correction protects the user’s autonomy instead of skimming it off.
How sharp this yardstick is shows in the current debate about agentic AI in customer service. It promises to fully automate routine contacts and to place cross-selling and upselling prompts in the same conversation, embedded so skilfully that customers don’t find them disruptive. By the measure of irritation, that is a success. By the transparency test, it is the opposite. A sale the user doesn’t notice passes one test and fails the other. Non-irritation is not consent, and this very confusion separates orchestration from extraction.
Here too, European lawmakers have already acted. The same distance-selling directive that brings the withdrawal button prohibits manipulative interfaces for financial services and gives customers the right to ask for a human being in an automated process. An agent that sells without anyone noticing is therefore not just an ethical question.
Ethics is the more demanding option here. The operational evidence is short-cycle and falsifiable: correction rate, exit symmetry, complaints on the cancellation path, all measurable within a quarter rather than at the end of a term of office. The lifetime of the customer relationship is the underlying thesis, not the KPI you steer by.
Economically, though, the argument holds only for part of the business. In retail banking, inertia supposedly holds the relationship together, not trust: switching costs, account links, standing orders. The cancellation rate barely reacts to a preselected checkbox. For the existing base, that is true. Inertia holds what is already there.
But it holds only that: the dormant account and the low-margin core. Main-bank status, share of wallet, mortgage mandate and investment portfolio do move; they depend on offer and trust. And inertia does not protect the next generation at all. Anyone choosing their first main bank at 25 has hardly any switching costs; that decision is made above all on trust and experience.
The Swiss figures show both: that people choose, and how slowly. Yuh gained around 400,000 customers in just over four and a half years; Neon reached about 250,000 by the end of 2025. Radicant, Kaspar& and Yapeal have discontinued or scaled back their retail business. Yet only just over one per cent of the population uses a neobank as their main bank; in the UK the share rose from one to around six per cent between 2019 and 2025. So people do choose; the main bank just moves slowly.
Extraction therefore costs different amounts. On the core of the existing base it is cheap, because customers stay anyway. It becomes expensive where the bank has to grow, within the existing base and beyond: in share of wallet, in acquisition, in the generation that still chooses.
An unticked checkbox can be copied overnight; there’s no advantage in that. The EU’s requirements for financial services since June are no longer a differentiator anyway. And in a commodity market you may lose market share before a trust premium sets in, if it sets in at all.
What is hard to copy is not the surface but the calculation behind it. A bank that measures relationship quality instead of funnel conversion and pays its teams accordingly builds something a competitor cannot take over overnight, because it would cannibalise that competitor’s current revenue logic. The moat lies in the operating model. The lead is real, but slow. Whoever builds it is working with a different discount rate: less weight on the next quarter, more on the relationship that will still bring margin in ten years. The argument is a temporal one and therefore vulnerable as soon as the bank’s time horizon is too short.
Hao’s most important idea is the quieter one: instead of only asking what you want to prevent, say what vision you have and what technology supports it. For financial services the answer is more concrete than it sounds. We build products that steer the user – that is unavoidable – but in their favour and in a way they can follow. We measure operational success in short cycles and honestly, and treat the lifetime of the relationship as a goal, not as an excuse for missing numbers. And we admit that this path changes little in the existing base. It counts where inertia no longer protects: in growth.
Regulation is catching up, on defaults and on exit. It will raise the floor. The alternative Hao speaks of won’t come from regulation alone.
The empires optimise the moment; that is both their strength and their limit. Whoever bets on trust is making a sober wager on the longer time horizon. It is not risk-free. But it aims at what is hardest to extract.
Further reading:
