I’ve started doing something slightly childish whenever a product describes itself as “AI-powered”. I picture the same box on the shelf with the sticker peeled off, and ask what’s left. My toaster could carry the label tomorrow and it would still burn the toast, and a surprising number of the business tools pitched to me this year would fare no better.

AI-washing is greenwashing’s younger sibling, and it’s older than ChatGPT. Back in 2019 the venture firm MMC Ventures looked at 2,830 European start-ups classified as AI companies and found no evidence that AI was an important part of the product in 40% of them. I wouldn’t like to guess what the figure is now, but I’d be surprised if it had gone down, because the pressure to use the word has only grown. Investors ask about the AI roadmap in the first meeting and a competitor’s press release rattles the sales team, while “AI” itself is vague enough to cover anything from a large language model to a set of if-then rules somebody wrote in 2017.

For most of us, the moment that matters is sitting across from a vendor with a single feature on the screen and a decision to make about it. (Whether a whole company has really changed calls for a different test.) Getting that moment wrong rarely makes headlines; it just means paying a premium for automation you already had, or letting unreliable output into a workflow your team then learns not to trust.

The most useful thing you can ask for is the toaster test made formal. Run the same task on your own data, once with the AI switched on and once with it switched off, and judge both against a measure you agree in advance — time saved, corrections needed, how often the answer was right. A feature that depends on a model should beat its own baseline by a margin you can see, while a rules engine wearing a new badge will score much the same either way. If a live comparison isn’t possible for security or contractual reasons, that’s fair enough, but then ask for a documented benchmark, a customer you can call or a short controlled trial. A vendor who can offer none of those is asking you to buy the sticker.

Two follow-up questions do most of the remaining work. Ask which model sits underneath and what data it draws on to produce its answers, then ask separately what happens to your data: where it’s stored, how long it’s kept and whether it’s used for training. If they can’t answer, they either don’t know or would rather you didn’t. Then ask what happens when it’s wrong. AI systems get things wrong some of the time, and the vendors worth buying from have planned for it, so they’ll show you how uncertain answers are flagged, where a person reviews the output and how easily you can correct it. A product that claims never to make mistakes is either not doing anything difficult or not being straight with you.

Regulators are asking the same questions, which turns an unsupported claim from a sales irritation into a due-diligence warning. In 2024 the FTC alleged that DoNotPay, which billed itself as “the world’s first robot lawyer”, had never tested whether its chatbot performed like a human lawyer, and the final order in February 2025 required $193,000 in monetary relief and barred the company from claiming lawyer-level performance without evidence. When a pitch leans on words like “human-level” or “accurate”, you’re entitled to see what sits behind them.

The people I feel for are the builders doing it properly.

When every product carries the same label, buyers learn to discount the word entirely, and the start-up with a useful model gets marked down alongside the toaster. Their best defence is to run the test on themselves before a buyer does, leading with the side-by-side and publishing the error rate, which is the same advice I’d give any small firm tempted to overclaim on its own shopfront.

The word “AI” has stopped telling anyone much by itself, so whether we’re buying or selling, the useful question is what changes when it’s switched off.