Take the weakest idea you’ve had this quarter and pitch it to your AI assistant as though you’re proud of it. Unless you’ve changed the defaults, there’s a fair chance you’ll get back a sharpened version of the plan and an offer to draft the announcement — everything except the thing a good adviser would have led with, which is a reason to stop. I’ve run that experiment a few times with ideas I already knew were flawed, and the enthusiasm coming back is remarkably consistent.

The behaviour has a name — sycophancy — and it flows directly from how these tools are made. The big general-purpose assistants are consumer products first, tuned on millions of thumbs-up ratings from people who, on average, prefer to be agreed with. What you’ve inherited is a personality optimised for somebody else’s evenings: closer to a companion than a colleague, pleasant and supportive and therapy-adjacent by design. An adviser you’d trust with a real business decision is a fundamentally different product, and nobody ships one by default. The good news is that the tuning sits close enough to the surface for a determined operator to override most of it. But you’ll need to do it deliberately, because the default won’t drift your way on its own.

Trained to be liked

The mechanics are worth understanding, because they explain why the problem keeps returning no matter how many times the labs apologise for it. Modern assistants learn their manners through reinforcement learning from human feedback: the model produces answers, people rate them, and the model is trained towards whatever earns the higher rating. The trouble is what actually wins that rating. Anthropic’s researchers studied this pipeline in 2023 and found that the people doing the rating picked convincingly written flattery over the correct answer often enough for the lesson to stick; the automated graders trained on those ratings inherited the same taste. A model optimised against judges like that learns the obvious strategy: agree, sound confident, reassure, complete the task. All five of the leading assistants the researchers tested had learned it, and they’d learned a related trick too: challenge a correct answer and the assistant will often abandon it just to keep you happy.

You don’t need the research to see the pull, because the product history of the past eighteen months has played it out in public. OpenAI shipped a GPT-4o update in April 2025 so eager to please that it validated plainly bad and even delusional ideas, and rolled it back within days. When GPT-5 arrived that August with a cooler, more matter-of-fact persona, enough users grieved for the warmth of the model it replaced that the old one was reinstated and the new one made friendlier. In March 2026 the pendulum swung back once more, with GPT-5.3 Instant’s deliberate tone reset landing to cheers from users tired of being soothed and lectured. Your business adviser’s personality is being A/B tested, live, against the preferences of a consumer audience that mostly wants a companion.

The cost of choosing warmth is measurable. Oxford researchers showed this year that when they trained a range of models to be warmer and more empathetic, error rates rose by 10 to 30 percentage points, and the warm models became roughly 40 per cent more likely to reinforce a user’s incorrect beliefs, most sharply when the user sounded sad. In the models they tested, warmth and reliability pulled against each other, and a consumer product facing a consumer market will keep picking warmth. A business at least needs to know that trade is being made on its behalf.

An old problem at new scale

Businesses have never needed software to produce agreement — we’ve always managed that with people. Every leader who’s built a team knows how easily a room fills with yes, and how expensive that yes becomes when it’s wrong: the flawed acquisition nobody questioned, or the product the whole meeting knew was late except the person who asked. For the seventeen years I ran my agency, the hardest discipline in the building was keeping people willing to tell me things I didn’t want to hear. What AI changes is the scale and the availability of the agreement. The yes-man used to be limited by headcount and courage; now every manager with a subscription has one on tap around the clock, and it never gets tired, never worries about its job, and never softens you up for bad news because it never delivers any.

That matters most at the exact moments the tools are most useful — decisions under uncertainty, where you’re reaching for a second opinion precisely because your own conviction is shaky. Pitch a ten per cent price cut to an untuned assistant and what usually comes back is a tidier version of your own reasoning, not the churn risk you didn’t mention or the competitor response you haven’t gamed out. An assistant that mirrors your framing back with added confidence will firm up positions that deserved to stay soft, at the very moment softness was the most useful thing about them. And the accountability doesn’t move an inch. As I argued when a British minister resigned over her tax affairs, “I was just following advice” has never been an excuse — the decision stays yours regardless of what the adviser said, and that holds whether the adviser bills by the hour or by the token.

The playbook

None of what follows requires anything more technical than writing instructions down and paying attention to the answers.

  1. License the pushback in writing. Every serious AI tool now supports standing instructions — custom instructions, system prompts, project notes. Use them to make disagreement part of the job description: “When I propose a plan, give me the strongest case against it before any case for it. If my framing contains an assumption you can’t verify, say so. Don’t soften bad news.” Because assistants fold under challenge, add a line instructing it to hold its position unless you produce new evidence. Be clear-eyed about the limits, though — nobody yet knows how far instructions can override what’s trained into the weights, and my experience is that you’re managing the tendency rather than curing it. Which is exactly why the next step exists.

  2. Evaluate for honesty, not satisfaction. Most of us assess AI output by how good it feels, which is the same metric that got the labs into trouble. Test your setup the way you’d test a new hire’s judgement, and keep score. Four checks cover most of it: does it correct you when you state something false, does it surface the assumption you left unstated, does it hold a correct position when you push back, and does it admit it doesn’t know when it can’t know. Feed it decisions you got wrong and see whether it catches them; pitch it plans with a known flaw buried inside. I’ve written about mapping where AI is brilliant and where it falls over, and honesty deserves a row on that map alongside competence. If your setup fails more than one of the four, retune before trusting it with anything that matters — an assistant that has never once disagreed with you is failing the whole test, however good its output feels.

  3. Separate the assistant from the adviser. Drafting an email wants a cooperative tool that takes your intent and runs; stress-testing an acquisition wants an adversarial one with no investment in your mood. Trouble arrives when one configuration does both jobs, because the cooperative register bleeds into the judgement work. Keep two setups — different projects, different standing instructions, or a ritual prompt that switches the mode — and make the adversarial one mandatory for any decision that would be expensive to reverse, with the final call staying human. If you only ever open the friendly one, that’s a tell about what you’re really asking for.

The prize for getting this right is the thing everyone claims to want from AI: a tool you can trust near real decisions, rather than a very expensive yes-man. The vendors will keep tuning their defaults towards whichever audience shouts loudest, and the loudest audience wants a friend, so a leader who wants the truth has to build for it, the same as ever, whether the advisers are people or software. Ask yours for an opponent instead. An assistant that never disagrees with you isn’t intelligent — it’s flattering you, and flattery doesn’t scale.