When your workforce becomes the training data
Staff using AI at work are training the systems that may replace them. Why the bargain deserves daylight and what workers and employers should do now.
AI adoption at work gets described as a one-way transaction. The business buys the tools, staff use them to get more done, and everyone argues about how the productivity gains should be shared. That story is true as far as it goes, but it’s half the ledger. The traffic runs in the other direction too, because every prompt written, every machine draft corrected, every process demonstrated to software capable of learning from it is a lesson — and a growing class of workplace systems is built, explicitly, to take lessons. Staff aren’t just using these tools — they’re training them, on judgement that took careers to build.
I don’t think the teaching should stop. I’ve argued at length that written-down operating knowledge is the most durable AI asset a business has. Handing that knowledge to agents is precisely what turns them from generic assistants into something worth paying for. What bothers me is the silence around it. When the systems being trained can eventually stand in for the people training them, the rollout memo’s talk of productivity tools is really describing a transfer of capability from the workforce to the employer, and a transfer of that size deserves to be discussed like one.
The capture is by design
There’s no small print to squint at here; the vendors put it in the brochure. Cresta, one of the larger contact-centre AI companies, sells real-time agent guidance built on models trained on your own conversations, learning from your top performers so that everyone else can be nudged towards what your best people already do. Microsoft’s Copilot Tuning, announced at its Build conference last year, lets an organisation fine-tune models on its own contracts, tickets and past proposals so that agents absorb the company’s tone, vocabulary and subject matter, with no data-science team required. Task-mining vendors go a layer deeper still, installing software on employee desktops that records clicks, keystrokes and screenshots to map how the work actually gets done, and that map is increasingly pitched as the starting point for agents that will do the work instead. The mechanics differ: some of this is training in the strict sense, some is material fetched for context while the system works, some is straightforward recording. Every version, though, moves working knowledge from people into systems the employer, or its vendor, controls.
Notice who’s worth the most in that arrangement. A system that learns from demonstration values its best teachers — the support engineer whose case notes actually resolve things, the salesperson whose call transcripts show how objections get handled. The better your work, the more completely it can be absorbed.
What it looks like at full scale
Salesforce is the clearest case we have, partly because its chief executive has been unusually willing to say the numbers out loud. Agentforce, the company’s AI agent platform, was grounded in material its own support organisation had spent years producing — case histories, help content, the accumulated record of how customer problems get solved — and by late 2025 it was handling roughly half of the company’s customer conversations. Marc Benioff told an interviewer in September 2025 that he had cut the support team “from 9,000 heads to about 5,000, because I need less heads”. Salesforce says hundreds of those people were redeployed into sales and professional services. Set the two facts side by side, though: the support team’s accumulated work made the system competent, and that competence, by the chief executive’s own account, is why four thousand of them were no longer needed.
To Salesforce’s credit, none of this was hidden: the numbers came from the chief executive, on the record. Most businesses adopting similar tools will run the same trade at smaller scale and never mention it. The capture simply happens in the defaults of software bought for other reasons.
Consent is thinner at work than anywhere else
Creative workers saw this coming and fought it. The Hollywood writers’ strike of 2023 ended with a contract that restricts what studios can do with AI around writers’ material and reserves the Guild’s right to challenge its use as training data, terms that have become a template for other creative fields. Employees in ordinary jobs stand on much weaker ground. Legally, your work product belongs to your employer in most circumstances — the case notes, the documents, the call recordings were never yours to withhold.
But the intellectual-property clause in an employment contract was drafted for a world of patents, copyright and confidential files, where what the employer kept was a thing you made. What’s being captured now is closer to the pattern of your judgement — how you decide, phrase, prioritise and recover — and no contract I’ve ever signed described that as a deliverable. The law will take years to catch up with the difference. And the usual instrument for situations like this, consent, works badly inside an employment relationship anyway; data-protection regulators — the UK’s Information Commissioner among them — have said for years that consent is rarely freely given when the person asking controls your livelihood, which is why “we asked and they clicked yes” means so little at work.
The union movement has started catching up. By one recent count, AI provisions now sit in collective agreements covering more than four million American workers, including notice requirements, bargaining before deployment and, in some contracts, consent rights over training data. That still leaves the large majority of workers everywhere facing the bargain as a software default. And an after-the-fact settling-up is unlikely ever to work, because your corrections dissolve into an aggregate and what the aggregate owes any one person may have no clean answer. That’s why the conversation has to happen upstream, before the teaching.
What you can do about it
None of this means the right response is refusal, because refusal mostly isn’t available, and hoarding your know-how makes you worse at your job while the capture carries on regardless. The realistic moves are smaller and more useful:
- Ask what’s captured: When a tool is rolled out, ask three questions: what does this system retain, what is it used to train or improve, and who benefits from that improvement? You may not love the answers, but knowing them changes how deliberately you can act.
- Teach with your eyes open: Using AI to tidy a draft email is a different act from demonstrating, step by step, the judgement that makes you hard to replace. Both can be worth doing, but know which one you’re doing at any given moment. If your employer has mandated the tools, adopt them strategically rather than compliantly.
- Put your teaching on the record: If a system has improved partly because you corrected it, that’s a contribution. Keep examples, and raise them where contributions get valued — reviews, pay conversations, and the discussion about what your role becomes next.
- Keep building what doesn’t transfer: Relationships, accountability, and judgement in situations the system has never seen. The corpus absorbs how you handled yesterday’s cases; what you’ll do with tomorrow’s is still yours.
What an honest employer does
The employer’s side of the bargain is simpler to state. Tell people what the tools capture and what the captured material trains, before rollout and in plain language rather than in clause fourteen of an acceptable-use policy. Check what the vendor’s terms allow it to keep or reuse, and consult the people affected before the switch is flipped rather than after. Treat the people whose work trains the systems as contributors to the asset being built, because that’s what they are: if a team’s accumulated expertise halves the headcount a function needs, the roles that remain should be visibly better — better paid, broader in scope — and any redeployment promise should come with names and dates attached rather than living in a press release. Where genuine choice is possible, offer it, and where it isn’t, say so honestly instead of dressing a default up as consent.
Most employers aren’t running a scheme, in my experience. They’re buying tools whose defaults do the capturing for them, and the silence is inherited rather than chosen. It only becomes a choice once you’ve understood that your workforce is teaching these tools while it uses them.
The systems will keep learning; that part isn’t in anyone’s gift to stop. What we can decide is whether the teaching happens in the open — with the people doing it aware they’re doing it, and the businesses benefiting from it saying so. A bargain both sides can see is rare in this technology cycle, but it’s available to any business willing to state plainly what its tools are learning, and to any worker willing to ask.
