Ninety-eight percent sounds brilliant. It sounds like the kind of number you’d put in a board deck with a green traffic light next to it. And if you’re running a single AI call — one question in, one answer out — it probably is fine.

But that’s not how agents work. Agents chain things together. Intent parsing, retrieval, planning, tool calls, validation, formatting — six, eight, ten steps in sequence, each with its own failure probability. And those probabilities compound. Several of those steps — retrieval and intent parsing especially — live or die on how well you’ve engineered the context feeding the model.

Andrej Karpathy has a framework for this that I keep coming back to. He calls it the march of nines — the exponentially harder journey from 90% reliability to 99% to 99.9% to 99.99%. “Every single nine is the same amount of work,” he told Dwarkesh Patel, drawing on his five years leading autopilot at Tesla. They got through maybe two or three nines in that time. Multiple nines of iteration were still ahead of them.

The concept comes from infrastructure engineering, where five nines (99.999%) uptime has been the gold standard for decades. AWS, Azure, every serious cloud provider measures itself this way. But AI teams tend to think in accuracy percentages, not reliability nines, and the gap between those two mental models is where production failures live.

Why 98% is actually terrible

Run the numbers on a ten-step workflow at 98% per step and your end-to-end success rate drops to roughly 82%. One in five workflows fails. At 95% per step, you're down to about 60%. That's a coin flip masquerading as a production system.

I’ve watched this play out with my own agent setups. You build something, test it, it works beautifully twenty times in a row. Then you deploy it and within a day you’re troubleshooting a failure that turns out to be step six timing out because step four returned something slightly unexpected. The individual steps all look solid. The chain is where it falls apart.

And it gets worse. Correlated failures — the kind that don’t show up in unit tests — can take out entire workflow categories at once. A shared authentication service goes down. A rate limit hits multiple tool calls simultaneously. A retrieval index goes stale and nobody notices for a week. The independence assumption behind the maths is generous. Real production environments are messier. And the reflex to fix all this by adding more agents tends to deepen the hole rather than fill it — there’s a swarm tax to pay, and it multiplies the failure surface instead of shrinking it.

The cost of each nine

Karpathy's insight, that each nine costs roughly the same engineering effort as the last, is counterintuitive but matches what anyone who's shipped reliable software already knows.

Getting from nothing to 90% — the first nine — is achievable with a strong demo. Good prompts, a capable model, some basic tool integration. This is where most proof-of-concept projects live, and it’s where most enterprise AI deployments are right now. I wrote recently about where most businesses actually sit on the agentic maturity scale, and the honest answer is level one or two. That maps almost exactly onto the first nine.

Reaching 99% demands actual systems engineering. Structured outputs with schema validation. Retry logic with backoff and jitter. Timeouts on every external call. Error classification so you know which failures are transient and which are structural. “It works on my laptop” dies here.

At 99.9%, you’re into observability and operational rigour. Tracing every step. Building golden test sets from production failures. Running shadow deployments. Canary releases with automatic rollback. None of this is glamorous and none of it makes product announcements, but it’s what determines whether the system is trustworthy at scale.

Beyond that, most organisations discover they need to fundamentally rethink their architecture. Risk-based routing, where high-stakes actions get more verification than low-stakes ones. Human-in-the-loop approval for irreversible operations. Deterministic fallbacks that keep the system safe when the model fails. Resumable workflows with idempotency keys so a retry doesn’t create duplicate actions.

Each step up the nines ladder requires a different kind of investment — not just more of the same engineering, but qualitatively different disciplines. And I think that’s what catches teams out. They budget for “making it more reliable” as if it’s a linear effort, when it’s actually a series of phase transitions.

Where most businesses actually are

Be honest: most enterprise AI deployments are somewhere between 90% and 95% reliable. That first nine, maybe nudging into the second. And for internal tools where a failure means someone clicks “retry” and moves on, that might be acceptable.

But the ambition is running ahead of the infrastructure. The Celonis 2026 Process Optimization Report found that 85% of enterprises want to become an “agentic enterprise” within three years, while 76% admit their operations can’t support it. That gap is, in large part, a reliability gap. Organisations want agents handling customer interactions, financial transactions, compliance workflows — but they’re building on foundations that would embarrass a traditional SRE team.

McKinsey’s 2025 global survey puts numbers to the damage. Fifty-one per cent of organisations using AI have experienced at least one negative outcome, and nearly a third report consequences tied directly to AI inaccuracy. All of this at the first or second nine. And accuracy isn’t the only failure mode — agents optimising the wrong metric can produce technically correct results that still miss the point entirely.

A Princeton group called Normal Technology adds something worth sitting with. Their paper, “Towards a Science of AI Agent Reliability,” tested 14 models across 18 months of releases and found that bigger models aren’t uniformly more reliable. Scaling up improves some dimensions like calibration, but can actually hurt consistency. Models with richer behavioural repertoires sometimes show more run-to-run variability. Which is… not ideal, if your reliability strategy is “wait for GPT-6.”

What this means in practice

None of this is an argument against deploying agents. It's an argument for being ruthlessly honest about where you're at on the journey and what it'll cost to get where you need to be.

Start by matching the nines to the stakes. An internal knowledge-search agent that occasionally returns a less-than-ideal result? Two nines is probably fine. An agent processing customer refunds or submitting regulatory filings? You need at least three, probably four. Most teams apply the same reliability standard everywhere, which means either over-engineering the trivial stuff or under-engineering the critical stuff. Neither is a good use of anyone’s time.

Then measure what actually matters: end-to-end completion, not per-step accuracy. Per-step metrics are seductive because they almost always look good. But the number that counts is the one the user experiences — did the full workflow complete successfully? Instrument for that from day one, not as an afterthought.

The infrastructure work itself isn’t novel, and I think that’s the part that’s easy to miss. Timeouts, circuit breakers, retry budgets, idempotency keys, canary deployments — these patterns exist because distributed systems have the exact same reliability problems. AI agents hit every one of them plus a few new ones. Teams that treat agent infrastructure as a greenfield AI problem instead of a distributed systems problem are learning expensive lessons.

And build the fallback path before you need it. I keep coming back to the autonomy slider concept — dial agent autonomy up or down depending on confidence and risk. Default to the safe path. Require confirmation for anything irreversible. Make it trivially easy to flip the system into supervised mode when something goes wrong. The organisations that skip this step are the ones generating the incident reports in McKinsey’s survey.

Proving it

Karpathy learned this at Tesla over five years and multiple nines of iteration. The rest of us are learning it now, compressed into months as agent deployments accelerate.

The insight that sticks with me is this: the distance between a demo and production isn’t a straight line measured in engineering hours. It’s a logarithmic curve where each increment of reliability costs as much as everything that came before it. Most teams underestimate that curve by an order of magnitude because the demo looked so good. And fair enough — demos are designed to look good. That’s the whole point. But the demo is the first nine. You’ve got at least two or three more ahead of you, and each one is a different kind of hard.

The nines will arrive. They always do, for technologies that matter. But they arrive through disciplined engineering — bounded workflows, strict interfaces, resilient dependencies, operational learning loops. Not through wishful thinking about the next model release. It’s the same pattern that shows up in how the AI 2027 forecasts actually held up — the direction of travel was right, but the optimistic timelines slipped.

If you’re deploying agents into production, the question isn’t whether your system works. The question is how many nines you can prove.

Quick answers

What is Karpathy's march of nines?

A framework describing how each additional “nine” of reliability — 90% to 99% to 99.9% — requires roughly the same engineering effort as the previous one. Karpathy developed the thinking during five years at Tesla, where multiple nines of iteration on autopilot still left further nines ahead. It explains why demos that work 90% of the time are nowhere near production-ready.

Why does 98% AI accuracy fail in multi-step agent workflows?

Because failure probabilities compound across steps. A ten-step workflow where each step succeeds 98% of the time has only about 82% end-to-end reliability. At 95% per step, you’re down to roughly 60%. Correlated failures from shared dependencies make the real numbers even worse.

How reliable do AI agents need to be for production use?

It depends on the stakes. Internal tools can often get by with two nines (99%). Customer-facing or financial workflows need three or four nines (99.9%+). The key is matching reliability targets to business impact rather than applying a single standard across all use cases.

Can you buy reliability by using a bigger AI model?

Not straightforwardly. Research from Normal Technology found that larger models improve on some reliability dimensions but can actually decrease consistency. Each nine requires different engineering disciplines — schema validation, observability, fallback paths, operational rigour — not just a more capable model.

What’s the difference between AI accuracy and AI reliability?

Accuracy measures whether individual outputs are correct. Reliability measures whether full workflows complete successfully end-to-end, consistently, under real-world conditions. You can have high per-step accuracy and still have low system reliability once you chain multiple steps together.