The $500 million invoice: how to stop LLM spend from running away

When the bill arrives

In May 2026, an AI consultant shared a story that made a lot of finance teams sit up. One of their clients had spent half a billion dollars on Claude in a single month. Not over a year. One month. The cause wasn't fraud or a pricing error. The company had handed out AI licenses to employees and forgotten to set any usage limits.

That number is large enough that you should treat it with some skepticism. The company is unnamed, and a single consultant's anecdote isn't an audited figure. But the pattern behind it is real, and it's showing up at companies whose numbers we can check.

Take Uber. It rolled out Claude Code to its engineers in December 2025. By February, usage had doubled as developers figured out what the tool could do on its own. By April, four months into the year, the company had burned through its entire 2026 AI coding budget. Monthly spend ran between $500 and $2,000 per engineer. Uber's COO went looking for the return on all that token consumption and couldn't find a matching jump in shipped features. The company now caps each engineer at $1,500 a month per tool.

Then there's Microsoft, which is about as close to the AI industry as a company can get. It gave thousands of engineers access to Claude Code in December 2025. Six months later it started cancelling those licenses inside the group that builds Windows, Office, Outlook and Teams, pushing people onto its own GitHub Copilot CLI instead. The official reason was "toolchain unification." The cancellation deadline landing on the last day of Microsoft's fiscal year tells a quieter story about cost.

Three different companies, same shape of problem. Give capable people a metered tool with no guardrails and spend climbs faster than anyone expects. Two things make it worse than a normal SaaS overrun. Agentic tools — the ones that run multi-step tasks by themselves — can consume roughly a thousand times more tokens than a single question. And a lot of that spend goes to low-value work. The consultant in the $500 million story noted employees using top-tier models to check the weather.

What Bedrock gives you out of the box

Amazon Bedrock is AWS's managed way to call foundation models, including the full Claude lineup. Because it sits inside your AWS account, it inherits the same billing, identity and budgeting machinery you already use for everything else. Four mechanisms matter most.

These four cover the "stop the bleeding" problem. They don't touch the more interesting question of whether each request is using the right model in the first place.

The default-to-expensive problem

Here's a scenario that plays out constantly. An employee opens an AI tool and sees a model picker. The options mean little to them. One is described as the most capable. So they pick that one, for everything — including tasks a far cheaper model would handle perfectly well.

The price gap is the part most people never see. On Bedrock today, Claude Opus runs about $5 per million input tokens and $25 per million output tokens. Claude Haiku is roughly $1 and $5. That's a 5× difference on both sides for the same request. Reformatting a list, drafting a short email, classifying a support ticket, summarizing a paragraph: none of these need a flagship model. But if the picker defaults to Opus and nobody knows the difference, you pay the 5× premium thousands of times a day.

So the real question isn't only "how do we cap spend." It's "how do we make sure each prompt lands on the cheapest model that can still do the job well, without asking every employee to become an expert in model selection."

You don't fix that with a training course. AWS will do the picking for you.

A deeper look at Bedrock Intelligent Prompt Routing

Bedrock Intelligent Prompt Routing is AWS's answer to the model-selection problem. Instead of pointing your application at a specific Claude model, you point it at a router. The router looks at each incoming prompt, predicts how well each model would answer it, and sends the request to the cheapest model that will still do the job. Your code calls one endpoint; AWS decides where each request actually lands.

How the routing decision is made

For every request, the router runs a quick prediction of the likely response quality from each model in the pair, before any of them actually run. A short, simple prompt looks like something the small model can handle, so it goes to the small model. A long, ambiguous, multi-step prompt looks like it needs the bigger model, so it gets routed up. The classifier doing this is lightweight, which is why it adds very little latency and, for many requests, actually lowers it by keeping easy work on the faster model.

A few design rules worth knowing:

• A router works across exactly two models from the same family — a smaller one and a larger one. For Claude, that's a pairing like Haiku and Sonnet.
• There's a fallback model for cases where the router isn't confident. By default that's Claude 3.5 Sonnet, so an uncertain call lands on the more capable option rather than the cheaper one. You're never trading quality for a guess.
• You can start with AWS's default router and change nothing, or define your own.

The one knob that matters: response quality difference

The setting worth understanding is the response quality difference threshold. It controls how much better the bigger model has to be before the router bothers sending the request up.

Think of it as a tolerance. Set the threshold to 10% and you're telling the router: only escalate to Sonnet if you predict its answer will be at least 10% better than Haiku's. If the gap is smaller than that, keep the request on Haiku and pocket the savings. A low threshold sends more traffic to the cheaper model. A high threshold is more cautious. You tune this against your own data until the quality and the bill both sit where you want them.

What it costs and what it saves

Routing itself is cheap — AWS charges about $1.00 per 1,000 routing requests, on top of the normal token cost of whichever model answers. Against that, the savings are large. AWS cites cost reductions of up to 30% with no drop in accuracy, and in its own internal test across hundreds of prompts with the Claude family, routing hit 60% savings while matching the response quality of Claude 3.5 Sonnet. Independent write-ups routing between Haiku and Sonnet have reported savings in the 48–56% range.

The reason it works is the shape of real traffic. Most prompts in a business are mundane, and the expensive flagship model is overkill for them. Routing quietly moves that bulk onto the cheap model and reserves the flagship for the requests that genuinely need it. The employee never sees a model picker. They ask a question and get a good answer; the system has already decided, per request, which model was worth paying for.

Where it fits, and where it doesn't

Prompt Routing is the cleanest fix for the high-volume, mixed-difficulty traffic that makes up most company usage: chat assistants, support tools, internal Q&A, document summarising. It is less suited to workloads that are uniformly hard — where almost everything would route up anyway — and to agentic coding tools, which often pin a specific model on purpose. For those, the budget, access and attribution controls above are doing the heavy lifting.