Most AI projects get budgeted like software projects. Scope the build, agree a number, deliver it. Then it goes live and starts costing you money every time somebody uses it.
An agent is metered. You pay per token, so the bill tracks adoption, which means a rollout people actually use costs more than one they ignore. That inverts how most of us are used to thinking about a delivered system.
We have seen environments land at roughly ten times their planned monthly spend, with nothing broken and nobody at fault. There was no ceiling in the subscription, and in most environments we inherit, watching the run-rate is nobody’s job. That is the default worth designing against.
The question to be able to answer at sign-off is a simple one: what does this cost us next month?
The cap you assume exists
The obvious fix is a hard spend limit. Cap it at X, and at X it stops.
Azure will not do that on pay-as-you-go AI resources. You can set a budget, and it will tell you where you are, but it does not stop consumption. Nothing switches off when you hit 100%. That is how the platform works, so treat it as a constraint to design around rather than a setting somebody forgot.
You can wire a budget alert into an action group that runs automation to disable keys or stop the resource, and people do. You are still building the cap yourself, which is the point.
Put a gateway in front of the model
The most useful thing we changed is that we no longer let the application talk to the model. Every AI workload we build calls an API gateway, and the gateway calls the model. On Azure that is API Management, where Microsoft now ships this as the AI gateway with a published reference architecture, so it is a platform pattern rather than a house opinion.
That one hop is the real shield against runaway cost, for three reasons.
You can change the model without touching the application. If a model gets expensive, or a cheaper one becomes good enough for the job, you reroute at the gateway and the app never knows. You can send the routine traffic to a small model and keep the expensive one for the work that needs it. Wire the model endpoint straight into your solution instead and every one of those decisions turns into a code change, a test cycle and a release, which in practice means nobody makes them. API Management will also front several providers behind one OpenAI-compatible endpoint and handle the format translation, which is the difference between swapping a model and rewriting an integration.
You get an actual kill switch. The llm-token-limit policy enforces a tokens-per-minute ceiling or a longer quota, daily through to yearly, keyed on whatever you choose: subscription key, caller IP, or a policy expression. It will also pre-calculate prompt tokens and refuse a request that already exceeds the limit before it reaches the model, so you are not billed for the call you rejected. That is the hard cap the platform will not sell you, built at the one layer where you own the traffic. When an alert fires on a Saturday, throttling at the gateway is a config change instead of a deployment.
You get to see the traffic. This is the part people are usually surprised by. Azure will tell you how many tokens you burned and what they cost, but by default it does not hand you the content of the requests and responses going to the model. You get usage statistics, not traffic. So if you want to know what was actually asked, keep a record for audit, notice a caller behaving oddly, or work out which prompt pattern is eating the budget, you have to capture it before it reaches the model. The gateway is where that happens: llm-emit-token-metric pushes token counts into Application Insights with whatever dimensions you define, per consumer or per API or per user, and API Management will log prompts and completions to Azure Monitor. It is also where the model keys stay, so your application holds a gateway credential and never the key itself.
Response caching sits there too, via llm-semantic-cache-lookup and llm-semantic-cache-store, which match on vector proximity so a near-identical prompt is served from cache instead of billed again. That takes cost out directly rather than just making it visible.
What we run alongside it
Four controls go onto every AI environment we deliver.
Budget per workload, not per subscription. Most subscriptions end up carrying more than one AI workload. A single combined budget tells you the number is moving without telling you which one is moving it. Split them and a spike points you somewhere.
Set those budgets on actual cost. Forecast alerts fire on a projection, they are noisy, and people quickly learn to ignore them. An actual-cost alert means the money is already gone, so when one lands it gets read. We start the limits deliberately low, then raise them once there is a real run-rate to argue from.
Every alert goes to the client and to us at the same time. Their IT and finance contacts, our delivery team, same email. An alert that lands with one person is an alert one person gets to quietly decide is fine. On busier workloads we also route breaches to an Azure Monitor action group so the engineer who owns the environment hears about it separately from the general notice.
Run health alerting next to the cost alerting. Exceptions, latency, availability, and an API ping test. This is the one people skip and it has saved us the most. Runaway spend usually is not people using the product too much. It is a retry loop, a stuck job, or an integration hammering the model in the background. Your cost alert tells you the bill moved. Your latency and exception alerts will usually have told you why, several hours earlier.
Then there is the part you cannot automate. When a threshold breaks, a named engineer looks at consumption and throttles or pauses the traffic. That response is what turns all of the above into a control rather than a dashboard. It works only if the name on it is a real person who knows the job is theirs.
Does it hold up?
Nothing has breached since the controls went in. The cost chart is boring, which is the whole ambition with a run-rate.
What to ask before you sign
A previous First Digital article in May argued that most agentic AI projects will be quietly cancelled. In my experience the model is rarely what kills them. They die because nobody can answer what this costs us next month, and a question the sponsor cannot answer is very easy to defund.
So get the answer early. Ask what sits between the application and the model, and what it would take to swap the model or throttle it. Ask who receives the threshold alert and who has the authority to pull the plug. A shrug on either of those is the real risk in the project.
Get the gateway and the alerting in at design time, not after the first surprising invoice. Then a busy month is something you already warned the sponsor about, and you get to keep building.
