We've written before about FinOps as a survival skill. That piece assumed your cloud bill was made of familiar things: compute, storage, data transfer, maybe a managed database having a bad month.
Then AI showed up with two new line items, and both of them misbehave.
Why your FinOps playbook misses AI
The first problem is shape. Traditional workloads are reasonably steady, which is exactly what rightsizing and reservations depend on. AI workloads spike. A training run devours GPU hours for a week and vanishes. An inference feature launches, gets picked up by one enthusiastic customer, and triples its costs without a single deployment.
The second problem is visibility. Token-based pricing from model APIs doesn't show up as infrastructure at all. It's a SaaS-looking charge that scales like compute, and it hides in a corner of the bill your dashboards weren't built to watch. We've seen teams with genuinely mature tagging discipline discover that their fastest-growing cost had no tag, no owner, and no budget, because it was an API key on a corporate card.
That's the third problem, actually: shadow AI. Engineers experiment. Product managers experiment. Someone's experiment quietly becomes a dependency.
Unit economics or nothing
Here's the test we apply before anything else: can you say what one of your AI features costs? One inference, one processed document, one assisted conversation. A number, not a shrug.
If you can't, every other conversation about AI cost is guesswork. The total bill going up tells you nothing by itself; maybe usage doubled and the business is thrilled. Cost per unit going up is the signal that something needs attention. Getting to that number takes unglamorous work, tagging, metering, a little pipeline glue. It's worth more than any dashboard you can buy.
The GPU trap
The instinct, inherited from classic FinOps, is to commit early: reserve the capacity, lock in the discount. With GPUs that instinct is expensive. Teams commit to hardware before they've profiled the workload, then discover the model runs fine on something half the size, or that the whole thing belongs on a serverless inference service that bills by the request.
Profile first. Then commit. And revisit model choice before hardware choice: routing the easy 80% of requests to a smaller, cheaper model and saving the big one for the hard 20% routinely cuts inference bills by more than any instance-family optimization ever will.
Tokens are the new data transfer
Small per-unit price, colossal aggregate, mostly invisible. Sound familiar?
The waste here is usually in the prompts. Whole documents stuffed into context when a summary would do. Conversation history dragged along forever. System prompts that grew for a year and were never trimmed. Prompt hygiene is not a glamorous discipline, but we've watched it take double-digit percentages off inference costs without touching a line of product logic. Add caching for repeated questions and the number gets better again.
If your agents can act, give them allowances
One more, connecting to something we've covered separately: as soon as AI agents can take actions, they can take expensive ones repeatedly. A stuck loop doesn't get bored or go home. Budget ceilings per agent, per day, enforced by the platform rather than the agent's own good judgment, belong in the first deployment, not the retrospective.
The first 30 days
Keep it small. Find every AI-related charge, including the API subscriptions hiding outside the cloud bill, and tag it with an owner. Stand up one view where GPU spend and token spend appear side by side. Pick one production feature and compute its unit cost. Set alerts at thresholds that would actually make you wince. That's it. A month of that beats a quarter of tool evaluation.
The honest take
AI spend isn't the problem. Unmeasured AI spend is. Some of the highest-return money your company spends this year may well go to GPUs and tokens. The teams that win aren't the ones spending least; they're the ones who can say what they're getting for it, per unit, per feature, per customer, and who find out about surprises from an alert instead of an invoice.