The Economics of Agent Optimization: From pilots to measurable returns

This blog post is the first of a four-part series called The Economics of Agent Optimization which shares the strategies, capabilities, and proof points to help you optimize agent costs and run AI as a managed investment system on Microsoft Foundry.

The AI conversation in most enterprises has moved from the whiteboard to the budget review. Two years ago, the question was whether AI could work. The question leaders are asking now is sharper and less comfortable: is it paying for itself?

For the teams now in production—including more than 100,000 organizations building on Microsoft Foundry that question has become urgent. Tokens have become the new unit of technology spend, and financial discipline (not model choice) is what decides whether a promising pilot ever scales. The money is already moving in: in a Microsoft-commissioned IDC study of more than 4,000 business leaders, 71% said they plan to increase AI budgets, funded from IT and non-IT sources alike. The budgets are growing. The question is whether the discipline grows with them. 

The teams pulling ahead did not go looking for a cheaper model. They stopped running AI as a string of one-off pilots and started running it as a managed investment system: every request sized to its job, every agent improved as it runs, and every dollar bounded and accounted for. That shift, from buying intelligence to managing it, is the whole game. This series is about how the system works and why Microsoft Foundry is built to run it.

Understand your AI costs and spending

Before you can manage AI spend, you need to understand what creates it. Cost is not determined only by the model you choose. It is also shaped by the application or agent built around that model.

Every request includes input tokens, such as system prompts, conversation history, tool definitions, and retrieved content, as well as output tokens generated by the model. Because models are stateless, the full context is sent with every request. Costs can increase over time even when the user asks only a simple follow-up question.

Agents introduce another layer of complexity. Instead of following a single path, an agent may evaluate options, retry actions, or call multiple tools before producing a response. A single user request can generate many model calls, making workflow design as important as model selection.

Improve AI cost visibility across teams

AI spend is difficult to manage when it appears as a single aggregate number. Teams need visibility into costs by application, agent, workflow, and model to understand what is driving usage and where optimization opportunities exist.

Without that level of attribution, it becomes difficult to explain costs, prioritize improvements, or measure the impact of optimization efforts.

Control and optimize spend

Visibility alone is not enough. AI workloads can scale quickly, and unexpected behavior can increase consumption in a short period of time. Organizations need controls that help manage spend before costs become a surprise.

Optimization also requires more than selecting a lower-cost model. Most AI workloads contain a mix of requests with different requirements. Better outcomes come from matching requests to the right models, reducing unnecessary context, limiting unneeded tool use, and improving agent workflows so they operate more efficiently.

Why Microsoft is the platform for AI FinOps

FinOps began as the discipline of bringing financial accountability to variable cloud spend, a shared operating model that puts engineering, finance, and product on one set of numbers. FinOps for AI comes down to four commitments:

1. Make AI predictable to fund
2. Efficient by design
3. Optimized at scale
4. Proven in value 

Microsoft’s answer is a single, first-party approach to FinOps for AI that spans the entire lifecycle—plan, build, manage, and measure. Cost visibility and control are built into the products teams already use: Microsoft Foundry and GitHub where agents are built and run, Microsoft Cost Management for allocation and chargeback, Azure pricing offers for commitment-based savings, and Azure API Management as the gateway that meters and governs AI traffic. Microsoft Agent 365 extends the same discipline to the tenant—unifying agent cost management across Microsoft and third-party platforms with spending policies, budget caps, and departmental chargeback in one place. Together they give organizations something no point tool can: comprehensive, best-in-class cost management across the whole AI estate, from the first prompt to the board-level ROI number.

Foundry is where that approach gets specific, because it’s where agents are run and optimized. It runs AI as a managed investment system across one closed loop: optimize each request at runtime, optimize each agent workflow over time, and govern the spend continuously.

AI cost optimization starts with visibility

A managed investment system makes three decisions, each at a different speed. You optimize the request in the moment it runs. You optimize the agent workflow over days and weeks, as you learn what works. And you govern the spend continuously, with limits and budgets that never sleep. Foundry is built to make all three. Each move has its own set of Foundry capabilities, and the map below shows how they fit together. 

Agent 365 will extend governance to the tenant, unifying cost management across Microsoft and third-party agents with spending policies, budget caps, and departmental chargeback. 

You can watch the runtime levers work live in our new Microsoft Mechanics episode on token economics.

The four questions AI leaders should be asking

If you take one thing from this post, take these four questions into your next AI or budget review. Each has a concrete answer in Foundry. If you cannot answer one today, that is where to start.

1. Do we know what we’re paying for?
   Spend should be visible by model, agent, and workflow, not hidden in a single invoice line. Foundry’s metering and traces make it easier to understand where costs originate.
2. Are we paying the right amount for each request?
   Most requests do not need a frontier model. Model router, deployment and pricing options, caching, fine-tuning, and Foundry IQ help match each request to the capability it needs.
3. Are our agents operating efficiently?
   Agent costs should improve over time as workflows become more effective. Agent optimizer and memory in Foundry Agent Service and Toolboxes in Foundry help reduce unnecessary token usage and improve execution quality.
4. Do our limits hold when usage spikes?
   Usage that expands rapidly needs controls that hold. Today, many teams put Azure API Management in front of their AI endpoints to enforce token rate limits and quotas at the AI Gateway layer. Native budgets and enforcement inside Foundry, plus tenant-wide controls through Agent 365, are where we are headed next.

The first question is about understanding AI spend. The next three are the areas this series explores in more detail: matching requests to the right models, improving agent efficiency, and applying governance controls to manage cost at scale.

Get started

This series will continue over the coming weeks, going one level deeper on each subsequent move: how to optimize the request at runtime, how to build agents that use tokens efficiently, and how to govern the spend as you scale. Each post pairs the thinking with the Foundry capabilities that make it real.

You don’t have to wait to start. The capabilities behind this framework are live in Microsoft Foundry today:

• Learn more about the ways to optimize model cost and performance in Microsoft Foundry.
• Watch the token economics episode on Microsoft Mechanics for a hands-on look at the levers in action.

Follow along as the series unfolds and bring the four questions to your next review.