Transformation begins at the line between an experiment that works and a capability the business depends on. Scaling intelligence requires production capabilities built for sustained business performance, with investments prioritized against financial thresholds, tied to measurable outcomes, instrumented for unit costs, and owned by accountable leaders. This approach manages cost, performance, governance, and reliability as adoption grows.
A new IDC InfoBrief, sponsored by Microsoft, Scaling AI Without Scaling Costs*, on how organizations are funding and governing AI, shows both the urgency of that shift and the friction holding it back.
Nearly 90% of organizations apply financial thresholds before
Scaling AI Without Scaling Costs
approving AI initiatives.
Only 17% of organizations build models to project how costs scale in production—even as about 78% prioritize cost efficiency when selecting a model. 86% expect financial results within six to 24 months—roughly one budget cycle to show something real. Together, those findings point to a new operating reality: AI must be selected, architected, and managed as a continuous business discipline, not a series of isolated projects. The market has moved from proving it works to proving it can scale economically.
Four questions to answer before your next funding decision
Run each candidate through four questions that follow a scaled workload’s life: fund it, run it, improve it, prove it.
1. How can we predict and track costs at scale?
A pilot proves the idea, rarely the economics: only 17% of organizations model how costs scale in production. Before you fund, project usage at 3 times and 10 times, and make sure spend traces to an application, team, and agent. Compare models on cost, quality, and latency, and commit steady demand to reserved capacity. If cost rises with usage while value flattens, you’ve found an architecture problem while it’s still cheap to fix.
2. How can we optimize model costs for every request?
One successful outcome can take a dozen model requests, so what matters is cost per outcome, not price per token. Ask:
- Is routine work like classification paying frontier-model prices?
- Are the same instructions reprocessed on every turn?
- Is work that can wait running at a lower-cost tier?
The goal is the right model for every request, with evaluation proving quality held.
3. How can we continuously improve agent performance and costs?
Retrieved documents, tool definitions, instructions, and history are often an agent’s largest cost and a common cause of weak answers. Check:
- Does the agent retrieve the relevant passage or the whole document?
- Does it carry every tool description on every turn?
- Is the same procedure copied into every agent?
- Does it replay full transcripts instead of remembering what matters?
Trimming what the model doesn’t need lowers cost without hurting quality.
4. How can we control spending and measure ROI?
A budget alert reports the fire after it starts; a runaway agent needs a circuit breaker on every request. Cost must tie to the outcome the business pays for, such as a resolved case, so leaders can optimize, scale, or retire an agent. The business owner also needs an evaluation suite, cost and latency budgets, a rollback path, and regular reviews. With 86% of organizations expecting financial results within six to 24 months, that evidence has about one budget cycle to appear.
All four answers depend on evidence captured where AI runs: per request, agent, and outcome. If each team builds it by hand, the discipline won’t survive the tenth agent. It has to come from the platform.
How Microsoft Foundry supports continuous cost optimization
When customers ask for a few practical ways to scale AI while controlling costs, I recommend:
- Use model routing across the over 11,000 models available in Microsoft Foundry.
- Implement Microsoft IQ to improve relevance and reduce costs.
- Use the plan, control, and optimize capabilities described below to strengthen operations across the organization.
Microsoft Foundry is the enterprise platform to build, ground, and govern AI apps and agents at scale. It brings together your full agent lifecycle with open development, built-in intelligence, and consistent security, compliance, and policy controls across every agent. Think of it as a repeatable production line rather than a set of one-off builds, where the same controls, evidence, and economics apply to the tenth agent as to the first.
This is what makes the operating loop practical at scale. Across the loop:
Plan: Compare models on cost, quality, and performance before committing to an architecture, and match predictable baseline demand to reserved capacity while keeping the flexibility to burst. The decisions that set your production cost curve get made with data rather than intuition.
Control: Observability that reports cost alongside quality, latency, and safety in one view, so the economics of a workload are visible while it’s running rather than reconstructed at quarter-end. That’s what makes cost per outcome a number you can put in front of your CFO.
Optimize: Model routing that sends each request to the smallest model that can serve it, an intelligence layer that provides context and retrieval centrally and addresses a common source of avoidable token spend, and evaluation tooling that catches quality regression when you make a cost decision—because a cheaper system that quietly degrades isn’t a saving.
The value compounds with volume. When every agent inherits the same grounding, governance, and cost telemetry from the platform, scaling from one use case to 20 doesn’t mean rebuilding the discipline 20 times. What Foundry does is make sure the evidence exists when the funding conversation comes around, and the levers are within reach when you need them.
Next steps for optimizing AI agent costs and ROI
Scaling AI isn’t a bigger experiment. It’s a different discipline: a shared threshold, a unit cost, a value curve that outruns the cost curve, and an owner whose name is on the result.
Go deeper with our four-part blog series on The Economics of Agent Optimization in Microsoft Foundry, which takes each of these questions from principle to implementation:
- AI cost management: From AI pilots to measurable ROI
- AI cost optimization: How to lower AI spend
- AI agent optimization: How context engineering lowers AI costs
- AI agent governance: How to measure AI value and ROI
Then join us at Microsoft Ignite, November 17–20 in San Francisco or online, to see it in action and go hands-on with the team that built it.
Microsoft Foundry
The AI app and agent factory.
*Source: doc ##US54538926-IB, July 2026
