{"id":40339,"date":"2025-05-12T16:00:00","date_gmt":"2025-05-12T23:00:00","guid":{"rendered":""},"modified":"2025-12-26T12:56:36","modified_gmt":"2025-12-26T20:56:36","slug":"announcing-new-fine-tuning-models-and-techniques-in-azure-ai-foundry","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/announcing-new-fine-tuning-models-and-techniques-in-azure-ai-foundry\/","title":{"rendered":"Announcing new fine-tuning models and techniques in Azure AI Foundry"},"content":{"rendered":"\n

Today, we\u2019re excited to announce three major enhancements to model fine-tuning in Azure AI Foundry\u2014Reinforcement Fine-Tuning (RFT) with o4-mini (coming soon), Supervised Fine-Tuning (SFT) for the GPT-4.1-nano <\/strong>and Llama 4 Scout model (available now)<\/strong>. These updates reflect our continued commitment to empowering organizations with tools to build highly customized, domain-adapted AI systems for real-world impact. <\/p>\n\n\n\n

With these new models, we’re unblocking two major avenues of LLM customization: GPT-4.1-nano is a powerful small model, ideal for distillation, while o4-mini is the first reasoning model you can fine-tune, and Llama 4 Scout is a best-in-class open source model. <\/p>\n\n\n\n

\n
Design, customize, and manage AI apps with Azure today<\/a><\/div>\n<\/div>\n\n\n\n

Reinforcement Fine-Tuning with o4-mini <\/h2>\n\n\n\n

Reinforcement Fine-Tuning introduces a new level of control for aligning model behavior with complex business logic. By rewarding accurate reasoning and penalizing undesirable outputs, RFT improves model decision-making in dynamic or high-stakes environments.<\/p>\n\n\n\n

Coming soon for the o4-mini<\/strong> model, RFT unlocks new possibilities for use cases requiring adaptive reasoning, contextual awareness, and domain-specific logic\u2014all while maintaining fast inference performance.<\/p>\n\n\n\n

\n
Learn how Reinforcement Fine-Tuning improves AI reasoning accuracy<\/a><\/div>\n<\/div>\n\n\n\n

Real world impact: DraftWise <\/h2>\n\n\n\n

DraftWise, a legal tech startup, used reinforcement fine-tuning (RFT) in Azure AI Foundry Models to enhance the performance of reasoning models tailored for contract generation and review. Faced with the challenge of delivering highly contextual, legally sound suggestions to lawyers, DraftWise fine-tuned Azure OpenAI<\/a> models using proprietary legal data to improve response accuracy and adapt to nuanced user prompts. This led to a 30% improvement in search result quality, enabling lawyers to draft contracts faster and focus on high-value advisory work. <\/p>\n\n\n\n

\n

Reinforcement fine-tuning on reasoning models is a potential game changer for us. It’s helping our models understand the nuance of legal language and respond more intelligently to complex drafting instructions, which promises to make our product significantly more useful to lawyers in real time.<\/p>\n\u2014James Ding, founder and CEO of DraftWise.<\/cite><\/blockquote>\n\n\n\n

When should you use Reinforcement Fine-Tuning?<\/h2>\n\n\n\n

Reinforcement Fine-Tuning is best suited for use cases where adaptability, iterative learning, and domain-specific behavior are essential. You should consider RFT if your scenario involves: <\/p>\n\n\n\n

    \n
  1. Custom Rule Implementation:<\/strong> RFT thrives in environments where decision logic is highly specific to your organization and cannot be easily captured through static prompts or traditional training data. It enables models to learn flexible, evolving rules that reflect real-world complexity. <\/li>\n<\/ol>\n\n\n\n
      \n
    1. Domain-Specific Operational Standards:<\/strong> Ideal for scenarios where internal procedures diverge from industry norms\u2014and where success depends on adhering to those bespoke standards. RFT can effectively encode procedural variations, such as extended timelines or modified compliance thresholds, into the model’s behavior. <\/li>\n<\/ol>\n\n\n\n
        \n
      1. High Decision-Making Complexity:<\/strong> RFT excels in domains with layered logic and variable-rich decision trees. When outcomes depend on navigating numerous subcases or dynamically weighing multiple inputs, RFT helps models generalize across complexity and deliver more consistent, accurate decisions.\u202f<\/li>\n<\/ol>\n\n\n\n

        Example: Wealth advisory at Contoso Wellness <\/h2>\n\n\n\n

        To showcase the potential of RFT, consider Contoso Wellness, a fictitious wealth advisory firm. Using RFT, the o4-mini model learned to adapt to unique business rules, such as identifying optimal client interactions based on nuanced patterns like the ratio of a client\u2019s net worth to available funds. This enabled Contoso to streamline their onboarding processes and make more informed decisions faster.<\/p>\n\n\n\n

        \n
        Demonstration and applications: Watch o4\u2011mini reinforcement fine\u2011tuning in action<\/a><\/div>\n<\/div>\n\n\n\n

        Supervised Fine-Tuning now available for GPT-4.1-nano <\/h2>\n\n\n\n

        We\u2019re also bringing Supervised Fine-Tuning (SFT)<\/strong> to the GPT-4.1-nano<\/strong> model\u2014a small but powerful foundation model optimized for high-throughput, cost-sensitive workloads. With SFT, you can instill your model with company-specific tone, terminology, workflows, and structured outputs\u2014all tailored to your domain. This model will be available for fine-tuning in the coming days. <\/p>\n\n\n\n

        Why Fine-tune GPT-4.1-nano? <\/h3>\n\n\n\n