Microsoft continues to develop and advance cloud services to meet the full spectrum of government needs while complying with United States regulatory standards for classification and security. The latest of these tools, generative AI capabilities through Microsoft Azure OpenAI Service, can help government agencies improve efficiency, enhance productivity, and unlock new insights from their data.

Many agencies require a higher level of security given the sensitivity of government data. Microsoft Azure Government provides the stringent security and compliance standards they need to meet government requirements for sensitive data. 

Currently, large language models that power generative AI tools live in the commercial cloud. For government customers, Microsoft has developed a new architecture that enables government agencies to securely access the large language models in the commercial environment from Azure Government allowing those users to maintain the stringent security requirements necessary for government cloud operations.

If you’re an Azure Government customer (United States federal, state, and local government or their partners), you now have the opportunity to use the Microsoft Azure OpenAI Service through purpose-built, AI-optimized infrastructure providing access to OpenAI’s advanced generative models.  

Azure OpenAI Service 

Azure OpenAI Service REST APIs provide access to OpenAI’s powerful language models, including GPT-4, GPT-3, and Embeddings. You can adapt these models to your specific task, including but not limited to content generation, summarization, semantic search, and natural language-to-code translation.

You can also access the service using REST APIs, Python SDK, or our web-based interface in the Azure AI Studio. As an Azure Government customer or partner, you can access and operationalize advanced AI models and algorithms at scale. Developers can use Azure OpenAI Service to access pre-trained GPT models to build and deploy AI-enabled applications more quickly and with minimal effort.  

Capability enhancements with Azure OpenAI Service

Azure OpenAI Services can help government customers accelerate their operations and unlock new insights to meet their mission needs. This service will enable key new functions to help customers:   

• Accelerate content generation: Automatically generate responses based on mission or project inquiries to help reduce the time and effort required for research and analysis, enabling teams to focus on higher-level decision-making and strategic tasks.   

• Streamline content summarization: Generate summaries of logs and rapid analysis of articles, analysts, and field reports.  

• Optimize semantic search: Enable enhanced information discovery and knowledge mining.  

• Simplify code generation: Build custom applications using natural language to query proprietary data models and rapidly generate code documentation.

One of the most effective ways to generate reliable answers is to prompt the model to draw its responses from grounding data. If your use case relies on up-to-date, reliable information and is not purely a creative scenario, we strongly recommend providing grounding data based on trusted internal data sources. In general, the closer you can get your source material to the final form of the answer you want, the less work the model needs to do, which means there is less opportunity for error.

Azure Government to Azure commercial networking

Azure Government peers directly to the commercial Microsoft Azure network, including routing and transport capabilities to the internet and the Microsoft Corporate network. Azure Government limits its exposed surface area by applying extra protections and communications capabilities of the commercial Azure network. Additional information highlighting Azure Government environment isolation can be found on our Azure Government security website.

Microsoft encrypts all Azure traffic within a region or between regions using MACsec, which relies on AES-128 block cipher for encryption. This traffic stays entirely within the Microsoft global network backbone and never enters the public internet. The backbone is one of the largest in the world with more than 250,000 km of lit fiber optic and undersea cable systems.

Access and reference architecture

Access to the Azure OpenAI Service is available through the Azure Government environment. Azure Government peers directly with the commercial Azure network and doesn’t peer directly with the public internet or the Microsoft corporate network. As shown in the reference architecture in Figure 1, connection to Azure OpenAI is over the Microsoft backbone network to access and operationalize advanced AI models and algorithms securely and at scale.

Protecting your data, privacy, and security​

Microsoft Azure Government provides stringent security and compliance standards necessary to meet government requirements for sensitive data. Through this architecture, government applications and data environments remain on Azure Government. Only the queries submitted to the Azure OpenAI Service transit into the Azure OpenAI model in the commercial environment through an encrypted network and do not remain in the commercial environment. Government data is not used for learning about your data or to train the OpenAI model.

Microsoft allows customers who meet additional Limited access eligibility criteria and attest to specific use cases to apply to modify the Azure OpenAI content management features. If Microsoft approves a customer’s request to modify data logging, then Microsoft does not store any prompts and completions associated with the approved Azure subscription for which data logging is configured off in Azure commercial.

As part of our reference architecture, it is recommended to complete the approval process to modify content filters and data logging via this online form to ensure no logging data exists in Azure commercial. An example of how to modify your data logging settings is available on our Data, privacy, and security for Azure OpenAI Service website.

Microsoft responsible AI principles

When you create technologies that can change the world, we believe you must also ensure that the technology is used responsibly. That’s why we are committed to creating responsible AI by design. Our work is guided by decades of research on AI, grounding, and privacy-preserving machine learning as well as our Responsible AI Standard and a core set of AI principles: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. We put these principles into practice across the company to develop and deploy AI that will have a positive impact on society. We take a cross-company approach through cutting-edge research, best-of-breed engineering systems, and excellence in policy and governance. Additional information on our Microsoft Responsible AI Principles is available at Our approach to responsible AI at Microsoft website.

Azure OpenAI Service Frequently Asked Questions

How does Microsoft recommend implementing this reference architecture? 

• Have an account and subscription in Azure Government and Azure Commercial. 
• Recommended steps per environment: 

Azure Commercial	Azure Government
Request access to Azure OpenAI.	Deploy your application utilizing your access to Azure OpenAI API.
Request to modify content filters and data logging.	Complete the required authorizations (IATT and ATO) for customer-specific workloads.
Only utilize prompts for inferencing—do not leverage fine-tuning with Controlled Unclassified Information (CUI) data.

When will access to Azure OpenAI be available for Azure Government customers? 

Access to the Azure OpenAI Service is available to approved enterprise customers and partners through the Microsoft Azure Government environment. Customers can access the Azure OpenAI Service REST APIs on Azure Commercial from Azure Government as highlighted in the reference architecture above.

How do the capabilities of the Azure OpenAI Service compare to OpenAI? 

Azure OpenAI Service gives customers advanced language AI with OpenAI GPT-4, GPT-3, and Embeddings. The Azure OpenAI API is compatible with the OpenAI API, providing efficiencies for developers and users. With Azure OpenAI Service, customers get the benefit of the security capabilities of Microsoft Azure Government powered by OpenAI’s models.

How do you enable secure access to Azure OpenAI Service? 

Access to Azure OpenAI Service is enabled through transport-layer security (TLS). Azure Government peers directly with the commercial Microsoft Azure network and doesn’t peer directly with the public internet or the Microsoft corporate network. Your data is never used to train the OpenAI model (your data is your data).  

Getting started with Azure OpenAI Service

Government enterprise workloads can be complex and mission-critical with requirements such as high throughput, low latency, compliance, availability, and data sovereignty. Azure OpenAI Service requires registration and is only available to approved enterprise customers and partners.

Sign up here to learn how AI can accelerate your mission and stay up to date on Microsoft’s AI for government advancements.

We published an Azure Government OpenAI Access QuickStart that uses Azure CLI to deploy an isolated Docker container to Azure Container Instances in Azure Government using code from the Azure OpenAI QuickStart.