GA Azure OpenAI Emit Token Metric policy in Azure API Management
Published date: May 21, 2024
We're thrilled to announce the General Availability of the Azure OpenAI Emit Token Metric policy in Azure API Management! This new feature enables customers to effortlessly configure token usage metrics to be sent directly to Azure Application Insights, providing insights into the utilization of Azure OpenAI models across various applications and teams.
With this feature, customers can easily set up a policy to transmit token usage metrics to Application Insights with a straightforward configuration. The policy captures prompt, completions, and total token usage metrics, offering a comprehensive view of model utilization.
Customers have the flexibility to specify the Application Insights namespace to which metrics should be sent, ensuring seamless integration with their existing monitoring infrastructure. Moreover, they can configure or select from pre-defined dimensions to segment token usage metrics, enabling granular analysis by Subscription ID, IP Address, User ID, and more.
This new policy streamlines the process and supports both streaming and non-streaming scenarios for Azure OpenAI. It empowers customers to gain deeper insights into token consumption effortlessly, facilitating better decision-making and resource allocation.
Click here to learn more.