This past Fall we introduced webhook for Azure Alerts on metrics, enabling custom notifications and automated actions on alerts. We are very happy to see the fast adoption and positive feedback. You also provided some excellent, actionable feedback to expand this capability to a wide range of use cases. Today, we are excited to announce a set of features on the core Azure platform that will allow you to configure and manage more notifications and automated actions.
I'd also like to share information about advanced alerting capabilities on data from your Azure resources via Microsoft and partner solutions. These include the capability to create custom metric alerts as well as log search based alerts on data across multiple Azure subscriptions.
Email and webhook notifications for Azure Autoscale actions
Many of you have requested the ability to be notified on Autoscale events. We are excited to announce the ability to configure email and webhook notifications on scale actions for Web Apps, Cloud Services and Virtual Machine Scale Sets. Email and webhook configuration will work exactly like it does for Azure alert on metrics.
Autoscale engine notifies you about scale failures as well as unavailability/recovery of metrics data for scale actions. With this new feature, you can opt-in to receive email notifications for all successful scale out and in actions. You can also configure webhook Uri to be triggered upon a successful scale action, routing key metadata about the resource that scaled, which includes its new and old instance count.
You can configure this via the Azure Portal for Web Apps and Cloud Services and via REST API (Azure Resources Explorer), Azure Insights PowerShell and Azure Insights CLI for Web Apps, Cloud Services and VM Scale Sets.
More information about configuring the notifications and the webhook notification payload (JSON) schema for it can be found here.
Preview of alert on Azure Audit Logs (alert on events)
A new feature we are excited to announce is the alert on Audit Logs, which is essentially alert on operational events in Azure. This is an early preview release and is currently supported only via Azure PowerShell and Azure CLI.
One of the core scenarios that many of you wanted support for was to alert on a particular operation on a resource in Azure and the ability to take an automated action on it. With this new alert on event, you can get notified via email and webhook on control-plane actions such as VM Start/Stop/Delete and many more.
Here is an example of PowerShell script to create an alert on an event on a VM:ca
# Scenario: Send email and trigger a webhook when any VM is successfully started in resource group "myrg1" in my subscription
# Create an Email action
$actionEmail = New-AzureRmAlertRuleEmail -CustomEmail email@example.com
# Create a webhook action
$actionWebhook = New-AzureRmAlertRuleWebhook -ServiceUri https://example.com?token=mytoken
# Create the rule and configure the email and webhook actions for it
Add-AzureRmLogAlertRule -Name superalert1 -Location "East US" -ResourceGroup myrg1 -OperationName Microsoft.ClassicCompute/virtualmachines/start/action -Status Succeeded -TargetResourceGroup myrg1 -Actions $actionEmail, $actionWebhook
# Retrieve the rule
Get-AzureRmAlertRule -Name superalert1 -ResourceGroup myrg1 -DetailedOutput
When this alert is activated, it can trigger a webhook passing on valuable event metadata in its payload. The webhook notification payload schema (JSON format) for Alert on events can be found here.
Please note the feature is still in preview and the alerts on events can incur a delay of fifteen minutes. In the coming months, we will make significant performance improvements to the alert engine for events and will release user interface support in the Azure Portal for configuring and managing these alerts.
Alerts with email and webhook notifications on custom metrics and web tests
The ability to generate custom metrics and get notified on them has been a top ask from many of you. The capability to generate custom metric and events already exists within Application Insights. Today, we are excited to announce webhook notifications for these alerts. Webhooks on Application Insights based custom metrics will work exactly like those for standard metrics, sharing the same JSON schema for webhook payload.
Using Application Insights, with minimal lines of code, you can emit any numeric value as a custom metric from anywhere (a desktop app, an on premise machine or even a script running on a different cloud platform) and let Application Insights manage the visualization, alerting and notification for you from within the Azure Portal. Application Insights has a variety of SDKs to choose from (C#, Java, PowerShell, Python, Ruby to name a few).
Here is a sample C# application that retrieves the length of an Azure Storage queue and sends that value to Application Insights as a custom metric. Once, this data is routed to Application Insights, it is viewable just like any other metric. You can create alerts on it setting numeric thresholds, configure email & webhook notifications.
Application Insights comes with SDKs for C#, Java, PowerShell, Python, Ruby and many more, allowing you to generate custom metrics from almost any application and scripts on on-premises or cloud hosted servers. To learn more about custom metrics and events using Application Insights click here.
The Application Insights team has also extended the webhook notifications support to Web Test alerts.
More alert integration with PagerDuty
PagerDuty, one of the leading incident management solutions, has had first-class support for Azure Alerts on metrics. Today, PagerDuty is also extending its incident support for Azure alerts on autoscale actions, custom metrics from Application Insights and notifications on Audit Log events.
The image below shows what the custom, metric-based alert looks like when it triggers a notification in PagerDuty, which enriches, aggregates, and correlates the events into something actionable. In this way, PagerDuty streamlines incident management by reducing noise and resolution times.
With just a few clicks, you can integrate Azure Alerts with PagerDuty in a to receive SMS, voice, email and manage escalations on your Azure Alerts on metrics, Autoscale actions and Audit Log events. To learn more about PagerDuty integration, checkout this video.
You can read more about the Azure integration announcement from PagerDuty here.
Preview of Azure Automation runbook integration for alerts on v2 VMs
Last Fall, we announced the ability to configure webhook on a metric based alert to trigger any Azure Automation runbook. You provided feedback that this experience could be much improved via better integration within the Azure Portal between Azure Alerts and Automation runbooks. The Azure Automation team jumped on this to release a preview feature by enabling this for alert on VMs. Please note that this is currently supported only for the v2 VMs (Azure Resource Manager based VMs)
You can now directly configure Azure Automation Runbooks to take actions on common tasks such as Restart, Start, Stop or Update a VM based on metric trigger. The team is eager to receive your feedback and plans to expand this capability to alerts on other Azure Services as well.
Check out this article from the Azure Automation team for a detailed walkthrough of this new feature.
Resize VMs (vertical scaling) using Azure alerts and Azure Automation
Another common feedback received by the VM and Autoscale team was about the ability to perform vertical scale actions on VMs (scale a VM or VMSS up or down) based on a trigger. While this is not yet an in-built feature on the platform, we wanted to share a few samples on what’s possible today using Azure Alert webhooks and Azure Automation.
In the Azure Automation gallery, you will find samples of Scale-Up and Scale-Down runbooks that can be triggered via the metric Alerts. These runbooks allow you to scale a VM up or down (resize) based on a threshold value set for the alert. For instance, you can now automatically scale down (change the size from A2 to A1) a VM when an alert is triggered indicating that there has not been any activity on it for a specified period of time (for example the CPU% is less than 2% for a long time).
Check out a detailed walkthrough from the VM team about these sample scale up/down runbooks for single VMs.
[Update, 4/15/2016] You can also vertically autoscale VM Scale Set using the alert integration with Azure Automation Runbooks.
Please follow this walkthrough for vertical autoscaling VMSS
Advanced alerts on Azure VM diagnostics data using Log Analytics (OMS)
Log Analytics, part of the larger Operations Management Suite (OMS), is accessible from the Azure Portal and enables you to view and analyze data from your Azure servers from one or more subscriptions. OMS can ingest diagnostics data collected by the Windows and Linux VMs such as IIS Logs, syslog data, ETW logs, Azure Diagnostics Agent counters, application logs and more data types from across your Azure subscriptions. You can then create custom searches for logs across your on-premises and Azure servers, and you can be alerted on the results of these searches. An alert can be configured to send an email, fire a WebHook or run an Automation runbook.
You can view all your alerts by running the query Type=Alert SourceSystem=OMS. OMS also comes with a rich Alert Management Pack to streamline all alerts, categorize them and monitor their aggregated count and history.
Log Analytics is offered with a free tier that includes all alerting capabilities.
To learn more about OMS configuration for Azure diagnostics click here.
With these new features, you now have more control on alert on metrics, autoscale actions and audit log events. You also know more about the advanced alerting capabilities on your Azure resources using Application Insights and OMS. You can configure PagerDuty to receive SMS, push notifications and additional custom notifications for your Azure alerts.
In the coming months, we will improve the preview features mentioned above. Meanwhile, take these for a spin - fire away alerts with webhook notifications and share your feedback.