{"id":3493,"date":"2017-09-25T00:00:00","date_gmt":"2017-09-25T07:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/blog\/new-in-stream-analytics-output-to-azure-functions-built-in-anomaly-detection-etc"},"modified":"2025-06-25T08:48:47","modified_gmt":"2025-06-25T15:48:47","slug":"new-in-stream-analytics-output-to-azure-functions-built-in-anomaly-detection-etc","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/new-in-stream-analytics-output-to-azure-functions-built-in-anomaly-detection-etc\/","title":{"rendered":"New in Stream Analytics: Output to Azure Functions, built-in anomaly detection, etc."},"content":{"rendered":"\n

Today at Microsoft Ignite, we are pleased to announce the preview of several new and compelling capabilities in Azure Stream Analytics. These include built-in inline machine learning based anomaly detection, egress to Azure functions, support for compressed data formats, JavaScript User defined aggregates, and support for CI\/CD in Visual Studio tooling. These new features will start rolling out over the course of the next several weeks. If you are attending Microsoft Ignite<\/a> conference this week, please attend sessions BRK3302 and BRK3303 to learn more.<\/p>\n\n\n\n

Built-in Machine Learning-based anomaly detection<\/h2>\n\n\n\n

Built-in machine learning-based operator \u2018ANOMALYDETECTION\u2019 is designed to help customers of Azure Stream Analytics who monitor data from applications or devices in real time, and need help to easily detect events or observations that do not conform to an expected pattern.<\/p>\n\n\n\n

Until now, Industrial IoT customers, and others who monitored streaming data relied on custom machine learning models. Implementers needed to have intimate familiarity with the scenarios and the problem domain. Also, integrating these models with the stream processing mechanisms required complex data pipeline engineering.<\/p>\n\n\n\n

With the new capability announced today, the complexity of machine learning has been simplified to a single SQL function call to a machine learning model. The underlying general-purpose machine learning model is abstracted and continuously learns over time to match your input streams. This new functionality is targeted towards numerical time series data, and will help customers detect anomalies, positive and negative trends. You can find more information in the Anomaly Detection documentation<\/a> here.<\/p>\n\n\n\n

\"Stream<\/figure>\n\n\n\n

Multiple ways to use the function in Stream Analytics queries<\/em><\/p>\n\n\n\n

Egress to Azure functions<\/h2>\n\n\n\n

Azure Functions is a ‘serverless’ compute service in Azure that helps users run code on demand without having to explicitly provision or manage infrastructure, and lets them implement code that is triggered by events occurring in Azure or third-party services. This ability of Azure Functions to respond to triggers makes it a natural output target for Azure Stream Analytics.<\/p>\n\n\n\n

Many Stream Analytics customers today leverage the rich capabilities offered by Azure functions by first writing the output to service bus queues, and then accessing Azure functions from there. To provide a lot more direct integration with Azure functions, today we are announcing an output adapter to Azure Functions from Azure Stream Analytics. This output adapter is designed to allow users to connect Stream Analytics to Azure Functions, and run a script or piece of code to trigger downstream workflows and accelerate response to key events.<\/p>\n\n\n\n

For example, it will now be a lot easier for developers to send out email alerts or notifications when certain pre-defined conditions such as overheating is observed on the factory floor. Developers can also author custom post processing logic that will allow Azure Stream Analytics to connect to a variety of other Azure services or apps outside the Azure ecosystem.<\/p>\n\n\n\n

\"Output<\/figure>\n\n\n\n

Support for input compression formats<\/h2>\n\n\n\n

Bandwidth issues and message size limitations are the key reasons why customers need support for compression input formats. Based on the growing number of requests from many of our existing and prospective customers, we are glad to announce that Azure Stream Analytics will now offer support for GZIP and Deflate streams.<\/p>\n\n\n\n

Our announcement today means Azure Stream Analytics is a viable option for many customers who currently face these limitations.<\/p>\n\n\n\n

Visual Studio Tooling for continuous integration and delivery<\/h2>\n\n\n\n

The need to manage stability of production code while promoting improvements and code updates as quickly as possible means that development teams spend a lot of time building code integration and deployment (CI\/CD) processes. To better help such teams, we are rolling out native support for CI\/CD from within Stream Analytics Visual Studio Tools<\/a>. Customers can perform automated builds, test scripts and deploy them with the Stream Analytics Visual Studio projects using the newly released NuGet package<\/a>. With this new capability, it becomes very easy to setup a continuous CI\/CD pipelines for Stream Analytics jobs in the developer\u2019s environment as well as within other automated processes.<\/p>\n\n\n\n

JavaScript User Defined Aggregates<\/h2>\n\n\n\n

Following the release of JavaScript User defined functions mid-spring, we are now ready to offer JavaScript User Defined Aggregates (UDA). With user defined aggregates, you can run your own stateful computations like time weighted averages, cumulative density, array cumulation, and much more.<\/p>\n\n\n\n

Microsoft Ignite (Sep 25th<\/sup>  \u2013 29th<\/sup>)<\/h2>\n\n\n\n

Want to see these features in action? If you are at Microsoft Ignite<\/a> this week, please do attend the following sessions on Tuesday. Also, please meet our team at the expo hall.<\/p>\n\n\n\n