Today, we are announcing the general availability of Azure Stream Analytics (ASA) on IoT Edge, empowering developers to deploy near-real-time analytical intelligence closer to IoT devices, unlocking the full value of device-generated data. With this release, Azure Stream Analytics enables developers to build truly hybrid architectures for stream processing, where device-specific or site-specific analytics can run on containers on IoT Edge and complement large scale cross-devices analytics running in the cloud.
Why run stream analytics on the Edge?
Azure Stream Analytics on IoT Edge complements our cloud offering by unlocking the power and ease-of-use of Azure Stream Analytics (ASA) for new scenarios, such as:
- Low-latency command and control: For example, manufacturing safety systems need to be able to respond to operational data with ultra-low latency. With ASA on IoT Edge, you can analyze sensor data in near real time and issue commands to stop a machine or trigger alerts when you detect anomalies.
- Limited connectivity to the cloud: Mission critical systems, such as remote mining equipment, connected vessels, or offshore drilling, need to analyze and react to data even when cloud connectivity is intermittent. With ASA on IoT Edge, your streaming logic runs independently of the network connectivity and you can choose what you send to the cloud for further processing.
- Limited bandwidth: The volume of data produced by industrial IoT equipment or connected cars can be so large that data must be filtered or pre-processed before being sent to the cloud. Using ASA on IoT Edge, you can filter or aggregate the data that needs to be sent to the cloud, or only send data when changes or anomalies are detected.
- Data privacy: Regulatory compliance may require some data to be locally anonymized or aggregated before being sent to the cloud. With ASA on IoT Edge, you can aggregate data from various sources, devices, or users, and remove personal information before sending the data to the cloud.
Customers have already been bringing these scenarios to life using ASA on IoT Edge and sharing positive feedback. Hiroyuki Ochiai, Director of the IT platform division for NEC Corporation said, “Azure Stream Analytics on IoT Edge increases the responsiveness of IoT solutions, while ensuring data privacy and sovereignty by processing data locally on IoT Edge. We see great potential to use this service across both our own IoT solutions, and those of our customers who benefit from NEC’s Azure Plus consultancy.”
What’s new in the GA version?
With the general availability of ASA on IoT Edge, there have been a number of powerful enhancements to support production workloads on the Edge:
- Reliability and performance improvements.
- Improved support for offline scenarios: After the initial deployment, ASA on IoT Edge can be restarted without any cloud connection.
- Improved logging: Developers can now enable verbose debug logs for troubleshooting purpose.
- Improved monitoring: We enabled ten new metrics such as input, output, error count, and OutputWatermarkDelay for example. Additionally, it is now possible to query the ASA job status from module twin.
- Simplified update flow for query logic: Updating ASA jobs previously deployed to IoT Edge devices is now much simpler. After updating the logic of your job in ASA, you can update your IoT Edge deployment in just a few clicks in the IoT Hub portal.
- Ability to update reference data without restarting the container: Updating reference data location can now be done through a new IoT Edge deployment.
- Programmatic deployments: ASA on IoT Edge jobs can now be created and packaged using rest APIs, allowing CI/CD (Continuous integration and continuous delivery).
- Better parity with cloud jobs: We added multiple options previously available only for cloud jobs (e.g. GZIP compression, JSON format options, etc.), making easier to move jobs between cloud and IoT Edge.
Looking forward
In addition to new GA capabilities, there are a number of new features for ASA on IoT Edge now available in preview:
- Visual Studio support for ASA on IoT Edge jobs: You can create ASA Edge jobs using Visual Studio. Learn more by reading the Visual Studio doc.
- User-defined functions in C#: With .NET standard user-defined functions, you can run .NET Standard code as part of your streaming pipeline. You can create simple C# classes or import full project and libraries. Full authoring and debugging experience is supported in Visual Studio. For more information, visit the documentation, “Develop .NET Standard user-defined functions for Azure Stream Analytics Edge jobs.”
- C# custom deserializer: Developers can implement custom deserializers in C# to deserialize events received by ASA. Examples of formats that can be deserialized include Parquet, Protobuf, XML, or any binary format. For more information, please visit the Azure Stream Analytics preview page.
- Built-in machine learning for anomaly detection: ASA on IoT Edge now supports built-in machine learning models with support for spike and dips detection in addition to bi-directional, slow positive, and slow negative trends detection. For more information, please visit the Azure Stream Analytics preview page.
Get started now
Get started in minutes by following our getting started documentation Stream Analytics Edge.
ASA on IoT Edge is free until February 1, 2019. The new pricing model will be applicable after February 1, 2019. Custom pricing is available for large scale deployments (over 5,000 devices). More details are available on the ASA pricing page.
