Late last year, we announced the preview of some of the foundational capabilities of our industrial IoT analytics platform with a scalable time series storage for trending decades of data, semantic model support to describe domain-specific metadata, and enhanced analytics APIs and UX. We are building on the power of this analytics platform with additional new capabilities that will add richness and flexibility, and open up new scenarios for our enterprise IoT customers. Today, we are announcing the following new capabilities:
- Warm and cold analytics support that builds on top of our existing preview and provides retention-based data routing between warm and cold stores. Customers can now perform interactive analytics over warm data as well as gain operational intelligence over decades of historical data stored in a customer-owned Azure Data Lake.
- A flexible analytics platform that enables attaching a customer-owned Azure Data Lake to Azure Time Series Insights for data archival, thereby allowing customers to have ownership of their IoT data. Customers can connect to and interop across a variety of advanced analytics scenarios such as predictive maintenance and machine learning using familiar technologies including Apache Spark™, Databricks, Jupyter, etc.
- Rich query APIs and user experience to support interpolation, new scalar and aggregate functions, categorical variables, scatter plots, and time shifting of time series signals for in-depth analysis.
- Significant scale and performance improvements at all layers of the solution including ingestion, storage, query, and metadata/model to support customers’ IoT solution needs.
- Azure Time Series Insights Power BI connector that enables customers to take the queries they do in Azure Time Series Insights directly into Power BI to get a unified view of their BI and time series analytics in a single pane of glass.
Azure Time Series Insights continues to provide a scalable pay-as-you-go pricing model enabling customers to tune their usage to suit their business demands and let Azure Time Series Insights analytics platform worry about scaling the infrastructure to meet their growing needs.
A comprehensive analytics platform for Industrial IoT
We released a preview of our first wave of capabilities last year in December. We have since had great customer adoption and feedback that has led us to the preview refresh today.
Our customers span all major industrial IoT segments including manufacturing, automotive, oil and gas, power and utility, smart buildings, and IoT consulting. These customers are telling us that IoT time series analytics is more than just the potential to achieve operational excellence. IoT time series data together with rich contextualization helps them drive dynamic transformation, enabling their businesses to become more agile and data-driven than ever before.
To help maximize the value of time series data and drive this digital revolution, we're updating the Azure Time Series Insights offering to support comprehensive and rich analytics over multi-layered storage, open file format and flexibility to connect to other data services for connected data scenarios, enterprise grade scale and performance, enhanced user experience and SDK support, and out-of-box connectors to data services such as Power BI to enable end-to-end analytics scenarios.
Details of the new features in preview refresh
Comprehensive and rich analytics over multi-layered storage
The majority of industrial IoT customers work with IoT data for a variety of data access scenarios. To satisfy these requirements, Azure Time Series Insights provides scalable multi-layered time series storage for warm and cold data analytics. When a customer provisions Azure Time Series Insights, upon selecting the PAYG pricing option, they can configure Azure Storage as the cold store, as well as enable warm store. Additionally, a customer can choose the retention period (configurable at any time) for the warm store. Azure Time Series Insights will automatically route ingested data based on the configured retention period to warm store, for example if retention was configured as 30d, as data is being streamed, 30d worth of data is stored in warm store. All data is, by default, routed to customer-owned Azure data lake for purposes of archiving and analytics. Queries done over the configured retention period are always served up from the warm store with no additional input from the user. Queries outside of the retention period are always served up from the cold store. This allows customers to do high volume, interactive, asset-based analytics over warm for monitoring, dashboarding, and troubleshooting scenarios. Customers can continue to do asset-based analytics over decades of cold data stored in their Azure Data Lake for operational intelligence including troubleshooting, golden batch analysis, predictive analytics, etc.
Flexible analytics platform for integrating with first and third party data services
A critical and powerful capability that is unleashed with our cold store is data connectivity to other data solutions for end-to-end scenario coverage. As mentioned earlier, the cold store is a customer-owned Azure Data Lake and is the source of truth for all their IoT data and metadata. Data is stored in an open source, Apache Parquet format for efficient data compression, space, query efficiency, and portability.
Azure Time Series Insights will provide out-of-box connectors for popular and familiar data services that our customers use, for example Apache Spark™ or Databricks for machine learning, and predictive analytics. This is a work in progress and will become available to customers shortly.
As part of this preview refresh, we are releasing the Azure Time Series Insights Power BI connector. This feature is available in the Azure Time Series Insights Explorer user experience through the ‘Export’ option, allowing customers to export the time series queries they create in our user experience directly into the Power BI desktop and view their time series charts alongside other BI analytics. This opens the door to a new class of scenarios for industrial IoT enterprises who have invested in Power BI. It provides a single pane of glass over analytics from various data sources including IoT time series, thereby unlocking significant business and operational intelligence.
Enhanced asset-based analytics API and user experience
Since our preview launch in December last year, we have worked with a number of key IoT enterprise customers to prioritize the set of requirements around query and user experience. The result is the following new capabilities we are announcing as part of our preview refresh today:
- Interpolation to reconstruct time series signals from existing data
- Discrete signal processing with categorical variables
- Trigonometric functions
- Scatter plots
- Time shifting time series signals to understand data patterns
- Model API improvements for hierarchy traversal, time series search, auto-complete, paths, and facets
- Improved search and navigation efficiency and continuation token to support query at scale
- Improved charting capabilities including support for step interpolation, minimum or maximum shadows, etc.
- Updated model authoring and editing experience
- Increased query concurrency to support up to 30 concurrent queries
We have a number of new capabilities coming in this space including support for time weighted averages, additional scalar and aggregate functions, dashboards, etc. over the coming months.
Azure Time Series Insights is committed to our customers’ success
We look forward to continuing to deliver on our commitment of simplifying IoT for our customers and empowering them to achieve more with their IoT data and solutions. For more information, please visit the Azure Time Series Insights product page and documentation. Also, try out the quickstart to begin using Azure Time Series Insights today.
Please provide feedback and suggestions on how we can improve the product and documentation by scrolling down to the bottom of each documentation page, where you can find a button for “product feedback” or sign in to your GitHub account and provide feedback. We value your input and would love to hear from you.