Today at Microsoft Ignite in Orlando we announced the preview of Azure Data Explorer in 41 regions, a lightning fast service optimized for data exploration. Azure Data Explorer is a PaaS service that you can embed in your own custom applications to add near real time analytics capabilities. The service helps organizations quickly discover insights from petabytes of structured and unstructured data. Azure Data Explorer can be used to query streaming data to drive instant insights for analyzing performance, identifying trends and anomalies, and diagnosing problems. Data engineers can also use it to explore data in the early stages of data analytics to better understand their data. Azure Data Explorer enhances the modern data warehouse by providing instant visibility and analytics over data spanning minutes ago to weeks or months in the past.
Azure Data Explorer was built by a team of developers with a need to understand the massive amount of data coming from telemetry streams from their service in order to better troubleshoot and resolve issues. They needed a solution that could easily to run ad-hoc queries, handle very large data sets, and return fast results. The service was quickly adopted by several individuals and teams within Microsoft. Currently, Azure Data Explorer is regularly used by over 25 thousand Microsoft employees.
Azure Data Explorer offers several benefits for harnessing your data in your digital transformation journey.
Azure Data Explorer’s fast indexing is coupled with a powerful and simple query language that is optimized for ad-hoc exploration. With Azure Data Explorer you can query 1 billion records in less than one second, making you more productive.
Azure Data Explorer can ingest 200MB per second per node, using linear scaling to deal with the most demanding workloads. The service can scale within minutes to hundreds of machines, with efficient processing for both data ingestion and querying.
Data exploration is simplified with Azure Data Explorer. Azure Data Explorer uses a modern query language that utilizes smart IntelliSense and color coding, making it easy to run queries. By design, it combines fast and maintenance-free text indexing, columnar storage, and time series capabilities, so you can spend more time in the data and less time engineering the solution.
Azure Data Explorer is the proven back-end for several data intensive Azure services such as Azure Log Analytics, Azure Application Insights, Azure Time Series Insights, as well as Windows Defender Advanced Threat Protection. Now offered as a PaaS service, you have the flexibility to build your own custom solutions that require near real time analytics.
Although we are announcing the availability of the preview today, you can have confidence in the service knowing that Azure Data Explorer is already running large scale loads at Microsoft, with a capacity of 200PB of data available for query and tables as big as 4 trillion rows. Azure Data Explorer can handle text, typed columns, and semi-structured data consistently, indexing and compressing all data types automatically. As data is ingested, Azure Data Explorer offers powerful analytical query capabilities with a unique combination of:
- Relational query capabilities, including aggregations, filtering, selections, joins, calculated columns, and advanced analytical and aggregation functions.
- Full text indexing and retrieval, regular expression evaluation, and text parsing.
- Time series analysis capabilities.
Finally, select partners and customers are also running Azure Data Explorer including connected cars, digital advertising and marketing platforms, diesel engines, chocolate making machines, health and digital imaging, online document, and authenticity services. Azure Data Explorer is also being used across various industries such as oil, gas, and energy including software companies creating SaaS solutions with the service.
To find out more about Azure Data Explorer you can: