HDInsight

Managed open-source Big Data analytics service for the enterprise

Azure HDInsight is a managed open-source Big Data analytics service for enterprises. You can create optimised clusters for Hadoop, Spark, Hive, HBase, Storm, Kafka and Microsoft R Server backed by a 99.9% SLA.

Managed-service open-source analytics with an industry-leading SLA

While others provide an SLA on the underlying VMs, HDInsight is the only service in the industry to provide an end-to-end SLA on the workload. Create optimised clusters for Hadoop, Spark, Hive, HBase Storm, Kafka and Microsoft R Server backed by a 99.9% SLA. Using these building blocks, you can complete scenarios that encompass ETL, warehousing, data science, IoT and streaming while extending your on-premises investments. Using HDInsight, you can run these as production-ready solutions with enterprise-level security and monitoring within minutes on Azure.

HDInsight works with Hadoop projects like Apache HBase, Apache Storm, Apache Hive, Apache Spark and Apache Kafka

Global reach

Available in >25 regions globally – more than any other Big Data Analytics offering. Also available in Azure Government cloud and China.

Secure and compliant

Protect your data assets and extend your on-premises security and governance controls to the cloud with HDInsight. Get single sign-on (SSO), multi-factor authentication, and seamless management of millions of identities through Azure Active Directory. Authorise users and groups with fine-grained access control policies over all your enterprise data with Apache Ranger. HDInsight meets Health Insurance Portability and Accountability Act (HIPAA), Payment Card Industry (PCI) and Service Organization Controls (SOC) compliance, helping you ensure that your enterprise data assets are always well-protected. To support the highest level of business continuity, HDInsight extends capabilities for alerting, monitoring and defining preemptive actions, and it gives you enhanced workload protection through native integration with Azure’s monitoring suite.

High-productivity platform for developers and scientists

Use rich productivity suites for Hadoop and Spark with your preferred development environment such as Visual Studio, Eclipse, and IntelliJ for Scala, Python, R, Java and .NET support. Data scientists can combine code, statistical equations and visualisations to tell a story about their data through integration with the two most popular notebooks, Jupyter and Zeppelin. HDInsight is also the only managed-cloud Hadoop solution with integration to Microsoft R Server. Multithreaded maths libraries and transparent parallelisation in R Server handle up to 1,000x more data and up to 50x faster speeds than open-source R, which helps you to train more accurate models for better predictions than before.

Cost-effective cloud scale

Cost-effectively scale workloads up or down with decoupled compute and storage. Local storage can still be used for caching and fast I/O. Spark and interactive Hive users can choose SSD memory for interactive performance, while Kafka users can retain all streaming data in premium managed disks. Choose any Azure virtual machine type that enables the best utilisation of resources, and only pay for the compute and storage that you use.

Most extensible platform

HDInsight partners with the leading ISVs to provide a one-click, easy-to-use, extensible app framework.

During cluster deployment, ISVs such as Cask, Streamsets, H20.AI and more can be deployed to extend the capabilities of the Hadoop, Spark and Kafka analytics platform.

What can you build with Azure HDInsight?

Learn about use cases below:

Internet of Things and Streaming applications

Toyota’s Connected Car, Office 365 and Bing Ads are processing millions of events/sec for real-time big data processing on HDInsight through Kafka, Storm and Spark Streaming.

Learn more

Data science and machine learning

Transform your business by adding intelligence to your applications and organisation.

Learn more

Data warehousing

Perform interactive query at petabyte scale over structured or unstructured data in any format; build models while connecting with your favourite BI tool.

Learn more

Hybrid with Azure HDInsight and on-premises

Extend your on-premises investments to the cloud and transform your business by leveraging the advanced analytics and BI offerings in the cloud.

Learn more

Customers powering Big Data Analytics through Azure HDInsight

HDInsight learning path