Skip Navigation

HDInsight

Easy, cost-effective, enterprise-grade service for open source analytics

What are the benefits of HDInsight?

Easily run popular open source frameworks—including Apache Hadoop, Spark, and Kafka—using Azure HDInsight, a cost-effective, enterprise-grade service for open source analytics. Effortlessly process massive amounts of data and get all the benefits of the broad open source ecosystem with the global scale of Azure.

Why HDInsight?

Easy

Quickly spin up open source projects and clusters, with no hardware to install or infrastructure to manage.

Cost-effective

Reduce costs by creating big data clusters on demand, easily scaling them up or down, and paying only for what you use.

Enterprise-grade

Get enterprise-grade security and industry-leading compliance, with more than 30 certifications.

Open

Create optimized components for Hadoop, Spark, and more. Keep up to date with the latest versions.

What comes with HDInsight?

Open source ecosystem

HDInsight supports the latest open source projects from the Apache Hadoop and Spark ecosystems. Stay up to date with the newest releases of open source frameworks, including Kafka, HBase, and Hive LLAP.

Security and compliance

Get enterprise-grade data protection with monitoring, virtual networks, encryption, Active Directory authentication, authorization, and role-based access control. HDInsight has more than 30 industry certifications, including ISO, SOC, HIPAA, and PCI, to meet compliance standards.

Native integration with Azure services

Seamlessly integrate with a wide variety of Azure data stores and services, including SQL Data Warehouse, Azure Cosmos DB, Data Lake Storage, Blob Storage, Event Hubs, and Data Factory.

Simplified monitoring

HDInsight integrates with Azure Log Analytics to provide a single interface where you can monitor all your clusters.

Broad application support

HDInsight supports a broad range of applications from the big data ecosystem, which you can install with a single click. Pick from more than 30 popular Hadoop and Spark applications for a variety of scenarios.

Multiple languages and tools

Use your preferred productivity tools, including Visual Studio, Eclipse, IntelliJ, Jupyter, and Zeppelin. Write code in familiar languages such as Scala, Python, R, JavaScript, and .NET.

Azure HDInsight Ecosystem

Tools

Apache Zeppelin

VS Code

IntelliJ

JDBC

Data access

Batch

SQL

NoSQL

Stream

Machine Learning

Others

Azure Data Lake Storage

Security

Apache Ranger

Azure Active Directory

Virtual Network

Customers using HDInsight

  • Roche
  • LG CNS
  • Reckitt Benckiser
  • Jet.com
  • Asos

What can you do with HDInsight?

Extract, transform, and load (ETL) using HDInsight

Extract, transform, and load your big data clusters on demand with Hadoop MapReduce and Apache Spark.

Extract, transform, and load (ETL) using HDInsightExtract, transform, and load your big data clusters on demand with Hadoop MapReduce and Apache Spark.

Streaming using HDInsight

Ingest and process millions of streaming events per second with Apache Kafka, Apache Storm, and Apache Spark Streaming.

Streaming using HDInsightIngest and process millions of streaming events per second with Apache Kafka, Apache Storm, and Apache Spark Streaming.

Interactive querying with HDInsight

Perform fast, interactive SQL queries at scale over structured or unstructured data with Apache Hive LLAP.

Interactive querying with HDInsightPerform fast, interactive SQL queries at scale over structured or unstructured data with Apache Hive LLAP.

Extend your on-premises big data investments with HDInsight

Extend your on-premises big data investments to the cloud and transform your business using the advanced analytics capabilities of HDInsight.

Extend your on-premises big data investments with HDInsightExtend your on-premises big data investments to the cloud and transform your business using the advanced analytics capabilities of HDInsight.

Use Cases

Customer insights

Help employees make data-driven decisions by building an end-to-end open source analytics platform. Easily process massive amounts of data from different sources.

Learn how Reckitt Benckiser uses HDInsight for consumer insights.

Personalized recommendations

Engage your customers in new ways by building personalized recommendation engines.

Learn how ASOS uses HDInsight for personalized recommendations.

Predictive maintenance

Predict and prevent failures and keep vital equipment running. Ingest and process data in real time to optimize operations.

Learn how Roche Diagnostics uses HDInsight for predictive maintenance.

Risk assessment

Build better models by transforming and analyzing your critical data, and help keep your data secure with enterprise-grade capabilities.

Learn how Roche Diagnostics uses HDInsight for predictive maintenance.

Related products and services

Azure Databricks

Fast, easy, and collaborative Apache Spark-based analytics platform

Azure Data Lake Storage

Massively scalable data lake storage

SQL Data Warehouse

Elastic data warehouse as a service with enterprise-class features

Start using HDInsight today