Apache Kafka on HDInsight with Azure Managed Disks
Updated: June 30, 2017
We're pleased to announce Apache Kafka with Azure Managed Disks Preview on the HDInsight platform. Users will now be able to deploy Kafka clusters with managed disks straight from the Azure portal, with no signup necessary. This will allow for the powerful advantage of exponentially higher scalability, and much lower costs as workloads scale.
A few months ago we announced a limited preview of Managed Kafka on Azure HDInsight. The addition of Kafka on HDInsight will complete the ingestion piece for scalable open-source streaming on Azure. In addition to the scale and performance benefits of Apache Kafka, HDInsight Kafka customers will get the following advantages:
· The promise of a managed open-source Kafka backed by a 99.9% uptime SLA.
· Managed rebalance of replicas and partitions across Azure update domains and fault domains. This will ensure high availability of Kafka partitions on environments with a multidimensional view of a rack. This tool is also open sourced here.
- An integrated experience to deploy a managed and secure streaming pipeline (Kafka, Storm, or Spark streaming) within minutes via prebuilt architectures on Azure Resource Manager templates.
Learn more about the advantages of Apache Kafka with Azure Managed Disks.