HDInsight HBase Accelerated Writes is now generally available
Published date: November 06, 2019
HDInsight HBase Accelerated Writes is now generally available.
Highlights
- Improve your Apache HBase and Phoenix write performance by up to 9 times.
- Best suited for high throughput and low latency workloads running on Azure Data Lake Storage Gen 2 and Azure Blob Storage.
The separation of compute and storage in HDInsight clusters offers the benefits of scalability, elasticity, agility, and economy for customers. However, this can sometimes incur the costs of higher latencies due to network layers.
In the current implementation, write ahead logs (WALs) in the HDInsight HBase cluster are stored as page blobs in Azure standard Storage. Thus, the Azure file system is one of the few cloud offerings that satisfy consistency requirements for HBase write ahead log.
This architecture works very well for most applications, but it may not be ideal for mission critical applications that require consistent low latency and high throughput I/O as inconsistent latencies for WAL writes can be a bottleneck for overall write performance in HDInsight HBase clusters.
The accelerated writes feature in the HDInsight HBase cluster attaches a Premium SSD managed disk to every region server (worker node). Write ahead logs are configured to be written to Hadoop Distributed File System (HDFS) mounted on premium managed disks instead of standard Azure page blobs. Premium managed disks are SSD-based and offer excellent I/O performance with fault tolerance. Using premium managed disks for WALs allows us to separate write ahead log storage from Apache HBase data storage.
Lean more Azure HDInsight accelerated writes for Apache HBase.