Query Acceleration for Azure Data Lake Storage is now generally available
Published date: September 10, 2020
Query Acceleration for Azure Data Lake Storage (ADLS) is now generally available in all Azure regions. Query Acceleration for ADLS empowers the explosion of data-driven decision making that is motivating businesses to have a data strategy to provide better customer experiences, improve operational efficiencies, and make real-time decisions based on data. Query Acceleration achieves this by improving both performance and cost when gaining insights, processing highly-scalable volumes of data.
How Query Acceleration for Azure Data Lake improves performance and cost
Big data analytics frameworks, such as Spark and Hive, work by reading all of the data using a horizontally-scalable distributed computing platform with techniques such as MapReduce. However, a given query or transformation generally does not require all of the data to achieve its goal. Therefore, applications typically incur the costs of reading, transferring over the network, parsing into memory and finally filtering out the majority of the data that is not required. Given the scale of such data lake deployments, these costs become a major factor that impacts the design and how ambitious you can be. Improving cost and performance at the same time enhances how much valuable insight you can extract from your data.
Query Acceleration for Azure Data Lake Storage allows applications and frameworks to push-down predicates and column projections, so they may be applied at the time data is first read, meaning that all downstream data handling is saved from the cost of filtering and processing unrequired data.
Getting Started with Query Acceleration
You can read the documentation for Query Acceleration. A guide that demonstrates the use of Query Acceleration in four different programming languages is also available. Finally, the SQL Reference documentation formally describes the supported SQL grammar to push-down predicates and projections.
For information on prices for Query Acceleration, please consult the Pricing Page.