Apache Spark for Azure Synapse In-cluster Caching and Shuffle Service (Preview)
Published date: September 22, 2020
Caching and shuffle are two of the components of infrastructure for Apache Spark that have the greatest impact on performance. These new services, which we have written from scratch, allow the optimization of performance for these components on modern hardware and operating systems. The service is enabled for Apache Spark Pools in Azure Synapse today.