Azure Analysis Services tabular models can store data in a highly-compressed, in-memory cache for optimized query performance. This provides fast user interactivity over large data sets.
Large datasets normally require table partitioning to accelerate and optimize the data-load process. Partitioning enables incremental loads, increases parallelization, and reduces memory consumption. The Tabular Object Model (TOM) serves as an API to create and manage partitions.
The Automated Partition Management for Analysis Services Tabular Models whitepaper is available for review. It describes how to use the AsPartitionProcessing TOM code sample with minimal code changes. It is intended to be generic and configuration driven.
The sample is compatible with Azure Analysis Services. The following diagram shows an example architecture.
Azure SQL Database is used for the configuration and logging database.
Azure Functions is used for execution, and can be triggered in various ways. The following list shows some of the many options available with Azure Functions.
- Scheduled using a Timer function CRON expression. In this case, it is not necessary to set up a separate scheduling system.
- Using a webhook request for a WebHook function, or an HTTP request for an HttpTrigger function. This allows integration with existing scheduling systems that can call a URL.
- Triggered from Azure Queue using built-in integration points in Azure Functions.
Thanks to Marco Russo (SQLBI) and Bill Anton (Opifex Solutions) for their contributions to the whitepaper and code sample.