Public Preview: Modernizing Azure Resource Manager Throttling

Starting in 2024, Azure Resource Manager is introducing a revamped throttling experience for Azure subscriptions. These changes bring fresh limits that enhance your API management. Let’s dive into the details:
 

1. Token Bucket Algorithm:

Azure now employs a token bucket algorithm to manage throttling. Here’s how it works:

• Your account has a bucket that holds tokens (representing requests).
  • For example, the initial bucket size is 250 tokens for read requests.
• Each API request consumes one token from the bucket.
  • For example, the read bucket size is 250 tokens, so you can make up to 250 read requests in one second. If you exceed 250 requests in a second, you are throttled and the remaining requests within that second fail.

As you make requests, the bucket depletes, but fear not—it automatically refills.

2. Automatic Refill:

• The bucket refills at a fixed rate.
• If the bucket is below capacity, tokens are added back every second until it reaches the maximum bucket size.
• It’s like topping up your gas tank—you’re always ready for the next journey.

When refill tokens arrive and the bucket is already at capacity, they are discarded. The bucket has a maximum limit and cannot hold more than its designated number of tokens.

For instance, consider the bucket size for write operations, which is currently set at 200 tokens. If you initiate 200 write requests within a second, the bucket immediately drops to zero tokens. Subsequently, it refills at a rate of 10 tokens per second until it reaches its maximum capacity of 200 tokens. Therefore, if no additional requests are made, the previously empty bucket will reach its maximum capacity after 20 seconds.

You do not have to wait for the bucket to be completely full before making API requests. As tokens are added to the bucket at the refill rate (each second), you can use them as soon as new tokens are added. If you promptly utilize the refill tokens, the bucket won't reach its maximum capacity. For example, with a write bucket size of 200 tokens and a refill rate of 10 tokens per second, even after depleting the bucket with 200 API requests in a second, you can continue making 10 API requests per second. The bucket can refill to its maximum capacity only when you generate fewer than 10 API requests per second.

3. Increased Limits and Scope:

Throttling limits have increased by roughly 30 times for writes, 2.4 times for deletes, and 7.5 times for reads. These represent significant increases compared to previous limits. More requests mean more productivity—go ahead, scale up your operations!

Here are the new limits:

The subscription limits apply per subscription, per service principal, and per operation type. There are also global subscription limits that are equivalent to 15 times the individual service principal limits for each operation type. The global limits apply across all service principals. Tenant requests don't include your subscription ID, such as retrieving valid Azure locations. Requests will be throttled if the global, service principal, or tenant specific limits are exceeded.

Throttling limits are now applied per region, rather than per instance of Azure Resource Manager. This change simplifies management and ensures consistency across regions.
 

4. Smooth Transition:

• The migration to the new architecture will be gradual, with a certain percentage of tenants being migrated at a time.
• Rest assured, your patience during this transition is appreciated.
• Once complete, the current architecture will retire, making way for a more efficient and dynamic throttling experience.

For detailed information and frequently asked questions, consult the Azure Resource Manager documentation. We’re committed to empowering your Azure journey, one token at a time! 🌟