Ingestion data isolation in Log Analytics
Published date: 26 August, 2020
Azure Monitor is a high scale data service built to serve thousands of customers sending terabytes of data each month at a growing pace. As for any multitenancy platform, some limits must be put to protect customers from sudden ingestion spikes that can affect customers sharing the environment and resources.
Up until now there was an ingestion volume rate limit for Azure resource data coming via Diagnostic Settings. Now we have added the limit to other Log Analytics data sources including Diagnostic Settings, agents, and data collector API. The limit is enforced on compressed data approximated as 6 GB/min uncompressed. The actual size can vary between data types, depending on the log length and its compression ratio.
How does this effect you?
We have scanned and increased the threshold for any workspace that was nearing the ingestion volume rate limit, so there should not be an immediate effect on you. When you send data to a workspace at a volume rate higher than 80% of the workspace threshold, or reaching it, an event is sent to the Operation table in your workspace every 6 hours while the threshold continues to be exceeded. We recommend that you set alert on the events to receive these notifications.
How to increase the ingestion volume rate limit threshold in your workspace?
There is no “hard limit” for ingestion volume rate in Log Analytics. When you get alerts for reaching 80% of ingestion volume rate threshold, or you plan to increase it close to 6 GB/min or more you can request to increase it by opening a support request.