Application Map: Triage distributed applications

Application maps represent the logical structure of a distributed application. Individual components of the application are determined by their "roleName" or "name" property in recorded telemetry. These components are represented as circles on the map and are referred to as "nodes." HTTP calls between nodes are represented as arrows connecting these nodes, referred to as "connectors" or "edges." The node that makes the call is the "source" of the call, and the receiving node is the "target" of the call.

Application Map helps you spot performance bottlenecks or failure hotspots across all components of your distributed application. Each node on the map represents an application component or its dependencies and has health KPI and alerts status. You can select any component to get more detailed diagnostics, such as Application Insights events. If your app uses Azure services, you can also select Azure diagnostics, such as SQL Database Advisor recommendations.

Application Map also features Intelligent view to assist with fast service health investigations.

What is a component?

Components are independently deployable parts of your distributed or microservice application. Developers and operations teams have code-level visibility or access to telemetry generated by these application components. For example:

  • Components are different from "observed" external dependencies, such as Azure SQL and Azure Event Hubs, which your team or organization might not have access to (code or telemetry).
  • Components run on any number of server, role, or container instances.
  • Components can be separate Application Insights resources, even if subscriptions are different. They can also be different roles that report to a single Application Insights resource. The preview map experience shows the components regardless of how they're set up.

Composite application map

You can see the full application topology across multiple levels of related application components. Components can be different Application Insights resources or different roles in a single resource. The application map finds components by following HTTP dependency calls made between servers with the Application Insights SDK installed.

This experience starts with progressive discovery of the components. When you first load Application Map, a set of queries is triggered to discover the components related to this component. A button at the upper-left corner updates with the number of components in your application as they're discovered.

When you select Update map components, the map is refreshed with all components discovered until that point. Depending on the complexity of your application, this update might take a minute to load.

If all the components are roles within a single Application Insights resource, this discovery step isn't required. The initial load for such an application will have all its components.

Screenshot that shows an example of an application map.

One of the key objectives with this experience is to be able to visualize complex topologies with hundreds of components.

Select any component to see related insights and go to the performance and failure triage experience for that component.

Diagram that shows application map details.

Investigate failures

Select Investigate failures to open the Failures pane.

Screenshot that shows the Investigate failures button.

Screenshot that shows the Failures screen.

Investigate performance

To troubleshoot performance problems, select Investigate performance.

Screenshot that shows the Investigate performance button.

Screenshot that shows the Performance screen.

Go to details

The Go to details button displays the end-to-end transaction experience, which offers views at the call stack level.

Screenshot that shows the Go to details button.

Screenshot that shows the End-to-end transaction details screen.

View in Logs (Analytics)

To query and investigate your applications data further, select View in Logs (Analytics).

Screenshot that shows the View in Logs (Analytics) button.

Screenshot that shows the Logs screen with a line graph that summarizes the average response duration of a request over the past 12 hours.

Alerts

To view active alerts and the underlying rules that cause the alerts to be triggered, select Alerts.

Screenshot that shows the Alerts button.

Screenshot that shows a list of alerts.

Set or override cloud role name

Application Map uses the cloud role name property to identify the components on the map.

Follow this guidance to manually set or override cloud role names and change what appears on the application map.

Note

The Application Insights SDK or Agent automatically adds the cloud role name property to the telemetry emitted by components in an Azure App Service environment.

Write custom TelemetryInitializer

using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;

namespace CustomInitializer.Telemetry
{
    public class MyTelemetryInitializer : ITelemetryInitializer
    {
        public void Initialize(ITelemetry telemetry)
        {
            if (string.IsNullOrEmpty(telemetry.Context.Cloud.RoleName))
            {
                //set custom role name here
                telemetry.Context.Cloud.RoleName = "Custom RoleName";
                telemetry.Context.Cloud.RoleInstance = "Custom RoleInstance";
            }
        }
    }
}

ASP.NET apps: Load initializer in the active TelemetryConfiguration

In ApplicationInsights.config:

    <ApplicationInsights>
      <TelemetryInitializers>
        <!-- Fully qualified type name, assembly name: -->
        <Add Type="CustomInitializer.Telemetry.MyTelemetryInitializer, CustomInitializer"/>
        ...
      </TelemetryInitializers>
    </ApplicationInsights>

An alternate method for ASP.NET Web apps is to instantiate the initializer in code, for example, in Global.aspx.cs:

 using Microsoft.ApplicationInsights.Extensibility;
 using CustomInitializer.Telemetry;

    protected void Application_Start()
    {
        // ...
        TelemetryConfiguration.Active.TelemetryInitializers.Add(new MyTelemetryInitializer());
    }

Note

Adding an initializer by using ApplicationInsights.config or TelemetryConfiguration.Active isn't valid for ASP.NET Core applications.

ASP.NET Core apps: Load an initializer to TelemetryConfiguration

For ASP.NET Core applications, to add a new TelemetryInitializer instance, you add it to the Dependency Injection container, as shown. You do this step in the ConfigureServices method of your Startup.cs class.

 using Microsoft.ApplicationInsights.Extensibility;
 using CustomInitializer.Telemetry;
 public void ConfigureServices(IServiceCollection services)
{
    services.AddSingleton<ITelemetryInitializer, MyTelemetryInitializer>();
}

Understand the cloud role name within the context of an application map

To help you understand the concept of cloud role names, look at an application map that has multiple cloud role names present.

Screenshot that shows an application map example.

In the application map shown, each of the names in green boxes is a cloud role name value for different aspects of this particular distributed application. For this app, its roles consist of Authentication, acmefrontend, Inventory Management, and Payment Processing Worker Role.

In this app, each of the cloud role names also represents a different unique Application Insights resource with its own instrumentation keys. Because the owner of this application has access to each of those four disparate Application Insights resources, Application Map can stitch together a map of the underlying relationships.

For the official definitions:

   [Description("Name of the role the application is a part of. Maps directly to the role name in azure.")]
    [MaxStringLength("256")]
    705: string      CloudRole = "ai.cloud.role";

    [Description("Name of the instance where the application is running. Computer name for on-premises, instance name for Azure.")]
    [MaxStringLength("256")]
    715: string      CloudRoleInstance = "ai.cloud.roleInstance";

Alternatively, cloud role instance can be helpful for scenarios where a cloud role name tells you the problem is somewhere in your web front end. But you might be running multiple load-balanced servers across your web front end. Being able to drill in a layer deeper via Kusto queries and knowing if the issue is affecting all web front-end servers or instances or just one can be important.

Intelligent view A scenario when you might want to override the value for cloud role instance could be if your app is running in a containerized environment. In this case, just knowing the individual server might not be enough information to locate a specific issue.

For more information about how to override the cloud role name property with telemetry initializers, see Add properties: ITelemetryInitializer.


Application Map Filters

Application Map filters allow the user to reduce the number of nodes and edges shown by applying one or more filters. These filters can be used to reduce the scope of the map, showing a smaller and more focused map.

Creating Application Map filters

To create a filter, select the "Add filter" button in the application map's toolbar.

A screenshot of the Add Filter button.

This pops up a dialog with three sections: 1) Select filter type, 2) Choose filter parameters, and 3) Review.

A screenshot with the Node Filter radio button selected.

A screenshot with the Connector Edge Filter radio button selected.

The first section has two options:

  1. Node filter
  2. Connector (edge) filter

The contents in the other sections change based on the option selected.

Node filters

Node filters allow the user to leave only selected nodes on the map and hide the rest. A node filter checks each node if it contains a property (its name, for example) with a value that matches a search value through a given operator. If a node is removed by a node filter, all of its connectors (edges) are also removed.

There are three parameters available for nodes:

  • "Nodes included" allows the user to select only nodes with matching properties or to also include source nodes, target nodes, or both in the resulting map.

    • "Nodes and sources, targets"--This means nodes that match the search parameters will be included in the resulting map, and nodes that are sources or targets for the matching node will also be included, even if they don't have property values that match the search. Source and target nodes are collectively referred to as "Connected" nodes.

    • "Nodes and sources"--Same as above, but target nodes aren't automatically included in the results.

    • "Nodes and targets"--Same as above, but source nodes aren't automatically included.

    • "Nodes only"--All nodes in the resulting map must have a property value that matches.

  • "Operator" is the type of check that will be performed on each node's property values:

    • contains

    • !contains (not contains)

    • == (equals)

    • != (not equals)

  • "Search value" is the text that has to be contained, not contained, equal, or not equal to a node property value. Some of the values found in nodes that are on the map are shown in a drop-down. Any arbitrary value can be entered by clicking "Create option ..." in the drop-down.

For example, in the screenshot below, the filter is being configured to select Node(s) that contain(s) the text "-west". Source and target nodes will also be included in the resulting map. In the same screenshot, the user is able to select one of the values found in the map or to create an option that isn't an exact match to one found in the map.

A screenshot with the filter configured to select nodes that contain the text west.

Connector (edge) filters

Connector filters examine the properties of a connector to match a value. Connectors that don't match the filter are removed from the map. The same happens to nodes with no connectors left.

Connector filters require three parameters:

  • "Filter connectors by" allows the user to choose which property of a connector to use:

    • "Error connector (highlighted red)" selects connectors based on their color (red or not). A value can't be entered for this type of filter, only an operator that is "==" or "!=" meaning "connector with errors" and "connector without errors."

    • "Error rate" uses the average error rate for the connector---the number of failed calls divided by the number of all calls---expressed as a percentage. For example, a value of "1" would refer to 1% failed calls.

    • "Average call duration (****ms)" uses just that: the average duration of all calls represented by the connector, in milliseconds. For example, a value of "1000" would refer to calls that averaged 1 second.

    • "Calls count" uses the total number of calls represented by the connector.

  • "Operator" is the comparison that will be applied between the connector property and the value entered below. The options change: "Error connector" has equals/not equals options; all others have greater/less than.

  • "Value" is the comparison value for the filter. There's only one option for the "Error connector" filter: "Errors." Other filter types require a numeric value and offer a drop-down with some pre-populated entries relevant to the map.

    • Some of these entries have a designation "(Pxx)" which are percentile levels. For example, "Average call duration" filter may have the value "200 (P90)" which indicates 90% of all connectors (regardless of the number of calls they represent) have less than 200 ms call duration..

    • When a specific number isn't shown in the drop-down, it can be typed, and created by clicking on "Create option." Typing "P" shows all the percentile values in the drop-down.

Review section

The Review section contains textual and visual descriptions of what the filter will do, which should be helpful when learning how filters work:

A screenshot of the Review section with node in focus.

A screenshot of the Review section depicting an average call duration greater than 42 milliseconds.

Using filters in Application Map

Filter interactivity

After configuring a filter in the "Add filter" pop-up, select "Apply" to create the filter. Several filters can be applied, and they work sequentially, from left to right. Each filter can remove further nodes and connectors, but can't add them back to the map.

The filters show up as rounded buttons above the application map:

A screenshot displaying the rounded filter buttons above the application map.

Clicking the A screenshot of a rounded X button. on a filter will remove that filter. Clicking elsewhere on the button allows the user to edit the filter's values. As the user changes values in the filter, the new values are applied so that the map is a preview of the change. Clicking "Cancel" restores the filter as it was before editing.

A screenshot displaying the Configure Connector Filter section with a Cancel button.

Reusing filters

Filters can be reused in two ways:

  • The "Copy link" button on the toolbar above the map encodes the filter information in the copied URL. This link can be saved in the browser's bookmarks or shared with others. "Copy link" preserves the duration value, but not the absolute time, so the map shown at a later time may be different from the one observed when the link was created.

  • The dashboard pin A screenshot displaying the dashboard pin button. is located next to the title bar of the Application Map pane. This button pins the map to a dashboard, along with the filters applied to it. This action can be useful for filters that are frequently interesting. As an example, the user can pin a map with "Error connector" filter applied to it, and the dashboard view will only show nodes that have errors in their HTTP calls.

Filter usage scenarios

There are many filter combinations. Here are some suggestions that apply to most maps and may be useful to pin on a dashboard:

  • Show only errors that appear significant by using the "Error connector" filter along with "Intelligent view":
    A screenshot displaying the Last 24 hours and Highlighted Errors filters. A screenshot displaying the Intelligent Overview toggle.

  • Hide low-traffic connectors with no errors to quickly focus on issues that have higher impact: A screenshot displaying the Last 24 hours, calls greater than 876, and highlighted errors filters.

  • Show high-traffic connectors with high average duration to focus on potential performance issues: A screenshot displaying the Last 24 hours, calls greater than 3057, and average time greater than 467 filters.

  • Show a specific portion of a distributed application (requires suitable roleName naming convention): A screenshot displaying the Last 24 hours and Connected Contains West filters.

  • Hide a dependency type that is too noisy: A screenshot displaying the Last 24 hours and Nodes Contains Storage Accounts filters.

  • Show only connectors that have higher error rates than a specific value A screenshot displaying the Last 24 hours and Errors greater than 0.01 filters.


Application Map Intelligent view

The following sections discuss Intelligent view.

Intelligent view summary

Application Map Intelligent view is designed to aid in service health investigations. It applies machine learning to quickly identify potential root causes of issues by filtering out noise. The machine learning model learns from Application Map's historical behavior to identify dominant patterns and anomalies that indicate potential causes of an incident.

In large distributed applications, there's always some degree of noise coming from "benign" failures, which might cause Application Map to be noisy by showing many red edges. Intelligent view shows only the most probable causes of service failure and removes node-to-node red edges (service-to-service communication) in healthy services. Intelligent view highlights the edges in red that should be investigated. It also offers actionable insights for the highlighted edge.

Intelligent view benefits

  • Reduces time to resolution by highlighting only failures that need to be investigated
  • Provides actionable insights on why a certain red edge was highlighted
  • Enables Application Map to be used for large distributed applications seamlessly (by focusing only on edges marked in red)

Enable Intelligent view in Application Map

Enable the Intelligent view toggle. Optionally, to change the sensitivity of the detections, select Low, Medium, or High. For more information, see the troubleshooting question about sensitivity.

Screenshot that shows the Application Map user interface with options to toggle Intelligent view and provide feedback.

After you enable Intelligent view, select one of the highlighted edges to see the "actionable insights." The insights appear in the pane on the right and explain why the edge was highlighted.

Screenshot that shows the Application Map user interface section about actionable insights.

To begin troubleshooting, select Investigate failures. In the Failures pane that opens, investigate if the detected issue is the root cause. If no edges are red, the machine learning model didn't find potential incidents in the dependencies of your application.

To provide feedback, select the Feedback button on the map.

How does Intelligent view determine where red edges are highlighted?

Intelligent view uses the patented AIOps machine learning model to highlight what's truly important in an application map.

Some example considerations include:

  • Failure rates
  • Request counts
  • Durations
  • Anomalies in the data
  • Types of dependency

For comparison, the normal view only utilizes the raw failure rate.

How does Intelligent view sensitivity work?

Intelligent view sensitivity adjusts the probability that a service issue will be detected.

Adjust sensitivity to achieve the desired confidence level in highlighted edges.

Sensitivity setting Result
High Fewer edges will be highlighted.
Medium (default) A balanced number of edges will be highlighted.
Low More edges will be highlighted.

Limitations of Intelligent view

Intelligent view has some limitations:

  • Large distributed applications might take a minute to load Intelligent view.
  • Time frames of up to seven days are supported.

To provide feedback, see Portal feedback.


Troubleshooting

If you're having trouble getting Application Map to work as expected, try these steps.

General

  1. Make sure you're using an officially supported SDK. Unsupported or community SDKs might not support correlation.

    For a list of supported SDKs, see Application Insights: Languages, platforms, and integrations.

  2. Upgrade all components to the latest SDK version.

  3. If you're using Azure Functions with C#, upgrade to Azure Functions V2.

  4. Confirm the cloud role name is correctly configured.

  5. If you're missing a dependency, make sure it's in the list of autocollected dependencies. If not, you can still track it manually with a track dependency call.

Too many nodes on the map

Application Map constructs an application node for each unique cloud role name present in your request telemetry. A dependency node is also constructed for each unique combination of type, target, and cloud role name.

If there are more than 10,000 nodes in your telemetry, Application Map can't fetch all the nodes and links, so your map will be incomplete. If this scenario occurs, a warning message appears when you view the map.

Application Map only supports up to 1,000 separate ungrouped nodes rendered at once. Application Map reduces visual complexity by grouping dependencies together that have the same type and callers.

If your telemetry has too many unique cloud role names or too many dependency types, that grouping will be insufficient and the map won't render.

To fix this issue, you'll need to change your instrumentation to properly set the cloud role name, dependency type, and dependency target fields.

  • Dependency target should represent the logical name of a dependency. In many cases, it's equivalent to the server or resource name of the dependency. For example, if there are HTTP dependencies, it's set to the hostname. It shouldn't contain unique IDs or parameters that change from one request to another.

  • Dependency type should represent the logical type of a dependency. For example, HTTP, SQL, or Azure Blob are typical dependency types. It shouldn't contain unique IDs.

  • The purpose of cloud role name is described in the Set or override cloud role name section.

Intelligent view

Common troubleshooting questions about Intelligent view.

Why isn't this edge highlighted, even with low sensitivity?

A dependency might appear to be failing but the model doesn't indicate it's a potential incident:

  • If this dependency has been failing for a while, the model might believe it's a regular state, and not highlight the edge for you. It focuses on problem-solving in RT.
  • If this dependency has a minimal effect on the overall performance of the app, that can also make the model ignore it.
  • If none of the above is correct, use the Feedback option and describe your experience. You can help us improve future model versions.

Why is the edge highlighted?

If an edge is highlighted, the explanation from the model should point you to the most important features that made the model give this dependency a high probability score. The recommendation isn't based solely on failures but on other indicators like unexpected latency in dominant flows.

Why doesn't Intelligent view load?

If Intelligent view doesn't load, set the configured time frame to six days or less.

Why does Intelligent view take a long time to load?

Avoid selecting Update map components.

Enable Intelligent view only for a single Application Insights resource.

Screenshot that shows the Update map components button in the Application Map user interface.

Portal feedback

To provide feedback, use the feedback option.

Screenshot that shows the Feedback option.

Next steps