Today, we are announcing the public preview of metrics in the Job Diagram for data driven monitoring and troubleshooting of Azure Stream Analytics jobs. This new functionality enables the quick and easy isolation of issues to specific components and query steps using metrics on the state of the inputs, outputs, and each step of the processing logic.

If for example, an Azure Stream Analytics job is not producing the expected output, metrics in the Job Diagram can be used to identify query steps that are receiving inputs but not producing any output to identify and isolate issues. Additionally, when one or more inputs to the Steam Analytics job stop producing events, the new capabilities can help identify the inputs and outputs that have pending diagnosis messages associated with them.

To access this capability, click on the “Job diagram” button in the “Settings” blade of the of the Stream Analytics job. For existing jobs, it is necessary to restart the job first.

Every input and output is color coded to indicate the current state of that component.

When you want to look at intermediate query steps to understand the data flow patterns inside Stream Analytics, the visualization tool provides a view of the breakdown of the query into its component steps and the flow sequence. Each logical node shows the number of partitions it has.

Clicking on each query step will show the corresponding section in a query editing pane as illustrated. A metrics chart for the step is also displayed in a lower pane.

Clicking the … will pop up the context menu allowing the expansion of partitions showing the partitions of the Event Hub input in addition to the input merger.

 

Clicking a single partition node will show the metrics chart only for that partition on the bottom.

 

Selecting the merger node will show the metrics chart for the merger. The chart below shows that no events got dropped or adjusted.

 

Hovering the mouse pointer on the chart will show details of the metric value and time.

 

We are very excited to hear your feedback. Please give it a try and let us know what you think.