We are excited to announce new Azure Media Services (AMS) telemetry platform features, generally available through our new Telemetry API. Media Services telemetry allows you to monitor and measure the health of your services through a suite of telemetry data. Telemetry data is written to your Azure Storage account and can be processed and visualized using a wide array of data visualization tools.
Consuming Telemetry Data
This release includes telemetry metrics for Channel, Streaming Endpoint, and Archive entities. Telemetry data is written to an Azure Storage table in the storage account specified when configuring telemetry for your media services account. Telemetry data is stored in aggregate in a table, “TelemetryMetricsYYYYMMDD,” for each day’s data (where “YYYYMMDD” denotes the date timestamp).
Each table entry contains a set of common fields and a record with a set of entity-specific fields. The entry identifying fields include the following:
| Property | Value | Example |
| PartitionKey | {Account ID}_{Entity ID} | e49bef329c29495f9b9570989682069d_64435281c50a4dd8ab7011cb0f4cdf66 |
| RowKey | {Seconds to Midnight}_{Random Value} | 01688_00199 |
| Timestamp | The time at which the row entry was created | 2016-09-09T22:43:42.241Z |
| Type | The type of the entity providing telemetry | Channel |
| Name | The name of the telemetry event | ChannelHeartbeat |
| ObservedTime | The time at which the event occurred | 2016-09-09T22:42:36:924Z |
| ServiceID | Service ID | f70bd731-691d-41c6-8f2d-671d0bdc9c7e |
| Entity-specific Properties | {Record as defined by the event} | {Record} |
The account ID is included in the partition key to simplify workflows where multiple media services accounts are writing data to the same storage account. The row key start with the number of seconds to midnight to allow top n style data queries within a partition (see the log tail table design pattern for more information). The observed timestamp is an approximate measure provided by the entity reporting telemetry.
Entity-Specific Telemetry
The data in each telemetry row represents an aggregation of telemetry events raised over an aggregation time window, listed below. Each entity pushes telemetry with the following frequencies:
- Channels: Every 60 seconds
- Streaming Endpoints: Every 30 seconds
- Archive: Every 60 seconds
Below is the schema description for channels, streaming endpoints, and archive entities.
Channels
| Property | Value | Example |
| TrackType | Type of track | video |
| TrackName | Name of the track | video |
| Bitrate | Expected bitrate of the track | 785,000 |
| IncomingBitrate | Incoming bitrate of the track | 784,548 |
| OverlapCount | Number of overlapping fragments received in ingest | 0 |
| DiscontinuityCount | Number of discontinuities detected in ingest | 0 |
| LastTimestamp | Last ingested data timestamp | 1800488800 |
| NonincreasingCount | Count of fragments discarded due to non-increasing timestamp | 0 |
| UnalignedKeyFrames | Boolean on whether we received fragments where key frames are not aligned | False |
| UnalignedPresentationTIme | Boolean on whether we received fragments where presentation time is not aligned | False |
| UnexpectedBitrate | Boolean on whether the IncomingBitrate and Bitrate differ by more than 50% or if IncomingBitrate for an audio or video track is less than 40 kbps if Bitrate is 0 | False |
| Healthy | Boolean on whether OverlapCount, DiscontinuityCount, NonincreasingCount, UnalignedKeyFrames, UnalignedPresentationTime, and UnexpectedBitrate are all zero or false | True |
| CustomAttributes | Placeholder for custom attributes |
Streaming Endpoints
| Property | Value | Example |
| HostName | Hostname of the endpoint | builddemoserver.origin.mediaservices.windows.net |
| StatusCode | HTTP status code | 200 |
| ResultCode | HTTP result code detail | S_OK |
| RequestCount | Total requests received within the last aggregation window | 3 |
| BytesSent | Total bytes sent within the last aggregation window | 2,987,358 |
| ServerLatency | Average server latency including storage in milliseconds | 130 |
| E2ELatency | Average end-to-end latency in milliseconds | 250 |
Archive
| Property | Value | Example |
| ManifestName | Name of the manifest | asset-eb149703-ed0a-483c-91c4-e4066e72cce3/a0a5cfbf-71ec-4bd2-8c01-a92a2b38c9ba.ism |
| TrackName | Name of the track | audio_1 |
| TrackType | Type of track | audio |
| Bitrate | Track bitrate | 785,000 |
| Healthy | Boolean on whether there were no discarded fragments or archive acquisition errors in storage | True |
| CustomAttributes | Placeholder for custom attributes |
The schema above is designed to give good performance within the limits of Azure Table Storage:
- Data is partitioned by account ID and service ID to allow telemetry from each service to be queried independently
- Partitions contain the date to give a reasonable upper bound on the partition size
- Row keys are in reverse time order to allow the most recent telemetry items to be queried for a given service
This should allow many of the common queries to be efficient:
- Parallel, independent downloading of data for separate services
- Retrieving all data for a given service in a date range
- Retrieving the most recent data for a service
Visualizing Telemetry Data
Your Azure Storage account can export data to data visualization tools such as PowerBI, Application Insights, and AMS Live Monitoring Dashboard, among many others. Below is an example of how this data can be imported into and visualized directly with PowerBI.
First, select the telemetry tables for the days you are interested in visualizing. To import your data, select Microsoft Azure Table Storage from the Get Data menu. Enter your storage account credentials and select the tables to import. Next, by selecting Content.ObservedTime as the time axis and Average of Content.Healthy (casted to decimal representation) as the value, plot the health of your channels and archive entities as a line graph. To plot channel health and archive health separately, add a filter on Content.Name. This visualization illustrates entity health where a value of 1 represents perfectly healthy and a value of 0 represents perfectly unhealthy for the given time. Below are examples of the channel and archive health plotted over time.
To visualize the overall health of your services, collapse the time dimension by plotting the data as a pie chart. Selecting Content.Healthy as the legend and Count of Content.Healthy as the value, produces the visualization below.
We’re excited to share these telemetry features and hope they provide useful information about the health of your Azure media services.
Thanks,
Azure Media Services Streaming Team