Telemetry Platform Features in Azure Media Services

Publisert på 6 desember, 2016

Program Manager, Azure Media Services

We are excited to announce new Azure Media Services (AMS) telemetry platform features, generally available through our new Telemetry API. Media Services telemetry allows you to monitor and measure the health of your services through a suite of telemetry data. Telemetry data is written to your Azure Storage account and can be processed and visualized using a wide array of data visualization tools.

Consuming Telemetry Data

This release includes telemetry metrics for Channel, Streaming Endpoint, and Archive entities. Telemetry data is written to an Azure Storage table in the storage account specified when configuring telemetry for your media services account. Telemetry data is stored in aggregate in a table, “TelemetryMetricsYYYYMMDD,” for each day’s data (where “YYYYMMDD” denotes the date timestamp).

Each table entry contains a set of common fields and a record with a set of entity-specific fields. The entry identifying fields include the following:

Property Value Example
PartitionKey {Account ID}_{Entity ID} e49bef329c29495f9b9570989682069d_64435281c50a4dd8ab7011cb0f4cdf66
RowKey {Seconds to Midnight}_{Random Value} 01688_00199
Timestamp The time at which the row entry was created 2016-09-09T22:43:42.241Z
Type The type of the entity providing telemetry Channel
Name The name of the telemetry event ChannelHeartbeat
ObservedTime The time at which the event occurred 2016-09-09T22:42:36:924Z
ServiceID Service ID f70bd731-691d-41c6-8f2d-671d0bdc9c7e
Entity-specific Properties {Record as defined by the event} {Record}

The account ID is included in the partition key to simplify workflows where multiple media services accounts are writing data to the same storage account. The row key start with the number of seconds to midnight to allow top n style data queries within a partition (see the log tail table design pattern for more information). The observed timestamp is an approximate measure provided by the entity reporting telemetry.

Entity-Specific Telemetry

The data in each telemetry row represents an aggregation of telemetry events raised over an aggregation time window, listed below. Each entity pushes telemetry with the following frequencies:

  • Channels: Every 60 seconds
  • Streaming Endpoints: Every 30 seconds
  • Archive: Every 60 seconds

Below is the schema description for channels, streaming endpoints, and archive entities.

Channels

Property Value Example
TrackType Type of track video
TrackName Name of the track video
Bitrate Expected bitrate of the track 785,000
IncomingBitrate Incoming bitrate of the track 784,548
OverlapCount Number of overlapping fragments received in ingest 0
DiscontinuityCount Number of discontinuities detected in ingest 0
LastTimestamp Last ingested data timestamp 1800488800
NonincreasingCount Count of fragments discarded due to non-increasing timestamp 0
UnalignedKeyFrames Boolean on whether we received fragments where key frames are not aligned False
UnalignedPresentationTIme Boolean on whether we received fragments where presentation time is not aligned False
UnexpectedBitrate Boolean on whether the IncomingBitrate and Bitrate differ by more than 50% or if IncomingBitrate for an audio or video track is less than 40 kbps if Bitrate is 0 False
Healthy Boolean on whether OverlapCount, DiscontinuityCount, NonincreasingCount, UnalignedKeyFrames, UnalignedPresentationTime, and UnexpectedBitrate are all zero or false True
CustomAttributes Placeholder for custom attributes  

Streaming Endpoints

Property Value Example
HostName Hostname of the endpoint builddemoserver.origin.mediaservices.windows.net
StatusCode HTTP status code 200
ResultCode HTTP result code detail S_OK
RequestCount Total requests received within the last aggregation window 3
BytesSent Total bytes sent within the last aggregation window 2,987,358
ServerLatency Average server latency including storage in milliseconds 130
E2ELatency Average end-to-end latency in milliseconds 250

Archive

Property Value Example
ManifestName Name of the manifest asset-eb149703-ed0a-483c-91c4-e4066e72cce3/a0a5cfbf-71ec-4bd2-8c01-a92a2b38c9ba.ism
TrackName Name of the track audio_1
TrackType Type of track audio
Bitrate Track bitrate 785,000
Healthy Boolean on whether there were no discarded fragments or archive acquisition errors in storage True
CustomAttributes Placeholder for custom attributes  

The schema above is designed to give good performance within the limits of Azure Table Storage:

  • Data is partitioned by account ID and service ID to allow telemetry from each service to be queried independently
  • Partitions contain the date to give a reasonable upper bound on the partition size
  • Row keys are in reverse time order to allow the most recent telemetry items to be queried for a given service

This should allow many of the common queries to be efficient:

  • Parallel, independent downloading of data for separate services
  • Retrieving all data for a given service in a date range
  • Retrieving the most recent data for a service

Visualizing Telemetry Data

Your Azure Storage account can export data to data visualization tools such as PowerBI, Application Insights, and AMS Live Monitoring Dashboard, among many others. Below is an example of how this data can be imported into and visualized directly with PowerBI.

First, select the telemetry tables for the days you are interested in visualizing. To import your data, select Microsoft Azure Table Storage from the Get Data menu. Enter your storage account credentials and select the tables to import. Next, by selecting Content.ObservedTime as the time axis and Average of Content.Healthy (casted to decimal representation) as the value, plot the health of your channels and archive entities as a line graph. To plot channel health and archive health separately, add a filter on Content.Name. This visualization illustrates entity health where a value of 1 represents perfectly healthy and a value of 0 represents perfectly unhealthy for the given time. Below are examples of the channel and archive health plotted over time.

Health Over Time

To visualize the overall health of your services, collapse the time dimension by plotting the data as a pie chart. Selecting Content.Healthy as the legend and Count of Content.Healthy as the value, produces the visualization below.

Overall Health Percentage

We’re excited to share these telemetry features and hope they provide useful information about the health of your Azure media services.

Thanks,

Azure Media Services Streaming Team