Microsoft Video Indexer expands functionality unlocking more video insights

Posted on September 13, 2017

Principal Program Manager, Azure Media Services

In May 2017, we announced the global public preview of the Video Indexer service at the Microsoft Build Developers Conference in Seattle. As part of Microsoft Cognitive Services, Video Indexer was introduced as a unique integrated bundling of Microsoft's cloud-based artificial intelligence and cognitive capabilities applied specifically for video content. It has since become the industry's most comprehensive Video AI service available, making it easy to extract insights from videos. For the last four months, we have received a tremendous, enthusiastic response from our global customers and partners who began using Video Indexer within their own in-house workflows and video solutions. With the International Broadcasters Conference (IBC) just around the corner, we are excited to share significant feature updates to the Video Indexer service.

Additionally, we are very pleased that several customers and partners have already integrated Video Indexer within their own video solutions and product offerings. A few examples are as follows:

  • Ooyala has integrated Video Indexer into their Flex Media Logistics platform to enable auto-transcription, translation, content-aware advertising insertion, and better content monetization and search.
  • Zone TV is using Video Indexer to automate the curation of a first-of-its-kind customizable suite of linear TV channels.
  • Axle Video is taking advantage of Video Indexer to automatically tag entire video files and video segments to greatly accelerate the post-production process and efficient search across video libraries.

Overall Service updates since May

  • Speech-to-text – Video Indexer now supports Egyptian Arabic for speech-to-text.
    • With the addition of Arabic, Video Indexer portal has been updated to support RTL (right to left) text in the transcript tab as well as the search page.
  • Captions – You can now start playing captions automatically in the Video Indexer player by adding showCaptions=true to the iframe embed player URL.
  • Download insights from portal You can now download the JSON insights extracted by Video Indexer via the portal.
  • Playback speeds The Video Indexer player was updated to support different playback speeds.
  • Inline transcript editing – The transcript panel has been updated to support inline editing.
  • Editing in insights widget – The insights widget has been updated to support editing in embedded mode (based on an access token).
  • Keywords editing – The insights widget has been updated to enable the addition and removal of keywords.
  • Annotations – The insights widget now shows the annotations extracted by the Video Indexer engine.

API updates

The Video Indexer API reference page provides details about the APIs. Some APIs are new, while some have been enhanced with new features.

  • Upload – The Upload API has been enhanced with the following:
    • callbackUrl – This enables you to receive a callback when the indexing operation finishes on a video.
    • indexingPreset – You can specify if you want to use the default preset or only perform audio indexing.
    • streamingPreset – With this parameter, you can control whether the video gets encoded to multiple bitrates or not. You may want to disable encoding if you are only interested in extracting insights from your videos. This would help improve the turn around on processing the video.
  • Re-Index Breakdown – This is a new API and will enable you to reprocess a video by re-triggering insights extraction on an existing video in your account.
  • Re-Index Breakdown by External Id – Similar to the above, but this API will enable you to use the external id versus the id provided by Video Indexer to trigger the re-indexing of the video.
  • Update Transcript – This API enables you to update the transcript for a video by providing a JSON string containing the full video VTT.
  • Update Face Name – This is an API for providing a friendly name to the faces detected by Video Indexer.
  • Get Insights Widget URL – This API has been updated with an optional parameter called “allowEdit” that would allow the user to get an access token for editing in embedded mode.
  • Get Insights Widget URL by External Id – Similar to the above, but this API will enable you to use the external id versus the id provided by Video Indexer.

Other updates

  • Logic Apps – With the Video Indexer connector for Logic Apps, you can now setup custom workflows connecting your most used apps with Video Indexer to further automate the process of extracting insights from your video. You can read more about this in the Video Indexer connector for Logic Apps blog.
  • Microsoft Flow With the integration between Video Indexer and Microsoft Flow, users can now extract insights from their business videos without writing a single line of code.
  • UI updates – The Video Indexer web portal UI has been updated with several enhancements based on feedback provided by customers.

Besides all the above listed updates, Video Indexer backend services have been updated to make the service more robust and efficient. Looking ahead, Video Indexer will be making progress along the following dimensions:

  • More video AI algorithms to extract additional metadata.
  • Improving accuracy of video AI technologies.
  • More intelligent correlation among the extracted metadata to provide human friendly insights.
  • User interface enhancement to showcase the extracted metadata and insights.
  • Paid offering of Video Indexer.

Please submit your feedback related to Video Indexer on uservoice. Also, you can track updates from Video Indexer team by following us on Twitter (@Video_Indexer). Members of the Video Indexer team will be present at the Microsoft Stand (Hall 15, Stand 1, 35, and 36). Come see us to learn more!