• 2 min read

Update: Azure Media Indexer full caption feature and SQL Server Add-on v2.6

Learn about new features available in Azure Media Indexer.

We have heard your feedback, and today we are ready to release a new feature for Azure Media Indexer. 

Force Full Captions

Based on our original algorithm, Azure Media Indexer omitted all words/phrases with a computed confidence score below a certain threshold as a way of reducing erroneous captions. You may be familiar with this if you ever see an ellipsis in your caption files. 

Example with ellipses:

My name is holly shore ...in austin texas

...blogger and now of windows phone developer I

This has been a helpful tool for many users using Indexer in an accessibility scenario. The presence of an ellipsis can denote to users this lack of confidence, potentially kicking off a manual caption auditing workflow to ensure perfect, compliant captions. Some users, however, have been reporting dissatisfaction with these “holes” in our captions. We have since learned that some scenarios call for full captions, that is to say, captions that do not contain these ellipses, and would rather have the “best-guess” word or phrase. 

Example with Force Full Caption:

My name is holly shore live in austin texas

I'm blogger and now of windows phone developer

To use this new feature, add the following key/value pair to your configuration features:

Empty Configuration

We also augmented the resilience of our processor, which now accepts empty configuration files for a good “default” job. In addition to our last release creating good “defaults” for faulty configuration files, we decided using a configuration file should not be a hard requirement to try Azure Media Indexer, but rather an added bonus for extra customization!

An Indexer job with no configuration will return the following files by default:

  • ​Captions
    • TTML
    • SMI
    • VTT
  • Metadata
    • kw.xml (keywords
    • AIB (speech lattice for use with SQL Server Add-On)
    • aib.xml (xml form of AIB speech lattice)
    • .info (word-by-word time stamps and confidence)

As before, you can customize the files returned to you by changing your configuration file as detailed in this previous post.

New SQL Server Add-On (v2.6)

We have received tons of feedback on the SQL Server Add-On designed to be used with the AIB search index output files of Azure Media Indexer. Users still find it to be the number one way to enable the discoverability scenario because it exposes all word-alternatives for video search, greatly reducing the occurrence of false-negative queries. There were, however, some limitations in the queries which we have addressed in this release.

The new version of the Indexer SQL Server Add-On contains the following changes:

  • Support to get​ ​the text snippet at the beginning of the given AIB
  • Support to get the text snippet at the end of the given AIB
  • Get the full snippet duration by specifying the forceMaxContent parameter in the Retrieval.ComputeScore, Retrieval.GetBeginSnippet, Retrieval.GetEndSnippet methods, otherwise the snippet duration will be automatically decided by word confidences within the maximum value
  • Not supported on version of SQL before SQL Server 2012

You can find the new Add-On for download here. 

Not sure how to use the SQL Server Add-On? Check out this blog post to learn more.

As always, feel free to reach out with any questions or comments at indexer@microsoft.com.

Not sure what Azure Media Indexer is? Check out the introductory blog post here.