Edit

Retail assistant with visual capabilities

Azure App Service
Bing Custom Search
Bing Visual Search
Azure AI Bot Service
Azure AI services

Solution ideas

This article is a solution idea. If you'd like us to expand the content with more information, such as potential use cases, alternative services, implementation considerations, or pricing guidance, let us know by providing GitHub feedback.

This solution idea uses Azure services with a bot assistant to improve interactions with customers and provide suggestions based on visual information.

Architecture

Architecture diagram that shows how data flows between a user, a bot, and other components, such as language understanding and search services.

Download a Visio file of this architecture.

Dataflow

  1. The user uses an application, which is hosted on Azure App Service, either via a web browser or a mobile device.
  2. App Service communicates with Azure Bot Service to facilitate the interaction between the user and the application.
  3. Bot Service uses Azure Cognitive Services Language Understanding to identify user intents and meaning.
  4. Language Understanding (LUIS) returns the identified user intent to the Azure bot.
  5. The bot passes a visual context input, such as an image, to the Bing Visual Search API.
  6. The API returns output to Bot Service.
  7. Optionally, the bot retrieves more information for user queries within the user's domain by using the Bing Custom Search API.
  8. The Custom Search API returns output to Bot Service.

Components

  • App Service provides a framework for building, deploying, and scaling web apps.
  • Bot Service provides an integrated development environment for bot building.
  • Cognitive Services consists of cloud-based services that provide AI functionality.
  • Azure Cognitive Service for Language is part of Cognitive Services that offers many natural language processing services.
  • Conversational language understanding is a feature of Cognitive Service for Language. This cloud-based API service offers machine-learning intelligence capabilities for building conversational apps. You can use language understanding (LUIS) to predict the meaning of a conversation and pull out relevant, detailed information.
  • The Bing Visual Search API returns data that's related to a given image, such as similar images, shopping sources for purchasing the item in the image, and webpages that include the image.
  • The Bing Custom Search API provides a way to create tailored ad-free search experiences for topics.

Scenario details

This solution features a bot assistant with search integration. The bot can help customers interact with a business application. It can also provide suggestions based on visual information.

Potential use cases

This solution can be used broadly, but is ideal for the retail industry and the travel and hospitality industries.

Next steps