The Direct Line Speech Client is a Windows Presentation Foundation (WPF) application in C# that makes it easy to test interactions with your bot before creating a custom client application. It demonstrates how to use the Azure Speech Services Speech SDK to manage communication with your Azure Bot-Framework bot. To use this client, you need to register your bot with the Direct Line Speech channel.
- Fully configurable to support any bot registered with the Direct Line Speech channel
- Accepts typed text and speech captured by a microphone as inputs for your bot
- Supports playback of audio response
- Supports use of custom wake-words
- Supports sending custom Bot-Framework Activities as JSON
- Displays Adaptive Cards sent from your bot (with some limitations)
- Exports the transcript and activity logs to a file
Let's review the hardware, software, and subscriptions that you'll need to use this client application.
- Windows 10 PC
- Visual Studio 2017 or higher
- Microphone access
- An Azure Speech Services Key
- A Bot-Framework bot service registered with Direct Line Speech channel
- The first step is to clone the repository:
bash git clone https://github.com/Azure-Samples/Cognitive-Services-Direct-Line-Speech-Client.git
- Then change directories:
bash cd Cognitive-Services-Direct-Line-Speech-Client
- Launch Visual Studio 2017, then open the solution for the Direct Line Speech Client:
DLSpeechClient.sln. The solution is in the root of the cloned repository.
- Run the executable. For example, for Release x64 build:
- When you first run the application, the Setting page will open. The first two fields are required (all others are optional):
- Enter Subscription key. This is your Azure Speech Services Key
- Enter Subscription key region. This is the Azure region of your key in the format specified by the "Speech SDK Parameter" column in this table (for example "westus")
- The default input language is "en-us" (US English). Update the Language field as needed to select a different language code from the "Speech-to-text" list.
- Press Ok when you're done.
- Your entires will be saved and populated automatically when you launch the app again.
- In the main window, enter your Bot Secret. This is one of the two channel secret keys provided when you registered your Bot-Framework bot with the Direct Line Speech channel.
- Press Reconnect. The application will try to connect to your bot via Direct Line Speech channel. The message New conversation started -- type or press the microphone button will appear below the text bar if the connection succeeded.
- You'll be prompted to allow microphone access. If you want to use the microphone, allow access.
- Press the microphone icon to begin recording. While speaking, intermediate recognition results will be shown in the application. The microphone icon will turn red while recording is in progress. It will automatically detect end of speech and stop recording.
If everything works, you should see your bot's response on the screen and hear it speak the response. You can click on lines in the Activity Log window to see the full activity payload from the bot in JSON. Note: You'll only hear the bot's voice response if the Speak field in the bot's output activity was set.
If an error messages was shown in red in the main application window, use this table to troubleshoot:
| Error | What should you do? |
|----App Error (see log for details): Microsoft.CognitiveServices.Speech.csharp : Value cannot be null. Parameter name: speechConfig | This is a client application error. Make sure you have a non-empty value for Bot Secret in the main app window (see section Register your bot with Direct Line Speech channel) |
|Error AuthenticationFailure : WebSocket Upgrade failed with an authentication error (401). Please check for correct subscription key (or authorization token) and region name| In the Settings page of the application, make sure you entered the Speech Subscription key and its region correctly.
Make sure your Bot Secret was entered correctly. | |Error ConnectionFailure : Connection was closed by the remote host. Error code: 1011. Error details: We could not connect to the bot before sending a message | Make sure you checked the "Enable Streaming Endpoint" box and/or toggled "Web sockets" to On.
Make sure your Azure App Service is running. If it is, try restarting your App Service.| |Error ConnectionFailure : Connection was closed by the remote host. Error code: 1011. Error details: Response status code does not indicate success: 500 (InternalServerError)| Your bot specified a Neural Voice in its output Activity Speak field, but the Azure region associated with your Speech subscription key does not support Neural Voices. See Standard and neural voices.| |Error ConnectionFailure : Connection was closed by the remote host. Error code: 1000. Error details: Exceeded maximum websocket connection idle duration(> 300000ms)| This is an expected error when the client left the connection to the channel open for more than 5 minutes without any activity |
See also Debugging section in Voice-first virtual assistants Preview: Frequently asked questions
Sending custom activities to your bot
Direct Line Speech Client allows you to author and send a custom JSON activity to your bot. This is done using the "Custom Activity" bar at the bottom of the main window and the "New", "Edit" and "Send" buttons. Enter a valid JSON format that conforms to the Bot-Framework Activity schema. An example is given in the file example.json.
- Bot Framework docs:
- Speech SDK docs:
- About custom voice-first virtual assistants
- Voice-first virtual assistants: Frequently asked questions
- Troubleshoot the Speech SDK
- Quickstart: Create a voice-first virtual assistant with the Speech SDK, UWP
- Quickstart: Create a voice-first virtual assistant with the Speech SDK, Java
- Quickstart: Create a voice-first virtual assistant in Java on Android by using the Speech SDK
- Create a custom wake word by using the Speech service
- Try Speech Services for free
- Language and region support for the Speech Services
- Adaptive Cards