Testing the dialog¶
In-app assistant testing can be a challenging task. While talking to your in-app assistant, users can say virtually anything. That is why it is hard to standardize how to test your multimodal interface.
To lift the burden of the assistant testing, Alan provides several testing tools out of the box. You can use them throughout the whole development cycle to verify your conversational experience.
1. Set up environments¶
As approaching the testing and release stages, set up Development, Testing and Production environments for your Alan Studio project and decide what script versions to publish to a specific environment. We recommend that you:
Keep the working draft in the Development environment
Publish a version to be tested to the Testing environment
Promote the tested script version to Production
Switching voice script versions while dialog sessions are active may cause errors in the assistant’s behavior. Plan to change versions during off-peak hours to avoid user disruption.
2. Test while coding¶
As you write a voice script, you need to check how the newly added features work fast. Alan’s Debugging Chat and logs help make sure your code and logic are correct. Think of this type of tests as unit testing: you can check single intents in your script locally and ensure the code works as expected.
3. Automate testing¶
When the voice script is ready, prepare a test plan and lay out test cases using Test View in Alan Studio. The Test View lets you cover basic scenarios and run tens and hundreds of tests without manual input.
In future, when a new version of your voice script is ready, you can run the existing test cases against it to make sure the previously added functionality works in a proper way.
4. Perform end-to-end testing¶
To ensure the entire system – the voice script and your app – work correctly, perform in-app testing using Alan’s Playground or your app directly. With this type of tests, you can check how your voice-enabled app operates in different environments and on different platforms.
5. Perform crowd testing¶
The more people test your in-app assistant, the more confident you can be to actually release it. Before you roll out your assistant to everyone, you can use Alan’s cohorts to involve a group of end users in beta testing.
Cohorts provide are a great way to collect feedback from the real audience using Alan’s analytics and polish your dialog before publishing it to all of your customers.