Passing the app state to the dialog script (Kotlin)¶

To provide relevant responses to users, the voice agent must know what is happening in the app: what tab is currently open, what options are enabled and so on.

To get information about the app state, you can use Alan AI’s visual state functionality. The visual state allows you to send an arbitrary JSON object informing about the current app context to the dialog script. You can access this data in the script through the visual object.

In this tutorial, we will get the information about the currently open tab in the app and use it to filter intents and differentiate the command logic for our voice agent.

YouTube

If you are a visual learner, watch this tutorial on Alan AI YouTube Channel.

What you will learn¶

How to get the information about the app visual context
How to access the data passed with the visual state in the dialog script
How to filter intents depending on the app state
How to play different responses depending on the app state

What you will need¶

For this tutorial, we will use the app created in the Building a voice agent for an Android Java or Kotlin app tutorial.

Step 1. Send the app state to the dialog script¶

To send the information about the app state to the script, we first need to call Alan AI’s setVisualState() client API method on the client app side.

In the IDE, open the MainActivity.kt file and add the setVisualState() function to the MainActivity class. In this function, we will use the setVisualState() method to pass the index of the currently open tab to the dialog script.

MainActivity.kt¶

class MainActivity : AppCompatActivity() {
  /// Setting up a visual state
  fun setVisualState(tabIndex: Int) {
    val params = JSONObject()
    try {
      params.put("tab", tabIndex)
    } catch (e: JSONException) {
      e.message?.let { Log.e("AlanButton", it) }
    }
    alanButton?.setVisualState(params.toString())
  }
}

In the onCreate() function, register the addOnTabSelectedListener. In the onTabSelected() method, call the added setVisualState() function and pass the selected tab index to it:

MainActivity.kt¶

class MainActivity : AppCompatActivity() {
  override fun onCreate(savedInstanceState: Bundle?) {
    val tabs: TabLayout = binding.tabs
    tabs.setupWithViewPager(viewPager)
    /// Registering the listener
    tabs.addOnTabSelectedListener(object : TabLayout.OnTabSelectedListener {
      override fun onTabSelected(tab: TabLayout.Tab) {
        /// Sending the current tab index
        val position = tabs.selectedTabPosition
        setVisualState(position)
      }
      override fun onTabUnselected(tab: TabLayout.Tab) {}
      override fun onTabReselected(tab: TabLayout.Tab) {}
    })
  }
}

Additionally, in the onCreate() function, call the setVisualState() function and pass 0 to it to make sure the visual state is set when the app is initially loaded.
MainActivity.kt¶
```
class MainActivity : AppCompatActivity() {
  setVisualState(0)
}
```

Step 2. Use the visual state as a filter¶

With the visual state set in the app, we can now use it in the dialog script. First, we will make sure our navigation commands can be given only on the tabs where they make sense.

In Alan AI Studio, update the navigation intents to the following:

Dialog script¶

const firstTab = visual({"tab": 0});

intent(firstTab, 'Open the second tab', p => {
    p.play({command: "openTab", tab: 1});
    p.play('Opening the second tab');
});

const secondTab = visual({"tab": 1});

intent(secondTab, 'Go back', p => {
    p.play({command: "openTab", tab: 0});
    p.play('Going back to the first tab');
});

Here we are using the visual state as a filter added to intents. This filter defines when the intent can be matched.

You can test it: in the app, try giving navigation commands on different tabs. If the filter conditions are not met, the voice agent will not be able to reply to you.

Step 3. Differentiate the command logic¶

Next, let’s use the visual state to provide different responses if the same question is asked on different tabs.

Add the following intent to the dialog script:

Dialog script¶

intent('What tab is currently open?', p => {
    let tab = p.visual.tab;
    switch (tab) {
        case 0:
            p.play('This is the first tab');
            break;
        case 1:
            p.play('This is the second tab');
            break;
        default:
            p.play('Sorry, I do not know the answer');
    }
});

Here, if the visual state is set to 0 or 1, the voice agent will name the currently open tab. If no visual state is set, the default response will be played to the user.

What’s next?¶

Have a look at the next tutorial: Sending data to the dialog script.