Step 17: Get the app visual context

We are building a context-aware voice assistant: it should differentiate responses depending on what is currently displayed on the page. Let’s update our voice script to let the user ask about the order details.

  • If no items are added to the cart on the page, the voice assistant will inform the user about it.

  • Otherwise, it will list all items added to the cart.

For this step, we will use Alan’s visual state. This predefined object allows you to get information about the app visual context, such as what page is currently open or what data is displayed, and send it to the voice script.

  1. In the app, update the changeOrder() function to call the setVisualState() method.

    With the help of this method, we will send the information about the visual state of the app to the voice script. To the method, you can pass any object; in our case, it will be the order object we have in the app.

    function changeOrder(item, quantity) {
        let number = (order[item] ? order[item] : 0) + quantity;
        // Removing the item or updating the number of items
        if (number <= 0) {
            delete order[item];
        } else {
            order[item] = number;
        }
        updateCart();
        // Sending the app visual state
        alanBtnInstance.setVisualState({order});
    }
    
  2. To the voice script in Alan Studio, add two voice commands:

    intent(visual(v => !_.isEmpty(v.order)), '(What are|) my order details?', p => {
        p.play('You have ordered:');
        for (let product in p.visual.order) {
            p.play(p.visual.order[product] + " " + product);
        }
    });
    
    intent(visual(v => _.isEmpty(v.order)), '(What are|) my order details?', p => {
        p.play('You have not ordered anything');
    });
    

    The visual state data sent from the app is accessible through Alan’s visual object. In these commands, the object is used as a filter: if the object contains any data, the first command is matched. Otherwise, Alan matches the second command.

  3. To make sure our voice assistant is context-aware, we should also update the voice command for removing items from the cart. If the item is not yet in the cart, Alan must inform the user about it; otherwise, it will remove the named items.

    In the voice script, replace the command for removing items with the following one:

    intent(`(Remove|Delete) $(ITEM ${itemList}), (please|)`,
           `(Remove|Delete) $(NUMBER) $(ITEM ${itemList}), (please|)`, p => {
        let order = p.visual.order || {};
        const item = p.ITEM.label;
        if (!order[item]) {
            p.play(`${p.ITEM.value} is not in the cart yet`);
        } else {
            let quantity = order[item] ? order[item] : 0;
            let deleteQnty = p.NUMBER ? p.NUMBER.number : quantity;
    
            if (quantity - deleteQnty <= 0) {
                p.play(`Removing all ${p.ITEM.value}`);
            } else {
                p.play(`Updating ${p.ITEM.value}`);
            }
            p.play({command: 'updateOrder', item: item, quantity: -deleteQnty});
        }
    });
    

    Web page source | Voice script

    Here is how this command works: Alan checks what is currently available in p.visual.order sent from the app. If the named item is not available, it plays: The item is not in the cart yet. If the named item is added to the order, Alan updates the order to remove the desired quantity of items.

Refresh the web page and ask about the order details. Add some items to the cart and ask about the order details again. Then try adding and removing items with voice.