Documentation

Powered by Algolia

Playing messages on actions in the app

In an app with the multimodal UX, where the voice and gestures are used together as input modes, it may be necessary to trigger Alan messages without voice commands given by the user. For example, you can let Alan play a message when some activity in the app is complete or the user clicks a control on the screen. To do this, you can use Alan's project API.

The project API is special Alan's functionality that allows you to send data from the client app to the voice script and perform activities without voice commands. In this tutorial, we will create a simple webpage on which a YouTube playlist will be displayed. Once the playlist is loaded in the player, Alan will inform the user about it. We will also let the user hear a message when the Show video title button is clicked on the page.

What you will learn

  • How to play messages without a voice command, both on the server and client side
  • How to use the project API
  • How to send data from the client app to the voice script

What you will need

To go through this tutorial, make sure the following prerequisites are met:

Step 1: Create a webpage

First, let's create a webpage with the Alan button:

  1. In Alan Studio, open your project.
  2. At the top of the code editor, click Integrations.
  3. In the Embed Code Example section, click Copy All. Save the copied code as an HTML page.

    <!DOCTYPE html>
    <html lang="en">
    	<head>
    		<meta charset="UTF-8">
    		<title>Alan Example</title>
    		<link href="https://fonts.googleapis.com/css?family=Lato:100,100i,300,300i,400,400i,700,700i,900,900i"
    			rel="stylesheet">
    	</head>
    	<body>
    		<div class="alan-btn"></div>
    		<script type="text/javascript"
    			src="https://studio.alan.app/web/lib/alan_lib.min.js"></script>
    		<script>
    			// Define alanBtnInstance
    			var alanBtnInstance = alanBtn({
    			key: "438c790695a25bd87d43485dbc3cb44a2e956eca572e1d8b807a3e2338fdd0dc/stage",
    			onCommand: function (commandData) {
    				if (commandData.command === "go:back") {
    					//call client code that will react to the received command
    				}
    			},
    			rootEl: document.getElementById("alan-btn"),
    			});
    		</script>
    	</body>
    </html>

Open the created webpage and make sure there is the Alan button in the bottom right corner.

Step 2: Add a command to display the playlist

Now we need to display a YouTube playlist when the user gives a voice command.

  1. To the voice script in Alan Studio, add a command to show the video on the page:

    intent('Show me a video', p => {
    	p.play({command: "displayPlaylist"});
    	p.play(`Here are some nice videos, please have a look`);
    });
  2. In the webpage, to the <body> tag, add a container for the video playlist:

    <body>
    	<div id="player"></div>
    	...
    </body>
  3. In the command handler, add the code to handle the displayPlaylist command received from the voice script:

    <script>
    	var alanBtnInstance = alanBtn({
    		key: "438c790695a25bd87d43485dbc3cb44a2e956eca572e1d8b807a3e2338fdd0dc/stage",
    		onCommand: function (commandData) {
    			if (commandData.command === "displayPlaylist") {
    				showPlayer();
    			}
    		},
    	rootEl: document.getElementById("alan-btn"),
    	});
    </script>
  4. And add the showPlayer() function itself:

    <script>
    	function showPlayer() {
    		var tag = document.createElement('script');
    		tag.src = "https://www.youtube.com/iframe_api";
    		var firstScriptTag = document.getElementsByTagName('script')[0];
    		firstScriptTag.parentNode.insertBefore(tag, firstScriptTag);
    		var player;
    	}
    
    	function onYouTubePlayerAPIReady() {
    		player = new YT.Player('player', {
    			height: '390',
    			width: '640',
    			playerVars: {
    				listType:'playlist',
    				list: 'PLR2FILWn40KgsGTcUprpU0FfcmkaMGN3Y'
    			}
    		});
    	}
    </script>

Open the webpage, click the Alan button and say: Show me a video. Alan will send the displayPlaylist command to the webpage, and the showPlayer function will be called to display the playlist on the page. Alan will also play: Here are some nice videos, please have a look.

Step 3: Play a message on playlist loading

The message about the videos is now played right after the command is sent to the webpage. However, we want Alan to play the message only after the playlist is loaded in the player. To do this, we will use the project API functionality.

To use the project API, we need to define a method in the voice script and then call this method in the webpage.

  1. From the Show me a video command, remove the last line for playing the message.
  2. To the voice script, add the following:

    projectAPI.playText = function(p, param, callback) {
    	callback(null, "Project API method has been called");
    	p.play('Here are some nice videos, please have a look');
    }
  3. In the webpage, update the onYouTubePlayerAPIReady function:

    function onYouTubePlayerAPIReady() {
    	player = new YT.Player('player', {
    		height: '390',
    		width: '640',
    		playerVars: {
    			listType:'playlist',
    			list: 'PLR2FILWn40KgsGTcUprpU0FfcmkaMGN3Y'
    		},
    		
    		events: {
    			'onReady': onPlayerReady
    		}	
    	});
    }

    Here, we are adding the onReady event that will help us check if the playlist is loaded.

  4. Add the onPlayerReady() function:

    function onPlayerReady(event) {
    	alanBtnInstance.callProjectApi("playText", {}, function(error, result) {
    		if (error) {
    			console.error(error);
    			return;
    		}
    		console.log(result);
    	});
    }	

Here is how it works: in the voice script, we've defined a new project API method named playText. This method is called with callProjectApi in the webpage when the playlist is loaded. The method plays the Here are some nice videos, please have a look message on the server side and logs the result of execution to the webpage console: Project API method has been called.

Step 4: Send data to the voice script

In addition to playing messages, you can use the project API functionality to send arbitrary data to the voice script. Let's send the title of the video that is being currently played.

  1. In the voice script, update the playText project API method to the following:

    projectAPI.playText = function(p, param, callback) {
    	p.userData.title = param.videoTitle;
    	console.log(`Now playing: ${p.userData.title}`);
    	callback(null, "Project API method has been called");
    	p.play('Here are some nice videos, please have a look');
    }
  2. In the webpage, update the onPlayerReady method to send the video title to the voice script. We will get the title with player.getVideoData().title. Pass it as the second argument to callProjectApi():

    function onPlayerReady(event) {
    	alanBtnInstance.callProjectApi("playText", {
    		videoTitle: player.getVideoData().title
    	}, function(error, result) {
    		if (error) {
    			console.error(error);
    			return;
    		}
    		console.log(result);
    	});
    }	

Here is how it works: when the playText method is called, the video title is sent to the voice script and the following message is displayed in Alan logs: Now playing: <video title>.

Open the webpage, click the Alan button and say: Show me a video. Make sure the playlist is displayed and the title of the first video is shown in logs in Alan Studio.

Step 5. Play a message on button click

Let's assume we also want to let the user know what video from the playlist is currently playing. We will add the Show video title button. When the user clicks this button, the video title will be displayed, and Alan will pronounce it.

  1. In the voice script, add a new project API method for playing the title:

    projectAPI.playTitle = function(p, data, callback) {
    	p.userData.title = data.videoTitle;
    	console.log(`Now playing: ${p.userData.title}`);
    	callback(null, "Project API method has been called");   
    }
  2. In the webpage, add a container for the button:

    ...
    <div id="player"></div>
    <div class="buttons"></div>
    ...
  3. Add the showButton method. The method will display the button on the page; once the button is clicked, the playTitle project API method will be called, and Alan will perform the following:

    a. Send the video title to the voice script and display it in Alan logs.

    b. Display the video title on the webpage.

    c. Use the Alan Client API methods — activate and playText — to activate the Alan button programmatically and play the message: You are watching now: <video title>.

    function showButton() {
    	var button = document.createElement("button");
    	button.innerHTML = "Show video title";
    	var body = document.getElementsByClassName("buttons")[0];
    	body.appendChild(button);
    
    	button.addEventListener("click", function() {
    		title = player.getVideoData().title;
    		alanBtnInstance.callProjectApi("playTitle", {
    			videoTitle: title
    		}, function(error, result) {
    			if (error) {
    				console.error(error);
    				return;
    			}
    			console.log(result);
    			var div = document.createElement("div");
    			div.innerHTML = title;
    			body.append(div);
    			alanBtnInstance.activate();
    			alanBtnInstance.playText("You are watching now: " + title);
    		});
    	});
    };

    In step 4, the message is played on the server side. Here we are using playText to play the message on the webpage (client) side.

  4. In the webpage, update the command handler in the Alan button block to call the showButton() method together with showPlayer():

    var alanBtnInstance = alanBtn({
    	key: "438c790695a25bd87d43485dbc3cb44a2e956eca572e1d8b807a3e2338fdd0dc/stage",
    	onCommand: function(commandData) {
    		if (commandData.command === "displayPlaylist") {
    			showPlayer();
    			showButton();
    		}
    	},
    	rootEl: document.getElementById("alan-btn"),
    });

You can now test it: in the webpage, click the Alan button and say: Show me a video. Alan will load the playlist and say: Here are some nice videos, please have a look. Now, click the Show video title button. The video title will be displayed, and Alan will name the title of the video playing. You will also see the video title in Alan logs.

Step 6. Control video playback with voice

Let's also add several commands to go forward and backward in the video list and start video playback with voice.

  1. In the voice script, add the following intents:

    intent('Go to the next video', p => {
    	p.play('Going to the next video in the list');
    	p.play({command: "nextVideo"});
    });
    
    intent('Go to the previous video', p => {
    	p.play('Going to the previous video in the list');
    	p.play({command: "previousVideo"});
    });
    
    intent('Start video playback', p => {
    	p.play('Playing the video');
    	p.play({command: "playVideo"});	
    });
  2. In the webpage, update the command handler to handle the sent commands:

    ...
    var alanBtnInstance = alanBtn({
    	key: "438c790695a25bd87d43485dbc3cb44a2e956eca572e1d8b807a3e2338fdd0dc/stage",
    	onCommand: function (commandData) {
    		if (commandData.command === "displayPlaylist") {
    			showPlayer();
    			showButton();
    		}
      	  
    		else if (commandData.command === "nextVideo") {
    			player.nextVideo();
    			player.stopVideo();
    		}
      
    		else if (commandData.command === "previousVideo") {
    			player.previousVideo();
    			player.stopVideo();
    		}
      
    		else if (commandData.command === "playVideo") {
    			player.playVideo();
    			alanBtnInstance.deactivate();
    		}	  
    	},
    	rootEl: document.getElementById("alan-btn"),
    });

You can test it: in the webpage, click the Alan button and say the following commands:

  1. Show me a video
  2. Go to the next video
  3. Go to the previous video
  4. Start video playback