In this post, we will see how to use the recently added component TextToSpeechManager from the HoloToolKit to use Cortana to speak to the user.
We will do a simple sample app that will speak to us when we click on a Hologram. This is a very simplistic but powerful sample to show this capability that was introduced in the version 1.5.7.0 of the HoloToolKit.
You can access the final project in this GitHub repository.
Shall we code?
1. Create the structure models in Unity3D
Let’s create a new project in Unity3D, named wisely TextToSpeechDemo.
Once created, import the latest HoloToolkit package (Assets -> Import Package -> Custom Package).
Lets create a simple Cube in the scene. Click on the GameObject menu, then 3D Object -> Cube.
Create a new a script name CubeSpeaker.cs (Make right click in the Project panel, then Create -> C# Script). Drag the CubeSpeaker.cs file to the Cube and it will be added as a component of it.
Open the just created file in the editor of your choice.
2. Implement the IInputClickHandler interface
To be able to interact when the user Click/Air Tap over our cube, we need to implement the IInputClickHandler in our script. We will need to use the HoloToolkit.Unity.InputModule namespace and implement the OnInputClicked method. This method will fire when the user click on a Collider. Fortunately, our default Cube has a Box Collider.
Add a private variable definition at class level to store the instance attached to the TextToSpeechManager component.
private TextToSpeechManager textToSpeechManager;
Also add a simple count variable to store the times the user has clicked the Cube.
private int count = 0;
The TextToSpeechManager component is which will make magic communicating with Cortana passing our text. In the Start method, we need to search for the component’s instance attached to the GameObject and store it to use later.
void Start ()
{
this.textToSpeechManager = this.gameObject.GetComponent<TextToSpeechManager>();
}
Finally, we will use this in the OnInputClicked method calling the SpeakText method. Which pass our plain text to Cortana. We either call the SpeakSsml, in which we can use SSML to emphasize the text.
public void OnInputClicked(InputClickedEventData eventData)
{
this.textToSpeechManager.SpeakText(string.Format("You have clicked on me {0} times.", ++count));
}
You can include the RequireComponent class decorator to ensure the TextToSpeechManager component is attached in the same GameObject. If not, Unity3D will add itself.
[RequireComponent(typeof(TextToSpeechManager))]
Your code should look similar as below.
3. Add missing component to the GameObject
Back to Unity3D to add the remaining pieces we need to put in order to get this working.
- Drag the InputManager prefab (HoloToolKit\Input\Prefab) to the Hierarchy panel. This component is the responsible to trigger the Click event from the user.
- Click on the GameObject menu. Then, click Audio -> Audio Source.
- Select from our Cube and the TextToSpeechManager component.
- Drag the Audio Souce GameObject created previously to the Audio Source property on the TextToSpeechManager component.
- You may notice that there is another property in the TextToSpeechManager component called Voice. As you may guest, there are all the available voice you can use in HoloLens’s Cortana. You can try anyone you want or just leave it as Default.
Your project should looks like the following:
4. Build & Try it
Save your scene with any name you want. Next, apply the HoloLens Settings to the Scene, Project, Project Capabilities from the HoloToolKit menu. Next, go to the Build menu and remember to check the Copy References & Unity C# Project options and finally, build the project.
Once the project is build, open in Visual Studio and chance the Configuration to x86 and choose to run it in the HoloLens Emulator or onto your HoloLens Device.
You can view the video of the project here.
Bonus track
In this branch, you can get a little more sophisticated sample.
.g 