🗣️Lesson 7: Start a conversation
Aim
In this lessons you will learn how to work with Natural Language Processing (NLP) concepts and Speech Recongition in Python and create your own conversations with Misty. By the end of lesson you will be able to master the full range of Misty's speech and conversational abilities.
Waking up
Challenge 1: Misty Wake-up
You may recall from the Blockly Lessons that before you start a conversation with Misty you can let her know that you want to get her attention with her default wake-up phrase "Hey, Misty". To do this, you will need to set up the start key-phrase recognition event. The code structure of the key-phrase recognized event is identical to any other event.
Once you have imported the libraries required, you can call the KeyPhraseRecognized event by building a callback function that will determine what you want Misty to do when the key-phrase is recognized. How would you choose to design your wake up sequence?
Building NLP conversations
Now that you've created your wake-up sequence in Python you can start building a conversation. In Blockly Lesson 8: NLP we used the conversation tree concept to understand the relationships between different elements of the conversation such as context, intents, sample, etc. In Python, you can use the same concepts to build your conversation. Now let's try rebuilding the YesNoQuestion conversation tree example from Blockly in Python.
The first step in order to build an NLP Conversation in Python is to create the conversation with the following command:
It includes the conversation name and the starting state.
Later on, you can think about the states that Misty will assume in the flow of your conversation. To do so you can use the misty.create_state command.
This state will be the one called when you will trigger its intent.
There are two types of states:
Start States
Flow States
The start states contain the context and so the “question” that you will answer, when the code runs, your answers will be recorded and the flow state associated with that intent will play.
You can find several properties for each state, like:
State name: the unique name for the state
Speak: what you would like Misty to say during the state
StartAction: A particular action associated with that specific state, you can find the list of Misty's pre-built actions here NLP Actions or build your own.
The Start state must contain an extra necessary parameter: the context.
It’s necessary to insert the context because Misty has in her memory several contexts and you can choose which one will be used in that specific scenario. The way that you can include a context in the start_state is "['context_name']", as you can see in the example:
For the flow states, instead, you’re free to use all the parameters you like and here is an example of working flow states:
Important: If you want Misty to listen to you speaking during a state, it's important to select 'listen' in the action field. This way Misty will recognize your speech and reference it with the intent. Of course a flow state can be the start state for a next question.
There are many more parameters you can modify while creating a conversation state:
Last but not least is to map the conversation. You will need to determine and filter how you want Misty to transit between one state and another. To achieve this you can use the misty.map_state() API call.
In the map state, you need to insert which is the conversation you want to map, which is the starting state for that transition and the next one and lastly which is the trigger and the trigger filter.
Important: The triggerFilter name should match the intent name.
Once you have built your conversation you can call it with the command:
Important: You don’t need to always re-build your conversation because it will be stored in Misty’s memory. So feel free to delete the block of code that creates it after feeling satisfied about it.
To check your states, your contexts and your conversation you can use Misty’s API links:
If you insert the following links in your browser you will be able to see your contexts, your conversations and your states in Misty’s current storage.
(When you are working on them remember to refresh the page to see the current results)
http://<your_robot_ip_address>/api/dialogs/contexts
http://<your_robot_ip_address>/api/conversations
http://<your_robot_ip_address>/api/states
Important: Misty may not distinguish "really well" capital and not-capital letters, so for your names we suggest to use something different each time (es not Start/start but StartConv1/StartConv2).
You can use pre-built actions and contexts for your conversations with Misty.
But if you want to create your actions you can use the misty.create_action API. In the parameters, you will need to define the unique name of the action and a script, which contains all of the actions you want Misty to perform, you can find a list of them in Action Commands.
You can check your action at:
http://<your_robot_ip_address>/api/actions
At the moment we’re facing some issues with the misty.train_nlp_engine. So if you want to use your contexts we would suggest to use Blockly to build it.
Example of creation of Misty NLP conversation :
You will need to run first the blockly workflow. In the blockly workflow remember to run first the context block and then the start_state, in this case, you’ll have available the context you just created. After, you will be able to use the Python Studio with the new elements you built.
Example of using NLP conversation:
Congratulations on completing your own conversation tree! You are ready to combine all that you've learned so far to create amazing human-robot interactions using event skills together with physical touch, face recognition, object recognition, audio localization, speech recognition and natural language processing.
Advanced conversation:
In this example, you'll be able to see how you can create a conversation with more interactions. The key concept is that you're building one conversation, so you need only one, that has all the states and the map that can control the flow between them. Code explanation: As usual, you can import the required libraries, create the robot and start the event that will start the conversation (key_phrase_recognized). The main steps to build a conversation are:
Initialization of one new conversation
Creation of states
States' mapping
In this case, the conversation's name is ColorConversation1, and the starting state will be StartColorConverasation1. When the state is a Start State it will need the context as well, as, in this case, it is in StartColorConverasation1 and YesColorConversation1. After, you can map the different states how you prefer. In the last code lines, you can find the function that will be called when "hey, Misty" is heard and you'll be able to start your conversation. In this case, there are three possibilities after the YesColorConversation1.
For the moment you would still need to create your context in Blockly. Python code:
Last updated
