# Welcome to Misty Lessons

You've arrived at the place where your journey into social robotics with Misty begins. As you explore Misty's capabilities, you'll find that she has a wealth of built-in functionality, all packaged in an accessible way. Misty is here to help you enter this incredible world, expand your horizons, and prepare you for the digital future ahead.

These lessons are specially designed for users of all skill levels, from those just learning to program at school or university, to the casual hobbyist, experienced programmer and researcher. As you go through the Misty lessons there's a progression of concepts that will take you to the level you feel most comfortable. 

Should you ever need any support from our humans, you are welcome to reach out to [hello@mistyrobotics.com](mailto:undefined) or write to us in the [Misty Slack Community.](https://join.slack.com/t/misty-community/shared_invite/zt-19hieb0yc-etlgR~uI5P4lfgoHkT80Tw) 

## &#x20;           <img src="/files/13CrShjpVQASlgIXQwoX" alt="" data-size="line">**Misty Blockly**                                <img src="/files/yMxaVorgjrjJo5aNNutf" alt="" data-size="line"> **Misty Python** <a href="#blockly-and-python" id="blockly-and-python"></a>

{% tabs %}
{% tab title="                                        Contents                                        " %}
{% content-ref url="/pages/zTKe81GeV8Te1NS6MY8a" %}
[Blockly Lessons](/blockly/blockly-lessons)
{% endcontent-ref %}

{% content-ref url="/pages/Rg9FqAzJ2BPIUzxaxGej" %}
[Lesson 1: Movement](/blockly/blockly-lessons/lesson-1-movement)
{% endcontent-ref %}

{% content-ref url="/pages/Apm7xKbwA3eEhghdO1h8" %}
[Lesson 2: Voice and Sound](/blockly/blockly-lessons/lesson-2-voice-and-sound)
{% endcontent-ref %}

{% content-ref url="/pages/ok9oWx5KwN0nwRxVKlDC" %}
[Lesson 3: Expressions](/blockly/blockly-lessons/lesson-3-expressions)
{% endcontent-ref %}

{% content-ref url="/pages/gHK25DNCBf89ryidyaUY" %}
[Lesson 4: Robot Cinema](/blockly/blockly-lessons/lesson-4-robot-cinema)
{% endcontent-ref %}

{% content-ref url="/pages/ccqYraVShxtjDQLngVaF" %}
[Lesson 5: Events](/blockly/blockly-lessons/lesson-5-events)
{% endcontent-ref %}

{% content-ref url="/pages/b2EtDuMAeOHJoaykPr2V" %}
[Lesson 6: Face recognition](/blockly/blockly-lessons/lesson-6-face-recognition)
{% endcontent-ref %}

{% content-ref url="/pages/9zPIKnIyHZaS7mghxThw" %}
[Lesson 7: Variables and Functions](/blockly/blockly-lessons/lesson-7-variables-and-functions)
{% endcontent-ref %}

{% content-ref url="/pages/WgV7UuDmJz8FvugFKOX0" %}
[Lesson 8: NLP](/blockly/blockly-lessons/lesson-8-nlp)
{% endcontent-ref %}
{% endtab %}

{% tab title="                                           Contents                                            " %}
{% content-ref url="/pages/GfS5YunOu2WOJVQqp3LB" %}
[Python Lessons](/python/python-lessons)
{% endcontent-ref %}

{% content-ref url="/pages/jMmR6zhWxyKwJaq3pm2S" %}
[Lesson 1: Loco-motion](/python/python-lessons/lesson-1-loco-motion)
{% endcontent-ref %}

{% content-ref url="/pages/r71Z2AUCoX75gq9ttCmo" %}
[Lesson 2 : Build a character](/python/python-lessons/lesson-2-build-a-character)
{% endcontent-ref %}

{% content-ref url="/pages/oN42i02192Z1CuCb9ynD" %}
[Lesson 3: Create memories](/python/python-lessons/lesson-3-create-memories)
{% endcontent-ref %}

{% content-ref url="/pages/y5BNGfJtIH2iRdQ6AKl4" %}
[Lesson 4: Event skills](/python/python-lessons/lesson-4-event-skills)
{% endcontent-ref %}

{% content-ref url="/pages/FQ34MRZxQovYvbDqUq6O" %}
[Lesson 5: Expand awareness](/python/python-lessons/lesson-5-expand-awareness)
{% endcontent-ref %}

{% content-ref url="/pages/1ap6Y7CO3Evbe9MHJ9zU" %}
[Lesson 6: Compact code](/python/python-lessons/lesson-6-compact-code)
{% endcontent-ref %}

{% content-ref url="/pages/RmIVK73p8i8tuCj1udpp" %}
[Lesson 7: Start a conversation](/python/python-lessons/lesson-7-start-a-conversation)
{% endcontent-ref %}

{% content-ref url="/pages/CWtz3kmOR8eq4TdVy1MS" %}
[MistyGPT](/python-projects/mistygpt)
{% endcontent-ref %}

{% content-ref url="/pages/MXCOMrHC26frvrCvsZ5g" %}
[Misty Intruder Alert](/python-projects/misty-intruder-alert)
{% endcontent-ref %}

{% content-ref url="/pages/LESOYhPEaWq1FSVFl7N5" %}
[Conference Assistant](/python-projects/conference-assistant)
{% endcontent-ref %}
{% endtab %}
{% endtabs %}


# Get to know your Misty

## Misty versions

Misty has many sisters out there that share many core functionalities and capabilities, but there are some differences between them:

## Misty II&#x20;

Previously known as the Misty Standard, this Misty can be easily identified by her single camera visor.

<figure><img src="/files/ebV37d6lRj6wuQDvqTpG" alt=""><figcaption></figcaption></figure>

### How to identify your Misty II

* The visor only has 1 Sony IMX 214 RGB camera.&#x20;
* Your Misty is charged with a cabled barrel jack charger.
* No wireless power receiver, which means she cannot automatically dock and use the wireless charging base.
* No Automatic Navigation
* No access to the Mapping Page in Misty Studio.

## Misty II Pro

Previously known as the Misty Advanced, this Misty stands out with additional cameras and sensors on her visor that allow for room mapping and automatic navigation.

<figure><img src="/files/k73pkfY4RRT71dWJnT1l" alt=""><figcaption></figcaption></figure>

### How to identify your Misty II Pro

* The visor has Occipital sensors and 3 cameras.
* Misty II Pro comes with wireless charger base (we call this a PTU, or Power Transmit Unit). The automatic docking and charging feature require the Occipital Sensor in the visor above.
* Automatic Navigation with SLAM
* You have access to the Mapping Page in Misty Studio.


# Connect to Misty

To get started with your Misty lessons you will need to connect Misty to your local Wi-Fi network. If you haven't connected to your Misty yet, then you can follow these quick, out-of-box setup steps.

## Mobile App

### Step 1. Download the Misty App&#x20;

You can download the Misty App from the [App Store (iOS)](https://apps.apple.com/us/app/misty-app/id1296946424) to connect to Misty.

### Step 2. Enable Bluetooth on your phone or tablet

<div align="center"><figure><img src="/files/YF143SpDuZ0YQr6A4Nu7" alt=""><figcaption></figcaption></figure></div>

You can connect to Misty via Bluetooth by standing close to her with the Misty App open on your device. When the Bluetooth broadcast succeeds, the app displays a list of Misty robots that your app can connect to. Select your Misty from the list.

Note: If you are in a room with more than one Misty, you may need to locate your Misty's serial number by checking the label on the bottom of your robot.

### Step 3. Connect to Wi-Fi

<figure><img src="/files/GwXq8D61I6e9SCNMUnvk" alt=""><figcaption></figcaption></figure>

When Misty connects to your device, the Misty App displays a list of nearby Wi-Fi networks. Select your network from the list, and follow the prompt to enter your password. Once the Wi-Fi connection succeeds, you should see the Misty App Home screen.

### Step 4. Access Misty Studio

<figure><img src="/files/265O2GQDCN0L06F5kUxr" alt=""><figcaption></figcaption></figure>

Confirm that the Wi-Fi status is connected and that a valid IP address (example: 10.10.10.46) for Misty appears on the screen. Type in the IP into your web browser to access Misty Studio where you can control and send commands to Misty. We suggest writing down this IP address as you will also need it to access Misty with other devices on the same network.

***

## USB drive

In cases where using a mobile application won't work, or if you're having issues with Bluetooth, you can also use a **USB drive** to connect to the wifi network. To do so, follow these steps:

1. Format the USB drive in FAT format.
2. In the root of that file system, create a directory named 'misty' (without the quotes).
3. In that directory, create a text file named wifi.txt.
4. Edit that text file. On the first line, add the name of the SSID. On the second line, add the network key. The resulting file should look something like this:

   ```
   MyNetworkName
   MyNetworkKey
   ```

   Text
5. Save the file and eject the USB drive. Insert the drive into Misty's USB port and give her 60 seconds to connect to the network.
6. Remove the USB drive from Misty. Within the misty directory should be a file called 'onboard.txt'. This file will contain the robot's IP address.

If you're connecting to an **enterprise wifi network**, you'll use a similar method to the USB drive method above:

1. Format the USB drive in FAT format.
2. In the root of that file system, create a directory named 'misty' (without the quotes).
3. Into that directory, you add a JSON formatted text file named enterprisewifi.txt with the format:

   <pre class="language-json" data-overflow="wrap"><code class="lang-json">{
    "SSID": "THE-SSID",
    "Identity": "THE-USERNAME",
    "Password": "THE-PASSWORD",
    "EapMethod": "{NONE},{PEAP},{TLS},{TTLS},{PWD},{SIM},{AKA},{AKA_PRIME},{UNAUTH_TLS}",
    "Phase2Method": "{NONE},{PAP},{MSCHAP},{MSCHAPV2},{GTC},{SIM},{AKA},{AKA_PRIME}",
    "AltSubjectMatch": "THE-ALT-SUBJECT-MATCH",
    "AnonymousIdentity": "THE-ANONYMOUS-IDENTITY",
    "DomainSuffixMatch": "THE-DOMAIN-SUFFIX",
    "Realm": "THE-REALM",
    "KeyManagementMethods": [
        "{None},{WPA_PSK},{WPA_EAP},{IEEE8021X},{WPA2_PSK},{OSEN},{FT_PSK},{SAE},{OWE},{SUITE_B_192},{WPA_PSK_SHA256},{WPA_EAP_SHA256}",
        "{None},{WPA_PSK},{WPA_EAP},{IEEE8021X},{WPA2_PSK},{OSEN},{FT_PSK},{SAE},{OWE},{SUITE_B_192},{WPA_PSK_SHA256},{WPA_EAP_SHA256}",
        ...
        "{None},{WPA_PSK},{WPA_EAP},{IEEE8021X},{WPA2_PSK},{OSEN},{FT_PSK},{SAE},{OWE},{SUITE_B_192},{WPA_PSK_SHA256},{WPA_EAP_SHA256}"
    ]
   }
   </code></pre>
4. The wifi file should be configured to match the requirements of the network. You'll likely need to work with your network administrator to determine the right options to use. The underlying system is Android, which supports [these configuration options](https://developer.android.com/reference/android/net/wifi/WifiEnterpriseConfig). Also, note that alues provided for EapMethod, Phase2Method, and KeyManagementMethods represent the valid options, and that KeyManagementMethods is an array.
5. Save the file and eject the USB drive. Insert the drive into Misty's USB port and give her 60 seconds to connect to the network.
6. Remove the USB drive from Misty. Within the misty directory should be a file called 'onboard.txt'. This file will contain the robot's IP address.

Lastly, there are cases in which you may want to connect Misty to a wired network. Misty doesn't have an RJ45 ethernet port, so you'd need a **USB to ethernet adapter** (not supplied) to make the connection.


# Misty Studio

A description of Misty Studio

The Misty Studio is a graphical interface for exploring Misty's functionalities and learning how to code her in Blockly and Python. It's the best place to become familiar with the tools that you will need to use throughout your programming journey. Let's get you familiarized with how to navigate around Misty Studio!

<figure><img src="/files/EGvEnrkXEJHsEQulAqUw" alt=""><figcaption></figcaption></figure>

### Top Navigation <a href="#top-navigation" id="top-navigation"></a>

At the top of Misty Studio, you'll see several prominent elements.&#x20;

* From left to right, there's a slider to adjust the robot's audio volume.&#x20;
* Next to that slider is the Halt! button. This button immediately releases all of the robot's motors. You can think of this as the 'panic' button for when things don't go according to plan.&#x20;
* There's also a battery level indicator that shows the charge percentage. The battery indicator will change to green when the robot is charging.
* &#x20;Finally, furthest right you will find the Live Data button which will open a tab that gives you data from all of Misty's sensors.

<figure><img src="/files/eUqjQVr4x9wRoa6VwzN0" alt=""><figcaption></figcaption></figure>

### Live Data

The live data tab is an extremely useful testing and diagnostic tool as it provides real-time information about the functionality of your sensors.. You can enable each sensor group individually to view your sensor data.

<figure><img src="/files/ryFN3A1h8mcIkc3TZukI" alt="" width="163"><figcaption></figcaption></figure>

### Left Navigation <a href="#left-navigation" id="left-navigation"></a>

On the left side of Misty Studio, you'll find the primary navigation. Each top level menu item is marked with an icon, and many expand to reveal additional items.&#x20;

* Wizard: Here you can try out Misty's capabilities, including moving her head and arms, enabling speech, expressions, locomotion and much more.
* Explore: learn more about Misty's capabilities, how to use face training, audio localization, mapping and upload/download custom files.
* Programming: here you can find the Blockly workspace, Python Interface , API Library, Skill manager and a link to developer documentation.
* Tutorials: contains an expandable list of tutorial content that will change as new content is made available.
* System: contains informational and diagnostic features that are sometimes useful, but won't be your go-to items in the beginning.


# Desktop Environment

If you're already an experienced roboticist and developer you can program Misty by setting up your own desktop environment and code editor. There you can program Misty using Python, C# or JavaScript. While you can do most of your programming in Misty Studio, for more complex programs that require external dependencies you may need to use your preferred code editor.

## Setting up a desktop environment for Python in Windows

### Step 1.  Power on Misty

Misty has to be powered on and connected to your WIFI network in order to receive commands.

### Step 1. Download your code editor and program packages

In order to start setting a desktop environment to program Misty in Python you will need to download a code editor, a Python installer and the Misty Python SDK package. For your code editor you can use Visual Studio Code or any other Python editor.

* [Download Visual Studio Code](https://code.visualstudio.com/download)
* [Download Python Installer](https://www.python.org/downloads/)
* [Download Misty Python SDK](https://github.com/MistyCommunity/Python-SDK)

### Step 3. Set up a Path

Once you've installed Visual Studio Code and Python on your computer, you will need to create a working directory where you can extract the Misty Python SDK package zip file. The extracted contents can be place in any folder on your desktop. (For example C:\Users\YourName\Desktop\ MistyPythonSDK) Now to be able to put the code editor, Python and the Misty SDK in same desktop environment and establish a working path, you will need to search and open  'Edit the system environment variables' function in the Windows Search Bar. You will see window called System Properties,  click on the button called Environment Variables.

<figure><img src="/files/wSl17H0V3Y0klKVDJCBs" alt="" width="302"><figcaption></figcaption></figure>

After you open the environment variables windows you will see a section with a list of folders, choose 'Path' and click on the button 'Edit'.

<figure><img src="/files/7HF7cgpnVPkQNyRsuekZ" alt="" width="375"><figcaption></figcaption></figure>

Inside the path folder you will need to add all the folders paths that contain your code editor and python executables, as well as python packages and the working directory with the Misty Python SDK.

<figure><img src="/files/8tdIU8fU5TKmPeRqQaFS" alt="" width="370"><figcaption></figcaption></figure>

### Step 4. Install Dependencies

In your Visual Studio Code you will need to set your working directory as the default directory for the terminal. This can be done in settings by searching "terminal.integrated.cwd" and writing in the folder path of the working directory.  Now you can navigate back to your terminal and install the following dependencies. Place them in your terminal and press Enter. In some case you will need to install additional dependencies if you want to use more Python libraries. Visual Studio Code will highlight the lines of code that need additional dependencies to work, by hovering over the line you can use 'Quick Fix' to find the dependency you need.

```python
pip install 'requests>=2.25.1'
pip install 'websocket-client<=0.57.0'
pip install 'yapf>=0.30.0'
```

<figure><img src="/files/DjXEd0ccTBW5PsjULuoH" alt=""><figcaption></figcaption></figure>

### Step 5. Create Robot Generator

To be able to connect to Misty via her IP address and use the API endpoints you will need to create a Python file in your working directory called `update.py` . Save the file and run it using the command python update.py in your terminal.&#x20;

```python
from mistyPy.GenerateRobot import RobotGenerator
RobotGenerator("ROBOT-IP_ADDRESS-GOES-HERE")
```

<figure><img src="/files/tsLJGBolJ16c3p23yfyi" alt=""><figcaption></figcaption></figure>

### Step 6. Create your Python Project

Congratulations! You have now set up your desktop environment for Misty. You can now create a Python project and start programming Misty. Remember to keep your project file in the same folder path as your Misty Python SDK. You can run your program by using the run button in the top right corner and cancel your program by deleting the terminal session or pressing Control + C.

<figure><img src="/files/9S5gMVVkShV8KnIrgewO" alt=""><figcaption></figcaption></figure>


# Update your Misty

In this page you can find instructions to update your Misty's version and have access to the Misty Studio.

In cases where Misty is connected to the internet, she should normally update automatically after some period of time. Not all Mistys update at the same time, so you'd want to leave your Misty on for a few hours, and plugged in. If the battery is below 50%, or if internet connectivity is absent, the robot won't try to update.

Now it's necessary to get the robot information:&#x20;

* If you got your Misty's IP address from the app you can click on the middle bottom button (my misty) to get the information of your robot.
* If you got your Misty's IP address in another way you can still look for the information of your robot by writing in your search bar : `http://{ROBOT-IP-ADDRESS}/api/device`

If the robot version - in the software section- is less than 2.0, the robot won't be able to host the Misty studio and has to be updated.

If the robot meets these conditions, you can attempt to force an update by issuing two HTTP commands:

* `POST http://{ROBOT-IP-ADDRESS}/api/system/update/allow` \[this allows automatic updates if the user had suppressed them]
* `POST http://{ROBOT-IP-ADDRESS}/api/system/update` \[this triggers the update request, which still needs to download the update package]

To issue the POST commands you can use curl, Visual Studio code or postman if you feel more comfortable but in this case, we'll work with HTTPie.

#### Instructions

* Download [HTTPie](https://httpie.io/download)
* Install the program on your device and login (with a GitHub account, for example)
* Double-check to be on the same network as the robot and have it plugged in
* Sending the command:

  * In the top label select POST as the method
  * in the writing area next to it type:

  `http://{ROBOT-IP-ADDRESS}/api/system/update/allow`&#x20;

once the request has succeded (you can read "status" = success),&#x20;

* submit the second one : `http://{ROBOT-IP-ADDRESS}/api/system/update`

After executing the above, give the robot some time and see if updates begin. The update package is around 600MB, which must be downloaded before the robot will notify you that the update is beginning.

In this picture, you can see the result after submitting the second command (this Misty was already updated so the "result" is false, but "status" is Success).

<figure><img src="/files/onFfIkki5yDvC1qnnZOL" alt=""><figcaption></figcaption></figure>


# Projects

## Community Projects

[This section](/community-projects/misty-weather-forecaster) will contain projects created by community members who are willing to share their projects with Misty!&#x20;

Send your project to this email address to have it published: <hello@mistyrobotics.com>

## Work in progress Projects

[This section](broken://pages/xqvI6M8Oqtf3eSxSzIWC) will contain all the projects that we're working on currently.&#x20;

In this way, the community will be able to take partial code, complete the project or suggest better paths to achieve the result!

**`Thank you so much for improving the Misty community!`**


# Blockly Lessons

## Learning with Blockly

Blockly is a popular choice for learning programming if you don't have any prior experience. If that is you, then you have come to the right place. Throughout Misty's Blockly lessons you will learn the basic concepts of programming without having to worry about the complexities of syntax. You can easily drag and drop blocks together to create programs, and Blockly will automatically generate syntactically correct code.

## Blockly Workspace

To access your Blockly workspace you will need to open the 'Programming' tab in the left side bar and click 'Blockly'.

<figure><img src="/files/3EKnXOPxbDlTbu227xej" alt=""><figcaption></figcaption></figure>

As you start exploring the Blockly workspace in Misty Studio you will find that the blocks are organized into 3 categories. If you would like to explore these in more detail before kicking off the lessons you can simply click the links below:

* [Misty Blocks:](/blockly-elements/misty-blocks) Movement, speech, audio, vision, events, miscellaneous, NLP and system.&#x20;
* [Basic Blocks:](/blockly-elements/basic-blocks) Logic, loops, math, texts and lists.&#x20;
* [Advanced Blocks](/blockly-elements/advanced-blocks): Variables and functions.&#x20;

## Types of blocks

There are six types of blocks you can use in your Blockly workspace. You can identify them by their shape, connectors and sockets, which will tell you how they are connected together:

<figure><img src="/files/cwX0FoqO9M0DQFdCUprA" alt=""><figcaption></figcaption></figure>

1. **Independent blocks**. As you can see in the example above this block can't be connected with other blocks because it has no connector or socket. It runs independently from the rest of the blocks.
2. **Action blocks:** These blocks come with a socket and a connector. They can be connected to other blocks with sockets of the same type to form a sequence. Inside some function blocks you can find empty spaces where you can insert action blocks.&#x20;
3. **Value blocks:** Stand out by their shape and colour (Red, Black or Blue)**.** These blocks are typically used to insert values into your action blocks, including text, list, variable, number or anything you need for your code.
4. **Independent Function blocks.** These blocks also can't be connected to other blocks and run independently, however, you can nest other blocks inside them to run a sequence based on a certain condition.
5. **Function blocks:** Can be connected to other blocks in a sequence and nest additional blocks that you want to run once the function is triggered.
6. **Function Value Blocks:** These are very similar to the regular function blocks, however, you can also attach value blocks.

## Selected blocks

<figure><img src="/files/WK5P4vtwvotlTkc8Lrei" alt=""><figcaption><p>selected blocks</p></figcaption></figure>

You can recognize which block you are working with at that moment because it will be bordered with a yellow line. When the code runs the blocks will have a whiter tone, so you can check which block is currently being used.&#x20;

## Disabled Blocks and Suggestions

In the lessons, you will find that some blocks are shown in a faded yellow colour. These are disabled blocks that are provided as examples of what you can use when building sequences. You can re-enable the blocks if you want to run them in your program. Below you will find instructions on how to manage your Blockly workspace and blocks.

<figure><img src="/files/risirqLRzE5u1geGWUPO" alt="" width="563"><figcaption></figcaption></figure>

## Managing Your Blockly Workspace and Program <a href="#managing-your-program" id="managing-your-program"></a>

Blockly allows you to drag and drop blocks directly into your workspace. Should you at any point decide you want to delete a block or a combination of blocks, you can drag them to the trash bin in the bottom right corner or right-click and select 'delete block'. You will find that you can also deactivate blocks, duplicate them or add comments.

<figure><img src="/files/fhlWLPC0u8yg6sxBaZSp" alt=""><figcaption></figcaption></figure>

At the top of your Blockly workspace, you'll see a set of controls that you can use to manage your Blockly program.

![Blockly](https://docs.mistyrobotics.com/assets/images/misty-studio-run-blockly.png)

* The 'Run' button is used to start program execution and will change to a 'Stop' button while your program is running.&#x20;
* The 'Stop' button stops the current program.&#x20;
* The 'New' button clears the current workspace.&#x20;
* The 'Open' button is used to load a saved program from your computer.&#x20;
* The 'Save' button is used to save the current program to your computer. By default, the program file is always saved as "misty\_blockly.json", but you can rename it.

## Start Lessons

{% content-ref url="/pages/Rg9FqAzJ2BPIUzxaxGej" %}
[Lesson 1: Movement](/blockly/blockly-lessons/lesson-1-movement)
{% endcontent-ref %}


# Lesson 1: Movement

## Aim

Welcome to your very first lesson in Misty Blockly! The aim of this lesson is to explore Misty's range of motion and help you understand how to program her to move. If you have any questions about the individual movement blocks, you can check out the [Movement](/blockly-elements/misty-blocks/movement) section under Blockly Elements.

## Arm Movements

### Challenge 1: Wave a hello

When you meet a robot for the first time it can be a little scary since you don't know what it will do, that's why it's important to start your interaction with a greeting. It's not only an ice-breaker, but also a fun way to let humans know that Misty is a friend. Let's get Misty to wave her arms! \
\
Select the arm block from the 'Movement' tab in Misty Blocks and place it in your Blockly workspace. With this block you can control both her arms, since her arm range is limited, you can choose values between -90 (up) and 90 (down).  You can also choose the speed of the arm movements with values between 0 and 100. Try out different combinations!

<figure><img src="/files/W78pwqB4nrKI3S4h3GbQ" alt="" width="368"><figcaption></figcaption></figure>

### Challenge 2: Sequence your waves

If you don't want to get stuck changing the values of your arms every time, you can create a sequence by adding a second arm movement block. To run your arms in a timed sequence you'll need to use the timer block between the two arm blocks. This block will help you to determine the amount of time you need for each block to run.  You can find the timer block in the 'Miscellaneous' tab in Misty Blocks. The minimum amount of time you can set is 0.1 seconds (100ms). Try starting with 1 second (1000ms).

*Note: If you don't use a timer block the program will try to run every block at the same time in the sequence and the sequence may not be completed. When you are testing your program adjust the timer to allow Misty to complete each action in the sequence.  It's not necessary to insert a timer block between two movement blocks of different kinds. For example, Misty's arms and head have different motors so they will still execute the code independently, but if you put two arm blocks together, they will both try to run at the same time.*

<figure><img src="/files/tuzblyB2kLQpOLr5bGwP" alt=""><figcaption></figcaption></figure>

Now you can connect the second arm block to your timer to make Misty raise her right hand and pull down her left hand. Wait one second and then switch them. You can also try out using different values. What's missing? Make the greeting as natural as you can!

<figure><img src="/files/scIKy1gahzRuNGtgMAEb" alt=""><figcaption></figcaption></figure>

## Head Movements

### Challenge 3: Look around the room

Now that you are comfortable with moving Misty's arms, let's explore how to get her to look around the room. Select the head block from the 'Movement' tab in Misty blocks and place it under your arm sequence, don't connect it just yet. Pitch is the first head range that you can use to make Misty look up or down. The pitch range values are -40 (up) to 25 (down). To make Misty roll her head left or right, you can change the roll range values from -90 (right) to 90 (left). Finally, if you want Misty to simply turn her head left or right, you can use the head yaw range values -42 (left) to 42 (right). Remember that you can choose the movement velocity. Give it a go!

<figure><img src="/files/PUXU59hMZf3d5mm684ux" alt=""><figcaption></figcaption></figure>

### Challenge 4: Robot Dance

Once you are familiar with how to move Misty's head, you can start combining different movement blocks, let's try to connect the head block to an arm sequence and program Misty to dance. Remember to use the timer block to time your sequence.&#x20;

<figure><img src="/files/yuhZoCGSZWXjecNwQKBB" alt=""><figcaption></figcaption></figure>

### Challenge 5: Let's do it again!

Do you want to run the sequence again? Instead of clicking the 'Run' button every time, you can use the repeat block found in the Basic Blocks 'Loops' tab and choose how many time you would like Misty to repeat the sequence. Repeat blocks are very useful if you would like to save time and avoid rebuilding the same sequences. Find out more about loops here: [Loops](/blockly-elements/basic-blocks/loops)

<figure><img src="/files/eNvBFQUJkFnQ14OzbtAK" alt=""><figcaption></figcaption></figure>

## Driving

### Challenge 6: Let's get Misty rolling

Now that you have a hang of how to move Misty's arms and head, let's get Misty rolling. Misty has tracks that allow her to drive anywhere in your room and turn at any angle, giving you full control of where you want her to go. Select the drive block and choose the values for linear velocity, angular velocity and time. The linear velocity sets Misty's speed in a straight line, its range is -100(backward) to 100(forward). The angular velocity sets Misty's speed in rotation, its range is -100(clockwise) to 100 (counter-clockwise). You can also set the duration in seconds. Try it out!

<figure><img src="/files/FH0UDYX86UNYgmDMj53J" alt=""><figcaption></figcaption></figure>

### Challenge 7: Create a path

Great job! You've got Misty rolling, let's give her a little guidance on where to go. Connect the blocks to make Misty drive forward for 3 seconds, turn right, and drive forward for 3 seconds more. Try out different combinations until you feel happy with your path plan.

<figure><img src="/files/zs4JMEFC0bA7NOkYJgWS" alt=""><figcaption></figcaption></figure>

### Challenge 8: Curvy road

If you don't like going in a straight line, well then Misty has the right block for you. The drive arc block allows you to make Misty drive at a curve. Select the block and choose a value for heading, radius and duration. For example, you can run this block to make Misty drive an arc of 0.5 meters of radius, doing half circumference in 5 seconds.  Try it out and find the values that you are most comfortable with.

<figure><img src="/files/M9XNqYW00czLuF7PZZXj" alt=""><figcaption></figcaption></figure>

### Challenge 9: Red Carpet

Finally, let's combine what we've learned and imagine Misty just rolled out onto a red carpet to show off her incredible talents. You can get her to drive forward and greet everyone on the left and right two times, and stop her one second after she has finished saying hello to all her fans.

<figure><img src="/files/KMeFlXNTpwIYpguoqnff" alt=""><figcaption></figcaption></figure>

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/Apm7xKbwA3eEhghdO1h8" %}
[Lesson 2: Voice and Sound](/blockly/blockly-lessons/lesson-2-voice-and-sound)
{% endcontent-ref %}


# Lesson 2: Voice and Sound

## Aim

The aim of this lesson is to understand how to enable Misty's voice and program her to speak and play audio. By the end of this lesson, you'll be able to understand how to use Misty's speech and audio capabilities in any way you prefer. If you have any questions about Misty's voice and sound capabilities, you can check out [Speech](/blockly-elements/misty-blocks/speech) and [Audio](/blockly-elements/misty-blocks/audio).

## Voice and Speech

### Challenge 1: Hello world!

Besides physical greetings you can also have Misty greet her human friends with words. Now that you've mastered Misty's movements, let's hear what she has to say. Select the speak block from the 'Speech' tab in Misty Blocks and nest a text block from the 'Text' tab in Basic Blocks. You can identify your text block by the double quotation marks ("  "). Try out, writing your first sentence in the text block and select your language. You can also adjust the pitch. The pitch value must be higher than 0, her default is 1.0. What would you like Misty to say?

<figure><img src="/files/gmdprQQ4xEJWlKHEE7JV" alt=""><figcaption></figcaption></figure>

<figure><img src="/files/kgYtZA0vCaLmmankpvZm" alt=""><figcaption></figcaption></figure>

### Challenge 2: Start a dialogue

Once you've found your favorite voice and pitch, let's create a dialogue where Misty plays two robot characters. You can select a different pitch and a different voice in the same language to create more characters. How would you continue this dialogue?

<figure><img src="/files/981eTY4xbatcuGJN4NaX" alt=""><figcaption></figcaption></figure>

### Challenge 3: Action goodbye

After all that talking it's nice to give Misty an exit out of the dialogue. Let's give her a chance to wave and say goodbye. Try combining the speech blocks and the movement blocks. Connect the blocks in a sequence that you are comfortable with. How would you customize it?

<figure><img src="/files/znciBtY6vxQslz1XsEuh" alt=""><figcaption></figcaption></figure>

## Sound and Audio

### Challenge 4: Vocal expressions

Besides her wonderful voice and speech capabilities, Misty can also be a little vocal about her feelings and has a number of vocal expressions that you can choose from. Select the play audio block in the 'Audio' tab in Misty Blocks to try out some of her vocal expressions. You can explore them in the drop down list or the resource database [Audio files](/resource-database/audio-files). Try out adjusting Misty's volume in a range from 0 to 100.

<figure><img src="/files/RamW6m8NCgzsRw0oTDdw" alt=""><figcaption></figcaption></figure>

### Challenge 5: Record audio

Misty can also record your voice and other sounds. Connect the recording block from the 'Audio' tab to let Misty record your voice. After you complete recording the audio file will be automatically saved in Misty's memory and you will be able to select it in the drop down list in the play audio block. Remember to keep the file-type .wav.&#x20;

<figure><img src="/files/kKTNESvB9XzV4RVNR2rV" alt=""><figcaption></figcaption></figure>

### Challenge 6: Upload your favourite song

Do you want Misty to become your personal DJ and play your favorite songs? Then she has just the block you need. Add the upload audio block in the 'Audio' tab and click on the check box, a desktop tab will appear. From your desktop, select your song and press upload/open, then a check mark will appear in your upload audio block and the name of the song in the \<filename> field.&#x20;

<figure><img src="/files/xGuscGpDd6Sb5NZS4O8T" alt=""><figcaption></figcaption></figure>

Press the run button to let Misty save the file in her memory. Success! Your song is now uploaded. &#x20;

**Important:** Before using the uploaded audio file you always need to run the program only with the upload audio block, then refresh the page. After that you will be able to find your song in the play audio file block.  If it doesn't show up in the drop down list of the play audio file block try running the upload audio block again.

<figure><img src="/files/6JZAtVFIT0VPzFe90Qds" alt=""><figcaption></figcaption></figure>

After you upload the audio you can delete the upload audio block because the audio is already saved.

*Note: Remember that you can't upload two contents with the same name, if so, the \<filename> will not change and your new block will not upload the audio, because Misty will see that she already has the same name in her memory.*&#x20;

### Challenge 7: Time your song

If you only want to play a short segment of your song you can connect a timer block and a stop audio block from the 'Audio' tab. Try out playing the song for only 10 seconds.

<figure><img src="/files/1ptqwLcUE1vmlAJZL7BQ" alt=""><figcaption></figcaption></figure>

### Challenge 8: Misty Dance

Now that Misty can play your favorite song, let's combine speech, movements and audio to teach her how to dance and sing!&#x20;

You can start by uploading the song, then try out making Misty drive forward, say the name of the song and create choreographed dance. Which dance will you create?

If you are unsure of how you want to build it, you can use the example below of Misty Dancing the Moonwalk.

<figure><img src="/files/zasAtdVPkANKe83rszgF" alt=""><figcaption></figcaption></figure>

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/ok9oWx5KwN0nwRxVKlDC" %}
[Lesson 3: Expressions](/blockly/blockly-lessons/lesson-3-expressions)
{% endcontent-ref %}


# Lesson 3: Expressions

## Aim

The aim of this lesson is to explore Misty's facial expressions and teach you how to customize the display in any way you like. By the end of this lesson, you'll be able to personalize every expression of your Misty. If you have any questions about the specific blocks, you can check out [Vision](/blockly-elements/misty-blocks/vision) and [Miscellaneous](/blockly-elements/misty-blocks/miscellaneous).

## Display Expressions

### Challenge 1: Change Misty's eyes

Expressions are essential to let humans know what you are feeling and it's no different for robots. Let's get Misty to show her range of expressions. Select the display image block from the 'Vision tab' in Misty Blocks and use it to change the eyes on Misty's display. You can explore them in the drop down list or go to [Image files](/resource-database/image-files). Remember that Misty's display is 480x272 pixels so your image file needs to be the same dimensions. You can also adjust the opacity of your image: 0 is all black and 1 is full colored picture.

<figure><img src="/files/f0fzkBDRplhzDpNISnKj" alt=""><figcaption></figcaption></figure>

### Challenge 2: Upload your own image

If you want to add your own image or design new eyes for Misty, you can up upload it in the same way as your song. Try designing and uploading your very own Misty eyes or a favorite photo. You will need to add the upload image block from the 'Vision' tab and click on the check box, upload the image, wait for the checkmark to appear and the name of the file.  After you run the block you will be able to find your image in the display image block.&#x20;

<figure><img src="/files/n17dE78kjuQIhKaZCnBU" alt="" width="301"><figcaption></figcaption></figure>

### **Challenge 3: Display texts**

If you want to also display a text you can use the display text block in the 'Vision' tab. You can time several texts to show up in a sequence or at a specific point in your program. When you want to delete it, you need to remove the text from the field as in the following example:

<figure><img src="/files/s9lnxRnQHllTSNKEYX6N" alt=""><figcaption></figcaption></figure>

## Chest LED

### Challenge 4: Choose your favorite LED color

The LED block is used a lot in Misty applications because it gives clear feedback on whether some actions were done or not, it fits everywhere! You can find your LED block in the 'Miscellaneous' tab in Misty Blocks. As you probably discovered, there are a lot of colors to choose from when you click on the colored square, choose your favorite one and run the block!

<figure><img src="/files/8zy7EvNU7kWUbKekT0H8" alt=""><figcaption></figcaption></figure>

### **Challenge 5: Create your own action**

If you've tried out the Wizard in Misty Studio you've probably discovered that Misty has lots of pre-programmed actions with expressions and movements. Let's try to rebuild one using the display, movement and LED blocks. If you are unsure how to build on you can start with the example of the action "Love".&#x20;

<figure><img src="/files/raXS5gOJuPjlsr7pgxJP" alt=""><figcaption></figcaption></figure>

### Challenge 6: Ambulance

As a robot Misty can take on a lot of features from other machines and vehicles. If you've stubbed your toe on your bed for example, you can make Misty rush over to you like an ambulance. Try out connecting the transition LED block with a play audio block and a drive block so Misty can simulate an ambulance for ten seconds. In the transition audio block you can select two colors and the time it takes to cycle between them In the drop-down you can also choose how you want the LED to transition. How would you improve the sequence below to make her more like an ambulance?

<figure><img src="/files/Y3z27gyvyaAKvJrWNoCI" alt=""><figcaption></figcaption></figure>

### Challenge 7: Recite a poem

Now that you've mastered Misty's display, expressions and LED, let's have Misty recite your favorite text and show it on her screen. You can either upload a custom background for your text or change it to just black. Add a display text block and write in your text. Here is an example of how you can have her recite a poem:&#x20;

In a realm where cogs entwine,                                                                                                                                Two robots found a love, design divine.                                                                                                               Their hearts in circuits hummed and spun,                                                                                                               In perfect sync, a dance begun.

Their love in gears and sparks was told,                                                                                                                    A connection timeless, coded bold.                                                                                                                             In this world of metal gleam,                                                                                                                                   Their love, an eternal dream machine .

<figure><img src="/files/KdlffTh6OLnqIaizyLjN" alt=""><figcaption></figcaption></figure>

### Challenge 8: Turn your room into a Disco with Misty

Now that you've been enlightened about Misty's chest LED, try combining what you've learned so far to transform your room into a robot disco! Don't be afraid to use more and more of Misty's action blocks together with her expressions and turn her chest LED into disco ball!

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/gHK25DNCBf89ryidyaUY" %}
[Lesson 4: Robot Cinema](/blockly/blockly-lessons/lesson-4-robot-cinema)
{% endcontent-ref %}


# Lesson 4: Robot Cinema

## Aim

The aim of this lesson is to explore how to turn your Misty into a robot cinema. By the end of this lesson, you'll be able to record your own videos and play them on Misty's display.  If you have any questions about the video blocks, you can check out [Vision](/blockly-elements/misty-blocks/vision).

## Uploading an image or movie

### Challenge 1: Prepare your intro

In the last lesson we learned how to upload custom images on Misty, but did you know that you can turn Misty into a mini cinema too? Every movie needs an introduction screen or image, let's try making one by uploading an image or expression and combine it together with a LED light.

<figure><img src="/files/OvbqWK1hnqzhav23gBUB" alt=""><figcaption></figcaption></figure>

### Challenge 2: Upload your video

To upload your video you can follow exactly the same steps as for the upload audio and upload image blocks. Once you made or found the video you want Misty to show, you can play it by using the play video asset block from the 'Vision' tab in your Misty Blocks. You can also add some special effects by selecting a different LED color.

<figure><img src="/files/venXXSOwKr9mtqFm2pEO" alt="" width="282"><figcaption></figcaption></figure>

## Shooting a new movie

### Challenge 3: Snap a photo with Misty

Misty loves to take pictures together with her friends and you are no exception. If you would like Misty to take a picture of you, an object or landscape, you can go the Vision tab under Explore in Misty Studio and press 'Display Photo'

<figure><img src="/files/wPIhsre9HhnthIDYkNe9" alt=""><figcaption></figcaption></figure>

Success!  You can now download the photo by pressing the 'Download photo' button. Once you've downloaded it you can give the file a unique name. Go back to your Blockly program and upload the image using the upload image block.&#x20;

<figure><img src="/files/vjkSDYtiOYf4boCPPjVM" alt=""><figcaption></figcaption></figure>

### Challenge 4: Record videos

Now let's imagine you are Misty's personal director and want to record a short movie with her. You can go to the Record Video section in the 'Vision' tab in under Explore. You can also choose the resolution of the video.

<figure><img src="/files/KsTPjhA728uIns3LFUal" alt=""><figcaption></figcaption></figure>

### Challenge 5: Create a trailer with Misty of your favorite movie

If you are here it means you are already familiar with most of Misty's capabilities! Compliments!

But you're only at the beginning of your journey. Before moving on, try recording a short movie and use what you've learned so far to create your very own movie introduction with Misty. Be creative and use all her functionalities. She'll be a perfect actor because she will listen to your every command.

Feel free to share it with the rest of the Misty community!&#x20;

***

### Advanced

If you are using the take picture or video recording in Blockly, you can access the files directly from the browser console ("Cmd + Option + J" (on a Mac) or "Ctrl +Shift +J" (on Windows). As an alternative, you can right-click on the webpage and click "Inspect" to open the developer console.).

Open it before running the code, it will look like this:&#x20;

<figure><img src="/files/pjImTvl0GvWlueuLEe7l" alt=""><figcaption></figcaption></figure>

Here you can see all what Misty is doing. To access the files you will need to find this area in your console:&#x20;

When she takes a picture or records a video she creates a link to that content. Click on the 7th line and expand it. The first link is your picture or video.

<figure><img src="/files/mQgPPgmyeYkaOUTbRZgd" alt=""><figcaption></figcaption></figure>

###

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/ccqYraVShxtjDQLngVaF" %}
[Lesson 5: Events](/blockly/blockly-lessons/lesson-5-events)
{% endcontent-ref %}


# Lesson 5: Events

## Aim

The aim of this lesson is to understand how Misty can interact with her surroundings and respond to user input using events. By the end of this lesson, you'll be able to create your first robot skills with multiple scenarios and outcomes. If you want to know more about Misty's events, you can check out [Events](/blockly-elements/misty-blocks/events).

## Events with touch

### Challenge 1: Program an event with physical touch

Up until now you've seen all of the amazing things you can do with Misty, but it doesn't end there. Misty can also receive commands through touch using various sensor event blocks. Let's try it out! \
\
Select the bump sensor event block and the touch sensor event found in the 'Events' tab in Misty Blocks, and the run until stopped block from the 'Miscellaneous' tab. It's important to remember that when you are programming Misty to react to the environment you need her to actively monitor her sensors. This is why it's necessary to put a run until stopped block after an event block. If you want her to monitor all the bump or touch sensors at the same time, you can choose 'any' in the drop-down field.\
\
For example if you look at the program below, you can see that when you trigger a bump sensor Misty will start the event "Love" , if your press any of her touch sensors you can trigger the event "Hi". &#x20;

Try to be as creative as you can! If you want to copy this code remember to enable the blocks.

<figure><img src="/files/qcZ3rskiWjxxmFNKV0Hc" alt=""><figcaption></figcaption></figure>

### Challenge 2: Creating events with bump sensors and touch sensors

To make Misty react to a specific touch or bump sensor you can choose which of the sensors you want to trigger in the drop-down list. Misty has four bump sensors above her tracks: Front right, Front left, Back right and Back left. On her head you will find six touch sensors: Head front, Chin, Head left, Head right, Head rear and Scruff (Back grip).

By combining different bump and touch sensors you can create and trigger more than one event. For example, you can create a skill where Misty drives forward for 5 seconds if touched on the front of her head and then drives backwards to her original position if you press the front right bumper.

The more capabilities you learn with Misty the more skill you can practice! How would you improve this program?

<figure><img src="/files/9TjHZnwm5Y3NDcatTqoh" alt=""><figcaption></figcaption></figure>

## Designing skills with event sensors

### Challenge 3: Create a multiple event skill

In most human-robot interactions you will find that robots usually have multiple reactions or answers to the same question, giving each person a unique experience.  These kind of interactions are built using a programming concept called 'skill tree'.  Imagine that Misty is asking you a question and you want to give more than one answer. The question in this case is the main body of your tree and the answers that you create are your branches growing out of the tree

Now that you've explored Misty's event sensors, let's try creating 4 different events using all of her  bump sensors. You can start by having Misty ask you a question like in the example below and create  events as responses. Use the drop-down to determine which sensor you want to trigger, remember to select each of the 4 bump sensor. The next stage is to nest your unique blocks in each of the event blocks, like the LED lights in the example below. You can also create more complex sequences with movement and expressions blocks, try it out!

<figure><img src="/files/ADHEea7EUEA7pLCyulWN" alt=""><figcaption></figcaption></figure>

### Challenge 4: A better way to program: the event message block

By using all of Misty's bump and touch sensors you can create more complex skills, conversations and interactions. Once you build up your program using multiple sensors you will find that it starts to occupy a lot of space and you will have to manually write in values for each sensor like in the example below.

<figure><img src="/files/5yIaOlG461n95jAktE9p" alt=""><figcaption></figcaption></figure>

This style of programming is time consuming and not very versatile, but there is a way to compact your program with an elegant solution. Instead of writing values manually each time, you could use the specific [Events](/blockly-elements/misty-blocks/events#event-message) for the touch sensor event. It will return the name of the sensor that you have triggered. If you continue with the example above, you can use the 'create text with' block to  build your texts however you prefer and have Misty return the name of the sensor at the end of the text. If you are unsure how to use this block you can check out [Text](/blockly-elements/basic-blocks/text).

<figure><img src="/files/ArxWnqiSIVCNho7LYUmH" alt=""><figcaption></figcaption></figure>

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/b2EtDuMAeOHJoaykPr2V" %}
[Lesson 6: Face recognition](/blockly/blockly-lessons/lesson-6-face-recognition)
{% endcontent-ref %}


# Lesson 6: Face recognition

## Aim

The aim of this lesson is to teach Misty how to recognize faces and remember her friends. It's a wonderful and interesting skill that most users want to play with and use for more complex interactions. You can find more details about her computer vision capabilities in [Vision](/blockly-elements/misty-blocks/vision) or ask our humans.

## Recognizing faces

### Challenge 1: Train Misty to recognize your face

Before exploring face recognition in Blockly it's important to understand some of the basics of working with face recognition. If you've already explore the Vision subsection in the Explore tab of Misty Studio, you most likely saw that there is a feature called Face Training.&#x20;

<figure><img src="/files/Y2PWRR3ArFI6ikwEKC5U" alt=""><figcaption></figcaption></figure>

Face training is a process in which your Misty builds up library of images with your face and maps them to create a unique FaceID associated with the name that you have written. You can start face training by clicking the start button, Misty will help you to guide you through it by displaying an orange label with a message. For instance, if it's too dark you are asked to move into the light. Misty's chest LED will turn green when your face is memorized. You can find a list of all the FaceIDs by clicking on the middle button 'Get Learned Faces', and you can also delete all of them by clicking on the last button ''Clear Faces'.

<figure><img src="/files/h4HGd9mWJAPPaejYcCtf" alt=""><figcaption></figcaption></figure>

Once you've created your FaceID, Misty will be able recognize your face anywhere, this function is called face recognition.  You can test if Misty recognizes you by clicking the 'Start Face Recognition' button under Face Recognition and you will see that your name will appear in the 'Label' field. The other fields will tell where you are in relation to Misty's camera.&#x20;

### Challenge 2: Create a Face Recognition Event

<figure><img src="/files/y0cBnsMgcn7XDAzUXwrI" alt=""><figcaption></figcaption></figure>

A face recognition works just as any other event, so it means that you will need to put the run until stopped block at the end of your program, however the main difference between a face recognition event and a bump sensor event is that you will need to have a start face recognition block to trigger it. \
\
The next step is to add your face recognition event block from the 'Vision' tab in Misty blocks and choose the name you want to be recognized in the first text field and Misty will only recognize the face associated with that name. If you leave it empty as show in the code above, she will choose any face in her system, as if  "any" was selected in the bump or touch sensor event blocks.&#x20;

If you check the second check box Misty will repeat the face recognition event every time she detects a face. Misty checks her surroundings several times per second so in this case she would say "Hi, I'm Misty" every time she detects a face. You can found out more in [Events](/blockly-elements/misty-blocks/events) when it's useful to check the second check box.

## Skills with face recognition

### **Challenge 3: Make a skill with face recognition**

Now that you understand how face recognition events work you can start building a multiple face recognition event skill to determine how your Misty will react when she sees specific individuals. Let's try creating two events for two individuals.

<figure><img src="/files/yHagECDZb385tZZaTWMT" alt=""><figcaption></figcaption></figure>

Can you see how this code can be improved? Imagine that you don't have to recognize only two faces but ten. This program would occupy a lot of space! In this case the[Events](/blockly-elements/misty-blocks/events#event-message)block comes to the rescue yet again. To compact the program into one face recognition event block, you will need to start with a condition block found in the 'Logic tab in Basic Blocks and a text block. By using the condition block you can let Misty know that when she sees a specific person, she has to trigger a sequence associated with that person. To do that you will need to set your condition to '='.

<figure><img src="/files/7FqM11LHhw7oRIDYww2u" alt="" width="370"><figcaption></figcaption></figure>

To compact the program into one face recognition event block and create multiple outcomes, you will also need to use the if-do and a else-if-do blocks found in the 'Logic' tab in Basic Blocks and a text block. To create and else if condition you can click on the gear icon in the if-do block. These blocks will tell Misty that she has to perform a sequence only if she sees a specific person. &#x20;

<figure><img src="/files/BDcmgP5g2xHOXLYNjQpK" alt="" width="544"><figcaption></figcaption></figure>

Now that you have all the condition blocks in place, try building your face recognition skill using the example below.

<figure><img src="/files/c75BKyfXiVkrw5G2KH2K" alt=""><figcaption></figcaption></figure>

### **Challenge 4: Don't repeat yourself**

We can do even more to improve the program. When learning programming you can find this amazing concept: DRY (Don't Repeat Yourself). The idea with this concept is that we need to repeat our code as few times as possible and make it as general as possible.&#x20;

In the example below you can see that we're combining our create text with block from the previous lesson and the condition blocks to generalize the program.  By separating the greeting 'Hi' from the names you can have hundreds of people memorized and each of them will be greeted with a 'Hi' followed by their name. If you want to give a special greeting to your friends, then you can create event messages for their names and add your own unique sequence for each person. You can easily expand this to everyone. Try it out using the example below!

<figure><img src="/files/CXLJdBgL7sbCw2ilZi8w" alt=""><figcaption></figcaption></figure>

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/9zPIKnIyHZaS7mghxThw" %}
[Lesson 7: Variables and Functions](/blockly/blockly-lessons/lesson-7-variables-and-functions)
{% endcontent-ref %}


# Lesson 7: Variables and Functions

###

## Aim

This lesson aims to give you an understanding of how you can improve your program with variables and functions.  You've already tried out some functions, but here you will learn how to combine them with variables to create more readable, expandable and more efficient code. If you have any questions about variable blocks, you can check out the [Advanced Blocks](/blockly-elements/advanced-blocks) section.

## How to use variables&#x20;

### Challenge 1:  Change movement values with variables

Variables are a fundamental concept in programming that allow you to store and manage data throughout your program. They are like containers that hold values, and you can refer to the values by using names. For example if you want to associate the variable with velocity, you can call it 'speed'.

Do you remember your first robot dance in Lesson 1?  Well, what if you want to modify the speed of your arms? Normally you would have to modify them one by one, but with variables it's enough that you change only the variable's value and the rest of your arm values will change accordingly as seen in the example below.

<figure><img src="/files/ailcsna1sWCRMDnSiXYS" alt=""><figcaption></figcaption></figure>

## Designing skills with variables

### Challenge 2: Create a skill tree with variables

In Lesson 5 and 6 we've already covered some of the basics of creating multiple events using the 'skill tree' concept. Here you will learn how to build more complex skill trees using variables.&#x20;

Instead of nesting your individual sequences in each bump sensor event block, this time you can create variables to control how events are triggered.  When you go to the 'Variables' tab in Advanced Blocks, you will find that it doesn't have any default variables, no worries, you can create one by clicking on 'Create Variable'. A desktop tab will appear where you can name your variable, for this example let's use the name 'Reaction'.&#x20;

<figure><img src="/files/UHLRilQP0roGuLndLRRz" alt="" width="317"><figcaption></figcaption></figure>

Once you have created your variable, Misty will automatically populate your 'Variables' tab with the variable block, a set-variable-to block and a change-variable-by block.

<figure><img src="/files/gPHXyV8TLHrIUsKwC0Ar" alt="" width="413"><figcaption></figcaption></figure>

Now you will need to decide what you want Misty to do, and then create multiple bump sensor events that will determine how Misty should respond when the events are triggered. By using variables you can create a more complex skill tree. To do that  you will need to add a set-variable-to block and define it's conditions. For example if we use Logic blocks  'True' or 'False', we can create additional events based on the condition if the first event is true or false. For example, if we nest a variable that is 'False'  in the front left bumper event, it will trigger the next event that is associated with the condition 'False', and if we do the same for the front right bump sensor using 'True' , it will trigger an event associated with that condition.&#x20;

<figure><img src="/files/pWvEq8uWIu7PsYncLpAX" alt="" width="563"><figcaption></figcaption></figure>

Once we set the value of the variables and the conditions, we can start creating the sequences associated with the 'True' and 'False' events.

For Misty to be able to monitor all sensors at the same time and detect the ones you've selected for the condition, you will need to set your next bump sensor event to 'Any'. The next stage is to use the if-do or else-if-do function blocks and connect them with our variables and conditions. In the example below you can see that if the Front Left bumper is triggered, this tells Misty that the condition is 'False' and she has to trigger the corresponding sequence. Remember that you have to make sure that the variable 'Reaction' is equal to the condition you want to associate it with.

<figure><img src="/files/pak7xNObZjAzcfq5kyw7" alt=""><figcaption></figcaption></figure>

Using the same principle as in the previous example, we could also build a different type of 'skill tree' with the if-do-else block. Instead of creating a condition for each sequence, you can create only one condition, in this case 'True'. If you click the Front right bumper as in the example above, Misty will run the first sequence, if you click any other bumper, it will not match the condition and automatically runs the second sequence.

<figure><img src="/files/mnSUbZ9nwYNZhDkZMYSI" alt=""><figcaption></figcaption></figure>

We suggest using LED lights for event feedback to let you know which condition you have triggered. This way you can easily follow visually where Misty is going.&#x20;

<figure><img src="/files/TewPTdZfiOAOumrpTBHP" alt=""><figcaption></figcaption></figure>

### Challenge 3: Call your functions

Now that you've mastered how to use variables, let's discover the power of functions.  If you every find that you want to re-use a particular group of sequences in your Blockly workspace, there is a neat solution that allows you to call that group and run it no matter where you are in your program. Functions only run when they are called, so you can easily modify them at any point. Now let's build your first set of functions!

To be able to call your sequence group you with you will need to nest it in a function block and give it a unique name.

<figure><img src="/files/EymJQu1OQir5GlzpEvpA" alt="" width="272"><figcaption></figcaption></figure>

Once you've written the name of your function, the call block with its name will appear in your 'Function' tab in Advanced blocks.&#x20;

<figure><img src="/files/aQmj77LVazPOVrUiSdQS" alt="" width="137"><figcaption></figcaption></figure>

This block is used to call the function in your program. Try creating your won sequence group in a function and insert its call black in a new sequence to see what happens!

<figure><img src="/files/LBZkkC7dzA1x1BBrMGcr" alt=""><figcaption></figcaption></figure>

### Challenge 4: Using functions in events

If you have a very small set of events you can write a block of code for each one of them, but if you want Misty to do many things, or write complex programs to diversify the interaction, you'll get tired of  scrolling or looking for a specific block every time. That's why functions are very useful in creating multiple events.  In the example below, you can see that the function 'Misty Waves' has been nested and called inside the face recognition event. Try building a new program where you call a function and use it in one of the event blocks.

<figure><img src="/files/t6q3CUcev8kIbOARKAWS" alt=""><figcaption></figcaption></figure>

Remember that you want to generalize it as much as possible, so to improve and compact your code even more, you can take the sequences associated with your 'Event messages' and simply group them with a function block.

<figure><img src="/files/CSZScSbWNNcokzwjGr1u" alt=""><figcaption></figcaption></figure>

### Challenge 5: Using variables in functions&#x20;

Functions can also include variables. This makes them very versatile, in this way, your program will get all the pros of the variables and the pros of the functions. Here is an example of how to use a variable in a function.  You can start programming cool skills with all the knowledge you have now. \
Just remember that you need to define your event message variable inside an event block.

<figure><img src="/files/wuGHC2N8Jt8MxSGrtKaN" alt=""><figcaption></figcaption></figure>

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/WgV7UuDmJz8FvugFKOX0" %}
[Lesson 8: NLP](/blockly/blockly-lessons/lesson-8-nlp)
{% endcontent-ref %}


# Lesson 8: NLP

## Aim

Natural Language Processing (NLP) is an incredibly powerful tool that allows you to verbally communicate with robots and have them understand you. Here you will discover the basics of how to work with Misty's NLP, and by the end of lesson you will be able to create conversations about any topic. If you want to know more about NLP before you start, check out [NLP](/blockly-elements/misty-blocks/nlp).

## Wake Misty up

### Challenge 1: Start with a wake-up word

Before you start a conversation with Misty you can let her know that you want to get her attention. To do this, you will need to use the start key-phrase recognition block found under the 'Speech' tab in Misty blocks. If for Alexa for example, the wake-up word is "Alexa", and for Google it's "Hey, Google", for Misty the wake-up word is "Hey, Misty". The next step is to decide what you want Misty to do when she recognizes the wake-up word and starts waking up. For this you will need the key-phrase recognized event block found in the 'Event' tab.  Here you can nest your unique wake-up sequence. Once you start your program, you will see that Misty's tally light turns blue, this means that she's listening to you. How would you build your wake-up sequence?&#x20;

<figure><img src="/files/K7bXSg4wnsPxnZcK4Roa" alt=""><figcaption></figcaption></figure>

## How to talk to Misty

### Challenge 2: Building your first conversation

Now that you've created your wake-up sequence to get Misty's attention, you can start familiarizing yourself with how a 'conversation tree' is built by using the YesNoQuestion conversation tree example below. A 'conversation tree' is similar to the 'skill tree' concept we explored in previous lessons. In a conversation tree the context of the conversation is the main body of the tree, and the various topics are branches that grow and branch off into subtopics. For this example we will build a very simple conversation tree with Yes and No branches.

The first step in building your conversation tree in Blockly is to select the conversation-starting state block from the 'NLP' tab. This is the block where you will nest your whole conversation tree. In the first text field you can define the name of your conversation, in this example we will use the 'YesNoQuestion' preset to create a conversation tree where your Misty will ask you questions and understand if you say 'Yes' or 'No'.

<figure><img src="/files/wpojSJoRbJicuTaGfPPH" alt="" width="308"><figcaption></figcaption></figure>

The next step is to nest your starting state block, this block will determine what Misty will say and do when the conversation starts.  After you have chosen what Misty should ask you, you will need to select a context. For this example we will use the 'furhat.context.en.yes-no' found in the drop-down field. A context in a conversation tree contains all the topics and words associated with them that Misty can understand. These are commonly referred to as 'intents'.  The context in this example contains only two intents: Yes and No. <br>

<figure><img src="/files/uvfBE1IKMxIwngzYTdOM" alt="" width="443"><figcaption></figcaption></figure>

**Important:** If you want Misty to listen to you speaking during a state, it's important to select 'listen' in the action field. This way Misty will recognize your speech and reference it with the intents.

You will also notice that this block has no match and no match count fields. In the event that Misty does not understand you, you can decide what she will say and how many times she will say it.

To let Misty know how the conversation should flow after you have initiated the start state, you need to use the state transition block and place it after all of your states.

<figure><img src="/files/wBaJPh3wg1NrtI11wsUb" alt="" width="251"><figcaption></figcaption></figure>

To trigger a state transition, Misty will need to reference your speech with the intents in the context. To let her know which intent should trigger a transition you will need to write the intent name in the trigger field. Remember that in this case we only have two intents: Yes and No.\
\
**Important:** Your state names in the state transition block have to match the state block names, otherwise the program won't work.\
\
Finally, you will need to add a start conversation block and write the name of the conversation tree you have created to get it going. Remember that just like with events, you will need to use a run until stopped block for Misty to be able to monito input.\ <br>

<figure><img src="/files/ahlLNJXl85neKqdC7AVK" alt="" width="343"><figcaption></figcaption></figure>

Now that you have explored all the blocks you need to construct a conversation tree, let's put it all together using the conversation tree example below.

<figure><img src="/files/v9h0muhrX71VxI3D01Tw" alt=""><figcaption></figcaption></figure>

### Challenge 3: Build your own context&#x20;

Great job on building your first conversation tree with Misty!  Let's try building a new conversation from the very beginning, but this time with your own context.

To get started, you will need to clear your Blockly Workspace and insert the context block from the 'NLP' tab in Misty Blocks. Choose a name for your context and think about what you want the intents to be. To add an intent you can use the gear icon in the top left corner and drag a new intent block into your context block.&#x20;

<figure><img src="/files/YcyuJYrh1gFMPoD0qxJI" alt="" width="346"><figcaption></figcaption></figure>

\
To name your intents and choose the words you want to associate with them you will need to connect a name-sample block.

<figure><img src="/files/Yv3LRs0WNrl2ZJRgUFzz" alt="" width="382"><figcaption></figcaption></figure>

In the example below we've named the context samplefood and created two intents. The first intent is pizza and the second is taco. In the sample fields you can write as many words as you like that you want to be associated with your intents.

<figure><img src="/files/lwX3LfAegvh5dZZVbFVv" alt=""><figcaption></figcaption></figure>

Once you build your new context you can run the code and your context will be uploaded to Misty's memory. \
\
**Important***:* Context names should not be re-used since Misty will not know which version you want to use. Every new context needs to have a unique name.

### Challenge 4: Build your own conversation tree

Congratulations! You are ready to start building your very own conversation tree. Just like in the example shown in Challenge 2, you will need to use the intents you just created in your context to trigger states and state transitions. If you want to continue building on the context from the previous example, you can use the following conversation tree.

<figure><img src="/files/voHvFrS1Xw8PhTAXf9QG" alt=""><figcaption></figcaption></figure>

### Challenge 5: Animate your conversation

Now that you've built your conversation tree, let's take it to the next level! You can also include all that you've learned about Misty in the previous lessons and include it in your conversation. Using the action block in the NLP tab, you can build new actions for your states. To select them in your action drop-down in the state block you will have to upload them by running the program first. Try it out using the example below!

<figure><img src="/files/A5eLTrVbHkDoDlajStlw" alt=""><figcaption></figcaption></figure>

Unfortunately, there are some issues with the action blocks.&#x20;

* It seems that you can't use negative parameters in the movement blocks.&#x20;

The solution is: instead of utilizing the Value Block, you can opt for the Operation Block and calculate your number. This should address the problem effectively.

<figure><img src="/files/9cWAYjobiZqXUXslWm3u" alt="" width="563"><figcaption></figcaption></figure>

* There are some blocks who generate an "Invalid diagram" in Blockly. The reasons are unclear, but Misty's actions are stored in the same space in her API. Indistinctly if they come from Blockly or from Python.&#x20;

So you can create new actions in Misty by writing in Python a code built in this way:

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events

misty = Robot()

misty.create_action(name="move", script = "HEAD:0,0,0,1750;ARMS:0,0,1750;HEADING:0,0.5,3000,false;", overwrite=True)
```

For example, this one leads Misty to move her head, move her arms and to a front movement of 0.5 meters.

create\_action has 3 parameters :\
-name (the name that you'll use to recall your action)\
-script (the list of commands that Misty will use)\
-overwrite (we use this to always have the latest upload)&#x20;

You can find the complete list to customize your Misty's actions here : [Action Commands](/resource-database/action-commands)


# Blockly projects

If you want to challenge your students with Misty blockly programming this is the right page!

## Object recognition

The first challenge for your students after they finish the Blockly lesson course is to create an Object recognition event with Misty entering in a routine and then recognizing at least two objects. \
In this case, we selected something that could be easily found in a classroom: a book and a person.

The first step is to create an initial Misty behaviour, like this one, including different [faces](/resource-database/image-files), [audio](/resource-database/audio-files) and colours to your Misty.

<figure><img src="/files/UVRrYXK3GsOQ458dci51" alt=""><figcaption></figcaption></figure>

Then it's necessary to create the events that will handle the objects that your Misty will recognize.&#x20;

For example, a book:

<figure><img src="/files/wPH9yk2e5fXcJrIXB0FJ" alt=""><figcaption></figcaption></figure>

or a person:

<figure><img src="/files/tpEFSKuJumL54tf3GMf4" alt=""><figcaption></figcaption></figure>

Make sure that your Misty will have a unique reaction to each recognized object and now you're ready to call the object recognition event:

<figure><img src="/files/kamu5NeRHzPWEcJ93Cpk" alt="" width="170"><figcaption></figcaption></figure>

You can stop Misty's object detector after completing the recognition by enabling the grey block ("stop object detector") at the end of the event handler for each recognized item.

After this event, you can create new animations or interactions with Misty!

<figure><img src="/files/8pJ8ZmKM10UffmW1mQm7" alt=""><figcaption></figcaption></figure>

Check all [Misty's known objects](/resource-database/known-objects)!

## Dance party

This challenge involves Misty's [NLP capabilities](/blockly/blockly-lessons/lesson-8-nlp) and her great skills in dancing and having fun!

In this challenge, Misty will have to learn which types of music/dance exist and also how to dance them!

For example, we selected Waltz and Headbang.

To make Misty learn which type of music/dance it's necessary to create a context that will train her AI capabilities to make her recognize the words in the sample label

<figure><img src="/files/WvZ9F8P4tZNAJYLFICQ3" alt="" width="347"><figcaption></figcaption></figure>

And to make Misty learn how to dance describe what you want her to do with an action block!\
Don't forget to upload the music that Misty will play on the Explore>Expression page and then select it.

<figure><img src="/files/996QJIv1BXja1RpCpFRS" alt="" width="462"><figcaption></figcaption></figure>

<figure><img src="/files/XBXkFRtWczajOanKUX6A" alt="" width="464"><figcaption></figcaption></figure>

You need to run this blockly code before your next ones, in this way, even if Misty will not do anything she'll learn all that she needs for your skill!

Now it's time to create some interaction:

Use NLP blocks to build your very original conversation with Misty including your context in the first state and your actions in the next ones

<figure><img src="/files/yesSpDWcWTzlCONMyha7" alt="" width="245"><figcaption></figcaption></figure>

After you run the code, also this block can be disabled because Misty will have learnt it.

If you want to apply changes to your structure feel free to keep it enabled and Misty will delete the previous conversation and create a new one with the same name.

When you're satisfied, you can use these blocks to run it!

<figure><img src="/files/fm8izinnQrZo0iSCOAKc" alt="" width="274"><figcaption></figcaption></figure>

<figure><img src="/files/lU2HProaJGza0DFFYL6q" alt=""><figcaption></figcaption></figure>

## Phrase recognition and random behaviour

With this latest challenge, your Misty will assume a random behaviour when suddenly woken up!

Also in this case is best to create a sequence of actions before the actual skill to create context.

In this case, Misty will pretend to sleep and she will extract a random number between 1 and 3, it will help us later.

<figure><img src="/files/jLejcMZA601GzLLNSF6m" alt="" width="372"><figcaption></figcaption></figure>

We'll use the extracted number to select one of the three actions that Misty can do while she listens to the[ Misty word](/blockly/blockly-lessons/lesson-8-nlp#wake-misty-up).

Then it's necessary to create functions that will handle every number:

<figure><img src="/files/vdeUc6lX8TCEpYj52kRu" alt=""><figcaption></figcaption></figure>

And last but not least the listening event that, once heard the word, will assign to each number its action.

<figure><img src="/files/O5jTELENYzFGxyEIYrHE" alt="" width="563"><figcaption></figcaption></figure>

In this case, you cannot disable the functions because they are not memorized in Misty's memory as it was for the NLP.

<figure><img src="/files/197I0JFTLnSxUBUHoEjE" alt=""><figcaption></figcaption></figure>


# Misty Blocks

In this section you will discover a world of Blocky blocks designed to effortlessly control and customize various actions of your Misty. These Misty Blocks serve as action and function blocks for crafting your interactions. These will be the tools that you can use to choreograph Misty's ever move and let your ideas flow through this digital realm of possibilities. Your Misty awaits your command, one block at a time.

## Content&#x20;

{% content-ref url="/pages/TcvUlq9VIJS3KUjBUIrK" %}
[Movement](/blockly-elements/misty-blocks/movement)
{% endcontent-ref %}

{% content-ref url="/pages/y0BeGQ4nqc0b711VUSvi" %}
[Speech](/blockly-elements/misty-blocks/speech)
{% endcontent-ref %}

{% content-ref url="/pages/AiboAR5V5kw0esnTpBnf" %}
[Audio](/blockly-elements/misty-blocks/audio)
{% endcontent-ref %}

{% content-ref url="/pages/g3S04OWoQ20Hupn34pMn" %}
[Vision](/blockly-elements/misty-blocks/vision)
{% endcontent-ref %}

{% content-ref url="/pages/0SL9ytXpx8Ql9EjlXe2C" %}
[Events](/blockly-elements/misty-blocks/events)
{% endcontent-ref %}

{% content-ref url="/pages/chTZXc9ofK57zQ9aavMy" %}
[Miscellaneous](/blockly-elements/misty-blocks/miscellaneous)
{% endcontent-ref %}

{% content-ref url="/pages/SqlAy4yAUkkonWYFpUXF" %}
[NLP](/blockly-elements/misty-blocks/nlp)
{% endcontent-ref %}

{% content-ref url="/pages/EOrF8h3yfxD2XME9dwQ4" %}
[System](/blockly-elements/misty-blocks/system)
{% endcontent-ref %}


# Movement

In this folder, you can find detailed information about the movement blocks and understand Misty's range of motion.

When you start building your first sequence with movement blocks, it's important to keep in mind that if you need to have a timed delay between each block, otherwise Misty won't have enough time to execute all of the action. You can set a delay using the timer block found in the 'Miscellanious' tab. &#x20;

<figure><img src="/files/YbQWNILSJ4j8jjfnctiy" alt=""><figcaption><p>movement blocks</p></figcaption></figure>

### Head movement&#x20;

<figure><img src="/files/j0UDBe59Ze28H0rklNZL" alt=""><figcaption><p>head movements block</p></figcaption></figure>

Misty's head has three ranges of motion head and the values are calculated in degrees.&#x20;

The first one is the pitch, its range is -40 (Up) to 25 (Down) . It represents the front movement up and down of the head, as you can see in the first vector.&#x20;

The second one is roll, its range is -42 (Left) to 42 (Right). It represents the side movement of the head, as you can see in the last doodle.&#x20;

The thirds is yaw,  its range is -90 (Left) to 90 (Right). It represents the lateral movement of the head, right to left, as you can see in the second vector.

The last parameter to set is the speed with which Misty will achieve that position. Its range is \[1,100].&#x20;

### Arm movement

<figure><img src="/files/3hudRSGaDhu3tKxHig3Q" alt=""><figcaption><p>arms movement block</p></figcaption></figure>

Misty has the possibility to move her arms on one singular plane. The values are also in degrees.&#x20;

<figure><img src="/files/71TIEUsQdIqcu8h1WQw2" alt=""><figcaption><p>right and left hand</p></figcaption></figure>

The range of Misty's arms is -90 (Up) to 90 (Down). The velocity parameter is the speed that Misty will use to achieve that position with a range between 1 and 100.

### Drive command

<figure><img src="/files/48KdTsYGZo457b817kJ4" alt=""><figcaption><p>drive block</p></figcaption></figure>

The drive block allows you to move Misty forward or backward at a specific speed.

When using this block, it's important to understand how linear velocity (speed in a straight line) and angular velocity (speed and direction of rotation) work together:

* Linear velocity (-100) and angular velocity (0) = driving straight backward at full speed.
* Linear velocity (100) and angular velocity (0) = driving straight forward at full speed.
* Linear velocity (0) and angular velocity (-100) = rotating clockwise at full speed.
* Linear velocity (0) and angular velocity (100) = rotating counter-clockwise at full speed.
* Linear velocity (non-zero) and angular velocity (non-zero) = Misty drives in a curve.&#x20;

Parameters:&#x20;

* Linear velocity - A percent value that sets the speed for Misty when she drives in a straight line. The default value range is from -100 (full speed backward) to 100 (full speed forward).
* Angular velocity- A percent value that sets the speed and direction of Misty's rotation. The default value range is from -100 (full-speed rotation clockwise) to 100 (full-speed rotation counter-clockwise).&#x20;

The maximum speed achievable from misty is about 0.5m/s.&#x20;

If the robot is not moving forward or backward, try to reboot it or refresh the interface.

&#x20;

### Timed Drive Command

<figure><img src="/files/43N71Eukh3PqOv6IbcqN" alt=""><figcaption><p>drive by time block</p></figcaption></figure>

Using a drive block with a timer allows you decide how long you want Misty to drive in a particular direction. While the timer is set in seconds, you can also write half of seconds and use the comma freely.&#x20;

### Drive arc

<figure><img src="/files/CR7lukXjnCw2oWbtN13q" alt=""><figcaption><p>drive arc block</p></figcaption></figure>

If you're not a fan of straight lines, you can drive Misty in an arc with the drive arc block. Here you need to calculate the amount of time you need to execute the whole arc depending on the heading and radius that you've chosen.&#x20;

There are several parameters:&#x20;

* Heading - This is the absolute heading Misty should obtain when the arc is complete. To set the absolute heading, use either: 0  to 360, where 0 is straight ahead, 90 is directly to the left, 180 is straight behind, and 270 is directly to the right, or: -180 to 180, where 0 is straight ahead, 90 is directly to the left, 180 and -180 are straight backwards, and -90 is directly to the right.
* Radius - The radius (in meters) of the arc.
* Duration - The duration (in seconds) that Misty needs to execute the arc.

### Motor controls&#x20;

<figure><img src="/files/gBbwz04PiY1liMWDNeOJ" alt=""><figcaption><p>stop driving and stop all motors blocks</p></figcaption></figure>

These two blocks control the movement and they are pretty self-explanatory, you will find that most Blockly language is. The first stops only the driving blocks and the second stops all other movement blocks . You can also use the 'Halt' button in the top right corner of  Misty Studio to stop the program.<br>

<figure><img src="/files/vbT9MWELs4I78yWofVZY" alt=""><figcaption></figcaption></figure>

*Note: We don't recommend using the halt button frequently. Misty's drive motors remain engaged after she stops moving and attempt to hold her in her current position. This can be useful when Misty needs to stop moving while she's on an incline; sending Halt keeps the motors engaged so that Misty does not roll down the slope. Stopping Misty's driving with the stop blocks should suffice under most circumstances and can prolong the life of your robot's drive motors.*


# Speech

In this folder, you will find more information about Misty's speech capabilities and how to use the speech blocks.

<figure><img src="/files/drPMz4qMfbIUsa8nREbc" alt=""><figcaption><p>speech blocks</p></figcaption></figure>

### Text-To-Speech (TTS)

Text-to-speech is one of the most important tools that you will need to give Misty a voice. You can select your favorite language and insert a text value block with your unique text to have Misty say it in the language that you've chosen.

<figure><img src="/files/jR1i3dWYTkK7Z5lh4Ioo" alt=""><figcaption><p>text to speech block</p></figcaption></figure>

You can adjust the pitch of Misty's voice from 0 (Deep), 1 (Default) to 2 (High).

Here you can find a list of all the languages that Misty can speak:&#x20;

Arabic, Bengali, Chinese, Czeck, Danish, German, Greek, English, Spanish, Estonian, Finnish, Filipino, French, Gujarati, Hindi, Hungarian, Indonesian, Italian, Japanese, Khmer, Kannada, Korean, Malayalam, Mandarin Chinese, Norwegian, Nepali, Dutch, Polish, Portuguese, Romanian, Russian, Sinhalese, Slovac, Sundanese, Swedish, Tamil, Thai, Turkish, Ukrainian, Urdu, Vietnamese.

In most of these languages, you can select a male or female voice, but we suggest using the female one because it matches Misty's character and personality more. Afterall, she was designed to be the power girl of the robot world.

### Stop speaking

<figure><img src="/files/nhcFESF9lgEbO4dNQIHk" alt=""><figcaption><p>stop speaking block</p></figcaption></figure>

If Misty is speaking and you want her to stop at a specific point, you can use a timer block and the stop speaking block to let her know.

### Speech and Keyphrase recognition

Speech recognition allows Misty to record your speech and convert it into text that she can understand. This tool will be essential when you start creating conversations with Misty. The keyphrase recognition blocks use speech recognition to record specific keywords or keyphrases. These can be useful when you want to trigger sequences using a word or a phrase. The default keyphrase for Misty is 'Hey Misty'.

<figure><img src="/files/yjvoTXrqWlbDlHwdxgpX" alt=""><figcaption><p>keyphrase blocks</p></figcaption></figure>

Once you've trained your Misty to recognize specific words you can use these two blocks in order to start or end Misty's "listening mode" for keyphrases. When Misty will show you that she is listening by turning her head tally light blue.&#x20;

### Capture speech&#x20;

<figure><img src="/files/yyUJ9d0aQZoE0vwfMcUC" alt=""><figcaption><p>capture speech block</p></figcaption></figure>

This block starts capturing speech in a new audio recording. By default, Misty's chest LED pulses blue when she is recording audio or listening for the key phrase. Misty's head tally light also turns on when she is recording audio or video.

Misty waits to start recording until she detects you speaking. She then records your speech until she detects the end of a question or statement. By default, Misty records speech up to 7.5 seconds.

To learn more about Misty's conversational capabilities, you can jump to [NLP](/blockly-elements/misty-blocks/nlp)


# Audio

Recording and playing audio can be useful if you want Misty to provide you feedback, give you notifications or just be your personal DJ. Audio also can enhance her personality and expressiveness, providing auditory cues and alerts.

There are a few limitations to consider when working with audio files:&#x20;

* Misty can't record longer than one minute
* The maximum size of the file is 3MB
* The file types supported are .wav, .mp3, .wma, .aac

<figure><img src="/files/eKAgOrlRn63eA0k8eFC5" alt=""><figcaption><p>audio blocks</p></figcaption></figure>

### Manage your audio&#x20;

<figure><img src="/files/UQmOJuhzBVa3FqIUsezy" alt=""><figcaption><p>manage audio blocks</p></figcaption></figure>

With these two blocks, you can upload an audio file directly from your computer. By clicking on the checkbox in the first block you will access your computer folders and be able to choose your audio. You always need to consider the limitations we wrote at the beginning of this paragraph. The default audio files will not be deleted.

Misty memorizes all the audio files you upload to her. With the second block, you will be able to delete a chosen audio file and free her memory.&#x20;

### Record audio

<figure><img src="/files/6ZSCyNcKxrnhdmaWs8nV" alt=""><figcaption><p>record audio blocks</p></figcaption></figure>

This block can be used to starts recording audio. Misty saves audio recordings in her local storage as .wav files. To stop recording, you must call the stop recording audio block. If you don't stop the "recording audio block" Misty will continue recording for 60 seconds.

You can name your audio file in the text field.

### Play audio

<figure><img src="/files/ztdv5bhxmekii36DnXd8" alt=""><figcaption><p>play audio blocks</p></figcaption></figure>

With the first block, you can choose which audio to play and its volume. The volume range is 0 to 100.&#x20;

With the second and the third block, you can pause or stop an audio file after a specific amount of time, similar to the stop speech block. The difference between the pause and the stop block is that if the audio is recalled the former will play the audio from the point it stopped before, the latter will play the audio again from the beginning.&#x20;


# Vision

One of the most awesome features in Misty is the customizability of her display. This allows her to show different facial expressions and visual information in the form of images, videos and text. Misty's display resolution is 480x272 pixels. The size of the image is calculated from the top right point, so in case the image is larger in size, Misty will try to resize the image to fit the screen, if the image is too narrow or too wide it will be cut. The display can reproduce a wide range of default Misty's eyes and actions. You can find a library of the default images in [Image files](/resource-database/image-files)

There are a few limitations  to consider when it comes to using new images and video files:&#x20;

* The maximum size of a video file is 6 MB
* The video file types supported are .MP4 or .wmv
* The maximum size of an image is 3 MB
* The images file types supported are .jpg, .jpeg, .gif, .png

In the 'Vision' tab in Misty Blocks, you will be able to explore all of her blocks related to vision such as taking images and videos, displaying them and recognizing faces and objects.

<figure><img src="/files/l0IrSzG8LjCdAQ064f4G" alt=""><figcaption><p>visual blocks</p></figcaption></figure>

<figure><img src="/files/ThR8xWtMT2nRJtdoNL77" alt=""><figcaption><p>visual blocks</p></figcaption></figure>

### Take picture

<figure><img src="/files/jgBEalglZiRfUIvH3DbD" alt=""><figcaption><p>take picture block</p></figcaption></figure>

With this block, Misty can take a picture with her RGB camera.

In order to download your image and re-upload it with a custom file name, we suggest to use the Take Photo section in Vision under the Explore tab in Misty Studio.

<figure><img src="/files/NLcRv52Zv5TVCkH19o1Z" alt=""><figcaption><p>take photo</p></figcaption></figure>

There are two different options: RGB camera  and Wide-Angle camera.

The former takes a color photo in 4K. You can either choose to download it or display it.&#x20;

The latter is a feature in Misty II Pro that allows you take a black-and-white photo with the wide-angle camera on Misty's Occipital Structure Core depth sensor. You can also download it or display it.&#x20;

### Images

<figure><img src="/files/d8RgnIt04tRsbNQVC6RJ" alt=""><figcaption><p>images blocks</p></figcaption></figure>

With the display image block you can explore the wide range of Misty's expressions found in [Image files](/resource-database/image-files). You can also choose the opacity of the image. Its range is 0 to 1, where 0 means all black and 1 means picture with no opacity.&#x20;

The upload image block allows you to add new custom expressions that you've designed or regular images. To save an image to Misty's memory and access it in the display image block, you will first have to run the upload image block.

With the delete image block you can easily remove any file you've uploaded from Misty's memory. Default images can't be deleted.&#x20;

&#x20;

### Text

<figure><img src="/files/1EBCoCBZrLHv1XpL9zbh" alt=""><figcaption><p>text block</p></figcaption></figure>

Misty has also the possibility to display text. Due to the size of Misty's display you can put on average 80 characters. In order to remove the text you can run this block with an empty field.

### Video recording

<figure><img src="/files/kLFIZNEatvXjduSmjFPA" alt=""><figcaption><p>video recording blocks</p></figcaption></figure>

Misty can record high-quality videos with her cameras.&#x20;

With the start video recording block, you can write the name of your video in the text field, select the duration and choose the quality. Misty has a wide range of video quality, depending on your needs: 3840x2160, 1920x1080, 1280x960, 640x480, 320x240.&#x20;

The second block stops Misty's video recording.&#x20;

The third block works in the same way as the delete image block.

The upload video block also works in the same way as a upload image block. It allows you to access your computer and choose video files to upload, remember that you have to run this block first for the video to show up your list of videos.

### Video asset

<figure><img src="/files/LWakwAwp3zGdtWZqD7M6" alt=""><figcaption><p>video asset blocks</p></figcaption></figure>

In order to play or delete the video you uploaded you will need to use these blocks.&#x20;

### Face detection

<figure><img src="/files/ef1I6t10N3cAFGL10Bfr" alt=""><figcaption><p>face detection blocks</p></figcaption></figure>

When Misty uses face detection, she uses algorithms to identify and locate human faces in front of her. This block is useful when your skill is supposed to count how many people there are in front of Misty and interact with her. The first block starts face detection, the second block stops it.&#x20;

You can also try out the face detection feature in the Vision tab under Explore.&#x20;

### Face training

<figure><img src="/files/AVdaNowawHIVVQqj4Ifh" alt=""><figcaption><p>face training blocks</p></figcaption></figure>

This is a very popular group of blocks in Misty because they allows her to learn the faces and names of the individual she interacts with. During face training, Misty takes several pictures of your face with her RGB camera. She then attempts to create a face matrix that she can use to recognize you in the future.

With the start face training block you can train Misty to recognize a face and associate it with a FaceID. To create your Face ID all you need to do is write your name in the text field, run the block and wait 30 seconds for her to learn your face.  The chest LED will turn green when your face is memorized.

The second block stops the face training, if you don't use it Misty will continue to look for more faces to learn.

The third block gives you access to all the learned faces and the last block clears them.&#x20;

You can also use the face training button in the Vision tab under Explore. During face training Misty will guide you with messages at the bottom. For instance, if it's too dark you are asked to move into the light.

<figure><img src="/files/uTT18qcEd2g6m5xGNLCk" alt=""><figcaption></figcaption></figure>

### Face recognition

<figure><img src="/files/XprwuhP4bRqeTJTM85hK" alt=""><figcaption><p>face recognition blocks</p></figcaption></figure>

After Misty has learned your face, she can recognize you any time you run the start face recognition block. To get information about the face that Misty is currently detecting you  can go to the Face Recognition section in the Vision tab under Explore. There you you have several labels tall tell you the FaceID she identified and the position of the face.

If face recognition is running on the robot, and a previously trained face is recognized, the label value is the name previously assigned to that face. The label value is unknown\_person if an untrained/unknown face is detected.

<figure><img src="/files/h4HGd9mWJAPPaejYcCtf" alt=""><figcaption></figcaption></figure>

### Object detector

<figure><img src="/files/c2spJpsSA7koHIDReeAv" alt=""><figcaption><p>object detector blocks</p></figcaption></figure>

The start object detector can give information about the type and position of objects.&#x20;

Misty has a list of pre-trained objects that she can recognize:

person, bicycle, car, motorcycle, airplane, bus, train, truck, boat, traffic\_light, fire\_hydrant, stop\_sign, parking\_meter, bench, bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe, backpack, umbrella, handbag, tie, suitcase, frisbee, skis, snowboard, sports\_ball, kite, baseball\_bat, baseball\_glove, skateboard, surfboard, tennis\_racket, bottle, wine\_glass, cup, fork, knife, spoon, bowl, banana, apple, sandwich, orange, broccoli, carrot, hot\_dog, pizza, donut, cake, chair, couch, potted\_plant, bed, dining\_table, toilet, tv, laptop, mouse, remote, keyboard, cell\_phone, microwave, oven, toaster, sink, refrigerator, book, clock, vase, scissors, teddy\_bear, hair\_drier, and toothbrush.

*Please Note: All the functions related to object detection and face training must be used together with their associated events. If we just start the object detection and don't use that information in another block the robot will learn it and forget it the moment after, and we can't build anything on it.*


# Events

The events blocks enable Misty to respond to changes in her environment and perform actions accordingly. Events provide a reactive programming model, allowing Misty to autonomously respond to you through real-time user input. In order to create this autonomous interaction, you must create skills and robot applications that are driven by events and with logic that listens for and responds to incoming environmental data.

When we are running an event we need to make sure Misty monitors her environment and sensor input at all times so she knows what's happening. To do this you will need to place a 'run until stopped' block from the 'Miscellaneous' tab in Misty Blocks.

<figure><img src="/files/zxNDgDVFe8yKSJHtvCOg" alt=""><figcaption><p>events blocks</p></figcaption></figure>

<figure><img src="/files/X9RqFH1zjOJ97OZvw7Ee" alt=""><figcaption><p>events blocks</p></figcaption></figure>

All the event blocks work in the same way:&#x20;

They wait for something to happen (bump sensor, keyphrase, face recognition, etc.) and if they are triggered they will execute the code nested inside them. If the checkbox is selected in an event block, it will let Misty know that she has to repeat it. It means that if you want to execute the event only once when its triggered you don't have to select the checkbox. Otherwise, you select it and the event will be executed as long as the event is verified.&#x20;

For example :&#x20;

* If our program is: to touch Misty's head and say "Hi", we want it to happen only once.
* If our program is: to record my voice as long as I press on the bump sensor, we want it to be repeated.&#x20;

### Event Message

This block is really important if we want to use information from an event. It transmits the event message associated with each event block. For example, when you triggering face recognition and Misty sees person's face that has a FaceID, you can use that information to start trigger another event with an action sequence. For example if Misty's sees you and recognizes your face, you can program her to send your name to trigger an event block with a greeting.

<figure><img src="/files/8FhTtYPXdV356zB5xZXM" alt=""><figcaption><p>event message block</p></figcaption></figure>

To receive event messages in an event block, you must create event listeners. We sometimes refer to the process of creating an event listener as *registering for an event*. You will find more information about how to do this in [Lesson 5: Events](/blockly/blockly-lessons/lesson-5-events). When you register for an event, you can:

* filter out unwanted data by specifying which properties an event message should include
* apply event conditions and validations or rules that define what kind of data an event listener can receive from a particular event type
* set a debounce value, or how frequently a particular event type should send event messages (this is useful for event types that provide a constant stream of new data, instead of sending data just once when a particular event occurs)

### Tactile sensors

<figure><img src="/files/w9A8XXZQY6QsU319Atl0" alt=""><figcaption><p>tactile sensors blocks</p></figcaption></figure>

Misty has 4 bump sensor that you can use to trigger events. You can either choose to trigger a sequence  by touching any of Misty's bumps or select one of them in the drop-down. The names of the bump sensors are: front left, front right, back left and back right.

The second block is the touch sensor event and you can also choose to trigger any sensor or select a specific sensor. There are six possibilities: head front, head back, head left, head right, chin and scruff.&#x20;

A triggered bump or touch sensor will create an event message that you can use in other events.

### Vision events

<figure><img src="/files/XAjgHQujyWi8ACUisKNl" alt=""><figcaption><p>vision events blocks</p></figcaption></figure>

With these two blocks, you can build events related to Misty's face and object recognition. You can use these blocks to build a skill that recognizes different objects or people. In the text field you can insert the name of the object or person you are looking for or leave it empty and don't limit the robot.&#x20;

The event message in the first block is the name of the object recognized, in the second block is the name of the face recognized.

### Audio events

<figure><img src="/files/b8MjAKxFXDpPjbHP0vpT" alt=""><figcaption><p>audio events blocks</p></figcaption></figure>

With these two blocks, you can build skills using Misty's audio and speech capabilties. With the first block, the event waits for the audio to complete, meanwhile, the second block is used to trigger an event when a keyphrase is recognized.

The event message in the first block is a boolean message, true if the audio is completed, false if not. In the second block the event message is the keyphrase that Misty has recognized.&#x20;

### Driving events

<figure><img src="/files/Os8zagx16sJ0SFZZ03Wt" alt=""><figcaption><p>driving events blocks</p></figcaption></figure>

You can use these two blocks to control Misty's position in a particular space.&#x20;

The first block waits for a drive encoders event. You can select several parameters to check:

left distance \[mm], right distance \[mm], left speed \[mm/s], right speed \[mm/s].&#x20;

You can also set a < or a > condition to be respected and of course, you need to insert the value in millimeters for the condition.&#x20;

In this case, your output will be true or false.&#x20;

The second block checks and regulates the time of flight sensors. Even in this case, there are several parameters like: Front right distance \[m], Front center distance \[m], Front left distance \[m], Back distance \[m], Front right downward \[m], Front left downward \[m], Back right downward \[m] and Back left downward \[m].&#x20;

Please note that this time the values are expressed in meters and not in millimetres. As it was for the previous block you can set a < or a > condition to be respected and of course, you need to insert the value in metres for the condition. Also in this case your output will be true or false.&#x20;

### Hazard event

<figure><img src="/files/44qWqHOTAnldyYn8MgLV" alt=""><figcaption><p>hazard event block</p></figcaption></figure>

You can use this block to run a sequence Misty encounters a "hazard". A hazard can be a bump sensor o motor malfunction, critical internal error, cliff edge sensors being triggered, excessive speed, among others.

If you want to find out more about possible hazard you go to Misty's developer documenation here: <https://docs.mistyrobotics.com/misty-ii/robot/sensor-data/#hazardnotification>


# Miscellaneous

Miscellaneous means "of mixed or varied character."

In this folder, you will find various useful blocks such as the timer block, chest LED blocks and run until stopped block.

<figure><img src="/files/a82wBdQpk2JG5Heo8RS4" alt=""><figcaption><p>miscellaneous blocks</p></figcaption></figure>

### Timer block

<figure><img src="/files/SwMeIE5Kw0m2eja0iJ4j" alt=""><figcaption><p>wait block</p></figcaption></figure>

If you are new to programming, you will discover that timers are a fundamental component when working with sequences. After you press run, machines typically don't wait to execute all parts of your code unless you tell them to. So if you want to have a smooth and controlled flow of your program, you will need to use the timer block. This block is set in milliseconds, 1000 milliseconds are equal to 1 second.&#x20;

### Chest LED

<figure><img src="/files/7O5Jlc8AHknSk29MFj82" alt=""><figcaption><p>LEDs blocks</p></figcaption></figure>

If you need clear feedback from Misty about the execution of your program, you can rely on the Chest LED. It acts as visual cue or signal that tells you if you've successfully triggered an event. There is a big palette of colors that you can choose from.

<figure><img src="/files/ijXlM4YBvi9chJwgIpqf" alt=""><figcaption><p>colours palette</p></figcaption></figure>

The transition led block allows you to cycle between two different colors. There are a couple of options in the drop-down:&#x20;

* Breath: does the transition gradually with other colors. For example, if you want to go from red to green it will display red, orange, light orange, yellow and green and backwards in the time you set.&#x20;
* Blink does the transition straight between one color and the other.&#x20;

### Log message

<figure><img src="/files/x5zHyNr1YwYxGuYVPjN5" alt=""><figcaption><p>log message block</p></figcaption></figure>

This block allows you to display a message on the console. If you are debugging your code or want to read a specific message on your console this is the block you need to use.&#x20;

### Run until stopped

<figure><img src="/files/OAdZxyFfp9mJ59AFPJN0" alt=""><figcaption><p>run until stopped block</p></figcaption></figure>

When you are working with events you shouldn't forget about this block.&#x20;

This block will continuously repeat your program in a loop until you press the stop button. It's fundamental in an event code because Misty needs to constantly monitor her environment and sensors to detect user input. If you don't put this block at the end of your code Misty will run it only once and even if your program is well done it won't really work.&#x20;


# NLP

Natural Language Processing (NLP) is a field of computer science that deals with the interaction between computers and human language. It's a subfield of artificial intelligence that enables computers to understand and generate human language.

NLP is used in robotics to allow robots to interact with humans in a natural way. This can be done through speech recognition, natural language understanding, and natural language generation.

* Speech recognition allows robots to understand human speech. This can be used for tasks such as following voice commands or providing information to users.
* Natural language understanding allows robots to understand the meaning of human language. This can be used for tasks such as answering questions or following instructions.
* Natural language generation allows robots to generate human language. This can be used for tasks such as providing information to users or engaging in conversations.

NLP is a powerful tool that can be used to make robots more intelligent and interactive.

<figure><img src="/files/CW4fQFeSXdrB0WgsnwVD" alt=""><figcaption><p>NLP blocks</p></figcaption></figure>

<figure><img src="/files/caFsdXAhCEIxbmvUjc5e" alt=""><figcaption><p>NLP blocks</p></figcaption></figure>

### Context creator

<figure><img src="/files/ycG13pVcErsqTyBrzYFI" alt=""><figcaption><p>context creator</p></figcaption></figure>

Before you start a conversation with Misty you need to define or create a context and give it a name. Using the context block you can add intents. You can consider intents as the main topics of your conversation. To add more intents you can use the gear icon.&#x20;

The next stage is to give your intents a name using the name-sample block. Once you've decided the name you will need to write in keywords that are associated with that name so Misty can recognize the intent. So for example if you are creating an intent called 'humans', you will need to write in words that let Misty know how to identify it: round head; two eyes; two legs; language; pink skin, etc. &#x20;

The context creation has to run before the conversation is designed. This is due to the fact that the contexts are a dropdown in the Blockly interface. Basically, you must run the context script, then refresh the page and the context will exist. \
\
*Note: You shouldn't re-use context names as it can create a conflict, Misty won't know which one to use. However you are free to re-use intent names and keywords as these are contained to the context.*

### Conversation-State Block

<figure><img src="/files/QqKEICUPPMm6Z79wVMUc" alt=""><figcaption><p>container conversation block</p></figcaption></figure>

This block contains the conversation that you will have with Misty. Each conversation has its own name and you can't use the same name in two conversations because Misty won't know which one is the correct one.

In the first text field you can write the name of your conversation. In the second text field, you can write which state block you want to run first. Typically the first state is a question or a greeting to get the conversation started. &#x20;

### State block

<figure><img src="/files/hBwgiXwBWvkc5FzCSJwr" alt=""><figcaption><p>NLP block</p></figcaption></figure>

This is a fundamental block in NLP conversations. In the name field you can write the name of your state. You can see the state as the name you associate with that conversation. In the speak field, you can write what you want Misty to say when that state is triggered. In the context drop down you need to select the context of your conversation. By default, you will find the furhat.context.en.yes-no. In the action drop-own, you can select your favorite Misty action, you can find[^1] them in the Wizard tab. If you are in a noisy room, it can happen that Misty doesn't recognize what you said, in this case you will need  write what she should say in the no match field. You can also define how many times she will repeat the text using the no match count field.

### State transition block

<figure><img src="/files/CsR2uHTmc7h6mxVQCQ3f" alt=""><figcaption><p>transition block</p></figcaption></figure>

If you are building a conversation with Misty you want her to go from one state to the other depending on the context that you are using. In the state field, you need to write the name of the state (for example: start). In the trigger field you need to write the intent that trigger a transition to the next state.

### Start conversation

<figure><img src="/files/yj0dmOFWyx7euWk5oCH8" alt=""><figcaption><p>start conversation block</p></figcaption></figure>

If there are several conversations in your program you may want to choose which conversation to start. You can use the start conversation block and input the name of the conversation that you written in the Conversation-State block.&#x20;

*Note:  Just like with event blocks, when you are using NLP blocks you need to add a run until stopped block at the end of your program.*

### Create action

<figure><img src="/files/UOa0AzYXxghfh0MNV4O9" alt=""><figcaption><p>action block</p></figcaption></figure>

With this block, you can name and create your own action for the State block. As you can notice it has the same structure as [Events](/blockly-elements/misty-blocks/events) blocks. In it, you can put all the movements or the blocks you want, including other NLP blocks and all what your imagination suggests to you. After you run it it will show up in your action drop-down in the State block.

### Intent&#x20;

<figure><img src="/files/tN5z71YSjZ2eSDSrkW0B" alt=""><figcaption><p>intent block</p></figcaption></figure>

With this block, you can select a specific intent from one of the contexts and insert it into speech blocks that utilize speech recognition.

&#x20;

### Speak and listen context

<figure><img src="/files/eRwZv7RYZXEyeqiiuoYH" alt=""><figcaption><p>speak and listen context</p></figcaption></figure>

With this block, you can program Misty to speak and listen to specific contexts, both default and customizable ones.&#x20;

[^1]: insert action files as well


# System

You could use these blocks to make cycles or set conditions. For example, if the battery level is lower than 20% you can use this block in a sequence to make Misty know she needs to perform a particular action. In Misty II Pro for example you can use this to make her go back to her charging station.

<figure><img src="/files/Ba8xQJ64jn4RaMeJw3I2" alt=""><figcaption><p>system blocks</p></figcaption></figure>

### Misty properties

<figure><img src="/files/vhiBvUHDym5WZjNSREoS" alt=""><figcaption><p>properties block </p></figcaption></figure>

With this block, you can get different information about your specific robot. By clicking on the drop-down you will see there are several options to choose from :&#x20;

Android Hardware, Android OS Version, Current Network ID, Current Pre Shared Key, Current Profile Name, IP Address, Mac Address, Network Connectivity, Robot ID, Robot Version, Windows OS Version, Sensory Service App Version, Serial Number, SKU, RTC board ID, RTC board firmware, RTC board hardware, mcBoard board id, mcBoard firmware, mcBoard hardware. &#x20;

### Battery properties

<figure><img src="/files/ZGZhzi44Bfae60EqK73I" alt=""><figcaption><p>battery properties block</p></figcaption></figure>

With this block, you can get different information about the battery of your robot. By clicking on the drop-down you can find the following options:&#x20;

Charge Percent, Current, Health Percent, Is Charging, Trained, Voltage.&#x20;

### Hazard settings

<figure><img src="/files/OO7VMGpC5evYD1uW17y0" alt=""><figcaption><p>hazard setting block</p></figcaption></figure>

With this block, you can update and reset the settings regarding the bump sensors and the time of flight sensors.&#x20;


# Basic Blocks

In this page you will be able to have an overview of the principal blockly blocks

Basic blocks are essential for writing any complex program, as they allow you to add and nest additional values, as well as create specific conditions for your program. As you progress throughout your lessons you will find that you will need more and more of these blocks.&#x20;

### Contents

{% content-ref url="/pages/XBBuiUi8r3fIC4g5QohY" %}
[Logic](/blockly-elements/basic-blocks/logic)
{% endcontent-ref %}

{% content-ref url="/pages/KDtPBFHKK7rEVZI1r5Ds" %}
[Loops](/blockly-elements/basic-blocks/loops)
{% endcontent-ref %}

{% content-ref url="/pages/5dNVWOuCQ3LEdk59hXgw" %}
[Math](/blockly-elements/basic-blocks/math)
{% endcontent-ref %}

{% content-ref url="/pages/zwDKCmkCu7drKxU3xXms" %}
[Text](/blockly-elements/basic-blocks/text)
{% endcontent-ref %}

{% content-ref url="/pages/VkB1Umqo5MLcOAj9ljQf" %}
[Lists](/blockly-elements/basic-blocks/lists)
{% endcontent-ref %}


# Logic

The Logic folder in Blockly contains blocks that are used to control the flow of execution of a program.These blocks include conditional statements (such as if and else), loops (such as while and for), and logical operators (such as and, or, and not).

Logic blocks are essential for writing any complex program, as they allow you to control how the program behaves based on different conditions. For example, you can use a conditional statement to check if a variable is equal to a certain value, and then execute different code depending on the result. Or, you can use a loop to repeat a block of code multiple times until a certain condition is met.

<figure><img src="/files/67SDapcWGtgPUPtcnw3S" alt=""><figcaption><p>Logic blocks</p></figcaption></figure>

### If, else, else if

<figure><img src="/files/WOo09rBT4UXWVAnRc1Hd" alt=""><figcaption><p>If, else, else if block</p></figcaption></figure>

The `if`, `else`, and `else if` statements are conditional statements in programming languages. They are used to control the flow of a program by executing different blocks of code depending on the value of a condition.

The `if` statement checks if a condition is true. If the condition is true, the code inside the `if` block is executed. If the condition is false, the code inside the `else` block (if it exists) is executed.

The `else if` statement is similar to the `if` statement, but it allows you to check multiple conditions. If the first condition is false, the second condition is checked. If the second condition is also false, the third condition is checked, and so on. If all of the conditions are false, the code inside the `else` block (if it exists) is executed.

In Misty Studio you can build your own if-else function by clicking on the gear and dragging else if and else blocks under the if one.&#x20;

In this case, Misty will check if the condition in the "if-gap" (red) is true. If it is, she will do the action in the "do-gap" (green), if not she will do what's inside the "else-gap" (yellow).

### Value comparison

<figure><img src="/files/dQuWD0tJwMnes3vAet2Y" alt=""><figcaption><p>value comparison block </p></figcaption></figure>

This block allows you to compare two values. If the condition is respected the block will return *true*, otherwise it will return *false*. These are useful in multiple event scenarios when you want Misty to perform different actions based on the type of event triggered.

For example, this block could be used as a condition in the if-else block seen previously. You can select his condition by clicking on it.

### Case Comparison

<figure><img src="/files/ARX2OBMAyzSTChjmVnOg" alt=""><figcaption><p>case comparison block</p></figcaption></figure>

This block allows you to compare two inputs and give an output.

*And-case*: This block will return *true* if both inputs are true.

*Or-case*: This block will return *true* if at least one of the inputs is true.

### Not&#x20;

<figure><img src="/files/bDOIjdq9y2ZZTZBGB3KB" alt=""><figcaption><p>not block</p></figcaption></figure>

With this block, you can reverse the value of the input. If the input is true it returns *false,* if the input is false it returns *true.*

### True-False

<figure><img src="/files/qGU5ooUIs69JFeOvN21I" alt=""><figcaption><p>true and false block</p></figcaption></figure>

This block returns either *true* or *false.*

In general, returning true or false is a good way to indicate the success or failure of a function, or to control the flow of a program.

### Null

<figure><img src="/files/oh9XeUziEDJn0QEwaDWE" alt=""><figcaption><p>null block</p></figcaption></figure>

The null block is used to represent the absence of a value. It can be used in a variety of situations, such as:

* When you need to initialize a variable.
* When you need to return a value from a function, but the function does not need to return a specific value.
* When you need to pass a value to a function, but the function does not expect a specific value.

In Misty Studio this block simply returns *null.*

### Test&#x20;

<figure><img src="/files/xxFIhFb7FvJ6p0T08fXI" alt=""><figcaption><p>test block</p></figcaption></figure>

This block checks the condition in the "test-gap" (yellow), if it is true it returns the if-true value (green), if it is false it returns the if-false value (red).&#x20;


# Loops

The Loop folder in Blockly contains blocks that are used to repeat a block of code multiple times. This can be useful for tasks such as iterating over a list, performing a calculation multiple times, or waiting for a certain amount of time.

To use a loop function, simply connect the block of code that you want to repeat to the loop function. Then, specify the condition for the loop function, or the number of times that you want the loop function to repeat.

<figure><img src="/files/qPDhC1SoXBawKdAiiOlE" alt=""><figcaption><p>loops blocks</p></figcaption></figure>

### Repeat n-times

<figure><img src="/files/20nvNM8F727rsgqXXdTj" alt=""><figcaption><p>repeat n-times block</p></figcaption></figure>

This block allows the user to repeat some statements a certain amount of times. You can intuitively choose the number of times and the actions of the robot.&#x20;

### Repeat (true-false)

<figure><img src="/files/YVOgl8dVwFHxax7MLnDk" alt=""><figcaption><p>repeat block</p></figcaption></figure>

This block has two options: while and until.&#x20;

If it is set to "while", the robot will repeat the statement set in the "do-gap" while the condition set is *true.*&#x20;

If it is set to "until", the robot will repeat the statement set in the "do-gap" while the condition set is *false.*&#x20;

### Customized repeat&#x20;

<figure><img src="/files/g0gOKzsSAbB5LqHl7oXZ" alt=""><figcaption><p>customized repeat block</p></figcaption></figure>

This block allows you to customize your own "repeat". In fact, you can choose the variable to use. Take it on the values from the start number to the end number, counting in specified intervals.

When the variable will have the same value as the end value the block will give a false output and will stop the execution, it means that while the condition is verified Misty will keep doing her actions.&#x20;

### Specific function for each term of a list&#x20;

<figure><img src="/files/VuAFQFjUKGMUs1SD0hMd" alt=""><figcaption><p>specific function for each term of a list block</p></figcaption></figure>

Lists in Blockly are ordered collections of items. They can be used to store any type of data, including numbers, strings, and even other lists. We'll explore this element of the programming on the dedicated page.&#x20;

This block will do a specific action for each term of the list. In the "list-gap" you can insert the name of the list and under, in the "do-gap", the actions.&#x20;

### Break-out

<figure><img src="/files/QxGjN7BorJRBBFZtmBW8" alt=""><figcaption><p>break out block</p></figcaption></figure>

This block can only be used inside a loop.&#x20;

You can use the break-out of the containing loop when the user wants to stop the loop early. This can be for a variety of reasons, such as:

* You have found what you were looking for.
* The loop has reached a certain condition.

Otherwise, you can select "continue with the next iteration". In this case, you will skip the rest of the loop and the program will continue with the next iteration.&#x20;

This can be for a variety of reasons, such as:

* You have already done what you needed to do in the current iteration.
* The current iteration does not meet the user criteria.


# Math

The Math folder in Blockly is supposed to be useful for performing mathematical operations. It contains blocks for basic arithmetic operations (+, -, \*, /), as well as more advanced operations such as trigonometry, exponents, and logarithms.

The Math folder also contains blocks for working with variables, functions, and conditional statements. This allows you to write complex mathematical expressions in Blockly.

The description of this page will be particularly summarized because it will contain mostly mathematical elements and numbers to use in the other functions described.

![](/files/pgty70wf9hngyR9h8fFb) ![](/files/iuyIVy3Ymi5RVc741s5N)

### Numbers

<figure><img src="/files/uKJoFJsTnLU9WDw6CDoA" alt=""><figcaption><p>type of numbers in Blockly</p></figcaption></figure>

The first block identifies a general number. It is also the most used one to program Misty because when you need to specify an action, like how many meters to drive, you will need to use this block to give the number.&#x20;

The second block (*π),* returns the common constants: *π* (3.141...), e (2.718...), Φ (1.618...)&#x20;

sqrt(2) (1.414...), or ∞ (infinity).&#x20;

The third block (sin 45), returns the trigonometry functions (sin, cos, tan, asin, acos, atan) in degrees, not radians.&#x20;

The difference between the black and the blue box for a number is that the blue one is bound to that expression, instead the black one is not.&#x20;

The fourth block (square root 9), returns some common mathematical functions (square root, absolute, negative, ln, log10, e^, 10^).&#x20;

### Operations

<figure><img src="/files/HLotHgvchUJKxJjAzID2" alt=""><figcaption><p>operations block</p></figcaption></figure>

With this block, you can do several operations like +,-,\*,/,^ between two numbers.&#x20;

### Check

<figure><img src="/files/krcoB5ENcAFcgQwHM1Ub" alt=""><figcaption><p>check block</p></figcaption></figure>

This block checks if a number is even, odd, prime, whole, positive, negative, or divisible by (it depends on what the user clicks) and returns *true* or *false.*

### Other operations&#x20;

<figure><img src="/files/pt0IGMJCqWVifEcaauD4" alt=""><figcaption><p>other operations blocks</p></figcaption></figure>

It is really intuitive to understand what these blocks do in Blockly:&#x20;

* The first block rounds up or down the number;
* The second block constrains a number between a chosen range;
* The third block returns the remainder of a division;
* The fourth block generates a random fraction;
* The fifth block generates a random integer in a range and is often used to create random sequences of lights with the misty RGB chest LED.

### Change variable&#x20;

<figure><img src="/files/UVMyE6KCshzPDUoTxuWf" alt=""><figcaption><p>change variable block</p></figcaption></figure>

With this block, you can select a variable in your code and change it according to the number written in the blue box.&#x20;

### Lists operations&#x20;

<figure><img src="/files/XJ4e3t66ichCAPynoBps" alt=""><figcaption><p>lists operations blocks</p></figcaption></figure>

This block is associated with a list of items that can return a sum, a minimum, a maximum, average, median, modes, standard deviation and a random item. So all the common statistic operations that you can do. This can be helpful in developing a program for Misty where you can extract a random name from a list or so.&#x20;


# Text

The Texts folder in Blockly is useful for manipulating text and contains Blocks that you will need to enable Misty's Text-To-Speech.  These blocks can be used for basic text operations, such as concatenation, splitting, and searching. It also contains blocks for more advanced text operations, such as regular expression matching and text formatting.

The Texts folder also contains blocks for working with variables, functions, and conditional statements. This allows you to write complex text manipulation expressions in Blockly.

<figure><img src="/files/YlFaJtYEt79yKYPAvkF8" alt=""><figcaption><p>text blocks</p></figcaption></figure>

<figure><img src="/files/Ppk04tMH9HW2aVJkXyXZ" alt=""><figcaption><p>text blocks</p></figcaption></figure>

### Text

<figure><img src="/files/UiMcLVpn3trgG9zlJ6r8" alt=""><figcaption><p>text block</p></figcaption></figure>

This is a simple text block. By clicking on the text field you can start writing numbers, letters or strings. This is often used together with Misty's speech blocks.&#x20;

### Build text

<figure><img src="/files/lbKObno4nhsUPA9lfaNh" alt=""><figcaption><p>build text block</p></figcaption></figure>

With this block, you can create your own text blocks. Its different from the previous one because here you can combine variables and normal text. It means it can be used in functions and in more advanced applications for Misty.

By clicking on the gear icon you can join multiple items and build the text as long as you like.

### Append text

<figure><img src="/files/k8j3lb1UpxFNdb4e4Wrd" alt=""><figcaption><p>append text in Misty Studio</p></figcaption></figure>

With this block, you can add some text to a specified item, as well as other text blocks.

### Other text block editors

<figure><img src="/files/1UsYsn7erOXp0zHhPAcZ" alt=""><figcaption><p>other text editor blocks</p></figcaption></figure>

These blocks provide a variety of functionalities and return different values:&#x20;

* The first block returns the length of the string.
* The second block prints the string contained.
* The third block returns true if the block is empty.
* The fourth block changes everything in the blue box to UPPER CASE or lower case or Title Case.
* The fifth block trims some space next to the block (both sides, right side or left side only). It can also be used for example next to variables in the build text block in order to create well-readable text.&#x20;

### Prompt the user for some text&#x20;

<figure><img src="/files/ovx1as3W5g98wiiuBb0V" alt=""><figcaption><p>prompt for user for some text block</p></figcaption></figure>

The "Prompt for Text" block in Blockly is a feature that allows a program to request and receive text input from a user. When this block is used within a Blockly program and executed, it triggers an interface or dialogue that asks you to input text.

The purpose of this functionality is to have the robot give requests for input to the user.

### Strings operators

<figure><img src="/files/bZKmAFjnBeAIbudjKsdk" alt=""><figcaption><p>strings operators blocks</p></figcaption></figure>

These three blocks in Blockly provide operations with strings :&#x20;

* The first block returns the letter in a specified position, 0# is the first letter.
* The second block returns a specified portion of the text.
* The third block returns the index of the first, or last, occurrence of the first text in the second text. It returns -1 if the variable text is not found.

*Note: Numbers written in the math blocks are "read" with their value, instead numbers written in the text strings are "read" as part of the language, in fact, in text strings type "five" gives exactly the same result as typing "5". This doesn't happen in the math blocks.*&#x20;


# Lists

The Lists folder in Blockly contains blocks that can be used to create and manipulate lists. Lists are ordered collections of items. They can be used to store any type of data, including numbers, strings, and even other lists.

Lists are useful in programming because they allow you to store and organize collections of data in a single place and perform operations on collections of data in a more efficient way than if you were to perform the operations on each item individually.&#x20;

<figure><img src="/files/bXZOqcH9JcjwZEFUHh0j" alt=""><figcaption><p>lists blocks</p></figcaption></figure>

### Create a list

<figure><img src="/files/qhi0FsxGKGmxdAX0WUVQ" alt=""><figcaption><p>create a list block</p></figcaption></figure>

With this block, you can create a list. By clicking on the gear icon you can modify the number of items and add as many as you like.&#x20;

### Operations with lists&#x20;

<figure><img src="/files/JzmnnLJ6uX6A88H4klzU" alt=""><figcaption><p>operations wth lists blocks</p></figcaption></figure>

These blocks are useful for operations with lists:&#x20;

* The first block creates a list with a specified value repeated in a chosen number of times.&#x20;
* The second block returns the length of the list associated with it.&#x20;
* The third block has two options :&#x20;

1. *List from text* splits the text into a list of texts, breaking at each delimiter chosen by the programmer.&#x20;
2. *Text from list* joins a list of text into one text, separated by a delimiter chosen by the programmer.&#x20;

* The fourth block returns true if the list is empty, otherwise, it returns false.&#x20;
* The last block can be used as a condition to execute or not a loop.&#x20;

### Sorting a list&#x20;

<figure><img src="/files/q5e3DH8XTyekjhcjr1lJ" alt=""><figcaption><p>sort a list block</p></figcaption></figure>

This is another operation with lists. If lists are collections of items they need to be ordered and this block can sort a copy of a list depending on a numeric or alphabetic order. The list can be sorted in ascending or descending ways.&#x20;

### Other operations with lists

<figure><img src="/files/RzQ89I7HyqEpXWb8rBiT" alt=""><figcaption><p>other operations with lists blocks</p></figcaption></figure>

These are other operations you can find interesting while working on lists:&#x20;

* The first block returns the index of the first or the last occurrence of the item in the list. It returns -1 if the item is not found.&#x20;
* The second block can return, remove or return and remove the item at the specified position.&#x20;
* Instead, the third block can set or insert the item at the specified position. There are several options in order to choose the position, and they are related to the second option:&#x20;

1. \#identify the position of a term. #0 is the first term.&#x20;
2. First is the first term in a list.
3. Last is the last term in a list.
4. Random is a random term in a list.


# Advanced Blocks

The Advanced blocks consists of two primary types of blocks: Variable Blocks and Function Blocks. These will be fundamental blocks to create more advanced and complex programs that require you to store and manage data throughout your program , as well as creating group related code together and execute it as a single unit. These blocks can help you make your code more modular, compact and easier to manage.

### Contents

{% content-ref url="/pages/gQzx2Wvgwqci1j1yjmTE" %}
[Variables](/blockly-elements/advanced-blocks/variables)
{% endcontent-ref %}

{% content-ref url="/pages/p8lLKjobzqox5gSgNtXS" %}
[Functions](/blockly-elements/advanced-blocks/functions)
{% endcontent-ref %}


# Variables

The Variables folder contains the blocks need to create variables. Variables are a fundamental concept in programming that allow you to store and manage data throughout your program. They are like containers that hold values, and you can refer to them by their names to access or modify the values they contain.

They are useful for many reasons, like storing data, sharing data between blocks, reusing values, making the code more readable and improving the code efficiency.

Once you click on the variables folder you will be able just to see this screen (See image below). In order to create a variable you need to click on the "Create variable..." block and a window will pop up on your screen. You need to write the name of your variable, in this case "x", and then press OK. After this sequence, the variables folder will add several variable blocks with the value "x" that you can use in your program.  If you want to add more variables the folder will put them in the order you create them, you will also be able to choose which variable you need in the drop-down.

<div align="center"><figure><img src="/files/JQuaP98RbYSXY4zfOy2K" alt=""><figcaption><p>image 1</p></figcaption></figure></div>

<figure><img src="/files/Io2vzpI8RQ8M5IGqTG02" alt=""><figcaption><p>image 2</p></figcaption></figure>

<figure><img src="/files/jxlbMzg9Fjrc0DikDR4O" alt=""><figcaption><p>image 3</p></figcaption></figure>

<figure><img src="/files/P4OIR5FvezQXEFKdNnxK" alt=""><figcaption><p>image 4</p></figcaption></figure>

We will show the way to delete a variable in the description of the different blocks.&#x20;

### Variable

<figure><img src="/files/GAlJVwhKyUzJTyRhWTZJ" alt=""><figcaption><p>variable block</p></figcaption></figure>

The variable block is used when you need to use your variable in your program. You can connect it wherever you can find a block with a socket. The variable block will represents a value or a text string.&#x20;

When you click on its options you can easily change which variable you are referring to, you can rename it or delete the variable itself.&#x20;

### Set variable&#x20;

<figure><img src="/files/E7PvD5I0RiNeuREkI9Gd" alt=""><figcaption><p>set variable block</p></figcaption></figure>

This is a fundamental block when you work with variables. You can use this block to assign a value, a text or an event message to a variable. By clicking on the drop-down you will find exactly the same options as in the variable block.

### Change variable

<figure><img src="/files/y6z5aRMYVMLPmTRnS1p4" alt=""><figcaption><p>change variable block</p></figcaption></figure>

With this block, you can change the value of the variable by adding the number contained in the blue block. By clicking on the arrow you will find exactly the same options as in the variable block.


# Functions

The functions folder contains all the blocks you need to create functions. Functions are a fundamental concept in programming that allow you to group blocks and code together to execute it as a single unit. They are like reusable blocks of code that can be called from different parts of your program, making your code more modular and easier to manage.

Functions are a powerful tool for writing structured, maintainable, and reusable code in Blockly. By mastering the use of functions, you can significantly improve the quality and efficiency of your programs.&#x20;

<figure><img src="/files/YvPwp2PS1rrj7Z6Pniui" alt=""><figcaption><p>function blocks</p></figcaption></figure>

### Function without output

<figure><img src="/files/xiel9YmtRuTvvkPnvsv7" alt=""><figcaption><p>function without output block</p></figcaption></figure>

This block represents a function without an input. Its built out of different parts:&#x20;

* In the image, the "do something" is the name of the functions that will be recalled in the code. It's important to use different names for the functions.
* By clicking on the question mark you can add a description of the function, this can be useful when you need to debug your code or explain to other programmers what your code is doing in that part.&#x20;
* By clicking on the gear you can choose which input to insert into your function.&#x20;
* The part inside the block, between the connect and the socket will contain all the sequences  that the function will need to perform.

### Function with outputs

<figure><img src="/files/5uA48G92wd01aIaxcRJ2" alt=""><figcaption><p>function with outputs block</p></figcaption></figure>

This blocks is virtually identical to the previous block described but it returns an output that you've chosen. Instead, when statements are not allowed this function "closes" the gap and when it's called it will return what you add to the return socket.

### Check condition

<figure><img src="/files/rBdMWDEoJxPf5YLIGJhp" alt=""><figcaption><p>check condition block</p></figcaption></figure>

Note: This block can only be used in a function.&#x20;

With this block, you can check a condition. If the value put in the 'if' socket is true then it will return the value in the 'return' socket. If the value is not true the function will end the program.

### Call function&#x20;

<figure><img src="/files/p0YbadX62ywJZWCo62Jm" alt=""><figcaption><p>call function block</p></figcaption></figure>

In order to call a function you must create a do something block and give it the same name as the function you want to call. &#x20;


# Python Lessons

## Learning with Python

In recent years Python has become one of the most popular programming languages for education in computer science and robotics. Its simplicity, versatility, extensive libraries and robust community support makes it an ideal choice for both robotics beginners and experienced programmers who are looking to elevate their knowledge about Human-Robot Interaction. Throughout Misty Python Lessons you will learn the basics of syntax coding in Python, as well as explore how to utilize Misty's capabilities to create fun and real-life applications for your projects.&#x20;

## Misty's Python Interface

Misty's Python Interface is designed to directly use Misty’s hosted API endpoints, acting as an API client for Misty. This API client is generated by calling Misty's API and producing Python code as a result. Misty currently runs Python 3.9, built for a 32 bit ARM architecture running on Windows. Misty also hosts a Python Language Server, which gives her the ability to analyze your Python scripts, providing code completion, highlighting, and syntax validation. Misty even has a Python editor that you can use to create and debug your code!

Misty's built-in Python interface uses the Python runtime, Python Language Server, and version of the Monaco editor to allow for simple Python development within Misty Studio.

To start using Python on Misty, open Misty Studio and navigate to Programming, then Python. You should see Misty's Python editor.

<figure><img src="https://lh7-us.googleusercontent.com/lrngPDbuo3A2n4Sm_bSTWCd-gnD4zLlEtJ28FISq0XzJXqPL3roAMY75REFppjUttnG4xmVkE7NwqM-MdYAMuD4ahKH-16lCSVGcd60hMryQR4KV-N4B9sbo-FxPPGIYGB0NJFbAMnoQgT05Wn2fe4o" alt=""><figcaption></figcaption></figure>

The buttons at the top of the page give you the tools necessary to start programming Misty. Here's what each of them does:

* The 'New' button clears the editor window of all text
* The 'Open' button lets you select an existing Python script to run on Misty
* The 'Save' button lets you save your Python code to your computer
* The templates dropdown give you some code to start with, which makes creating your first programs easier
* The 'Load Template' button loads the contents of the selected template into the editor. Be careful, it'll replace anything currently in the editor!
* The 'Run' button runs your script on Misty
* The 'Stop' button stops a currently running script

At the bottom of the editor, you'll see three boxes labelled 'Messages', 'Output', and 'Errors'.<br>

<figure><img src="https://lh7-us.googleusercontent.com/uNjR2c83rxMy2LJzLRWlIA2esfs8IZpsibbnW3mpovK9lnEA0YRKBK2lgrtazJjhWou522-6cK0B-hdv5JGDBa2YRwaGGETBOWpKlbvb0ob5A8clRx_aRoPzspaPrLoHkQ8uzTT5jw4JtHIhMKSGX8c" alt="" width="563"><figcaption></figcaption></figure>

Because Python scripts are run using the Python runtime on Misty, some of the outputs are what you'd expect to see from any shell script. The 'Output' box captures anything that your script outputs when using the Python 'print' function. These statements are reported after the script has completed execution. Similarly, the 'Errors' box captures any unhandled error produced by your script, which you'll be presented with after the completion of the script. Lastly, the 'Messages' box receives debug messages sent from your code in real time.

## Building Python Skills&#x20;

Every Python program on Misty starts with importing the Misty Python library and the robot as shown in the example below. When you start building a new program these lines are already in place.

```python
from mistyPy.Robot import Robot
misty = Robot()
# Your code here
```

To start programming Misty in Python and building skills you will  also need to familiarize yourself with her API endpoints in the API Explorer under the Programming tab in Misty Studio and [Misty Python API](/python-elements/misty-python-api).There are approximately 150 endpoints that you can discover.

## Calling Misty’s API endpoints in Python

Once you've imported the libraries and the robot you will be able to call your APIs. To call Misty’s API endpoints in Python and have her execute a certain command you will need to start your code with ‘misty’ followed by a dot. After the dot you can write in the name of the API endpoint you want to call, for example if you want to call the head or arm movement APIs, you will need to use the following syntax to select a list of actions when typing an API call.&#x20;

<figure><img src="/files/eujhOr7b3Ii652ISWd4y" alt="" width="471"><figcaption></figcaption></figure>

Once you've determines the APIs you want to use, you can place them after misty=Robot().

```python
from mistyPy.Robot import Robot
misty = Robot()
# Your code here
misty.move_head()
```

Since Misty’s Python uses the same API endpoints as Misty’s Blockly, you can use the same values in the parameters, however they are not as easy to identify. To check the name and order in which your values need to be set in Python, you can add a comma in the brackets as shown in the example below.<br>

<figure><img src="/files/6a7XlU4TVK61xOEFUlTQ" alt="" width="544"><figcaption></figcaption></figure>

There are different ways you can write your values in the parameters. For example you can follow the parameter structure in the brackets or just write the values in the corresponding order.

The programming term to refer to the name of the box that contains the value is “label”. When you start writing the parameters in Misty’s Python interface Misty will give you suggestions and you can select your label by clicking on the tab.&#x20;

This is really useful when your commands can contain a lot of parameters and you need to name only some of them while leaving the rest as default.

<figure><img src="/files/PhLjb3Dk9sdlY4PN99tu" alt="" width="482"><figcaption></figcaption></figure>

Of course if you are naming them manually you can choose which ones to insert and which ones to omit.

```python
from mistyPy.Robot import Robot
misty = Robot()
# Your code here
misty.move_head(pitch= -40, roll= 0, yaw= 0,velocity=80)
```

Or

```python
from mistyPy.Robot import Robot
misty = Robot()
misty.move_head(-40, 0, 0, 80)
```

## Start Lessons

{% content-ref url="/pages/jMmR6zhWxyKwJaq3pm2S" %}
[Lesson 1: Loco-motion](/python/python-lessons/lesson-1-loco-motion)
{% endcontent-ref %}


# Lesson 1: Loco-motion

## Aim

Welcome to your first lesson in Misty Python! The aim of this lesson is to introduce you to some of the basics of building interactions with Misty using Python, starting with movement and locomotion. If you have any questions about the individual APIs to trigger locomotion and movement, you can check out the [Motion and Mobility](/python-elements/misty-python-api/motion-and-mobility) section in Python Elements.

## Motion Sequences

### Challenge 1: Hello again human!

If you've completed the Blockly Lessons you might remember that Misty started her first lesson by teaching you how to make her greet humans. In this challenge you can rebuild the Movement blocks using Python and modify Misty's head and arm positions. Let's dive into Misty Python!

Below are three lines of API calls that you can use, the API `misty.move_head` on the first line controls Misty's head movement. Like the Head movement block in Blockly, the parameters are pitch, roll and yaw. If you don't recall the range of values for this API you can check them out in [Motion and Mobility](/python-elements/misty-python-api/motion-and-mobility).

The second line is an arm movement API called `misty.move_arm`, in the parameters you need to specify which arm Misty has to move and in to which position, the first value is a string and can be "right", "left" or "both" and the second is the value to define the degree of movement.

The third line is another arm movement line with the API `misty.move_arms`, which allows you to control two arms simultaneously and set them to different degrees. Let's try using these APIs to have Misty wave a hello again!

```python
from mistyPy.Robot import Robot

misty = Robot()

#first line
misty.move_head(pitch= 0, roll= 20, yaw= 0)

#second line 
misty.move_arm(arm= "both", position= 0)

#third line 
misty.move_arms(leftArmPosition= 0, rightArmPosition= 70)
```

### Challenge 2 : Remember to sequence

Just like in Blockly, when you build a sequence of two or more movements Misty needs time to achieve each position and execute the code in a consecutive order. If you don't give her time between movements, she will try to execute the movements at the same time. While in Blockly we could simply insert a timer block, in Python you will need to import a library with the time functions you need.&#x20;

A library is a group of functions created be programmers for other programmers to use. It's a really powerful tool because you can implement predefined functions instead of building your own from scratch. In this case, we'll use the library "time", where we can find many functions related to timing sequences.

To use a the time library you need to import it as shown in the example below and then call the specific function you want to use.&#x20;

```python
import time 

time.sleep(#timeinseconds)
```

In this case we will use the `time.sleep` function which works much like the timer block in Misty Blockly. The parameter's unit is in seconds.&#x20;

Now try out timing your sequence to have Misty wave her right arm up and down.

```python
from mistyPy.Robot import Robot
import time   # importing the library

misty = Robot()

misty.move_head(0, 30, 0)
misty.move_arms(70, -50)
time.sleep(0.4)  #using the function in the library
misty.move_arms(70, 0)
time.sleep(0.4)  #sleep for 0.4 seconds or 400 milliseconds
misty.move_arms(70, -50)
time.sleep(0.4)
misty.move_arms(70, 0)
```

If you want you can also define which arm you want to use.

```python
from mistyPy.Robot import Robot
import time

misty = Robot()

misty.move_head(0, 30, 0)
misty.move_arm("left", 70) # choosing Misty's arm and its position
misty.move_arm("right", -50)
time.sleep(0.4)
misty.move_arm("right", 0)
time.sleep(0.4)
misty.move_arm("right", -50)
time.sleep(0.4)
misty.move_arm("right", 0)
```

### Challenge 3: Loop your sequences

Just like in Blockly you can also create loops in Python to save space and compact your code. Let's give it a try by having Misty repeat your sequence 5 times!

There are man ways you can code loops in Python, for this example you can use the `for i in range ()` method.  Within the brackets you can choose the number of times you want the sequence to be repeated.

To create a loop with this method you need 4 elements:&#x20;

* initialization: where you define the variables that you'll use in the loop. In this case the "i"
* check: where you control if the loop is verified and so go ahead. With this method, you don't need to insert the check because it's automatic.
* body: where you have all the action that your loop does. In this code all that's after the ":" and one tab away from the "for"
* update: where you update your variables, in this way your variable will be increased and there will be a moment where the check is not verified anymore, in this way, you can stop the loop. With this method, the update is automatic.

To exit from the loop and insert commands after the loop you can keep writing your code on the same line as the "for"

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

for i in range(5): #init, check, update
    misty.move_arms(70, -50) #body
    time.sleep(0.4) 
    misty.move_arms(70, 0)
    time.sleep(0.4)
#other actions outside the loop
misty.move_arms(0, 0)
```

## Driving

### Challenge 4: Need for speed

Now that you've mastered Misty's arm and head movement APIs, let's explore how to make her drive using the `misty.drive_time` and `misty.drive_arc` API calls.&#x20;

In the code example below, Misty is driving forward for 3 seconds, then she does a turn and goes back to her initial position.  In the first two lines of code you can see that the parameters clearly defined, while in the second set of lines they are shown as regular values. In the future you can choose to highlight the parameters or memorize their order and write values.  Now let's test to see how she executes the pattern on the ground, if you want her to perform it like a street car, you can always satisfy your need for speed by increasing the velocity.&#x20;

```python
from mistyPy.Robot import Robot

misty = Robot()

misty.drive_time(linearVelocity= 80, angularVelocity= 0, timeMs= 3000)
misty.drive_arc(heading= 180, radius= 0.5,timeMs= 4000)
misty.drive_time(80, 0, 3000)
misty.drive_arc(180, 0.5, 4000)
```

### Challenge 5: Plan a path

Misty's has many more driving options than just `drive_time` and `drive_arc`. This range of driving function allow to create path plans for Misty. In later lessons you will also learn how to use AR tags and QR code to help Misty navigate around your space.

Let's start by creating a simple path plan that uses `misty.drive_track` and `misty.drive_heading.`In this case Misty will go backwards for two seconds then will turn left, she'll move just her left track for one second, forward for one meter,  and then drive like there was a curve for five seconds more.&#x20;

When you want Misty to stop driving or moving between different stages of your path plan, you will need to use the `misty.stop` function which will stop all of her motors.

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

misty.drive(linearVelocity= -70, angularVelocity= 0)
time.sleep(2)
misty.stop()
misty.drive_arc(90, 0, 3000)
misty.drive_track(leftTrackSpeed= 100,rightTrackSpeed= 0)
time.sleep(1)
misty.stop()
misty.drive_heading(heading= 0,distance= 1,timeMs= 4000)
misty.drive_time(20, 60, 5000)
```

### Challenge 6: Returning to the red carpet

Remember when Misty went out on a red carpet in the Misty Blockly Lessons? Let's try building that sequence again using all the Misty API's you've learned so far. You can use the following example for inspiration.

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

misty.drive(70, 0)
misty.move_head(0, 0, 20)
misty.move_arms(70, -50)
time.sleep(0.4) 
misty.move_arms(70, 0)
time.sleep(0.4)
misty.move_arms(70, -50)
time.sleep(0.4)
misty.move_arms(70, 0)
misty.move_head(0, 0, 0)
time.sleep(0.5)
misty.stop()
```

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/r71Z2AUCoX75gq9ttCmo" %}
[Lesson 2 : Build a character](/python/python-lessons/lesson-2-build-a-character)
{% endcontent-ref %}


# Lesson 2 : Build a character

## Aim

The aim of this lesson is to introduce you to the concept of building robot characters using Misty's API calls that allow you to enable Misty's speech,  as well as display and vocalize expressions. If you have any questions about the APIs in this lesson you can check out the [Display and LED](/python-elements/misty-python-api/display-and-led) and [Speech and NLP](/python-elements/misty-python-api/speech-and-nlp) sections.

## Expressions and Speech

### Challenge 1:  Eyes to the skies

Imagine that Misty is a robot astronaut visiting Mars and she raised her head to look at the stars, suddenly a UFO flies by. Is it a plane, a helicopter, an alien bird? The first step is to decide how you want Misty to express her reaction, by using the `misty.display_image` API you can choose which default expressions Misty should show. In this example we chose 'Terror'. You can find a full list of them here: [Image files](/resource-database/image-files). Now try out combining the display and movement APIs to build the described sequence.

For more information about the display and its parameters, you can check the [Display and LED](/python-elements/misty-python-api/display-and-led)section in Misty Python API. The parameter "alpha" represents the opacity of the image.&#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

#the alpha parameter determines the opacity of the image
misty.display_image(fileName= "e_Contempt.jpg", alpha= 1) 
time.sleep(1)
misty.move_head(-40, 0, 0)
misty.move_arm("both", -90)
time.sleep(2)
misty.display_image("e_Terror2.jpg")
```

{% endcode %}

Misty' has many other display capabilities which you can uncover by typing the keyword "display". If you want her to display a video or text instead of an image, you can use the same arguments within the parameters.

<figure><img src="/files/QsNxujBJvwgufusVz2lI" alt="" width="494"><figcaption></figcaption></figure>

### Challenge 2: I spy with my little eye

Great job! You have built your first robot character sequence, but it's still missing an important human element to really bring your character to life. The `misty.speak` API works much in the same way as the Speak block in Blockly and shares many of the same parameters.  You can choose what Misty will say, in what language and with which pitch. Misty uses US English by default if nothing else is specified. Let's try inserting the API into the sequence and have Misty tell you what she sees.

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

misty.display_image("e_Contempt.jpg")
time.sleep(1)
misty.move_head(-40, 0, 0)
misty.move_arm("both", -90)
misty.speak(text= "I spy with my little eye", pitch= 0.8)
time.sleep(0.5)
misty.display_image("e_Terror2.jpg")
misty.speak("a UFO!!")
```

### Challenge 3: Uh oh!

You can also have Misty emphasize her reaction with one of her vocal expressions by using the `misty.play_audio` API. You can find all Misty's default audio in [Audio files](/resource-database/audio-files) and if you've uploaded audio files on Misty in Blockly, you can use their names in the parameters. How would you customize it?

As shown in the example below, the first parameter is a string, and it contains the name of the file you want to use, the second is a float and it represents the volume.&#x20;

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

misty.display_image("e_Contempt.jpg")
time.sleep(1)
misty.move_head(-40, 0, 0)
misty.move_arm("both", -90)
misty.display_image("e_Terror2.jpg")
misty.speak("I spy with my little eye")
time.sleep(0.5)
misty.speak("a UFO!!")
time.sleep(0.5)
misty.play_audio(fileName= "s_PhraseUhOh.wav",volume= 80)
```

## Language

### Challenge 4: Can you speak my language?

What if your robot astronaut ends lands by accident in a different country and suddenly you need Misty to speak a different language? In the Speak Block the language could be selected from a drop-down list, however in Python you will need to set the language as voice parameter with a string of code. You can find the strings for each language in [Languages](/resource-database/languages). Let's try having Misty repeat the previous sequence in Italian or in one of your favorite languages.

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

misty.display_image("e_Contempt.jpg")
time.sleep(1)
misty.move_head(-40, 0, 0)
misty.move_arm("both", -90)
misty.display_image("e_Terror2.jpg")
misty.speak("Spio con i miei occhietti", voice= "it-it-x-itb-local")
time.sleep(0.5)
misty.speak("un UFO!!", voice= "it-it-x-itb-local")
time.sleep(0.5)
misty.play_audio(fileName= "s_PhraseUhOh.wav",volume= 80)
```

## Chest LED

### **Challenge 5: Houston we have a problem**

To emphasize your robot character's expressions even more you can use the `misty.change_led` API to control the Chest LED. For example you can add a yellow alert light to the previous example to emphasize Misty's terror of seeing the UFO. You can also use the `misty.transition_led` API to cycle between different colors.

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

misty.display_image("e_Contempt.jpg")
time.sleep(1)
misty.move_head(-40, 0, 0)
misty.move_arm("both", -90)
misty.speak(text= "I spy with my little eye", pitch= 0.8)
time.sleep(2)
misty.display_image("e_Terror2.jpg")
misty.change_led((red= 255, green= 255, blue=0)
misty.speak("a UFO!!")
misty.play_audio(fileName= "s_PhraseUhOh.wav",volume= 80)
misty.transition_led(255,0,0,255,255,0,"blink")
misty.speak(text= "Houston, we have a problem", pitch= 0.8)
```

### Challenge 6: E.T go home

Remember how Misty created a disco ball effect in Blockly Lessons? We can adapt the same sequence to create a scenario where your robot astronaut gets sucked up by a colorful beam of light from the UFO.

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

misty.move_head(40,0,0,100)
time.sleep(0.5)
misty.move_head(-40,0,0,100)
time.sleep(0.5)
misty.move_head(0,0,0,100)
misty.move_arm("both",-60,100)
misty.display_image("e_EcstacyHilarious.jpg")
misty.play_audio("s_PhraseNoNoNo.wav",60)
misty.change_led(red= 0, green= 255, blue=40)
time.sleep(0.2)
misty.change_led(red= 150, green= 30, blue=0)
time.sleep(0.2)
misty.change_led(red= 48, green= 42, blue=79)
time.sleep(0.2)
misty.change_led(red= 231, green= 25, blue=41)
time.sleep(0.2)
misty.change_led(red= 94, green= 200, blue=194)
time.sleep(0.2)
misty.change_led(red= 3, green= 147, blue=240)
time.sleep(0.2)
misty.change_led(red= 149, green= 36, blue=236)
time.sleep(0.2)
misty.stop()
```

### **Challenge 7: Compact your code**

Inserting values for each of the color parameters and adding new lines of code can be tiresome, this is where importing libraries can be incredibly helpful. By importing the "random" library you can extract a random number in a range from 0 to 255. in this way yo can randomize the color values, save lines of code and space for the sequence.

With a cleaner code and the ability to randomize your values, you can now generate how many colors you want without creating an enormous code. To do this, you will need to create a loop and insert the `random.randrange` function inside your `misty.change_led` API as shown in the example below. We can select the range and Misty will do the rest. Amazing isn't it?

{% code overflow="wrap" fullWidth="false" %}

```python
from mistyPy.Robot import Robot
import random
import time
misty = Robot()

misty.move_head(40,0,0,100)
time.sleep(0.5)
misty.move_head(-40,0,0,100)
time.sleep(0.5)
misty.move_head(0,0,0,100)
misty.move_arm("both",-60,100)
misty.display_image("e_EcstacyHilarious.jpg")
misty.play_audio("s_PhraseNoNoNo.wav",60)
for i in range (100):
    misty.change_led(random.randrange(0, 255), random.randrange(0, 255), random.randrange(0, 255))
    time.sleep(0.2)
misty.stop()
```

{% endcode %}

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/oN42i02192Z1CuCb9ynD" %}
[Lesson 3: Create memories](/python/python-lessons/lesson-3-create-memories)
{% endcontent-ref %}


# Lesson 3: Create memories

## Aim

The aim of this lesson is to learn how to manage content on Misty and access external media online. By the end of the lesson you will be able to capture content, upload custom images, video and audio as well as stream online content directly on your Misty's display. If you need detailed information about the API calls for this lesson you can check out [Record Assets](/python-elements/misty-python-api/record-assets)and [Display and LED](/python-elements/misty-python-api/display-and-led).

## Capturing content

### Challenge 1: Snap a photo with Misty

Misty is a master at capturing information about her surroundings, especially with her visor camera. With the `misty.take_picture` API you can have Misty take a photo of you and objects in your room. Let's try to snap a picture with Misty and display it. In the parameters you can define the image name, choose to display it automatically, set its resolution and even overwrite it if you want snap a new version of the same image.&#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
misty = Robot()

misty.take_picture(base65=True, fileName="myportrait", width=3200,height= 2400,displayOnScreen=True,overwriteExisting=False)

#short version 
# misty.take_picture(True,"myportrait",3200,2400,True,False)
```

{% endcode %}

## Upload content&#x20;

### **Challenge 2: A new pair of eyes**

As you've probably discovered in Misty Studio, one of Misty's greatest talents is display customizability, allowing you to create your very own robot characters by uploading custom eyes and expressions. Typically local files on your computer or tablet can be uploaded in the Expressions tab under Explore in Misty Studio or by running the Upload Block in Blockly. But what if you want to give Misty your own customized eyes using Python code?&#x20;

To upload an new pair of eyes directly from your computer in Python you would need to create a new programming environment outside of Misty Studio, since this is a more complicated process that we will explore in later lessons, the quickest way to upload images is using HTTP endpoints from your cloud such as Google Drive or directly from a website. To achieve this we will need to rely once more on the power of libraries. For Misty to read and encode data from an HTTP endpoint you will need to import the base64 library and use the `misty.save_image` and `misty.display_image`API calls. Let's try it out!

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
import base64
import requests

#binary-to-text encoding schemes
misty = Robot()

# Encode the image as base64 and UTF-8 so it's safe for the API, 
encoded = result = base64.b64encode(requests.get("https://static.wixstatic.com/media/d181e9_b56a1c7cd8104742af8b00cc716caacd~mv2.gif").content).decode("utf8")

# Send it to Misty with matching dimensions. In this case, displaying immediately and overwriting any existing file
misty.save_image("customeyes.gif", encoded, 640, 480, False, True)
time.sleep(2)
misty.display_image("customeyes.gif")

```

{% endcode %}

### **Challenge 3: Create a robot movie theater**

As you continue to design your robot character and interactions with Misty, don't forget to document all the fun moments. You can do this by letting her record them or by uploading your own footage of your interactions. You can also fill her memory banks with your favorite short movies and create a robot cinema. The  API calls for uploading and displaying videos work in the same way as for images.

**Note:** Accepted video file types are .mp4 and .wmv. Maximum file size is 6 MB.

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
import base64
import requests
import time

#binary-to-text encoding schemes
misty = Robot()

# Encode the video as base64 and UTF-8 so it's safe for the API, 
encoded = result = base64.b64encode(requests.get("https://video.wixstatic.com/video/d181e9_837968f95e3f4cd2a6210ec36ff87f58/360p/mp4/file.mp4").content).decode("utf8")

# Send it to Misty with matching dimensions. In this case, displaying immediately and overwriting any existing file
misty.save_video("mistywow2.mp4", encoded, False, True)
time.sleep(2)
misty.display_video("mistywow2.mp4")
time.sleep(2)
misty.stop(1)

```

{% endcode %}

### **Challenge 4: Add a new vocal expression**&#x20;

In some cases you maybe want to have a Misty play your music or use custom vocal expressions for you robot character. In that case you can download an audio file to Misty's storage from a cloud drive and try out playing it.&#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
import base64
import requests
import time

#binary-to-text encoding schemes
misty = Robot()

# Encode the image as base64 and UTF-8 so it's safe for the API, 
encoded = result = base64.b64encode(requests.get("https://assets.mixkit.co/active_storage/sfx/93/93-preview.mp3").content).decode("utf8")

# Send it to Misty with matching dimensions. In this case, displaying immediately and overwriting any existing file
misty.save_audio("newvoice1.mp3", encoded, False, True)
time.sleep(1)
misty.play_audio("newvoice1.mp3",30)

```

{% endcode %}

## Stream content

### **Challenge 5: Stream you favorite content to Misty**

Now that you've mastered how to upload new content to Misty, let's learn how to program Misty to stream online content directly on her display. To achieve this you can use the misty.display\_web\_view API, this is a great feature if you want Misty to show you the latest news, weather, statistics or other live information. What would you choose to stream?

Misty uses the default webview layer settings the first time she draws content with the `DisplayWebView` command.&#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
import time
misty = Robot()

misty.speak("Good morning my human friend, do you wanna know how cold it is outside today? Take a look!")
time.sleep(2)
misty.display_web_view("https://www.yr.no/en/forecast/daily-table/2-5574991/United%20States/Colorado/Boulder/Boulder")
time.sleep(5)
misty.stop(1)
```

{% endcode %}

**Important!** Displaying webviews can consume a lot of computational resources. If you notice Misty's performance decrease while multiple webviews layers are active, you may consider deleting one or more webview layers. You can use the `SetWebViewDisplaySettings` command to adjust the settings and change the appearance for a specific webview layer. Issuing a `SetWebViewDisplaySettings` command redraws the updated webview layer on Misty's display.

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/y5BNGfJtIH2iRdQ6AKl4" %}
[Lesson 4: Event skills](/python/python-lessons/lesson-4-event-skills)
{% endcontent-ref %}


# Lesson 4: Event skills

## Aim

The aim of this lesson is to learn how to make Misty more aware of her surroundings using event-based  skills.  By the end of the lesson, you will be able to build basic skills in Python and utilize most of Misty's sensory capabilities to create more complex social interactions with your robot character.

## Defining Events

### Challenge 1: Import, register and define an event

You've probably already discovered in Blockly all the fun interactions you can build using Misty's capacitive touch and bump sensors. Now let's try building them in Python!

To use events in Python you will need to import the relevant library that contains all the events that Misty can trigger. This library works as any other library in Python and it’s called “mistyPy.Events”, it contains the APIs for your events such as BumpSensors, Facerecognition, ARTagdetection etc. You can find out more about them in the [Events](/python-elements/misty-python-api/events) section in Python Elements. Let's start with importing the library.

```python
from mistyPy.Robot import Robot #libraries
from mistyPy.Events import Events
```

Once you've imported this library the next step is to register the event you want in your code using the `misty.register_event` API. To have Misty continously monitor her sensors you can use the equivalent to the Run Until Stopped block in Blockly called `misty.keep_alive.`

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot #libraries
from mistyPy.Events import Events

#events definitions
misty.register_event(event_name='bump_event', event_type=Events.BumpSensor, callback_function=bumped, keep_alive=False)

#"run until stopped" of blockly
misty.keep_alive()
```

{% endcode %}

In the first parameter you need to insert the the name of the event you want to use. The second parameter identifies which event in that category you want to trigger. You need to use one of the names provided in [Events](/python-elements/misty-python-api/events). The third parameter is the callback function which contains the name of the function you need to use to define which commands Misty should execute when an event is triggered. The last parameter is `keep_alive`, if you set it to “False”, once your event has been triggered the code will stop, instead, if you set it to “True” you can decide when to stop your code by clicking the Stop button. In the default line, the last parameter is set as False.&#x20;

Finally, the third step is to define and create the sensor functions associated with your event using `def function_name(data)` and `print(data)` . This can be placed between your library import and the registered event. For your sequence to be contained within the function you need to place it after `print(data)`. &#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot #libraries
from mistyPy.Events import Events

#function
def bumped(data):   
    print(data)
    # Function code here
    # You can have multiple lines of code inside the function
    # The function's logic goes here

#events definitions
misty.register_event(event_name='bump_event', event_type=Events.BumpSensor, callback_function=bumped, keep_alive=False)

#"run until stopped" of blockly
misty.keep_alive()
```

{% endcode %}

## Event skills

### Challenge 2: Python event skills with a human touch

Great work! Now that you've successfully created a template for your event, you can start building event based skills in Python for your robot character using the skill tree concept that we covered in Blockly Lessons. Human touch, like a handshake or pat on the back, is an important element to social interactions, and human-robot interactions are no exception.&#x20;

Using the template you created, you can now register other events such as the capacitive touch sensor event in the example below. When you are building functions you can choose to specify which sensors you want to use from a group, if you don't specify the sensor Misty will trigger the function when any of the sensors in the group are triggered, similar to when you select  “any” when creating sensor events in Blockly. Let's try using all of the the movement and expression API calls you've learned so far to have Misty respond to your touch.

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot #libraries
from mistyPy.Events import Events

misty = Robot()

def bumped(data):   #functions
    print(data)
    misty.display_image("e_Love.jpg", 1)
    misty.move_head(0, -10, 0, 50)
    misty.move_arms(90, -70, 50, 50)
    misty.change_led(255, 0, 0)

def touched(data):
    print(data)
    misty.display_image("e_Joy.jpg", 1)
    misty.move_head(0, 0, 0, 50)
    misty.move_arms(-40, 90, 50, 50)
    misty.change_led(0, 255, 0)

#events definitions
misty.register_event(event_name='bumped', event_type=Events.BumpSensor, callback_function=bumped, keep_alive=True)
misty.register_event(event_name='touch', event_type=Events.TouchSensor, callback_function=touched, keep_alive=True)

#"run until stopped" of blockly
misty.keep_alive()

```

{% endcode %}

### Challenge 3: Expand your event skill tree

Congratulations! You have built your first event skill tree in Python. Now you can grow your skill tree by creating more branches in your interaction with Misty using individual sensors in a group.  To do this you will need to build a condition in your function that will help Misty verify which sensor to use.&#x20;

For example in the bump sensor function, to select a specific bumper in the group you need to create a condition using an if statement where you specify which of them you want to use:&#x20;

```python
if data["message"]["sensorId"] == 'bfr':
#'bfr' is short for Front-Right Bumper
```

For the bump sensors and the touch sensor, you can use their name ID. You can find all the name ID's for the sensors in [Sensor Events](/python-elements/misty-python-api/sensor-events)

When you create a condition it's always good practice to account for the possibility that it may not be verified due to syntax errors in your code. For example we can use an else statement to have Misty show a red light when something goes wrong.

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events

misty = Robot()

def bumped(data):
    print(data)
    if data["message"]["sensorId"] == 'bfr': #condition
        misty.drive_time(0, -50, 8000)
        misty.change_led(0, 255, 255)
        time.sleep(1)
        misty.drive_time(30, 0, 5000)
        misty.change_led(255, 255, 0)
    else: #what happens if the condition is not verified
        misty.change_led(255,0,0)

def touched(data):
    print(data)
    if data["message"]["sensorPosition"] == 'HeadFront' :
        misty.change_led(0, 255, 0)
        misty.drive_time(30, 0, 5000)
        misty.display_image("e_EcstacyHilarious.jpg", 1)
    else:
        misty.change_led(255,0,0)

misty.register_event(event_name='touch', event_type=Events.TouchSensor, callback_function=touched, keep_alive=True)
misty.register_event(event_name='bumped', event_type=Events.BumpSensor, callback_function=bumped, keep_alive=True)

misty.keep_alive()

```

{% endcode %}

### Challenge 4: Compact your event skill tree

As you grow your event skill tree you will find that your event functions can take up a lot of space. To save space and time try compacting your skill tree using the `elif` (else if) statement. This will allow you to contain all of the conditions for each sensor within a single function group as shown in the example below.

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events

misty = Robot()

misty.speak("What's your favourite colour?", 1)
misty.move_head(0, 0, 20, 50)
misty.display_image("e_Contempt.jpg", 1)

def bumped(data): # function with more condition
    print(data)
    if data["message"]["sensorId"] == 'bfr':
        misty.change_led(0,0,255)
    elif data["message"]["sensorId"] == 'bfl':
        misty.change_led(255,0,0)
    elif data["message"]["sensorId"] == 'brr':
        misty.change_led(255,0,255)
    elif data["message"]["sensorId"] == 'brl':
        misty.change_led(0,255,0)
    else:
        misty.change_led(255,255,255)

misty.register_event(event_name='bumped', event_type=Events.BumpSensor, callback_function=bumped, keep_alive=True)

misty.keep_alive()

```

{% endcode %}

## Event message

### Challenge 5: Generalize your event skill tree

If you've worked with event messages in Blockly you may remember that it's an elegant way to compact and generalize your event skill. Instead of writing values in each condition within a function group,  you can simply use an event message to return the value from the sensor that you have triggered. You can use the data from an event as strings in your code and make it more flexible and slim. Event messages can be used in a number of events such as Face Recognition, Object Recognition, BumpSensor, TouchSensors, etc. For example if we use the `misty.speak` API together with an event message we can have Misty say which of her touch sensors was triggered.&#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events

misty = Robot()

def touched(data):
    print(data)
    misty.speak("Hello there, this is my " +  data["message"]["sensorPosition"], 1)

misty.register_event(event_name='touch', event_type=Events.TouchSensor, callback_function=touched, keep_alive=True)
misty.keep_alive()

```

{% endcode %}

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/FQ34MRZxQovYvbDqUq6O" %}
[Lesson 5: Expand awareness](/python/python-lessons/lesson-5-expand-awareness)
{% endcontent-ref %}


# Lesson 5: Expand awareness

## Aim

The aim of this lesson is to learn how to make Misty more aware of her surroundings using event-based  skills with face and object recognition, as well as audio localization.  By the end of the lesson, you will be able to build more complex skills in Python for your robot character.

## Face Recognition

### **Challenge 1: Remember me?**

Now that you've explored how to make Misty more aware of her environment and respond to human touch in Python, you can also make her recognize faces using her face recognition capabilities just like in Blockly. If you've trained Misty to recognize your face in [Lesson 6: Face recognition](/blockly/blockly-lessons/lesson-6-face-recognition) you don't need to repeat the process for Python, your FaceID is already in her memory.

The amazing thing about programming is that once you have learned a way to solve a problem if you you can use the same solution for similiar problems. So even for the face recognition event the structure of the code for the function and for the event is the same as for the bump and touch sensors events with one exception. Some event functions require you to first enable the service you are using, in this case we need to use the `misty.start_face_recognition` API call to enable the service for the face recognition event.&#x20;

In order to distinguish faces in Python you can write the FaceID name in the “Label” parameter and build your conditions with sequences as shown in the example below.  Give it a try!&#x20;

<pre class="language-python" data-overflow="wrap"><code class="lang-python">from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot()
misty.start_face_recognition()

def recognized(data):
    print(data)  
    if data["message"]["label"] == 'Simone':
        misty.speak("Hi Simone!", 1)
        misty.play_audio("s_Awe.wav", 50)
        misty.transition_led(0, 255, 0, 255, 127, 0, "TransitOnce", 1000)
    for i in range(2):
        misty.move_arms(80, -80, 50, 50)
        time.sleep(1)
<strong>        misty.move_arms(80, 0, 50, 50)
</strong>        time.sleep(1)
    elif data["message"]["label"] == 'Denis':
        misty.speak("Hi Denis!", 1)
        misty.play_audio("s_Awe2.wav", 50)
        misty.transition_led(255, 0, 0, 0, 0, 255, "TransitOnce", 1000)
    for i in range(2):
        misty.move_arms(80, -80, 50, 50)
        time.sleep(1)
        misty.move_arms(80, 0, 50, 50)
        time.sleep(1)
    else :
        misty.change_led(0, 0, 0)

misty.register_event(event_name='face_recognition_event', event_type=Events.FaceRecognition, callback_function=recognized, keep_alive=False)
misty.keep_alive()

</code></pre>

### **Challenge 2: Recognize more people**

If you want Misty say hello to everyone she already recognizes you can generalize the code using event messages that we covered in the previous lesson. This will not only compact your code, but open up a new realm of interactions that you can create. Try editing the previous code using the example below.

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot()
misty.start_face_recognition()
def recognized(data):
    print(data)  
    misty.speak("Hi " + data["message"]["label"], 1)
    if data["message"]["label"] == 'Simone':
        time.sleep(1)
        misty.play_audio("s_Awe.wav", 50)
        misty.transition_led(0, 255, 0, 255, 127, 0, "TransitOnce", 1000)
    elif data["message"]["label"] == 'Denis':
        time.sleep(1)
        misty.play_audio("s_Awe2.wav", 50)
        misty.transition_led(255, 0, 0, 0, 0, 255, "TransitOnce", 1000)
    else :
        misty.change_led(0, 0, 0)
    for i in range(2):
        misty.move_arms(80, -80, 50, 50)
        time.sleep(1)
        misty.move_arms(80, 0, 50, 50)
        time.sleep(1)

misty.register_event(event_name='face_recognition_event', event_type=Events.FaceRecognition, callback_function=recognized, keep_alive=False)
misty.keep_alive()
```

{% endcode %}

## Object Detection

### **Challenge 3: What's that over there?**

Well done so far! You can now make Misty  recognize you and your friends, this is an essential element for robots to be able to establish and remember relationships just like us humans. But it doesn't stop there, you can even have her recognize objects like a TV, Toothbrush, Bicycle and much more. See [Known objects](/resource-database/known-objects)for more information. Using the same event skill structure as for face recognition and the example below, try building a skill where Misty tells you what object she sees.

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot()
misty.start_object_detector()
def recognized(data):
    print(data)  
    misty.speak("Oh sweet, I think I see a" + data["message"]["description"], 1)
    if data["message"]["description"] == 'person':
        time.sleep(2)
        misty.play_audio("s_Awe.wav", 20)
        time.sleep(1)
        misty.speak("I love humans, they are my best friends")
        misty.transition_led(0, 255, 0, 255, 255, 0, "TransitOnce", 1000)
    elif data["message"]["description"] == 'tv':
        misty.speak("Lets watch some sci-fi robot movies together")
        misty.transition_led(255, 0, 0, 255, 127, 0, "TransitOnce", 1000)

misty.register_event(event_name='object_detection_event', event_type=Events.ObjectDetection, callback_function=recognized, keep_alive=False)
misty.keep_alive()
```

{% endcode %}

## Audio Localization

### Challenge 8: Marco Polo

Besides recognizing faces and objects, you can also teach Misty to understand where your voice or a sound is coming from using audio localization. This is set up like any other event using the source tracking event together with the `misty.start_recording_audio` API. The tricky part about setting up an audio localization event is using the audio data to tell Misty in which direction she should move her head. To solve this you can set a variable called `doa` (degreeofarrival). Since Misty will create an audio recording, you will need to delete it once the event is complete. To do this, you can set up sensor event with one of her bumpers. Let's try playing the game Marco Polo using audio localization together with a face recognition event!

{% code overflow="wrap" %}

```python
from mistyPy.Events import Events
from mistyPy.Robot import Robot

misty = Robot()
audio_file_name = "deleteme.wav"

def bumper_press(event):
    if(event["message"]["sensorId"] == "bfr" and event["message"]["isContacted"] == True):
        misty.stop_recording_audio()
        misty.delete_audio(audio_file_name)
        misty.unregister_all_events()
        print("finished")

def source_tracking(event):
    doa = event["message"]["degreeOfArrivalSpeech"]
    # Calculate the head angle based on the DOA (adjust the scaling factor as needed)
    head_angle = 90 - doa  # Subtract 90 degrees to center the head
    # Set the head movement duration and units as needed
    movement_duration = 1.0  # You can adjust the duration as needed
    movement_units = "degrees"
    # Move the head towards the sound source
    misty.move_head(0, 0, head_angle, duration=movement_duration, units=movement_units)
    print(f"DOA: {doa}, Head Angle: {head_angle} degrees")

def recognized(data):
    print(data)  
    if data["message"]["label"] == 'yourname':
        misty.speak("Polo")
        misty.transition_led(0, 255, 0, 255, 127, 0, "TransitOnce", 1000)

# Register for the FaceRecognition event
misty.register_event(event_name='face_recognition_event', event_type=Events.FaceRecognition, callback_function=recognized, keep_alive=True)

misty.register_event(event_type=Events.BumpSensor, event_name="bump pressed", keep_alive=True, callback_function=bumper_press)
misty.register_event(event_type=Events.SourceTrackDataMessage, event_name="some name", keep_alive=True, callback_function=source_tracking, debounce=1000)
misty.start_recording_audio(audio_file_name)
misty.start_face_recognition()
misty.keep_alive()
```

{% endcode %}

### Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/1ap6Y7CO3Evbe9MHJ9zU" %}
[Lesson 6: Compact code](/python/python-lessons/lesson-6-compact-code)
{% endcontent-ref %}


# Lesson 6: Compact code

## Aim

The aim of this lesson is to learn how to use variables and functions in Python.  While you've already learned how to use some functions, here you will be able to expand your capabilities with variables to create more readable, expandable and efficient code.

## Variables

### Challenge 1 : Control speed with variables

While getting into variables in Python may be scary at first, they are actually quite simple to use. You may recall from [Lesson 7: Variables and Functions](/blockly/blockly-lessons/lesson-7-variables-and-functions)that they are a fundamental concept in programming that allow you to store and manage data throughout your program. They are containers that hold values, and you can refer to the values by using names.&#x20;

A value in a variable can be both a number and a string. If you put the double quotes “ ” or single ‘ ‘ it will be identified as a string. &#x20;

```python
x = 50
y = "Hello"
```

You can use variables to simplify your code and make quick changes. Let's try rebuilding the variable blocks from Blocky in Python to control Misty's movement speed.

```python
from mistyPy.Robot import Robot
import time

misty = Robot()
x = 40
speed = 60

misty.move_head(0, 0, 0, speed)
misty.move_arms(0, x, speed, speed)
time.sleep(1)
misty.move_arms(x, 0, speed, speed)
time.sleep(1)
misty.move_arms(0, x, speed, speed)
time.sleep(1)
misty.move_arms(x, 0, speed, speed)
time.sleep(1)
```

Remember to insert valid values in each comma space in a function. For example, a string where `is` expected an `int` is not accepted.&#x20;

### Challenge 2:  Let's go global

Variables can be both defined inside a function (local variables) or made global. A global variable in programming is a variable that is accessible from any part of the program, unlike local variables which are accessible only within the scope they are declared as in the example above. Global variables are useful for storing data that needs to be accessed by multiple functions or parts of a program and can help you build larger and more complex event skill trees that use bump and touch sensors, face recognition, object detection, among others.

When you’re using the same variable in multiple functions you will need to define it as a global variable and define event handlers using the following syntax:

{% code overflow="wrap" %}

```python
#first function that defines the conditions of an event
def eventname_1(data):
global variablename
    print(data)
    if statement
    variablename= True
    elif statement
    variablename= False
    

#second function that defines the outcomes of an event based on the conditions set in the first function
def eventname_2(data):
    print(data)
    if globalvariablename == True:
    #misty.display_image("e_Joy2.jpg", 1)
    else:
    #misty.display_image("e_Sadness.jpg", 1)
```

{% endcode %}

As you can see event handlers contain the code that defines what commands Misty should execute when a specific event is trigger. For example we can define our first event function as `eventname_1` to handle a group of bump sensor events. The second event function can be defined as `eventname_2`, which will handle the responses for the events that are triggered in the first function. Both functions are linked together by a global variable called `bumped`. The next step is to determine how you want the global variable to handle reponses to a triggered event. You can do this by using conditions `True` or `False` for your global variable. This way you can create multiple responses based on whether one of the conditions is met. Now let's try setting up an event skill with global variables using the example below. &#x20;

<pre class="language-python" data-overflow="wrap"><code class="lang-python">from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot()

misty.display_image("e_Contempt.jpg", 1)
misty.change_led(255, 255, 255)
time.sleep(1)

def bumped_1(data):
    global bumped #global variable
    print(data)
    if data["message"]["sensorId"] == 'bfr':
        misty.change_led(255, 255, 0)
        bumped = True #using global variable
        misty.transition_led(255, 255, 0, 255, 255, 255, "TransitOnce", 1000)
    elif data["message"]["sensorId"] == 'bfl':
        misty.change_led(0, 100, 255)
        bumped = False
        misty.transition_led(0, 100, 255, 255, 255, 255, "TransitOnce", 1000)
    else:
        misty.change_led(255, 255, 255)

def bumped_2(data):
    print(data)
    if bumped == True:
        misty.transition_led(255, 255, 255, 0, 255, 0, "TransitOnce", 1000)
        misty.display_image("e_Joy2.jpg", 1)
    else:
        misty.transition_led(255, 255, 255, 255, 0, 0, "TransitOnce", 1000)
        misty.display_image("e_Sadness.jpg", 1)

misty.register_event(event_name='bumped_1', event_type=Events.BumpSensor, callback_function=bumped_1, keep_alive=False)
<strong>misty.register_event(event_name='bumped_2', event_type=Events.BumpSensor, callback_function=bumped_2, keep_alive=True)
</strong>
misty.keep_alive()
</code></pre>

## Functions

### Challenge 3: Make a call to your functions

As you discovered throughout your programming journey, functions are fundamental to building any complex event skill both in Blockly and in Python. Since they usually require a lot of space you will eventually find yourself going back and forth between lines of code to find that perfect sequence that you really want to use again in another part of your program. To avoid let's build up a list of functions that you can call anywhere in your code.&#x20;

In Python a function is defined using the `def` keyword followed by a custom name. For example you can define your function as  'Greeting', 'Driving', 'Dancing' and so on. If the function has no arguments you can just open and close your parenthesis “()”, otherwise if you have arguments you can insert their name in the parenthesis and separate them with a comma “ , ”.

To call a function in your code you just need to write in it's name followed by paranthesis as shown in the example below. There isn't a specific order for defining the functions. You can define them both before and after your working code.&#x20;

```python
from mistyPy.Robot import Robot
import time

misty = Robot()

#functions local library
def Misty_waves(): #creating a function
    for i in range(2):
        misty.move_arms(0, -40, 50, 50)
        time.sleep(1)
        misty.move_arms(-40, 0, 50, 50)
        
def Hello_world():
    misty.speak("Hello world!", 1)
    misty.move_head(0, 0, 0, 50)
    misty.move_arms(0, -40, 50, 50)
    misty.display_image("e_Joy.jpg", 1)
    misty.transition_led(255, 255, 255, 0, 255, 0, "TransitOnce", 1000)
# end of functions local library

#code
misty.drive_time(30, 0, 5000)
Hello_world()#calls your function
time.sleep(1)
Misty_waves() #calls your function
#end of the code
```

### Challenge 4: Combine variables and functions

Now that you've covered all the basic of using variables and functions, let's combine them to create a slim and neat event skill that can call various functions when an event is triggered.&#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot()
misty.start_face_recognition()

def Misty_waves():
    for i in range(2):
        misty.move_arms(0, -40, 50, 50)
        time.sleep(1)
        misty.move_arms(-40, 0, 50, 50)

def reaction_to_Simone():
     #misty.play_audio("s_Awe.wav", 50)
     misty.transition_led(0, 255, 0, 255, 127, 0, "TransitOnce", 1000)

def reaction_to_Denis():
     #misty.play_audio("s_Awe2.wav", 50)
    misty.transition_led(255, 0, 0, 0, 0, 255, "TransitOnce", 1000)

def reaction_to_new_person(item):
    misty.speak("I'm Misty, nice to meet you" + item + "Why don't you come here to memorize your face?" + "So next time I can recognize you and we can play!", 1)
    misty.display_image("e_Joy.jpg", 1)
    misty.transition_led(70, 100, 160, 127, 255, 0, "TransitOnce", 1000)

def recognized(data):
    print(data)  
    global item #definition variable
    item = data["message"]["label"] #assigning a value to a variable
    misty.speak("Hi " + data["message"]["label"] + "Great to see you again", 1)
    if data["message"]["label"] == 'Simone':
        reaction_to_Simone() #calls your function
    elif data["message"]["PersonName"] == 'Denis':
        reaction_to_Denis()
    else :
        misty.change_led(0, 0, 0)
    Misty_waves() 
    reaction_to_new_person(item)
   
misty.register_event(event_name='face_recognition_event', event_type=Events.FaceRecognition, callback_function=recognized, keep_alive=False)

misty.keep_alive()

```

{% endcode %}

### &#x20;Up Next [👇](https://emojipedia.org/backhand-index-pointing-down)

{% content-ref url="/pages/RmIVK73p8i8tuCj1udpp" %}
[Lesson 7: Start a conversation](/python/python-lessons/lesson-7-start-a-conversation)
{% endcontent-ref %}


# Lesson 7: Start a conversation

## Aim

In this lessons you will learn how to work with Natural Language Processing (NLP) concepts and Speech Recongition in Python and create your own conversations with Misty. By the end of lesson you will be able to master the full range of Misty's speech and conversational abilities.

## Waking up

### Challenge 1: Misty Wake-up

You may recall from the Blockly Lessons that before you start a conversation with Misty you can let her know that you want to get her attention with her default wake-up phrase "Hey, Misty". To do this, you will need to set up the start key-phrase recognition event. The code structure of the key-phrase recognized event is identical to any other event.&#x20;

Once you have imported the libraries required, you can call the KeyPhraseRecognized event by building a callback function that will determine what you want Misty to do when the key-phrase is recognized. How would you choose to design your wake up sequence?

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot()

misty.display_image("e_SleepingZZZ.jpg", 1)
misty.play_audio("e_Sleepy.wav", 50)
misty.move_head(60, 0, 0, 80)
misty.move_arms(85, 85, 80, 80)
misty.change_led(0, 0, 255)
misty.start_key_phrase_recognition()

def Key_Phrase_Recognized(data):
    print(data)
    misty.display_image("e_Surprise.jpg", 1)
    misty.play_audio("s_PhraseHello.wav", 50)
    time.sleep(2)
    misty.speak("Good morning")
    misty.move_head(0, 0, 0, 80)
    misty.move_arms(-85, -85, 80, 80)

misty.register_event(event_name='Key_Phrase_Recognized', event_type=Events.KeyPhraseRecognized, callback_function=Key_Phrase_Recognized, keep_alive=True)
misty.keep_alive()
```

{% endcode %}

## Building NLP conversations&#x20;

Now that you've created your wake-up sequence in Python you can start building a conversation. In Blockly [Lesson 8: NLP](/blockly/blockly-lessons/lesson-8-nlp) we used the conversation tree concept to understand the relationships between different elements of the conversation such as context, intents, sample, etc. In Python, you can use the same concepts to build your conversation. Now let's try rebuilding the YesNoQuestion conversation tree example from Blockly in Python.

The first step in order to build an NLP Conversation in Python is to create the conversation with the following command:&#x20;

```python
from mistyPy.Robot import Robot

misty = Robot()

misty.create_conversation(name="name_of_your_new_conversation", startingState="new_start_state", overwrite=True)
```

It includes the conversation name and the starting state.&#x20;

Later on, you can think about the states that Misty will assume in the flow of your conversation. To do so you can use the misty.create\_state command.

This state will be the one called when you will trigger its intent.&#x20;

There are two types of states:&#x20;

* Start States
* Flow States

The start states contain the context and so the “question” that you will answer, when the code runs, your answers will be recorded and the flow state associated with that intent will play.

You can find several properties for each state, like:&#x20;

* State name: the unique name for the state
* Speak: what you would like Misty to say during the state
* StartAction: A particular action associated with that specific state, you can find the list of Misty's pre-built actions here  [NLP Actions](/resource-database/nlp-actions) or build your own.

The Start state must contain an extra necessary parameter: the context.&#x20;

It’s necessary to insert the context because Misty has in her memory several contexts and you can choose which one will be used in that specific scenario. \
\
The way that you can include a context in the start\_state is "\['context\_name']", as you can see in the example:&#x20;

```python
misty.create_state(name= "new_start_state", speak="new question", contexts="['furhat.context.en.yes-no']", listen=True, overwrite=True)
```

For the flow states, instead, you’re free to use all the parameters you like and here is an example of working flow states:

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot

misty = Robot()

misty.create_state(name="flow_state_1", speak="Yay, that's awesome!")
misty.create_state(name="flow_state_2", speak="Wow, it's so obvious that humans will eventually be replaced by robots.")
```

{% endcode %}

**Important:** If you want Misty to listen to you speaking during a state, it's important to select 'listen' in the action field. This way Misty will recognize your speech and reference it with the intent.\
\
Of course a flow state can be the start state for a next question.

There are many more parameters you can modify while creating a conversation state:

<figure><img src="/files/0qhxSpdFX8iZeNgC7BJN" alt="" width="438"><figcaption></figcaption></figure>

Last but not least is to map the conversation. You will need to determine and filter how you want Misty to transit between one state and another. To achieve this you can use the misty.map\_state() API call.

In the map state, you need to insert which is the conversation you want to map, which is the starting state for that transition and the next one and lastly which is the trigger and the trigger filter.

**Important:** The triggerFilter name should match the intent name.&#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot

misty = Robot()

misty.map_state(conversation= "name_of_your_new_conversation", state= "new_start_state", trigger= "SpeechHeard", nextState= "flow_state_1", triggerFilter="trigger1", reEntry=False, overwrite=True)
misty.map_state(conversation= "name_of_your_new_conversation", state= "new_start_state", trigger= "SpeechHeard", nextState= "flow_state_2", triggerFilter="trigger2", reEntry=False, overwrite=True)
```

{% endcode %}

Once you have built your conversation you can call it with the command:&#x20;

```python
from mistyPy.Robot import Robot

misty = Robot()

misty.start_conversation("name_of_your_new_conversation")
misty.keep_alive()
```

**Important:** You don’t need to always re-build your conversation because it will be stored in Misty’s memory. So feel free to delete the block of code that creates it after feeling satisfied about it.&#x20;

To check your states, your contexts and your conversation you can use Misty’s API links:&#x20;

If you insert the following links in your browser you will be able to see your contexts, your conversations and your states in Misty’s current storage.&#x20;

(When you are working on them remember to refresh the page to see the current results)

* http\://\<your\_robot\_ip\_address>/api/dialogs/contexts&#x20;
* http\://\<your\_robot\_ip\_address>/api/conversations
* http\://\<your\_robot\_ip\_address>/api/states

**Important**: Misty may not distinguish "really well" capital and not-capital letters, so for your names we suggest to use something different each time (es not Start/start but StartConv1/StartConv2).

You can use pre-built **actions** and **contexts** for your conversations with Misty.

But if you want to create your actions you can use the `misty.create_action` API. In the parameters, you will need to define the unique name of the action and a script, which contains all of the actions you want Misty to perform, you can find a list of them in [Action Commands](/resource-database/action-commands).

```python
misty.create_action(name="question_action", script= "LED-PATTERN:0,0,255,40,0,112,1200,breathe;IMAGE:e_ApprehensionConcerned.jpg;ARMS:29,29,1000;HEAD:10,0,0,1000;", overwrite= True)
```

You can check your action at:&#x20;

* http\://\<your\_robot\_ip\_address>/api/actions

At the moment we’re facing some issues with the `misty.train_nlp_engine`.  So if you want to use your contexts we would suggest to use Blockly to build it.

<figure><img src="/files/i53xPzoe54P4moPASDt1" alt=""><figcaption></figcaption></figure>

#### Example of creation of Misty NLP conversation :

You will need to run first the blockly workflow. In the blockly workflow remember to run first the context block and then the start\_state, in this case, you’ll have available the context you just created. After, you will be able to use the Python Studio with the new elements you built.

<figure><img src="/files/Zmu7MeVNDwCpz4qNDvoA" alt=""><figcaption></figcaption></figure>

```python
from mistyPy.Robot import Robot

misty = Robot()

misty.create_conversation(name="ColorConversation", startingState="StartColor", overwrite=True)
misty.create_state(name= "StartColor", speak=" Which color do you prefer between green and blue?", contexts="['color_contexs']", listen=True, noMatchSpeech= "Could you please repeat that?",  repeatMaxCount= 3, startAction="None", overwrite=True)
misty.create_state(name= "GreenState", speak= "I love green", listen=True, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="admire", overwrite=True)
misty.create_state(name= "BlueState", speak= "I love blue", listen=True, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="admire2", overwrite=True)

misty.map_state(conversation= "ColorConversation", state= "StartColor", trigger= "SpeechHeard", nextState= "GreenState", triggerFilter="green", reEntry=False, overwrite=True)
misty.map_state(conversation= "ColorConversation", state= "StartColor", trigger= "SpeechHeard", nextState= "BlueState", triggerFilter="blue", reEntry=False, overwrite=True)
```

#### Example of using NLP conversation:

```python
from mistyPy.Robot import Robot

misty = Robot()

misty.start_conversation("ColorConversation")
misty.keep_alive()
```

Congratulations on completing your own conversation tree! You are ready to combine all that you've learned so far to create amazing human-robot interactions using event skills together with physical touch, face recognition, object recognition, audio localization, speech recognition and natural language processing.

### Advanced conversation:

In this example, you'll be able to see how you can create a conversation with more interactions. The key concept is that you're building one conversation, so you need only one, that has all the states and the map that can control the flow between them.\
\
**Code explanation:**\
As usual, you can import the required libraries, create the robot and start the event that will start the conversation (key\_phrase\_recognized). The main steps to build a conversation are:

* Initialization of one new conversation
* Creation of states
* States' mapping

In this case, the conversation's name is ColorConversation1, and the starting state will be StartColorConverasation1. When the state is a Start State it will need the context as well, as, in this case, it is in StartColorConverasation1 and YesColorConversation1. After, you can map the different states how you prefer. \
\
In the last code lines, you can find the function that will be called when "hey, Misty" is heard and you'll be able to start your conversation.\
In this case, there are three possibilities after the YesColorConversation1.

For the moment you would still need to create your context in Blockly.\
\
**Python code:**

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot()
misty.start_key_phrase_recognition()
misty.change_led(255, 0, 0)

misty.create_conversation(name="ColorConversation1", startingState="StartColorConverasation1", overwrite=True)

misty.create_state(name= "StartColorConversation1", speak="Do you like colors?", contexts="['furhat.context.en.yes-no']", listen=True, noMatchSpeech= "Could you please repeat that?",  repeatMaxCount= 3, startAction="listen", overwrite=True)
misty.create_state(name= "YesColorConversation1", speak= "I like that one, which color do you prefer between red, green or blue?", listen=True, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="admire", overwrite=True, contexts="['color_context_3_colors']")
misty.create_state(name= "NoColorConversation1", speak= "Okay, don't play with me then", listen=True, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="mad2", overwrite=True)
misty.create_state(name= "RedStateConversation1", speak= "I'm so excited, I love red, it's love's color", listen=True, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="love", overwrite=True)
misty.create_state(name= "GreenStateConversation1", speak= "I love green, it's hope's color", listen=True, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="hi", overwrite=True)
misty.create_state(name= "BlueStateConversation1", speak= "Blue! Wow! I love to look at the blue sky", listen=True, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="hug", overwrite=True)

misty.map_state(conversation= "ColorConversation1", state= "StartColorConversation1", trigger= "SpeechHeard", nextState= "YesColorConversation1", triggerFilter="yes", reEntry=False, overwrite=True)
misty.map_state(conversation= "ColorConversation1", state= "StartColorConversation1", trigger= "SpeechHeard", nextState= "NoColorConversation1", triggerFilter="no", reEntry=False, overwrite=True)
misty.map_state(conversation= "ColorConversation1", state= "YesColorConversation1", trigger= "SpeechHeard", nextState= "RedStateConversation1", triggerFilter="red", reEntry=False, overwrite=True)
misty.map_state(conversation= "ColorConversation1", state= "YesColorConversation1", trigger= "SpeechHeard", nextState= "GreenStateConversation1", triggerFilter="green", reEntry=False, overwrite=True)
misty.map_state(conversation= "ColorConversation1", state= "YesColorConversation1", trigger= "SpeechHeard", nextState= "BlueStateConversation1", triggerFilter="blue", reEntry=False, overwrite=True)

def Key_Phrase_Recognized(data):
    misty.change_led(0, 255, 0)
    misty.play_audio("s_PhraseHello.wav")
    time.sleep(2)
    misty.start_conversation("ColorConversation1")

misty.register_event(event_name='KPR_event', event_type=Events.KeyPhraseRecognized, callback_function=Key_Phrase_Recognized, keep_alive=True)
misty.keep_alive()
```


# Misty Python API

This section contains all of the Misty's Python API calls with example codes and parameters for Misty's Python SDK.

This section contains all of the Misty's Python API calls with example codes and parameters for Misty's Python Interface.

Misty's Python library is designed to directly use the API that Misty already hosts, acting as an API client for Misty. This API client is generated by interrogating Misty's API, and producing Python code as a result. The actual generator script can be found here on [Github](https://github.com/MistyCommunity/Python-SDK), and is the best place to start if you want to control Misty from another device. The generator has several dependencies, which can be seen in the setup.py script. The method used to generate the API client is really straightforward and could be replicated in other programming languages without much difficulty.&#x20;

### Contents

{% content-ref url="/pages/wk1ao23bNp39Tc9xtil9" %}
[Motion and Mobility](/python-elements/misty-python-api/motion-and-mobility)
{% endcontent-ref %}

{% content-ref url="/pages/eJtk01KgqwlDna0aK8QV" %}
[Display and LED](/python-elements/misty-python-api/display-and-led)
{% endcontent-ref %}

{% content-ref url="/pages/g0AqkCkbxxVLQTXGUByY" %}
[Record Assets](/python-elements/misty-python-api/record-assets)
{% endcontent-ref %}

{% content-ref url="/pages/Ff3tD5E3n419Rf7iNMzY" %}
[Change/Remove Assets](/python-elements/misty-python-api/change-remove-assets)
{% endcontent-ref %}

{% content-ref url="/pages/M7bmtP5VMlYn8ReuPHfB" %}
[Stream Assets](/python-elements/misty-python-api/stream-assets)
{% endcontent-ref %}

{% content-ref url="/pages/cUqt4vSbQmNqnHb0C3G6" %}
[Get Assets](/python-elements/misty-python-api/get-assets)
{% endcontent-ref %}

{% content-ref url="/pages/0SL9ytXpx8Ql9EjlXe2C" %}
[Events](/blockly-elements/misty-blocks/events)
{% endcontent-ref %}

{% content-ref url="/pages/fdqPbvd1XmqMIEo3exXI" %}
[Sensor Events](/python-elements/misty-python-api/sensor-events)
{% endcontent-ref %}

{% content-ref url="/pages/DhlVAUCXsiGP2VNmo8tZ" %}
[Speech and NLP](/python-elements/misty-python-api/speech-and-nlp)
{% endcontent-ref %}

{% content-ref url="/pages/AYpgeepQAHDoYIFFQUHv" %}
[Arduino Backpack](/python-elements/misty-python-api/arduino-backpack)
{% endcontent-ref %}

{% content-ref url="/pages/rIttziphPdSGNHnC8CIQ" %}
[System](/python-elements/misty-python-api/system)
{% endcontent-ref %}


# Motion and Mobility

The following API calls allow you to programmatically drive and stop Misty and move her head and arms.

In this section you can find all of Misty's API calls for motion and mobility.

### **MoveArm**

Moves one or both of Misty's arms up and down.

**Example Code**

{% hint style="success" %}

```python
misty.move_arm("left", 50, 100)
```

{% endhint %}

When moving Misty's arms, it's helpful to understand their orientation.

* At 0 degrees, Misty's arms point straight forward along her X axis, parallel to the ground.
* At +90 degrees, Misty's arms point straight down towards the ground.
* At +/- 180 degrees, Misty's arms would face straight back, pointing toward her backpack; however, Misty's arms are not currently configured to move to this position.
* At -90/+270 degrees, Misty's arms would point straight up towards her head, perpendicular to the ground; however, the upward movement of Misty's arm movement is currently limited to -29 degrees.

**Parameters**

{% code overflow="wrap" %}

```python
misty.move_arm(self, arm : str = None, position : float = None, velocity : float = None, duration : float = None, units : str = None)
```

{% endcode %}

* `Arm` (string) - The arm to move. You must use either `left`, `right`, or `both`.
* `Position` (double) - The new position to move the arm to. Use the `Units` parameter to determine whether to use `position`, `degrees`, or `radians`. Defaults to `degrees`.
* `Velocity` (double) - Optional. A value of 0 to 100, specifying the speed with which the arm should move. Defaults to `null`.
* `Units` (string) - Optional. A string value of `degrees`, `radians`, or `position` that determines which unit to use in moving Misty's arms.

### **MoveArms**

Moves one or both of Misty's arms. You can control both arms simultaneously or one at a time.

**Example Code**

{% hint style="success" %}

```python
misty.move_arms(40, 40, 100, 100)
```

{% endhint %}

When moving Misty's arms, it's helpful to understand their orientation.

* At 0 degrees, Misty's arms point straight forward along her X axis, parallel to the ground.
* At +90 degrees, Misty's arms point straight down towards the ground.
* At +/- 180 degrees, Misty's arms would face straight back, pointing toward her backpack; however, Misty's arms are not currently configured to move to this position.
* At -90/+270 degrees, Misty's arms would point straight up towards her head, perpendicular to the ground; however, the upward movement of Misty's arm movement is currently limited to -29 degrees.

**Parameters**

{% code overflow="wrap" %}

```python
misty.move_arms(self, leftArmPosition : float = None, rightArmPosition : float = None, leftArmVelocity : float = None, rightArmVelocity : float = None, duration : float = None, units : str = None)
```

{% endcode %}

* `LeftArmPosition` (double) - Optional. The new position of Misty's left arm. Use the `Units` parameter to determine whether to use `position`, `degrees`, or `radians`. Defaults to `degrees`.
* `RightArmPosition` (double) - Optional. The new position of Misty's right arm. Use the `Units` parameter to determine whether to use `position`, `degrees`, or `radians`. Defaults to `degrees`.
* `LeftArmVelocity` (double) - Optional. A value of 0 to 100 specifying the speed with which the left arm should move. Defaults to `null`.
* `RightArmVelocity (`double) - Optional. A value of 0 to 100, specifying the speed with which the right arm should move. Defaults to `null`.
* `Units` (string) - Optional. A string value of `degrees`, `radians`, or `position` that determines which unit to use in moving Misty's arms.

### **MoveHead**

Moves Misty's head to a new position along its pitch, roll, and yaw axes.

**Example Code**

{% hint style="success" %}

```python
misty.move_head(-10, 30 ,0 ,100)
```

{% endhint %}

For the value ranges (by unit) for each direction of head movement, see the table below:

| Dimension | Degrees                  | Position               | Radians                       |
| --------- | ------------------------ | ---------------------- | ----------------------------- |
| pitch     | -40 (up) to 26 (down)    | -5 (up) to 5 (down)    | -0.1662 (up) to 0.6094 (down) |
| roll      | -40 (left) to 40 (right) | -5 (left) to 5 (right) | -0.75 (left) to 0.75 (right)  |
| yaw       | -81 (right) to 81 (left) | -5 (right) to 5 (left) | -1.57 (right) to 1.57 (left)  |

**Parameters**

{% code overflow="wrap" %}

```python
misty.move_head(self, pitch : float = None, roll : float = None, yaw : float = None, velocity : float = None, duration : float = None, units : str = None)
```

{% endcode %}

* Pitch (double) - Value that determines the up or down position of Misty's head movement.
* Roll (double) - Value that determines the tilt ("ear" to "shoulder") of Misty's head.
* Yaw (double) - Number that determines the left to right turn position of Misty's head.
* Velocity (double) - Optional. The percentage of max velocity that indicates how quickly Misty should move her head. Value range: 0 to 100. Defaults to 10.
* Duration (double) - Optional. Time (in seconds) Misty takes to move her head from its current position to its new position.
* Units (string) - Optional. A string value of `degrees`, `radians`, or `position` that determines which unit to use in moving Misty's head. Defaults to `degrees`.

### **Drive**

Drives Misty forward or backward at a specific speed until cancelled.

**Example Code**

{% hint style="success" %}

```python
misty.drive(60,60)
```

{% endhint %}

When using the Drive command, it helps to understand how linear velocity (speed in a straight line) and angular velocity (speed and direction of rotation) work together:

* Linear velocity (-100) and angular velocity (0) = driving straight backward at full speed.
* Linear velocity (100) and angular velocity (0) = driving straight forward at full speed.
* Linear velocity (0) and angular velocity (-100) = rotating clockwise at full speed.
* Linear velocity (0) and angular velocity (100) = rotating counter-clockwise at full speed.
* Linear velocity (non-zero) and angular velocity (non-zero) = Misty drives in a curve.

**Parameters**

```python
misty.drive(self, linearVelocity : float = None, angularVelocity : float = None)
```

* `LinearVelocity` (double) - A percent value that sets the speed for Misty when she drives in a straight line. Default value range is from -100 (full speed backward) to 100 (full speed forward).
* `AngularVelocity` (double) - A percent value that sets the speed and direction of Misty's rotation. Default value range is from -100 (full speed rotation clockwise) to 100 (full speed rotation counter-clockwise). **Note:** For best results when using angular velocity, we encourage you to experiment with using small positive and negative values to observe the effect on Misty's movement.

### **DriveArc**

Drives Misty in an arc. Misty continues driving until her current heading matches the desired absolute heading passed into this command.

**Example Code**

{% hint style="success" %}

```python
misty.drive_arc(0, 1, 4000, "false")
```

{% endhint %}

To get Misty's current heading, use the value for `yaw` from the [`IMU`](https://docs.mistyrobotics.com/misty-ii/robot/sensor-data/#imu) named object. To calculate Misty's velocity, use: `((desired_heading - current_heading) * (π/180) * radius) / (timeMs/1000)`.

**Parameters**

{% code overflow="wrap" %}

```python
misty.drive_arc(self, heading : float = None, radius : float = None, timeMs : float = None, reverse : bool = None)
```

{% endcode %}

* `Heading` (double) - The absolute heading Misty should obtain when the arc is complete. To set the absolute heading, use either: 0 - 360, where 0 is straight ahead, 90 is directly to the left, 180 is straight behind, and 270 is directly to the right, or: -180 to 180, where 0 is straight ahead, 90 is directly to the left, 180 and -180 are straight behind, and -90 is directly to the right.
* `Radius` (double) - The radius (in meters) of the arc.
* `TimeMs` (double) - The duration (in milliseconds) that Misty drives.
* `Reverse` (boolean) - Optional. If `true`, Misty drives in reverse. Default is `false`.

### **DriveHeading**

Drives Misty forward or backward in a straight line. While driving, Misty continuously adjusts her current heading to maintain the desired absolute heading.

**Example Code**

{% hint style="success" %}

```python
misty.drive_heading(0, 10, 4000, "false")
```

{% endhint %}

For a smooth driving experience, Misty's current heading should be within two degrees of the desired absolute heading before she executes the `DriveHeading` command. Variations of greater than two degrees result in large correction velocities. You can use the `DriveArc` command to face Misty in the direction of the heading you want her to maintain. Then, use the `DriveHeading` command to drive Misty forward or backward in a straight line. To calculate Misty's velocity, use `distance / (timeMs/1000)`.

**Parameters**

{% code overflow="wrap" %}

```python
misty.drive_heading(self, heading : float = None, distance : float = None, timeMs : float = None, reverse : bool = None)
```

{% endcode %}

* `Heading` (double) - The absolute heading Misty should maintain. To set the absolute heading, use either: 0 - 360, where 0 is straight ahead, 90 is directly to the left, 180 is straight behind, and 270 is directly to the right, or: -180 to 180, where 0 is straight ahead, 90 is directly to the left, 180 and -180 are straight behind, and -90 is directly to the right.
* `Distance` (double) - The distance (in meters) that Misty should drive.
* `TimeMs` (double) - The duration (in milliseconds) that Misty should drive.
* `Reverse` (boolean) - If `true`, Misty drives in reverse. Default is `false`.

### **DriveTime**

Drives Misty forward or backward at a set speed, with a given rotation, for a specified amount of time.

**Example code**

{% hint style="success" %}

```python
misty.drive_time(40,40,1000,30)
```

{% endhint %}

When using the DriveTime command, it helps to understand how linear velocity (speed in a straight line) and angular velocity (speed and direction of rotation) work together:

* Linear velocity (-100) and angular velocity (0) = driving straight backward at full speed.
* Linear velocity (100) and angular velocity (0) = driving straight forward at full speed.
* Linear velocity (0) and angular velocity (-100) = rotating clockwise at full speed.
* Linear velocity (0) and angular velocity (100) = rotating counter-clockwise at full speed.
* Linear velocity (non-zero) and angular velocity (non-zero) = Misty drives in a curve.

**Parameters**

{% code overflow="wrap" %}

```python
misty.drive_time(self, linearVelocity : float = None, angularVelocity : float = None, timeMs : int = None, degree : float = None)
```

{% endcode %}

* `LinearVelocity` (double) - A percent value that sets the speed for Misty when she drives in a straight line. Default value range is from -100 (full speed backward) to 100 (full speed forward).
* `AngularVelocity` (double) - A percent value that sets the speed and direction of Misty's rotation. Default value range is from -100 (full speed rotation clockwise) to 100 (full speed rotation counter-clockwise). **Note:** For best results when using angular velocity, we encourage you to experiment with using small positive and negative values to observe the effect on Misty's movement.
* `TimeMs` (integer) - A value in milliseconds that specifies the duration of movement. Misty will not drive if you pass in a value of less than 100 for this parameter.
* `Degree` (double) - (optional) The number of degrees to turn. **Note:** Supplying a `Degree` value recalculates linear velocity.

### **DriveTrack**

Drives Misty left, right, forward, or backward, depending on the track speeds specified for the individual tracks.

**Example Code**

{% hint style="success" %}

```python
misty.drive_track(30,70)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.drive_track(self, leftTrackSpeed : float = None, rightTrackSpeed : float = None)
```

{% endcode %}

* `LeftTrackSpeed` (double) - A value for the speed of the left track, range: -100 (full speed backward) to 100 (full speed forward).
* `RightTrackSpeed` (double) - A value for the speed of the right track, range: -100 (full speed backward) to 100 (full speed forward).

### **Halt**

Stops all motor controllers, including drive motor, head/neck, and arm.

**Example code**

{% hint style="success" %}

<pre class="language-python"><code class="lang-python"><strong>misty.halt()
</strong></code></pre>

{% endhint %}

**Parameters**

```python
misty.halt(self, motorMask : int = None) 
```

`motorMask` (int): A bitmask representing the motors to be halted. Each bit in the mask corresponds to a specific motor or group of motors. The exact definition of each bit depends on Misty's hardware and firmware configuration.

### **Stop**

Stops Misty's movement.&#x20;

**Example Code**

{% hint style="success" %}

```python
misty.stop()
```

{% endhint %}

**Important!** Under most circumstances, it is best to avoid calling the `Stop` command with a `Hold` value of `true`. Holding Misty's position can strain Misty's drive motors. Stopping Misty's driving without holding her position should suffice under most circumstances and can prolong the life of your robot's drive motors.

**Parameters**

```python
misty.stop(self, hold : bool = None)
```

* `Hold` (boolean) - Optional. Defaults to `false`. If `true`, Misty's drive motors remain engaged after Misty stops moving and attempt to hold the robot in its current position. This can be useful when Misty needs to stop moving while she's on an incline; sending a `Stop` command with a `Hold` value of `true` keeps the motors engaged, so that Misty does not roll down the slope. It is generally recommended to ignore the `Hold` parameter. Stopping Misty's driving without holding her position should suffice under most circumstances and can prolong the life of your robot's drive motors.

### **DriveToLocation (Misty II Pro)**

Drives to a designated waypoint.

**Important!** Make sure to use `StartTracking` before using this command to have Misty start tracking her location, and use `StopTracking` to have her stop tracking her location after she arrives at the specified location.

**Example Code**

{% hint style="success" %}

```python
misty.drive_to_location("destination":"10:25")
```

{% endhint %}

**Parameters**

```python
misty.drive_to_location(self, destination : GridCell = None)
```

* `Destination` (string) - A colon-separated integer pair that represents the X and Y coordinates of the destination. **Note:** `GetMap` obtains the occupancy grid for the most recent map Misty has generated. Use this grid to determine the X and Y coordinates of the destination. The X coordinate of a given cell is the index of the array for the cell. The Y coordinate of a cell is the index of that cell within its array

### **FollowPath (Misty II Pro)**

Drives Misty on a path defined by coordinates you specify. Note that Misty must have a map and be actively tracking before starting to follow a path. Misty will not be able to successfully follow a path if unmapped obstacles are in her way.

**Example Code**

{% hint style="success" %}

```python
misty.follow_path(4:3,8:8,10:15)
```

{% endhint %}

**Important!** Make sure to use `StartTracking` before using this command to have Misty start tracking her location, and use `StopTracking` to have her stop tracking her location after she arrives at the specified location

**Parameters**

{% code overflow="wrap" %}

```python
misty.follow_path(self, path : str = None, velocity : float = None, fullSpinDuration : float = None, waypointAccuracy : float = None, rotateThreshold : float = None)
```

{% endcode %}

* `Path` (string) - A string of comma-separated X:Y coordinates representing waypoints on a path for Misty to track through her currently active map. Each waypoint is a colon-separated integer pair representing the X and Y coordinates of a location on Misty's currently active map. Use `GetMap` to obtain the occupancy grid for Misty's current map, and use this grid to determine the X and Y coordinates of the destination.
* `Velocity` (double) - Optional. A fraction of Misty's max velocity. Determines how fast Misty moves when driving straight while following a path. Expects a decimal value greater than 0 and less than 1. Defaults to `0.5` (50% of max velocity) if not specified.
* `FullSpinDuration` (double) - Optional. Number of seconds it takes for Misty to complete a full spin (360 degrees) while following a path. Determines how fast Misty pivots or spins when changing direction. Defaults to `15` if not specified.
* `WaypointAccuracy` (double) - Optional. How close (in meters) the robot gets to a waypoint before considering itself to have reached that waypoint. Defaults to `0.1` if not specified.
* `RotateThreshold` (double) - Optional. The angle (in degrees) Misty's path following algorithm uses to determine when Misty should pivot toward a waypoint instead of continuing to drive straight. When following a path, Misty drives straight toward her next waypoint until the bearing between the waypoint and her current heading is greater than `RotateThreshold` degrees. When the bearing reaches this threshold, Misty pivots in the direction of the waypoint until the bearing is lower than `RotateThreshold`. When Misty reaches a waypoint, she spins to face the next waypoint and drives straight. As she approaches the waypoint, any error in the original spin causes the bearing angle to grow, causing Misty to stop and turn toward the waypoint; thus, Misty may stop and pivot multiple times between one waypoint and the next. Defaults to `10` if not specified.


# Display and LED

This section contains all of Misty's APIs related to changing the display and LED.

### **DisplayImage**

Displays an image on Misty's screen. You can use this command to display images from Misty's local storage or to display images that are hosted on the web.

**Example Code**

{% hint style="success" %}

```python
misty.display_image("e_Joy.jpg",1)
```

{% endhint %}

Misty uses the default image layer settings the first time she draws content with the `DisplayImage` command. You can use the `SetImageDisplaySettings` command to adjust the settings and change the appearance for a specific image layer. Issuing a `SetImageDisplaySettings` command redraws the updated image layer on Misty's display.&#x20;

**Parameters**

{% code overflow="wrap" %}

```python
misty.display_image(self, fileName : str = None, alpha : float = None, layer : str = None, isURL : bool = None)
```

{% endcode %}

* FileName (string) - Filename for the image to display. Valid image file types are `.jpg`, `.jpeg`, `.gif`, `.png`. Alternately, if `IsUrl` is true, the URL path for the image to display.
* Alpha (double) - Optional. Opacity for the layer on which the image displays. A value of 0 is completely transparent; 1 is completely opaque. When you specify a value greater than 0 and less than 1, the layer appears but is transparent. Defaults to 1.
* Layer (string) - Optional. The display layer to create or update with this command. If `null` or not supplied, the image displays on the default image layer (named `DefaultImageLayer`).
* IsUrl (boolean) - Optional. If `true`, the system treats the string you pass in for `FileName` as the URL address for an image hosted online.

### **DisplayText**

Misty uses the default text layer settings the first time she draws content with the `DisplayText` command. You can use the `SetTextDisplaySettings` command to adjust the settings and change the appearance for a specific text layer. Issuing a `SetTextDisplaySettings` command redraws the updated image layer on Misty's display.

**Example Code**

{% hint style="success" %}

```python
misty.display_text("Hello world, my name is Misty")
```

{% endhint %}

**Parameters**

```python
misty.display_text(self, text : str = None, layer : str = None)
```

* Text (string) - The text to display.
* Layer (string) - Optional. The layer on which to display the text. You can use this parameter to create a new text layer or to update an existing text layer. If not supplied, the text displays on the default text layer (named `DefaultTextLayer`).

### **DisplayVideo**

Plays a video on Misty's screen.You can use this command to play videos you upload to Misty or videos that are hosted on the web. Use the `SaveVideo` command to upload a new video asset to your robot.

**Example Code**

{% hint style="success" %}

```python
misty.display_video("MyHomeVideo.mp4")
```

{% endhint %}

Misty uses the default video layer settings the first time she draws content with the `DisplayVideo` command. You can use the `SetVideoDisplaySettings` command to adjust the settings and change the appearance for a specific video layer. Issuing a `SetVideoDisplaySettings` command redraws the updated video layer on Misty's display.

The `DisplayVideo` command has the following limitations at this time:

* You cannot use the `DisplayVideo` command to play video recordings that Misty creates with the `StartRecordingVideo` command. Misty can only play user-uploaded videos on her display.
* Misty does not play audio for the videos she plays on her display.

**Parameters**

{% code overflow="wrap" %}

```python
misty.display_video(self, filename : str = None, layer : str = None, isURL : bool = None)
```

{% endcode %}

* FileName (string) - Filename for the video to play, with the file type extension. Valid video file types are .`mp4` and `.wmv`. Alternatively, if `IsURL` is `true`, the URL path for the video to play.
* Layer (string) - Optional. The display layer to create or update with this command. If `null` or not supplied, the video plays on the default video layer (named `DefaultVideoLayer`).
* IsUrl (boolean) - Optional. If `true`, the system treats the string you pass in for `FileName` as the URL address for a video hosted online.

### **DisplayWebView**

Misty uses the default webview layer settings the first time she draws content with the `DisplayWebView` command. You can use the `SetWebViewDisplaySettings` command to adjust the settings and change the appearance for a specific webview layer. Issuing a `SetWebViewDisplaySettings` command redraws the updated webview layer on Misty's display.

**Example Code**

{% hint style="success" %}

```python
misty.display_web_view("https://www.youtube.com/")
```

{% endhint %}

**Important!** Displaying webviews can consume a lot of computational resources. If you notice Misty's performance decrease while multiple webviews layers are active, you may consider deleting one or more webview layers.

**Parameters**

{% code overflow="wrap" %}

```python
misty.display_web_view(self, url : str = None, layer : str = None)
```

{% endcode %}

* URL (string) - The URL for the web page to display.
* Layer (string) - Optional. The display layer to create or update with this command. If `null` or not supplied, the webview displays on the default webview layer (named `DefaultWebViewLayer`).

### **ChangeLED**

Changes the color of the LED light behind the logo on Misty's torso.

**Example Code**

{% hint style="success" %}

```python
misty.change_led(125, 80, 30)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.change_led(self, red : bytes = None, green : bytes = None, blue : bytes = None)
```

{% endcode %}

* Red (byte) - The red RGB color value (range 0 to 255).
* Green (byte) - The green RGB color value (range 0 to 255).
* Blue (byte) - The blue RGB color value (range 0 to 255).

### **TransitionLED**

Sets Misty's LED to transition between two colors. When you use this command, Misty will continue the transition you specify until she is powered off or receives another command to change or transition her LED.

**Example Code**

{% hint style="success" %}

```python
misty.transition_led(200,24,0,100,80,30,"Breathe",500)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.transition_led(self, red : bytes = None, green : bytes = None, blue : bytes = None, red2 : bytes = None, green2 : bytes = None, blue2 : bytes = None, transitionType : str = None, timeMs : float = None)
```

{% endcode %}

* Red (byte) - The red RGB color value for the first color (range 0 to 255).
* Green (byte) - The green RGB color value for the first color (range 0 to 255).
* Blue (byte) - The blue RGB color value for the first color (range 0 to 255).
* Red2 (byte) - The red RGB color value for the second color (range 0 to 255).
* Green2 (byte) - The green RGB color value for the first color (range 0 to 255).
* Blue2 (byte) - The blue RGB color value for the first color (range 0 to 255).
* TransitionType (string) - The transition type to use. Case sensitive. Accepts `Blink` (continuously blinks LED between the specified colors), `Breathe` (continuously fades LED between the specified colors), and `TransitOnce` (blinks LED from first color to second color only once).
* TimeMs (int) - The duration (in milliseconds) between each transition. Must be greater than `3`.


# Record Assets

This section describes API calls to upload your own content to Misty.

### **SaveImage**

Saves an image to Misty. Optionally, proportionately reduces the size of the saved image.

Valid image file types are `.jpg`, `.jpeg`, `.gif`, `.png`. Maximum file size is 3 MB.

**Example Code**

{% hint style="success" %}

```python
misty.save_image("myimage.jpg","30,190,40,24,...",300, 300, false, true)
```

{% endhint %}

**Note:** Images can be reduced in size but not enlarged. Because Misty does not adjust the proportions of images, for best results use an image with proportions similar to her screen (480 x 272 pixels).

**Parameters**

{% code overflow="wrap" %}

```python
misty.save_image(self, fileName : str = None, data : str = None, width : int = None, height : int = None, immediatelyApply : bool = None, overwriteExisting : bool = None)
```

{% endcode %}

* FileName (string) - The name of the image file to upload.
* Data (string) - The image data, passed as a base64 string. You must either supply a value for `Data` **or** specify a `File` to upload.
* File (object) - The image file to save to Misty. Valid image file types are `jpg`, `.jpeg`, `.gif`, and `.png`. **Note:** Make sure to set the content-type in the header of the POST call to `multipart/form-data`. Uploading files to Misty this way does not work with JQuery's AJAX, but does work with XHR (XMLHttpRequest). You must either supply a value for `Data` **or** specify a `File` to upload.
* Width (integer) - Optional. A whole number greater than 0 specifying the desired image width (in pixels). **Important:** To reduce the size of an image you must supply values for both `Width` and `Height`. Note that if you supply disproportionate values for `Width` and `Height`, the system uses the proportionately smaller of the two values to resize the image.
* Height (integer) - Optional. A whole number greater than 0 specifying the desired image height (in pixels). **Important:** To reduce the size of an image you must supply values for both `Width` and `Height`. Note that if you supply disproportionate values for `Width` and `Height`, the system uses the proportionately smaller of the two values to resize the image.
* ImmediatelyApply (boolean) - Optional. A value of `true` tells Misty to immediately display the uploaded image file, while a value of `false` tells Misty not to display the image.
* OverwriteExisting (boolean) - Optional. A value of `true` means the uploaded file should overwrite a file with the same name, if one currently exists on Misty. A value of `false` means the uploaded file should not overwrite any existing files on Misty

### **SaveVideo**

Saves a video to Misty.

Accepted video file types are `.mp4` and `.wmv`. Maximum file size is 6 MB.

**Example Code**

{% hint style="success" %}

```python
misty.save_video("myhomevideo.mp4","30,190,40,24,...", false, true)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.save_video(self, fileName : str = None, data : str = None, immediatelyApply : bool = None, overwriteExisting : bool = None)
```

{% endcode %}

* FileName (string) - The name of the video file to upload, with the file type extension.
* Data (string or file) - **Option 1**: A Base64-encoded string of the video file data. **Option 2**: The video file. Valid video file types are `.mp4` and `.wmv`. When using option 2, make sure to set the `content-type` in the header of the `POST` call to `multipart/form-data`. Uploading files to Misty this way does not work with JQuery's AJAX, but does work with XHR (XMLHttpRequest).
* ImmediatelyApply (boolean) - Optional. A value of `true` tells Misty to immediately play the uploaded video, while a value of `false` tells Misty not to play the video.
* OverwriteExisting (boolean) - Optional. A value of `true` means the uploaded file should overwrite a file with the same name, if one currently exists on Misty. A value of false means the uploaded file should not overwrite any existing files on Misty.

### **SaveAudio**

Saves an audio file to Misty. Maximum size is 3 MB.

Accepts audio files formatted as `.wav`, .`mp3` ,`.wma`, and `.aac`.

**Example Code**

{% hint style="success" %}

<pre class="language-python"><code class="lang-python"><strong>misty.save_audio("example.wav", "34,88,90,49,56", false, true)
</strong></code></pre>

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.save_audio(self, fileName : str = None, data : str = None, immediatelyApply : bool = None, overwriteExisting : bool = None)
```

{% endcode %}

* FileName (string) - The name of the audio file to upload.
* Data (string) - The audio data, passed as a string containing base64 data. You must either supply a value for `Data` **or** specify a `File` to upload.
* File (object) - The audio file to save to Misty. Valid audio file types are `.wav`, `.mp3`, `.wma`, and `.aac`. **Note:** If uploading a file instead base64 data for the asset, make sure to set the `content-type` in the header of the POST call to [`multipart/form-data`](https://developer.mozilla.org/en-US/docs/web/HTTP/Basics_of_HTTP/MIME_types#multipartform-data). Uploading files to Misty this way does *not* work with JQuery's AJAX, but does work with XHR (XMLHttpRequest). You must either supply a value for `Data` **or** specify a `File` to upload.
* ImmediatelyApply (boolean) - Optional. A value of `true` tells Misty to immediately play the uploaded audio file, while a value of `false` tells Misty not to play the file.
* OverwriteExisting (boolean) - Optional. A value of `true` means the uploaded file should overwrite a file with the same name, if one currently exists on Misty. A value of `false` means the uploaded file should not overwrite any existing files on Misty.

### **TakePicture**

Takes a picture with Misty's RGB camera. Optionally, saves the picture to Misty's local storage.

Valid resolutions (as `Width` x `Height`) for taking pictures are: 4160 x 3120, 3840 x 2160, 3264 x 2448, 3200 x 2400, 2592 x 1944, 2048 x 1536, 1920 x 1080, 1600 x 1200, 1440 x 1080, 1280 x 960, 1024 x 768, 800 x 600, 640 x 480, and 320 x 240.

These width and height values are reversed for the actual image that Misty returns when you call this command. The pictures Misty takes with her RGB camera are rotated 90 degrees counterclockwise. Misty reorients each picture 90 degrees clockwise during the encoding process.

**Example Code**

{% hint style="success" %}

```python
misty.take_picture("true","mypicture", 800, 600,"true","false")
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.take_picture(self, base64 : bool = None, fileName : str = None, width : int = None, height : int = None, displayOnScreen : bool = None, overwriteExisting : bool = None)
```

{% endcode %}

* Base64 (boolean) - Sending a request with `true` returns the image data as a downloadable Base64 string, while sending a request of `false` displays the photo in your browser or REST client immediately after it is taken. Default is `false`.
* FileName (string) - Optional. The filename to assign to the image file for the captured photo. If you do not supply a filename, Misty does not save the photo.
* Width (integer) - Optional. The desired image width (in pixels). When you specify a resolution, you must pass in values for both width and height. See the command description for a list of valid resolutions.
* Height (integer) - Optional. The desired image height (in pixels). When you specify a resolution, you must pass in values for both width and height. See the command description for a list of valid resolutions.
* DisplayOnScreen (boolean) - Optional. If `true` **and** a `FileName` is provided, displays the captured photo on Misty's screen. If `false` or no `FileName` value is provided, does nothing.
* OverwriteExisting (boolean) - Optional. Indicates whether Misty should overwrite an image with the same filename as the captured photo if one exists on her local storage. Passing in `true` overwrites a file with the same name. Passing in `false` prevents an existing file with the same name from being overwritten. In the case that `OverwriteExisting` is set to `false` and a photo already exists with the same filename as the newly captured photo, the new photo is not saved to Misty. Defaults to `false`.

### **StartRecordingAudio**

Starts Misty recording audio. Misty saves audio recordings to her local storage as .wav files. To stop recording, you must call the `StopRecordingAudio` command.

**Important!** If you do not issue a `StopRecordingAudio` command, Misty will continue recording until the audio file is 1 GB. Attempting to retrieve a file this large from Misty can cause the system to crash.

There are a few limitations to consider when recording audio:

* Misty cannot record audio and listen for the "Hey, Misty!" key phrase at the same time. Recording audio automatically disables key phrase recognition.
* Misty cannot use her microphones to record audio while actively streaming audio and video.

**Example Code**

{% hint style="success" %}

```
misty.start_recording_audio("MyBestSong.wav")
```

{% endhint %}

**Parameters**

```python
misty.start_recording_audio(self, fileName : str = None) -> Response:
```

* FileName (string) - The name to assign to the audio recording. This parameter must include a `.wav` file type extension at the end of the string.

### **StopRecordingAudio**

Directs Misty to stop the current audio recording. You must use this command after calling the `StartRecordingAudio` command. If you do not call `StopRecordingAudio`, Misty automatically stops recording after 60 seconds.

**Example Code**

{% hint style="success" %}

```
misty.stop_recording_audio
```

{% endhint %}

### **StartRecordingVideo**

Starts recording video with Misty's 4K Camera.

Valid resolutions (as `Width` x `Height`) for recording videos are: 3840 x 2160, 1920 x 1080, 1280 x 960, 640 x 480, and 320 x 240.

The videos Misty records with her RGB camera are rotated 90 degrees counterclockwise, and the width and height values listed above may be swapped for the actual video that Misty returns when you call this command. Video recordings that Misty creates have orientation information that enables media players to rotate and play back the video using the correct orientation; however, if you play the video in certain players, you will see that the actual video file itself is rotated 90 degrees counterclockwise.

Recording videos at 3840 x 2160 changes the max resolution for taking pictures to 3840 x 2160. If you record video at 1920 x 1080 (or lower), then Misty can use the highest resolution for taking pictures. If you try to record at 3840 x 2160 while using the highest resolution for taking pictures, the system automatically lowers the resolution for taking pictures to 3840 x 2160.

When Misty powers on, she starts a new camera session with a default resolution setting of 1920 x 1080 for recording videos. If you record a video without specifying a resolution, Misty uses the resolution that's already set in the current camera session.

**Example Code**

{% hint style="success" %}

```python
misty.start_recording_video("MyHomeVideo", "false", 60, 1920, 1080)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.start_recording_video(self, fileName : str = None, mute : bool = None, duration : int = None, width : int = None, height : int = None) -> Response:
```

{% endcode %}

* FileName (string) - Optional. The filename for the recorded video. Video recordings can only include uppercase and lowercase alphanumeric characters, hyphens, and underscores (`[a-zA-Z0-9_-]`). Do not supply a file type extension; the system automatically adds an extension of `.mp4`. If you do not supply a filename, the video recording is saved with the default filename of `misty_video`. **Important:** When you record a video with the same filename of a video that already exists on the robot, the new video recording automatically overwrites the existing recording.
* Mute (bool) - Optional. Whether to mute audio while recording. Default is `false`.
* Duration (int) - Optional. How long (in seconds) to record. Must be greater than `0`. The max duration for a video recording is 180 seconds (3 minutes). If you do not specify a value, Misty automatically stops recording after 30 seconds (default), or upon receiving a `StopRecordingVideo` command.
* Width (int) - Optional. Sets the resolution width (in pixels) for the video recording. When you specify a resolution, you must pass in values for both `Width` and `Height`. See the command description for a list of valid resolutions.
* Height (int) - Optional. Sets the resolution height (in pixels) for the video recording. When you specify a resolution, you must pass in values for both `Width` and `Height`. See the command description for a list of valid resolutions.

### **StopRecordingVideo**

Stops recording video with Misty's 4K camera.

If you do not call the `StopRecordingVideo` command, Misty automatically stops recording after the duration for the recording has elapsed. You set this duration when you call the `StartRecordingVideo` command. The default video recording duration is 30 seconds.

{% hint style="success" %}

```
misty.stop_recording_video()
```

{% endhint %}


# Change/Remove Assets

Here you can find a set of API calls to change the name of your assets and remove them if need be.

### RenameVideoRecording

Renames an existing video recording.

**Note:** This command only renames a video recording that Misty has created. You cannot use this command to rename a user-uploaded video file.

**Example Code**

{% hint style="success" %}

```python
misty.rename_video_recording("myhomevideo","partyvideo")
```

{% endhint %}

**Parameters**

```python
misty.rename_video_recording(self, oldName : str = None, newName : str = None)
```

* OldName (string) - The current (old) filename of the video recording to rename, without the file type extension.
* NewName (string) - The new filename to associate with the video recording, without the file type extension. The name of a video recording can only include uppercase and lowercase alphanumeric characters, hyphens, and underscores (`[a-zA-Z0-9_-]`). Do not supply a file type extension; the system automatically uses the `.mp4` extension for Misty's video recordings.

### **DeleteAudio**

Deletes audio files on your Misty

**Example Code**

{% hint style="success" %}
misty.delete\_audio("examplesong.wav")
{% endhint %}

**Parameters**

```python
misty.delete_audio(self, fileName : str = None) -> Response:
```

* FileName (string) - The name of the file to delete, including its file type extension.

### **DeleteImage**

Deletes an image file from Misty's local storage.

**Example Code**

{% hint style="success" %}

```python
misty.delete_image("mybaddimage.png")
```

{% endhint %}

**Note:** This command cannot delete Misty's default image files.

**Parameters**

* FileName (string) - The name of the file to delete, including its file type extension.

```python
misty.delete_image(self, fileName : str = None) -> Response:
```

### **DeleteVideo**

Deletes a user-uploaded video file from Misty's storage.

**Example Code**

{% hint style="success" %}

```python
misty.delete_video("myvideo.mp4")
```

{% endhint %}

**Note:** This command only deletes user-uploaded video assets. To delete a video recording that Misty has created, you must use the [`DeleteVideoRecording`](https://docs.mistyrobotics.com/misty-ii/web-api/api-reference/#deletevideorecording) command.

**Parameters**

```python
misty.delete_video(self, fileName : str = None) -> Response:
```

* FileName (string) - The name of the video file to delete, with the file type extension.

### **DeleteVideoRecording**

Deletes a video recording.

**Note:** This command only deletes video recordings that Misty has created. To delete a user-uploaded video asset, you must use the [`DeleteVideo`](https://docs.mistyrobotics.com/misty-ii/web-api/api-reference/#deletevideo) command.

**Example Code**

{% hint style="success" %}

```python
misty.delete_video_recording("myhomevideo.mp4")
```

{% endhint %}

**Parameters**

* Name (string) - The filename of the video to delete. Does not include the filetype extension.

```python
misty.delete_video_recording(self, name : str = None) -> Response:
```

### **ForgetFaces**

Removes records of trained faces from Misty's memory.

**Example Code**

{% hint style="success" %}

```python
misty.forget_faces("12142")
```

{% endhint %}

**Parameters**

```python
misty.forget_faces(self, faceId : str = None)
```

* FaceId (string) - Optional. The ID of the face to remove. If you do not pass in a value for this parameter, clears all trained faces from Misty's memory.


# Stream Assets

In this section you can find Misty's APIs for streaming content to and from Misty.

### **StartVideoStreaming**

Starts Misty video streaming.

To use the websocket, you can connect to it from a browser using:ws\://\<robot-ip-address>:\<port>

Video streaming consumes extra resources, so when you are done using the stream, you should call the `StopVideoStreaming` command.

**Example Code**

{% hint style="success" %}

```python
misty.start_video_streaming(5678,0,0,0,100,"false")
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.start_video_streaming(self, port : int = None, rotation : int = None, width : int = None, height : int = None, quality : int = None, overlay : bool = None) -> Response:
```

{% endcode %}

* Port (int) - Optional. The websocket port. Must be between 1024 and 65535, default is 5678.
* Rotation (int) - Optional. Stream rotation. Must be between 0 and 180, default is 0.
* Width (int) - Optional. Must be between 200 and 1600. Default is 0 and does not adjust the image.
* Height (int) - Optional. Must be between 200 and 1600. Default is 0 and does not adjust the image.
* Quality (int) - Optional. Image quality, which can affect performance. Must be between 1 and 100. Default is 100.
* Overlay (bool) - Optional. Whether to add overlay information, like face tracking, to the stream. Default is false.

### **StopVideoStreaming**

Stops Misty video streaming.

{% hint style="success" %}

```python
misty.stop_video_streaming
```

{% endhint %}

### **StartAVStreaming**

Starts Misty streaming audio and video from her microphones and RGB camera to an external source.

**Important!** Misty's AV stream is **NOT** encrypted at this time. Devices on the same network as your robot (or in between your robot and the streaming server or client) can intercept the stream, play back the content, and re-publish the stream outside of your local network. Additionally, you are responsible for securing and encrypting any media content you choose to stream from Misty to services or devices outside of your local network.

Valid resolutions (as `width` x `height`) for AV streaming are: 1920 x 1280, 1280 x 960, 640 x 480, and 320 x 240.

* By default, Misty's AV streaming service is disabled when the robot boots up. You must enable this service before you can start AV streaming. You can enable the AV streaming service with the `EnableAvStreamingService` command.
* Enabling the AV streaming service automatically disables Misty's camera service. Misty cannot take pictures, record videos, or use computer vision functionality (such as face detection or face recognition) while the AV streaming service is enabled. For more information, see AV Streaming Service.
* Misty cannot use her microphones for wake word detection, recording audio, or capturing speech while actively streaming audio and video.
* Misty's AV streaming service is unidirectional at this time. You can stream audio and video from Misty to an external device, but the robot cannot play live media streams.
* Misty's video stream is rotated 90 degrees counterclockwise. You can rotate the stream to the orientation you prefer by changing the settings in your streaming client.
* This command is currently in **Beta**, and related hardware, firmware, or software is still under development. Feel free to use this command, but recognize that it may behave unpredictably at this time.

Misty supports the following modes for AV streaming:

* Misty can transmit a live audio and video data stream to an external media server that you configure to run on the same network as the robot. Misty supports streaming over Real-Time Messaging Protocol (RTMP) or Real Time Streaming Protocol (RTSP). You must create and host the media server yourself and configure the server to publish a stream you can view with a streaming client (like [VLC](https://www.videolan.org/vlc/)).
* Misty can serve an RTSP stream herself, and you can view the stream with a client connected to the same network as the robot. Misty's server can stream to clients that use TCP or UDP to receive AV streaming data.

**Note**: For lowest latency use RTSP instead of RTMP. To decrease latency further, adjust the network caching settings for your streaming client.

**Example Code**

{% hint style="success" %}

```python
// This example sets Misty up to act as her own media server. Connect
// to this stream from a client on the same network as Misty. The URL
// for this stream would be: rtsp://<robot-ip-address>:1936

misty.start_av_streaming("rtspd:1936", 640, 480)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.start_av_streaming(self, url : str = None, width : int = None, height : int = None, frameRate : int = None, videoBitRate : int = None, audioBitRate : int = None, audioSampleRateHz : int = None, userName : str = None, password : str = None)
```

{% endcode %}

* URL (string) - **Option 1**: If transmitting a stream from Misty to an external media server, this value is the URL address for the streaming server. This value **must** match the URL used when you set up the streaming server. You must prefix the URL with either `rtmp://` or `rtsp://`, depending on the streaming protocol you want to use. **Option 2:** To use Misty as her own media server, use `rtspd:<port-number>`, where `<port-number>` is the port through which to publish the stream (for example, `rtspd:1935`). When the stream is live, you can view it in your media client by connecting to `rtsp://<robot-ip-address>:<port-number>` (for example, `rtsp://192.168.7.30:1935`).
* Width (int) - Optional. The width (in pixels) of the video stream. The default resolution for video streaming (as `width` x `height`) is 1920 x 1080.
* Height (int) - Optional. The height (in pixels) of the video stream. The default resolution for video streaming (as `width` x `height`) is 1920 x 1080.
* FrameRate (int) - Optional. The frame rate at which Misty streams video. You must use a value greater than `1` and less than `30`. Default is `30`.
* VideoBitRate (int) - Optional. The bitrate (in bits per second) at which to encode streamed video data. Defaults to `5000000` (5 mbps). Valid values are between `256000` (256 kbps) and `20000000` (20 mbps).
* AudioBitRate (int) - Optional. The bitrate (in bits per second) at which to encode streamed audio data. Defaults to `128000` (128 kbps). Valid values are between `32000` (32 kbps) and `1000000` (1 mbps).
* AudioSampleRateHz (int) - Optional. The sample rate (in hz) at which to record audio for the audio stream. Defaults to `44100` (44.1 kHz). Supported sample rates include: `11025`, `12000`, `16000`, `22050`, `24000`, `32000`, `44100`, and `48000`.
* UserName (string) - Optional. The username a stream must supply to transmit media to your external server. Not all servers require a username and password. You can change whether to require credentials when you set up your server.
* Password (string) - Optional. The password for connecting to your external media server.

### StopAVStreaming

Stops Misty streaming audio and video.

{% hint style="success" %}

```
misty.stop_av_streaming
```

{% endhint %}


# Get Assets

In this section you can find all of Misty's API calls related to fetching information about her stored assets. To display a list of assets in the output field in the Python Interface you will need to use the syntax shown in the example below:

```python
from mistyPy.Robot import Robot

# Initialize Misty
misty = Robot()

# Get the list of videos
video_list = misty.get_video_list()

# Check if video list is retrieved successfully
if video_list is not None:
    # Print the list of videos
    print("List of Videos:")
    for video in video_list:
        print(video)
else:
    print("Failed to retrieve the video list.")
```

### **GetAudioFile**

Obtains a system or user-uploaded audio file currently stored on Misty.

**Example Code**

{% hint style="success" %}

```python
misty.get_audiofile("myaudio.mp4", false)
```

{% endhint %}

**Parameters**

```python
misty.get_audio_file(self, fileName : str = None, base64 : bool = None)
```

* FileName (string): The name of the audio file to get, including its file type extension.
* Base64 (boolean): Optional. Sending a request with `true` returns the audio file data as a downloadable Base64 string. Sending a request with `false` returns the audio file to your browser or REST client. Defaults to `false`.

### **GetAudioList**

Lists all audio files (default system files and user-uploaded files) currently stored on Misty.

{% hint style="success" %}

```
misty.get_audio_list()
```

{% endhint %}

### **GetImage**

Obtains a system or user-uploaded image file.

**Example Code**

{% hint style="success" %}

```python
misty.get_image("e_Amazement.jpg", false)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.get_image(self, fileName : str = None, base64 : bool = None)
```

{% endcode %}

* FileName (string) - The name of the image file to get, including the file type extension.
* Base64 (boolean) - Optional. Sending a request with `true` returns the image data as a downloadable Base64 string. Sending a request with `false` displays the image in your browser or REST client immediately after the image is taken. Default is `true`.

### **GetImageList**

Obtains a list of the images currently stored on Misty.

{% hint style="success" %}

```
misty.get_image_list()
```

{% endhint %}

### **GetVideo**

Obtains a user-uploaded video file currently stored on Misty.

**Note:** This command only obtains user-uploaded video assets. To obtain a video recording that Misty created, you must use the [`GetVideoRecording`](https://docs.mistyrobotics.com/misty-ii/web-api/api-reference/#getvideorecording) command.

**Example Code**

{% hint style="success" %}

```python
misty.get_video("myvideo.mp4", false)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.get_video(self, fileName : str = None, base64 : bool = None)
```

{% endcode %}

* FileName (string) - The name of the video to obtain, with the file type extension.
* Base64 (boolean) - Optional. Sending a request with `true` returns the video data as a Base64-encoded string. Sending a request with `false` downloads the video file to your REST client. Default is `false`.

### **GetVideoList**

Obtains a list of the user-uploaded video assets saved to Misty's storage.

**Note:** This command only obtains a list of user-uploaded video assets. To obtain a list of video recordings that Misty created, you must use the [`GetVideoRecordingsList`](https://docs.mistyrobotics.com/misty-ii/web-api/api-reference/#getvideorecordingslist) command.

{% hint style="success" %}

```python
misty.get_video_list()
```

{% endhint %}

### GetVideoRecording

Downloads a video that Misty has created. Optionally, returns the video data as a Base64 string.

You can only use this command to download videos that Misty recorded. To get videos that you or another user has uploaded, use the [`GetVideo`](https://docs.mistyrobotics.com/misty-ii/web-api/api-reference/#getvideo) command.

**Example Code**

{% hint style="success" %}

```python
misty.get_video_recording("myrecording.mp4",true)
```

{% endhint %}

**Tip:** Misty records videos in .mp4 format. Video recordings have a maximum resolution of 3840 x 2160 pixels and can be up to 3 minutes long. A single video file can be up to 225 MB and can take several minutes to download.

**Parameters**

```python
misty.get_video_recording(self, name : str = None, base64 : bool = None)
```

* Name (string) - Optional. The filename of the video to download. If not supplied, defaults to `misty_video`.
* Base64 (boolean) - Optional. Sending a request with `true` returns the video data as a Base64-encoded string. Sending a request with `false` downloads the video file to your REST client. Defaults to `false`.

### GetVideoRecordingList

Obtains a list of filenames for each video recording saved to Misty's local storage.

{% hint style="success" %}

```python
misty.get_video_recording_list()
```

{% endhint %}

### **GetKnownFaces**

Obtains a list of the names of faces on which Misty has been successfully trained.

{% hint style="success" %}

```
misty.get_known_faces()
```

{% endhint %}

### **GetConversations**

Obtains a list of conversations stored in Misty's memory.

{% hint style="success" %}

```
misty.get_conversations()
```

{% endhint %}

### GetLoadedContexts

Obtains a list of loaded contexts used for NLP.

{% hint style="success" %}

```
misty.get_loaded_contexts()
```

{% endhint %}

### **GetStates**

Obtains a list of states used in your conversation.

{% hint style="success" %}

```
misty.get_states()
```

{% endhint %}


# Events

In this section you will find all of Misty's API calls for triggering different events. You can upload the sample code for an event by loading the event sample template. If you need information on how to trigger events with bump and touch sensors you can check out [Sensor Events](/python-elements/misty-python-api/sensor-events).

<figure><img src="/files/zFx4oeNdu2rgOh8cve67" alt="" width="365"><figcaption></figcaption></figure>

### **StartFaceDetection**

Initiates Misty's detection of faces in her line of vision. This command assigns each detected face a random ID.

{% hint style="success" %}

```
misty.start_face_detection()
```

{% endhint %}

When you are done having Misty detect faces, call `StopFaceDetection`.

### **StopFaceDetection**

Stops Misty's detection of faces.

{% hint style="success" %}

```
misty.stop_face_detection()
```

{% endhint %}

### **StartFaceRecognition**

Directs Misty to recognize a face she sees, if it is among those she already knows. To use this command, you previously must have used either the `StartFaceDetection` command or the `StartFaceTraining` command to detect and store one or more face IDs in Misty's memory.

{% hint style="success" %}

```
misty.start_face_recognition()
```

{% endhint %}

When you are done having Misty recognize faces, call `StopFaceRecognition`.

### **StopFaceRecognition**

Stop Misty looking for face to recognize.

{% hint style="success" %}

```
misty.stop_face_recognition()
```

{% endhint %}

### **StartFaceTraining**

Trains Misty to recognize a specific face and applies a user-assigned ID to that face.

This process should take less than 15 seconds and will automatically stop when complete. To halt an in-progress face training, you can call `CancelFaceTraining`.

**Example Code**

{% hint style="success" %}

```python
misty.start_face_training("Simone")
```

{% endhint %}

**Parameters**

```python
misty.start_face_training(self, faceId : str = None)
```

* FaceId (string) - A unique string of 30 characters or less that provides a name for the face. Only alpha-numeric, -, and \_ are valid characters.

### **CancelFaceTraining**

Halts face training that is currently in progress. A face training session stops automatically, so you do not need to use the `CancelFaceTraining` command unless you want to abort a training that is in progress.

{% hint style="success" %}

```python
misty.cancel_face_training("Simone")
```

{% endhint %}

### **StartObjectDetector**

Starts Misty locating the position and type of objects.&#x20;

**Example Code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.start_object_detector(0.6,0,5)
```

{% endcode %}
{% endhint %}

When you issue a `StartObjectDetector` command, Misty will start streaming `ObjectDetection` events when an appropriate object is seen.

To receive those events, you must also register for the `ObjectDetection` event type.

Object detection consumes extra resources, so when you are done using the events, you should call the `StopObjectDetector` command.

You can find a list of known objects in [Known objects](/resource-database/known-objects)

**Parameters**

{% code overflow="wrap" %}

```python
misty.start_object_detector(self, minimumConfidence : float = None, modelId : int = None, maxTrackerHistory : int = None, delegateType : int = None)
```

{% endcode %}

* MinimumConfidence (double) - The minimum confidence to trigger sending an event. From 0 to 1.0.
* ModelId (int) - The TensorFlow Lite object model to use. Valid model ids are 0-3.
* MaximumTrackerHistory (int) - How long to hold previous object history across frames.

### **StopObjectDetector**

Stop Misty locating the position and types of objects.

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.stop_object_detector()
```

{% endcode %}
{% endhint %}

### **StartArTagDetector**

Starts Misty locating the position and values of Ar Tags.

**Example Code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.start_ar_tag_detector(7,144)
```

{% endcode %}
{% endhint %}

When you issue a `StartArTagDetector` command, Misty will start streaming `ArTagDetection` events when an appropriate tag is seen.

To receive those events, you must also register for the [`ArTagDetection`](https://docs.mistyrobotics.com/misty-ii/robot/sensor-data/#artagdetection) event type.

**Parameters**

{% code overflow="wrap" %}

```python
misty.start_ar_tag_detector(self, dictionary : int = None, tagSizeMm : float = None)
```

{% endcode %}

* Dictionary (int) - The Ar Tag dictionary to use.
* TagSizeMm (double) - The size of the printed tags. Used in assessing distance.

### **StopArTagDetector**

Stops Misty locating position and values of AR tags.

{% hint style="success" %}

```
misty.stop_ar_tag_detector()
```

{% endhint %}

### **StartQrTagDetector**

Starts Misty detecting QR tags.

{% hint style="success" %}

```
misty.start_qr_tag_detector()
```

{% endhint %}

### **StopQrTagDetector**

Stops Misty detecting QR tags.

{% hint style="success" %}

```
misty.stop_qr_tag_detector()
```

{% endhint %}

### **StartRobotInteractionEvent**

Initiates a robot interaction event in Misty, enabling her to engage in interactions that may involve vision data processing.

**Example code**

{% hint style="success" %}

```python
misty.start_robot_interaction_event(True)
```

{% endhint %}

**Paratmeters**

```python
misty.start_robot_interaction_event(self, useVisionData : bool = None)
```

* `useVisionData` (bool): Specifies whether the interaction event should utilize Misty's vision capabilities. When set to `True`, Misty will process visual data (such as facial recognition or object detection) as part of the interaction.

### **StopRobotInteractionEvent**

Stops the interaction event.

{% hint style="success" %}

```python
misty.stop_robot_interaction_event()
```

{% endhint %}


# Sensor Events

The following list contains the names of all sensor events' parameters. With these names, you can filter your events and build amazing functions and interactions with Misty!

For example in the bump sensor Python function, to select a specific bumper you need to create an if statement where you verify:&#x20;

```python
if data["message"]["sensorId"] == 'bfr':
    #actions
```

&#x20;On this page, you'll get all the information about the name of the parameter, the comparison operator and the comparison value:&#x20;

### ActuatorPosition&#x20;

```python
 ActuatorPosition Event:
        ArmLeft = ("SensorId", "=", "ala")
        ArmRight = ("SensorId", "=", "ara")
        HeadPitch = ("SensorId", "=", "ahp")
        HeadRoll = ("SensorId", "=", "ahr")
        HeadYaw = ("SensorId", "=", "ahy")
```

### BumpSensor

```python
BumpSensorPosition Event:
        BackLeft = ("SensorId", "=", "brl")
        BackRight = ("SensorId", "=", "brr")
        FrontLeft = ("SensorId", "=", "bfl")
        FrontRight = ("SensorId", "=", "bfr")
```

### CapTouch Sensor

```python
CapTouchPosition Event:
        Chin = ("SensorPosition", "=", "Chin")
        Scruff = ("SensorPosition", "=", "Scruff")
        Right = ("SensorPosition", "=", "HeadRight")
        Left = ("SensorPosition", "=", "HeadLeft")
        Back = ("SensorPosition", "=", "HeadBack")
        Front = ("SensorPosition", "=", "HeadFront")
```

### TimeOfFlight

**Distance**

```python
    TimeOfFlightDistance Event:
        MinDistance = ("DistanceInMeters", ">=", value)
        MinDistance = ("DistanceInMeters", ">=", value)
```

**Position**

```python
TimeOfFlightPosition Event:
        FrontLeft = ("SensorPosition", "=", "Left")
        FrontRight = ("SensorPosition", "=", "Right")
        FrontCenter = ("SensorPosition", "=", "Center")
        Back = ("SensorPosition", "=", "Back")
        DownwardBackLeft = ("SensorPosition", "=", "DownBackLeft")
        DownwardBackRight = ("SensorPosition", "=", "DownBackRight")
        DownwardFrontLeft = ("SensorPosition", "=", "DownFrontLeft")
        DownwardFrontRight = ("SensorPosition", "=", "DownFrontRight")
```

**Status**

```python
TimeOfFlightStatus Event:
        MinStatus = ("Status", ">=", value)
        MaxStatus = ("Status", "<=", value)    
        StatusEqual = ("Status", "=", value)
```

### Face Recognition

```python
FaceRecognition Event:
        Label = ("Label", "=", "your_name")
```

### Object Detection

```python
ObjectDetection Event:
Description = ("description","=","knownobject_name")
```


# Speech and NLP

In this section you can find all of the API calls related to triggering Misty's speech functions and utilizing her Natural Language Processing capabilities.

### **Speak**

Starts Misty speaking text using her onboard text-to-speech engine.

By default, Misty speaks in US English. You can find the full list of languages and their reference codes in [Languages](/resource-database/languages)

**Example Code**

{% hint style="success" %}

```python
misty.speak("Buongiorno, mi chiamo Misty", 1, 1, "it-it-x-itb-local")
```

{% endhint %}

To stop Misty speaking before she reaches the end of a text-to-speech utterance, use the `misty.StopSpeaking` command.

Misty raises a `TextToSpeechComplete` event when she finishes speaking a text-to-speech utterance. To receive a `TextToSpeechComplete` event message for a given utterance in your skills, you must set an `utteranceId` when you issue the `Speak` command, and you must register a listener for `TextToSpeechComplete` events.

The `Speak` command uses the text-to-speech (TTS) engine on Misty's 820 processor. At this time Misty's TTS engine supports a limited subset of [Speech Synthesis Markup Language (SSML) Version 1.0](https://www.w3.org/TR/2004/REC-speech-synthesis-20040907/).&#x20;

**Parameters**

{% code overflow="wrap" %}

```python
misty.speak(self, text : str = None, pitch : float = None, speechRate : float = None, voice : str = None, flush : bool = None, utteranceId : str = None, language : str = None)
```

{% endcode %}

* Text (string) - The text to speak, along with any relevant SSML tags to customize speech synthesis.
* Flush (bool) - Optional. Whether to flush all previously enqueued `Speak` commands. Default is `false`.
* UtteranceId (string) - Optional. An identifier of your choosing for this instance of the `Speak` command. You must set a value for `UtteranceId` in order to receive a [`TextToSpeechComplete`](https://docs.mistyrobotics.com/misty-ii/robot/sensor-data/#texttospeechcomplete) event when Misty stops speaking this utterance.

### **SpeakAndListen**

Triggers Misty to listen for speech after speaking.

**Example Code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.speak_and_listen("Hi,are you human?", true, none, "yes-no-questions")
```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.speak_and_listen(self, text : str = None, flush : bool = None, utteranceId : str = None, context : str = None)
```

{% endcode %}

* Text (string) - The text to speak, along with any relevant SSML tags to customize speech synthesis.
* Flush (bool) - Optional. Whether to flush all previously enqueued `Speak` commands. Default is `false`.
* UtteranceId (string) - Optional. An identifier of your choosing for this instance of the `Speak` command. You must set a value for `UtteranceId` in order to receive a `TextToSpeechComplete` event when Misty stops speaking this utterance.
* Context (string) - the name of the context file which Misty should reference when listening to speech

### **StopSpeaking**

Stops Misty speaking the currently playing text-to-speech utterance.

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.stop_speaking
```

{% endcode %}
{% endhint %}

### **StartKeyPhraseRecognition**

Starts Misty listening for the "Hey, Misty!" key phrase. Additionally, configures Misty to record speech she detects after recognizing the key phrase. Misty's chest LED blinks blue when she is recording audio or listening for the key phrase.

**Example Code**

{% hint style="success" %}

```python
misty.start_key_phrase_recognition(true,3000,false,600)
```

{% endhint %}

Misty waits to start recording until she detects speech. She then records until she detects the end of the utterance. By default, Misty records an utterance up to 7.5 seconds in length. You can adjust the maximum duration of a speech recording with the `MaxSpeechLength` parameter.

There are two event types associated with key phrase recognition:

* Misty triggers a `KeyPhraseRecognized` event each time she recognizes the "Hey, Misty" key phrase.
* Misty triggers a `VoiceRecord` event when she captures a speech recording.

**Note:** Misty cannot use her microphones for wake word detection or recording speech while actively streaming audio and video.

<details>

<summary>Additional Notes</summary>

* When you issue a `StartKeyPhraseRecognition` command, Misty listens for the key phrase by continuously sampling audio from the environment and comparing that audio to her trained key phrase model ("Hey, Misty!"). Misty does **not** create or save audio recordings until **after** she recognizes the key phrase.
* Because Misty cannot record audio and listen for the "Hey, Misty!" key phrase at the same time, she stops listening for the key phrase when issued a separate command to start recording audio. To have Misty start listening for the key phrase after capturing speech, you must issue another `StartKeyPhraseRecognition` command.
* When Misty recognizes the key phrase, she automatically stops listening for key phrase events. In order to start Misty listening for the key phrase again, you need to issue another `StartKeyPhraseRecognition` command.

Follow these steps to code Misty to respond to the "Hey, Misty!" key phrase:

1. Invoke the `StartKeyPhraseRecognition` command. If needed, use the optional parameters to configure Misty's speech capture settings.
2. Register an event listener for `KeyPhraseRecognized` event messages to trigger a callback function when Misty recognizes the key phrase.
3. Register an event listener for `VoiceRecord` event messages to trigger a callback function when Misty captures a speech recording.
4. Write the code to handle what Misty should do when she recognizes the key phrase and captures a speech recording. For example, you might have Misty send the captured speech off to a third-party service for additional processing.

</details>

**Parameters**

{% code overflow="wrap" %}

```python
misty.start_key_phrase_recognition(self, overwriteExisting : bool = None, silenceTimeout : int = None, maxSpeechLength : int = None, captureSpeech : int = None, speechRecognitionGrammar : str = None)
```

{% endcode %}

* CaptureSpeech (bool) - Optional. If `true`, Misty starts recording speech after recognizing the "Hey, Misty" key phrase. By default, Misty saves speech recordings under the filename `capture_HeyMisty.wav`. Defaults to `true`.
* MaxSpeechLength (int) - Optional. The maximum duration (in milliseconds) of the speech recording. If the length of an utterance exceeds this duration, Misty stops recording after the duration has elapsed, and the system triggers a VoiceRecord event with a message that Misty did not detect the end of the recorded speech. Range: 500 to 20000. Defaults to 7500 (7.5 seconds).
* OverwriteExisting (bool) - Optional. If `true`, the captured speech recording overwrites any existing recording saved under the filename `capture_HeyMisty.wav`. If `false`, Misty saves the speech recording under a unique, timestamped filename: `capture_HeyMisty_{Day}-{Month}-{Year}-{Hour}-{Minute}.wav`. Defaults to `true`. **Note:** If you program Misty to save each unique speech recording, you should occasionally delete unused recordings to prevent them from filling the memory on the robot's 820 processor.
* SilenceTimeout (int) - Optional. The maximum duration (in milliseconds) of silence that can precede speech before the speech capture mechanism times out. If Misty does not detect speech before the `SilenceTimeout` duration elapses, she stops listening for speech and triggers a `VoiceRecord` event with a message that she did not detect the beginning of speech. Range: `500` to `10000`. Defaults to `5000` (5 seconds).

### **StopKeyPhraseRecognition**

Stops Misty listening for the "Hey,Misty!" key phrase.

{% hint style="success" %}

```python
misty.stop_key_phrase_recognition
```

{% endhint %}

### **CaptureSpeech**

Starts capturing speech in a new audio recording. By default, Misty's chest LED pulses blue when she is recording audio or listening for the key phrase. Misty's head tally light also turns on when she is recording audio or video.

{% hint style="success" %}

```python
misty.capture_speech(true, 500, 2000, true)
```

{% endhint %}

Misty waits to start recording until she detects speech. She then records until she detects the end of the utterance. By default, Misty records an utterance up to 7.5 seconds in length. You can adjust the maximum duration of a speech recording by using the `MaxSpeechLength` parameter.

Misty triggers a [`VoiceRecord`](https://docs.mistyrobotics.com/misty-ii/robot/sensor-data/#voicerecord) event when she captures a speech recording.

**Parameters**

{% code overflow="wrap" %}

```python
misty.capture_speech(self, overwriteExisting : bool = None, silenceTimeout : int = None, maxSpeechLength : int = None, requireKeyPhrase : bool = None, speechRecognitionGrammar : str = None)
```

{% endcode %}

* OverwriteExisting (bool) - Optional. If `true`, the captured speech recording overwrites any existing recording saved under the default speech capture filename. (**Note:** Misty saves speech recordings she captures with this command under one of two default filenames: `capture_HeyMisty.wav` when `RequireKeyPhrase` is true, or `capture_Dialogue.wav` when `RequireKeyPhrase` is `false`.) If `OverwriteExisting` is `false`, Misty saves the speech recording under a unique, timestamped filename: `capture_{HeyMisty or Dialogue}_{Day}-{Month}-{Year}-{Hour}-{Minute}.wav` Defaults to `true`.&#x20;

**Note:** If you program Misty to save each unique speech recording, you should occasionally delete unused recordings to prevent them from filling the memory on her 820 processor.

* SilenceTimeout (int) - Optional. The maximum duration (in milliseconds) of silence that can precede speech before the speech capture mechanism times out. If Misty does not detect speech before the `SilenceTimeout` duration elapses, she stops listening for speech and triggers a `VoiceRecord` event with a message that she did not detect the beginning of speech. Range: `500` to `10000`. Defaults to `5000` (5 seconds).
* MaxSpeechLength (int) - Optional. The maximum duration (in milliseconds) of the speech recording. If the length of an utterance exceeds this duration, Misty stops recording after the duration has elapsed, and the system triggers a `VoiceRecord` event with a message that Misty did not detect the end of the recorded speech. Range: `500` to `20000`. Defaults to `7500` (7.5 seconds).
* RequireKeyPhrase (bool) - Optional. If `true`, Misty waits to start recording speech until she recognizes the key phrase. If `false`, Misty immediately starts recording speech. Defaults to `true`.

### **StartConversation**

**Example Code**

{% hint style="success" %}

```python
misty.start_conversation("FoodConversation")
```

{% endhint %}

**Parameters**

```python
misty.start_conversation(self, name : str = None)
```

* `name` (string): The unique name of the conversation.

### **StopConversation**

Stops the ongoing conversation.

{% hint style="success" %}

```
misty.stop_conversation()
```

{% endhint %}

### **StartDialog**

Initiates a dialogue session with Misty, enabling her to engage in interactive speech-based communication using pre-defined states and contexts.

{% hint style="success" %}

```python
misty.start_dialog("session_12345")
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.start_dialog(self, sessionId : str = None)
```

{% endcode %}

* `sessionId` (string): An optional identifier for the dialogue session. This can be used to manage or reference specific dialogue interactions, particularly useful in scenarios where multiple dialogue sessions might be occurring or tracked.

### **StopDialog**

Stops the ongoing dialog.

{% hint style="success" %}

```
misty.start_dialog(self)
```

{% endhint %}

### **ConfigureDialog**

Configures Misty's dialogue services, including natural language processing (NLP), automatic speech recognition (ASR), and text-to-speech (TTS), by setting up the necessary service providers and their respective access credentials and endpoints.

**Example code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.configure_dialog(nlpService="YourNLPService", nlpServiceKey="YourNLPKey", nlpServiceRegion="YourNLPRegion",
                       asrService="YourASRService", asrServiceKey="YourASRKey", asrServiceRegion="YourASRRegion",
                       ttsService="YourTTSService", ttsServiceKey="YourTTSKey", ttsServiceRegion="YourTTSRegion")

```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.configure_dialog(self, nlpService : str = None, nlpServiceKey : str = None, nlpServiceRegion : str = None, nlpServiceEndpoint : str = None, asrService : str = None, asrServiceKey : str = None, asrServiceRegion : str = None, asrServiceEndpoint : str = None, ttsService : str = None, ttsServiceKey : str = None, ttsServiceRegion : str = None, ttsServiceEndpoint : str = None
```

{% endcode %}

* `nlpService` (string): The name of the natural language processing service provider.
* `nlpServiceKey` (string): The access key for the NLP service.
* `nlpServiceRegion` (string): The region or location of the NLP service.
* `nlpServiceEndpoint` (string): The endpoint URL for the NLP service.
* `asrService` (string): The name of the automatic speech recognition service provider.
* `asrServiceKey` (string): The access key for the ASR service.
* `asrServiceRegion` (string): The region or location of the ASR service.
* `asrServiceEndpoint` (string): The endpoint URL for the ASR service.
* `ttsService` (string): The name of the text-to-speech service provider.
* `ttsServiceKey` (string): The access key for the TTS service.
* `ttsServiceRegion` (string): The region or location of the TTS service.
* `ttsServiceEndpoint` (string): The endpoint URL for the TTS service.

### **CreateConversation**

Defines a new conversation flow for Misty, setting up a structured sequence of dialog states and interactions.

**Example code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.create_conversation(name="FoodConversation", startingState="Foodstart", description="A simple greeting and chat conversation", useVisionData=True, overwrite=False)
```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.create_conversation(self, name : str = None, startingState : str = None, description : str = None, useVisionData : bool = None, overwrite : bool = None
```

{% endcode %}

* `name` (string): The unique identifier for the conversation. It's used to reference and manage the conversation within Misty's system.
* `startingState` (string): The name of the initial state from which the conversation begins. This state sets the stage for the conversation's flow.
* `description` (string): An optional description of the conversation's purpose and flow. This is useful for documentation and for understanding the conversation's design.
* `useVisionData` (bool): Determines whether the conversation should utilize data from Misty's vision capabilities, such as facial recognition or object detection. This allows for more interactive and responsive conversations based on visual cues.
* `overwrite` (bool): If set to `True`, any existing conversation with the same name will be overwritten. This is useful for updating or modifying existing conversations.

### **DeleteConversation**

Deletes your saved conversations.&#x20;

**Example code**

{% hint style="success" %}

```python
misty.delete_conversation("myconversation")
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.delete_conversation(self, name : str = None)
```

{% endcode %}

* `name` (string): The unique name of the conversation.

### **UpdateConversation**

Changes the name of your conversation and starting state.

**Example code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.update_conversation(currentName="FoodConversation", newName="TacoPizzaConversation",startingState="Foodstart", description="A simple conversation about Tacos and Pizza", useVisionData=True, overwrite=False)
```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.update_conversation(self, currentName : str = None, newName : str = None, startingState : str = None, useVisionData : bool = None, description : str = None)
```

{% endcode %}

* `currentName` (string): The current name of the conversation to be updated. This is the identifier that Misty uses to locate the existing conversation.
* `newName` (string): The new name for the conversation. This allows you to rename the conversation for clarity or organizational purposes.
* `startingState` (string): The new starting state for the conversation. Changing this alters the initial interaction or response when the conversation begins.
* `useVisionData` (bool): Indicates whether the updated conversation should utilize data from Misty's vision capabilities (like facial recognition or object detection). This can make the conversation more dynamic and responsive to visual inputs.
* `description` (string): A new description for the conversation. This is useful for detailing the purpose, changes, or flow of the updated conversation.

### **SetContext**

Configures Misty's speech recognition capabilities to understand and respond to specific phrases or words based on a given context.

**Example Code**

{% hint style="success" %}

<pre><code><strong>misty.set_context("yes-no-questions", "YesIntent, NoIntent", True, True)
</strong></code></pre>

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.set_context(self, context : str = None, filteredIntents : str = None, overlapContexts : bool = None, retrain : bool = None) 
```

{% endcode %}

* `context` (string): The name of the context file Misty should use when processing speech. This file defines the phrases or words Misty will recognize and respond to.
* `filteredIntents` (string): A comma-separated list of specific intents to filter from the context. Misty will only listen for and respond to these intents.
* `overlapContexts` (bool): Optional. Specifies whether the new context should overlap with any previously set contexts. If `True`, Misty considers both the new and existing contexts when recognizing speech. If `False`, Misty uses only the new context. Default is `False`.
* `retrain` (bool): Optional. Indicates whether Misty should retrain her speech recognition model with the new context. Setting this to `True` can improve accuracy but might require additional processing time. Default is `False`.

### **CreateState**

Sets up a customized behavioral state for Misty, defining how she should act, speak, and respond during conversations.

**Example Code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.create_state(name="TacoState", speak="Yeah, tacos are yummy, but pizza is way better", speakingAction="Wave", noMatchSpeech="I'm sorry,I didn't get that",repeatMaxCount=3)
```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.create_state(self, name : str = None, speak : str = None, followUp : str = None, audio : str = None, listen : bool = None, contexts : str = None, preSpeech : str = None, startAction : str = None, speakingAction : str = None, listeningAction : str = None, processingAction : str = None, transitionAction : str = None, noMatchAction : str = None, noMatchSpeech : str = None, noMatchAudio : str = None, repeatMaxCount : int = None, failoverState : str = None, retrain : bool = None, overwrite : bool = None, reEntrySpeech : str = None, filters : str = None, requiredContext : str = None)
```

{% endcode %}

* `name` (string): The unique name for the state being created.
* `speak` (string): Text or speech synthesis markup language (SSML) for Misty to speak when in this state.
* `followUp` (string): The next state for Misty to transition to after completing this state.
* `audio` (string): File name of an audio clip for Misty to play in this state.
* `listen` (bool): If `True`, Misty listens for speech input while in this state.
* `contexts` (string): Comma-separated list of speech recognition contexts to be active in this state.
* `preSpeech` (string): Text or SSML for Misty to speak before executing the primary speech command.
* `startAction` (string): The action for Misty to perform upon entering this state.
* `speakingAction` (string): The action for Misty to perform while speaking.
* `listeningAction` (string): The action for Misty to perform while listening.
* `processingAction` (string): The action for Misty to perform while processing input.
* `transitionAction` (string): The action for Misty to perform during state transitions.
* `noMatchAction` (string): The action for Misty to perform if no matching speech input is recognized.
* `noMatchSpeech` (string): Text or SSML for Misty to speak if no matching speech input is recognized.
* `noMatchAudio` (string): Audio file for Misty to play if no matching speech input is recognized.
* `repeatMaxCount` (int): Maximum number of times to repeat this state if no match is found.
* `failoverState` (string): The state to transition to if this state fails or is not matched.
* `retrain` (bool): If `True`, retrains Misty's speech model for this state.
* `overwrite` (bool): If `True`, overwrites any existing state with the same name.
* `reEntrySpeech` (string): Text or SSML for Misty to speak if re-entering this state.
* `filters` (string): Comma-separated list of filters to apply in this state.
* `requiredContext` (string): Context required for this state to be active.

### **StartState**

Initiates a specific state in Misty's behavior, optionally utilizing vision data for enhanced interaction.

**Example code**

{% hint style="success" %}

```python
misty.start_state(name="Foodstart", useVisionData=False)
```

{% endhint %}

**Parameters**

```python
misty.start_state(self, name : str = None, useVisionData : bool = None)
```

* `name` (string): The name of the state to be started. This state should be predefined in Misty's system.
* `useVisionData` (bool): Determines whether the started state should make use of Misty's vision capabilities, such as facial recognition or object detection. This allows for more dynamic and context-aware interactions.

### **MapState**

Defines the navigation and flow between different states within a specified conversation for Misty, allowing for detailed control over how Misty transitions from one state to another based on triggers and conditions.

**Example code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.map_state(conversation="FoodConversation", state="FoodStart", trigger="taco", nextState="TacoState")
```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.map_state(self, conversation : str = None, state : str = None, trigger : str = None, triggerFilter : str = None, nextState : str = None, detail : str = None, nextConversation : str = None, reEntry : bool = None, includeFollowUp : bool = None, overwrite : bool = None)
```

{% endcode %}

* `conversation` (string): The name of the conversation to which the state mapping belongs.
* `state` (string): The current state from which Misty will transition based on the defined trigger.
* `trigger` (string): The trigger that initiates the state transition. This could be a specific command, user response, or other input.
* `triggerFilter` (string): Additional filter criteria to refine how the trigger is evaluated. For example, categorizing user responses as "Positive" or "Negative".
* `nextState` (string): The state Misty transitions to upon the trigger being activated.
* `detail` (string): A description or detail about the state transition, useful for documentation or debugging.
* `nextConversation` (string): Optionally specify a different conversation to transition to, instead of just a new state within the current conversation.
* `reEntry` (bool): Indicates whether Misty can re-enter this state if the conditions are met again.
* `includeFollowUp` (bool): Specifies whether to include any follow-up action or response after transitioning to the new state.
* `overwrite` (bool): If set to `True`, this allows overwriting any existing state mapping with the same parameters.

### **RemoveMapState**

{% code overflow="wrap" %}

```python
misty.remove_map_state(self, conversation : str = None, state : str = None, trigger : str = None, triggerFilter : str = None, detail : str = None)
```

{% endcode %}

### **DeleteState**

Deletes a state from your context.

**Example code**

{% hint style="success" %}

```python
misty.delete_state("pizzastate")
```

{% endhint %}

**Parameters**

```python
misty.delete_state(self, name : str = None)
```

* `name` (string): The unique name of your state.

### **CreateAction**

Define and store custom actions for Misty.

**Example code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.create_action(name="question_action", script= "LED-PATTERN:0,0,255,40,0,112,1200,breathe;IMAGE:e_ApprehensionConcerned.jpg;ARMS:29,29,1000;HEAD:10,0,0,1000;", overwrite= True)
```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.create_action(self, name : str = None, script : str = None, overwrite : bool = None
```

{% endcode %}

* `name` (string): The unique name assigned to the action. This name is used to identify and execute the action in your code.
* `script` (string): A string that describes the sequence of commands that make up the action. The commands are separated by semicolons (`;`) and can include various actions such as LED patterns, image displays, arm movements, and head movements.
* `overwrite` (bool): An optional parameter that determines whether an existing action with the same name should be overwritten. If set to `True`, the new action replaces any existing action with the same name.

### DeleteAction

Deletes a specific action in Misty's memory.

**Example code**

{% hint style="success" %}

```python
misty.delete_action("question_action")
```

{% endhint %}

**Parameters**

```python
misty.delete_action(self, name : str = None)
```

`name` (string): The unique name of your action.

### **TrainNLPEngine**

Trains Misty's natural language processing (NLP) engine with specified contexts and intents.

**Example code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.train_nlp_engine(context="samplefood", intents=blob, save=True, overwrite=True)
```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.train_nlp_engine(self, context : str = None, intents : object = None, save : bool = None, overwrite : bool = None)
```

{% endcode %}

* `context` (string): The name of the context within which the NLP training will occur. This context groups together various intents and phrases that Misty should understand.
* `intents` (object): A dictionary or similar object containing the intents to be trained. Each intent is mapped to a list of phrases or utterances that exemplify that intent.
* `save` (bool): If set to `True`, the trained data is saved in Misty's system for future use. This is essential for persisting the training across different sessions or interactions.
* `overwrite` (bool): If `True`, existing training data for the specified context will be overwritten. This is useful for updating or refining Misty's NLP capabilities.

### **DeleteNLPContext**

Deletes  a saved context

**Example code**

{% hint style="success" %}

```python
misty.delete_nlp_context(context="samplefood")
```

{% endhint %}

**Parameters**

```python
misty.delete_nlp_context(self, context : str = None)
```

* `context` (string): The name of the context within which the NLP training will occur. This context groups together various intents and phrases that Misty should understand

### **PlayAndListen**

Directs Misty to play an audio file and then listen for a response or input, typically within a specified context.

**Example code**

{% hint style="success" %}

```python
misty.play_and_listen("s_Awe.wav","samplefood")
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.play_and_listen(self, audioFile : str = None, context : str = None)
```

{% endcode %}

* `audioFile` (string): The name of the audio file that Misty will play. This file should be preloaded or accessible to Misty. It can be a spoken phrase, question, or any other audio prompt.
* `context` (string): Optionally specifies the context within which Misty should listen and interpret the response after playing the audio. The context helps Misty understand the expected types of responses or commands and process them appropriately

### **RestoreNLPModel**

Restores Misty's natural language processing (NLP) model to its default state or a previously saved state. This function is used to revert any customizations or training that have been applied to Misty's NLP capabilities.

{% hint style="success" %}

```
misty.restore_nlp_model()
```

{% endhint %}

### **TriggerConversationEvent**

Triggers a specific event within Misty's conversation flow, allowing for manual control over the progression of a conversational interaction

**Example code**

{% hint style="success" %}

```python
misty.trigger_conversation_event(name="GreetUserEvent")
```

{% endhint %}

**Parameters**

```python
misty.trigger_conversation_event(self, name : str = None)
```

* `name` (string): The name of the conversation event to be triggered. This name corresponds to a predefined event within Misty's conversational capabilities.

***


# Arduino Backpack

Here you can find API calls to to set up communication between Misty, the Arduino backpack and any additional sensory hardware.

### GetSerialSensorValues <a href="#getserialsensorvalues" id="getserialsensorvalues"></a>

Obtains a list of the most recent messages Misty has received through the universal asynchronous receiver-transmitter (UART) serial port on her back. This list of messages clears each time the system reboots.

{% hint style="success" %}
misty.get\_serial\_sensor\_values()
{% endhint %}

### WriteSerial <a href="#writeserial" id="writeserial"></a>

Sends data to Misty's universal asynchronous receiver-transmitter (UART) serial port. Use this command to send data from Misty to an external device connected to the port.

Misty can also receive data through the UART serial port. To use this data you must subscribe to [`SerialMessage`](https://docs.mistyrobotics.com/misty-ii/robot/sensor-data/#serialmessage) events.

**Example code**

{% hint style="success" %}

```python
misty.write_serial("your-data")
```

{% endhint %}

**Parameters**

```python
misty.write_serial(self, message : str = None)
```

* `message` (string): The message or command to be sent through the serial port. This string is transmitted exactly as provided to any device connected to Misty's serial interface.


# System

This section contains a API calls to enable/disable parts of Misty's system.

### EnableCameraService

Enables the camera service running on Misty's 820 processor.

{% hint style="success" %}
misty.enable\_camera\_service()
{% endhint %}

For more information about disabling and enabling the camera service, see the `DisableCameraService` command description.

### DisableCameraService

Disables the camera service running on Misty's 820 processor.

Disabling a specific service frees up memory on the 820 processor for other tasks, and can improve the performance of other services that use the same processor. As an example, you may consider disabling the audio and camera services before you start mapping or tracking within a map to improve the performance of Misty's simultaneous localization and mapping (SLAM) activities.

{% hint style="success" %}
misty.disable\_camera\_service()
{% endhint %}

### CameraServiceEnabled

Describes whether the camera service running on Misty's 820 processor is currently enabled.

{% hint style="success" %}
misty.camera\_service\_enabled()
{% endhint %}

### EnableAudioService

Enables the audio service running on Misty's 820 processor.

{% hint style="success" %}
misty.enable\_audio\_service()
{% endhint %}

### DisableAudioService

Disables Misty's audio service.

{% hint style="success" %}
misty.disable\_audio\_service()
{% endhint %}

### EnableAVStreamingService

Enables Misty's AV streaming service for other API calls in [Stream Assets](/python-elements/misty-python-api/stream-assets)

{% hint style="success" %}
misty.enable\_av\_streaming\_service()
{% endhint %}

### DisableAVStreamingService

Disables Misty's AV streaming service.

{% hint style="success" %}
misty.disable\_av\_streaming\_service()
{% endhint %}

### EnableSlamService (Misty II Pro)

Enables the SLAM service running on Misty's 820 processor.

{% hint style="success" %}
misty.enable\_slam\_service()
{% endhint %}

For more information about disabling and enabling the SLAM service, see the `DisableSlamService` command description.

### DisableSlamService

Disables Misty's Slam service.

{% hint style="success" %}
misty.disable\_slam\_service()
{% endhint %}

### GetAvailableWiFiNetworks

Obtains a list of local Wi-Fi networks and basic information regarding each.

{% hint style="success" %}

```python
misty.get_available_wifi_networks
```

{% endhint %}

### ConnectWiFi

Connects Misty to a new WiFi.

**Example Code**

{% hint style="success" %}

```python
misty.connect_to_saved_wifi("dlink-5256","home1234")
```

{% endhint %}

**Parameters**

```python
misty.connect_wi_fi(self, networkName : str = None, password : str = None)
```

* NetworkId (string) - The name of the network to connect to.
* Password (string) - The password for the network.

### ConnectToSavedWiFi

Connects Misty to a saved Wi-Fi network.

{% hint style="success" %}

```python
misty.connect_to_saved_wifi("tp-link1234")
```

{% endhint %}

**Parameters**

```python
misty.connect_to_saved_wifi(self, networkId : int = None)
```

* NetworkId (string) - The name of the network to connect to.

### ForgetWiFi

Deletes information about a Wi-Fi network from Misty's list of saved networks. If you send this command without any parameters, Misty deletes information for all of her saved networks.

**Example Code**

{% hint style="success" %}

```python
misty.connect_to_save_wifi("tp-link1234")
```

{% endhint %}

**Parameters**

```python
misty.forget_wifi(self, networkId : int = None)
```

* NetworkId (string) - The name of the network to connect to.

### StartWifiHotspot <a href="#startwifihotspot" id="startwifihotspot"></a>

Starts Misty II broadcasting its own wireless network.

This command lets you use Misty II as a soft access point, which is useful in environments with no local networks, or networks that Misty can't connect to (such as captive networks).

{% hint style="success" %}

```
misty.start_wifi_hotspot
```

{% endhint %}

Follow these steps to use Misty as a WiFi hotspot:

1. Boot up the robot.
2. Issue a `StartWifiHotspot` command.
3. Issue a `GetDeviceInformation` command to access the network ID and password for Misty's access point. In the `GetDeviceInformation` response data, the network ID is stored in the `currentProfileName` field, and the password is stored in the `currentPreSharedKey` field. Use these credentials to connect your computer or another WiFi enabled device to Misty's access point.
4. Use Misty's standard IP address - `192.168.43.1` - to connect to the robot and issue commands from your connected device.
5. When you are finished using Misty as an access point, issue a `StopWifiHotspot` command.

**Tip:** If you plan to use Misty as a hotspot in an environment where you are unable to connect to any wireless networks, you may consider writing a JavaScript or .NET skill that runs on startup to issue the commands that create the access point. You can code Misty to display the credentials for the access point on her screen, or even to speak them out loud. Otherwise you must find a way to issue the HTTP API commands to start broadcasting WiFi over a separate network connection.

### StopWifiHotspot <a href="#stopwifihotspot" id="stopwifihotspot"></a>

Stops Misty II broadcasting its own wireless network.

{% hint style="success" %}

```
misty.stop_wifi_hotspot
```

{% endhint %}

To enable Misty as a soft access point, follow the steps in the documentation for the [`StartWifiHotspot`](https://docs.mistyrobotics.com/misty-ii/web-api/api-reference/#startwifihotspot) command.

### &#x20;GetLogFile

Obtains log file data.

{% hint style="success" %}

```python
misty.get_log_file(2024/1/1)
```

{% endhint %}

If no date is specified, pulls up to 3MB of the most recent log data from log files up to 14 days old. Log data returns in ascending order by date and time. If all log data exceeds 3MB, the oldest entry returned may be truncated.

If a date is specified, pulls up to 3MB of log data from that date. If log data from that date exceeds 3MB, the oldest entry may be truncated.

**Parameters**

```python
misty.get_log_file(self, date : datetime = None)
```

* `Date` (string) - Optional. The date of the log file to obtain. Dates must be formatted as: `MonthName/Date/FourDigitYear` or `FourDigitYear/MonthNumber/Date`.


# Slam

This section contains API calls for using the Occipital Core Structure sensors and Slam capabilities on Misty II Pro.&#x20;

### GetMap

Obtains the occupancy grid data for Misty's currently active map.To obtain a valid response from `GetMap`, Misty must first have successfully generated a map. To change the currently active map, use the [`SetCurrentSlamMap`](https://docs.mistyrobotics.com/misty-ii/web-api/api-reference/#setcurrentslammap) command.

<details>

<summary>Additional Notes</summary>

Misty's maps are squares that are constructed around her initial physical location when she starts mapping. When a map is complete, it is a square with Misty's starting point at the center.

The occupancy grid for the map is represented by a two-dimensional matrix. Each element in the occupancy grid represents an individual cell of space. The value of each element (0, 1, 2, or 3) indicates the nature of the space in those cells (respectively: "unknown", "open", "occupied", or "covered").

Each cell corresponds to a pair of X,Y coordinates that you can use with the `FollowPath`, `DriveToLocation`, and `GetSlamPath` commands. The first cell in the first array of the occupancy grid is the origin point (0,0) for the map. The X coordinate of a given cell is the index of the array for the cell. The Y coordinate of a cell is the index of that cell within its array.

</details>

{% hint style="success" %}

```
misty.get_map()
```

{% endhint %}

### GetCurrentSlamMap

Obtains the key for the currently active map.

{% hint style="success" %}

```
misty.get_current_slam_map()
```

{% endhint %}

### GetSlamIrExposureAndGain

Obtains the current exposure and gain settings for the infrared cameras in the Occipital Structure Core depth sensor.

{% hint style="success" %}

```
misty.get_slam_ir_exposure_and_gain()
```

{% endhint %}

### GetSlamVisibleExposureAndGain

Obtains the current exposure and gain settings for the fisheye camera in the Occipital Structure Core depth sensor.

{% hint style="success" %}

```
misty.get_slam_visible_exposure_and_gain()
```

{% endhint %}

### GetSlamMaps

Obtains a list of keys and names for Misty's existing maps.

**Example code**

{% hint style="success" %}
misty.get\_slam\_maps()
{% endhint %}

**Parameters**

### GetSlamNavigationDiagnostics

Obtains diagnostic information about Misty's navigation system.

The information in the data object for this command is primarily used by the Misty Robotics engineering and support staff to troubleshoot and root-cause issues with Misty's SLAM system. The contents of this data object are likely to change without notice in future system updates.

{% hint style="success" %}
misty.get\_slam\_navigation\_diagnostics()
{% endhint %}

### GetSlamPath

Obtain a path from Misty's current location to a specified set of X,Y coordinates. Pass the waypoints this command returns to the path parameter of `FollowPath` for Misty to follow this path to the desired location.

This command is not functional with the Misty II Basic Edition.

**Important!** Make sure to use `StartTracking` before using this command to have Misty start tracking her location, and use `StopTracking` to have her stop tracking her location after she arrives at the specified location

**Example code**

{% hint style="success" %}
{% code overflow="wrap" %}

```python
misty.get_slam_path(x=13,y=37)
```

{% endcode %}
{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.get_slam_path(self, x : int = None, y : int = None, minGap : float = None, wallCostDistance : float = None, unknownIsOpen : bool = None)
```

{% endcode %}

* X (integer) - The X coordinate of the destination.
* Y (integer) - The Y coordinate of the destination.

### GetSlamStatus

Obtains values representing the current activity and status of Misty's SLAM system. Check these values for information about the current status of Misty's depth sensor, the SLAM system, and to see information relevant to any ongoing mapping or tracking activities.

{% hint style="success" %}
misty.get\_slam\_status()
{% endhint %}

### SetCurrentSlamMap

Sets a map to be Misty's currently active map for tracking and relocalization.

**Example code**

{% hint style="success" %}

```python
misty.set_current_slam_map("Map_20190912_21.16.32.UTC")
```

{% endhint %}

**Parameters**

```python
misty.set_current_slam_map(self, key : str = None)
```

* Key (string) - The unique `key` of the map to make currently active. **Note:** This command does not work when passed the value for the `name` associated with a map.

### SetSlamIrExposureAndGain

Sets the exposure and gain settings for the infrared cameras in the Occipital Structure Core depth sensor.

**Example code**

{% hint style="success" %}

```python
misty.set_slam_ir_exposure_andGain(
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.set_slam_ir_exposure_andGain(self, exposure : float = None, gain : float = None)
```

{% endcode %}

* Exposure (double) - Exposure levels for the infrared cameras in the depth sensor (in seconds). Range: `0.001` - `0.033`.
* Gain (integer) - Gain levels for the infrared cameras in the depth sensor (in dB). Range: `0` - `3`.

### SetSlamVisibleExposureAndGain

Sets the exposure and gain settings for the fisheye camera in the Occipital Structure Core depth sensor.

**Example code**

{% hint style="success" %}

```python
misty.set_slam_visible_exposure_andGain(exposure=0.007987,gain=2)
```

{% endhint %}

**Parameters**

{% code overflow="wrap" %}

```python
misty.set_slam_visible_exposure_andGain(self, exposure : float = None, gain : float = None)
```

{% endcode %}

* Exposure (double) - Exposure levels for the fisheye camera in the depth sensor (in seconds). Range: `0.001` - `0.033`
* Gain (integer) - Gain levels for the fisheye camera in the depth sensor (in dB). Range: `1` - `8`

### RenameSlamMap

Renames an existing map.

**Example code**

{% hint style="success" %}

```python
misty.rename_slam_map("Map_20190911_21.47.16.UTC", "Kitchenmap2")
```

{% endhint %}

**Parameters**

```python
misty.rename_slam_map(self, key : str = None, name : str = None)
```

* `Key` (string) - The unique `key` value of the map to rename.
* `Name (`string) - A new `name` value for the map.

### ResetSlam

Resets Misty's SLAM sensors

{% hint style="success" %}
misty.reset\_slam()
{% endhint %}

### StartMapping

Starts Misty mapping an area.

Misty saves each map she creates to local storage. Each map is associated with a unique key at the time of the map's creation. Map keys are formatted as date timestamps in UTC (i.e. `Map_20190911_21.47.16.UTC`). To obtain a list of Misty's existing maps, use the [`GetSlamMaps`](https://docs.mistyrobotics.com/misty-ii/web-api/api-reference/#getslammaps) command.

{% hint style="success" %}
misty.start\_mapping()
{% endhint %}

### StopMapping

Stops Misty from mapping the environment.

{% hint style="success" %}
misty.stop\_mapping()
{% endhint %}

### DeleteSlamMap

Deletes a specified Slam map.

**Example code**

{% hint style="success" %}

```python
misty.delete_slam_map("Map_20190912_21.16.32.UTC")
```

{% endhint %}

**Parameters**

```python
misty.delete_slam_map(self, key : str = None)
```

* Key (string) - The unique `key` value of the map to delete. **Note:** This command does not work when passed the value for the `name` associated with a map.

### StartSlamStreaming

Opens the data stream from the Occipital Structure Core depth sensor, so you can obtain image and depth data when Misty is not actively tracking or mapping.

**Important!** Always use `StopSlamStreaming` to close the depth sensor data stream after sending commands that use Misty's Occipital Structure Core depth sensor. Using `StopSlamStreaming` turns off the laser in the depth sensor and lowers Misty's power consumption.

**Example code**

{% hint style="success" %}
misty.start\_slam\_streaming()
{% endhint %}

### StopSlamStreaming

Stops Misty's streaming data from the Occipital Structure Core depth sensor.

{% hint style="success" %}
misty.stop\_slam\_streaming()
{% endhint %}

### StartTracking

Enables Misty to start tracking the mapped enviornment.

{% hint style="success" %}

```
misty.start_tracking()
```

{% endhint %}

### StopTracking

Stops Misty from tracking the mapped environment.

{% hint style="success" %}

```
misty.stop_tracking()
```

{% endhint %}

### TakeDepthPicture

Provides the current distance of objects from Misty's Occipital Structure Core depth sensor. Note that depending on the scene being viewed, the sensor may return a large proportion of "unknown" values in the form of `NaN` ("not a number") values.

{% hint style="success" %}

```python
misty.take_depth_picture()
```

{% endhint %}

### TakeFisheyePicture

Takes a photo using Misty's Occipital Structure Core depth sensor..&#x20;

{% hint style="success" %}

```
misty.take_fisheye_picture()
```

{% endhint %}


# Python Definitions

Python, being a dynamically typed language, offers a variety of value types (data types) and keywords that are integral to its programming structure. Here's a list of some common value types and keywords along with their definitions:

### Value Types (Data Types)

1. **`int`**: Represents integer values. Example: `5`, `-3`.
2. **`float`**: Represents floating-point numbers (decimals). Example: `3.14`, `-0.001`.
3. **`str`**: String type for text. Example: `"Hello"`, `'Python'`.
4. **`bool`**: Boolean type, representing `True` or `False`.
5. **`list`**: An ordered, mutable (changeable) collection of items. Example: `[1, 2, 3]`, `['a', 'b', 'c']`.
6. **`tuple`**: An ordered, immutable collection of items. Example: `(1, 2, 3)`, `('a', 'b', 'c')`.
7. **`dict`**: Dictionary type, an unordered collection of key-value pairs. Example: `{'name': 'Alice', 'age': 25}`.
8. **`set`**: An unordered collection of unique items. Example: `{1, 2, 3}`.
9. **`NoneType`**: Special type representing the absence of a value or a null value. Example: `None`.

### Keywords

1. **`def`**: Used to define a function. Example: `def my_function():`.
2. **`return`**: Used to return a value from a function.
3. **`class`**: Used to define a class.
4. **`import`**, **`from`**: Used to import modules or specific functions, classes, etc., from modules.
5. **`as`**: Used in import statements to give an imported module a different alias, or with `try...except` blocks to rename exceptions.
6. **`if`**, **`elif`**, **`else`**: Conditional statements.
7. **`for`**, **`while`**: Loop control statements.
8. **`break`**: Exits a loop.
9. **`continue`**: Skips the current iteration in a loop.
10. **`try`**, **`except`**, **`finally`**, **`raise`**: Exception handling keywords.
11. **`with`**: Used for exception handling in resource management (like file reading/writing).
12. **`global`**: Declares a variable as global.
13. **`nonlocal`**: Declares a variable as non-local (useful in nested functions).
14. **`lambda`**: Used to create an anonymous function.
15. **`yield`**: Used in a function like `return`, but for generator functions.
16. **`del`**: Deletes an object.
17. **`pass`**: A null statement, a placeholder for future code.
18. **`assert`**: Used for debugging purposes to check conditions that should always be `True`.
19. **`in`**, **`not in`**: Membership operators.
20. **`is`**, **`is not`**: Identity operators.


# MistyGPT

Welcome to MistyGPT! Using this project template you can learn how to integrate ChatGPT in your Misty to have autonomous conversations and combine it with Langchain to give Misty verbal commands as well as ask for information. \
\
**Note:** Can only be used in a desktop environment, not compatible with the Misty Studio Python Interface.&#x20;

{% tabs %}
{% tab title="MistyGPT.py" %}
{% code overflow="wrap" %}

```python
import os
import sys
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time
import random
import requests

# Initialize Misty with your robot's IP address
misty = Robot("YOUR_IP_ADDRESS")
#Robot default
misty.set_default_volume(50)

#To install dependencies use pip install langchain_community,  pip install openai and pip install constants
import constants
from langchain_community.document_loaders import TextLoader
from langchain.indexes import VectorstoreIndexCreator
from langchain_community.llms import OpenAI
from langchain_community.chat_models import ChatOpenAI

os.environ["OPENAI_API_KEY"] = constants.APIKEY
print("API key loaded")

#query = sys.argv[1]

loader = TextLoader('Python-SDK-main\data.txt')
print("data.txt is loaded")
#loader = DirectoryLoader(".", glob="*.txt")
index = VectorstoreIndexCreator().from_loaders([loader])

def speech_captured(data):
    if data["message"]["step"] == "CompletedASR":
        user_input = data["message"]["text"]
        process_user_input(user_input)
        print(user_input)


def process_user_input(user_input):
    mistyOutput = index.query(user_input, llm=ChatOpenAI())
    #print(mistyOutput)
    moveArms = "move my arms"
    moveHead = "move my head"
    moveForward = "go forward"
    moveBackward = "go backward"
    moveForGesture1 = "intelligence"
    lowerVolume = "lower my volume"
    higherVolume = "higher my volume"
    changeDisplay= "change my display"
    print(mistyOutput)
    misty.speak_and_listen(mistyOutput)
    if moveForGesture1 in mistyOutput:
        misty.move_arms(-70, 50, 40, 40)
        time.sleep(1)
        print("left arm moved")
        misty.move_arms(50, 50, 40, 40)
    elif moveArms in mistyOutput:
        misty.move_arms(-50, -50, 40, 40)
        time.sleep(2)
        misty.move_arms(50, 50, 40, 40)
        print("arms moved")
    elif moveHead in mistyOutput:
        misty.move_head(0, -25, 0, 100, None, None)
        time.sleep(2)
        misty.move_head(0, 25, 0, 100, None, None)
        time.sleep(2)
        misty.move_head(0, 0, 0, 100, None, None)
        print("arms moved")
    elif moveForward in mistyOutput:
        misty.drive_time(5000,1,5000,0)
        print("moving forward")
    elif moveBackward in mistyOutput:
        misty.drive_time(-5000,1,5000,0)
        print("moving forward")
    elif lowerVolume in mistyOutput:
        misty.set_default_volume(50)
    elif higherVolume in mistyOutput:
        misty.set_default_volume(100)
    elif changeDisplay in mistyOutput:
        misty.display_image("e_JoyGoofy3.jpg")
        time.sleep(3)
        misty.display_image("e_EcstacyHilarious.jpg")
        time.sleep(3)
        misty.display_image("e_defaultcontent.jpg")

def recognized(data):
    print(data)  
    misty.speak("Yay, Hi " + data["message"]["label"], 1)
    misty.stop_face_recognition()
    time.sleep(2)
    misty.start_dialog()
    misty.speak_and_listen("How can I help you today", utteranceId="required-for-callback")

#If Misty is lifted she gets a bit touchy about that.
def touch_sensor(data):
    if data["message"]["sensorId"] == "cap" and data["message"]["isContacted"] == True:
        touched_sensor = data["message"]["sensorPosition"]
        print(touched_sensor)
        if touched_sensor == "Scruff":
            misty.play_audio("s_Rage.wav")
            misty.display_image("e_Anger.jpg")
            time.sleep(3)
           #Triggers face recognition event to initate ChatGPT
        if touched_sensor == "HeadFront": 
            misty.move_head( -5, 0, 0, 85, None, None)
            misty.display_image("e_Joy2.jpg")
            misty.speak("Aha")
            time.sleep(1)
            misty.start_face_recognition()
            #Stops ChatGPT event
        if touched_sensor == "Chin":
            misty.move_head(0, -50, 0, 150, None, None)
            misty.play_audio("s_Love.wav")
            misty.display_image("e_Love.jpg")
            time.sleep(2)
            misty.display_image("e_DefaultContent.jpg")
            misty.unregister_event("arbitrary-name")

misty.register_event(event_name="touch-sensor",
                     event_type=Events.TouchSensor,
                     callback_function=touch_sensor,
                     keep_alive=True)


misty.register_event(event_name="arbitrary-name",
                     event_type=Events.DialogAction,
                     callback_function=speech_captured,
                     keep_alive=True)

misty.register_event(event_name='face_recognition_event', 
                     event_type=Events.FaceRecognition, 
                     callback_function=recognized, 
                     keep_alive=False)

#misty.speak(index.query(query, llm=ChatOpenAI()))

x = 4
while (x > 3):
    misty.display_image("e_DefaultContent.jpg")
    misty.move_arms(30, 30, 40, 40)
    misty.move_head(0, 0, 0, 85, None, None)
    time.sleep(5)
    misty.display_image("e_ContentLeft.jpg")
    time.sleep(3)
    misty.move_arms(20, 10, 40, 40)
    time.sleep(2)
    misty.move_head(0, -10, 0, 60, None, None)
    time.sleep(5)
    misty.display_image("e_ContentRight.jpg")
    time.sleep(3)
    misty.move_head(0, 10, 0, 60, None, None)
    time.sleep(5)
    misty.move_arms(10, 20, 40, 40)
    

print("testing")
misty.keep_alive()
```

{% endcode %}
{% endtab %}

{% tab title="Constants.py" %}

```python
APIKEY = "YOUR_OPEN_AI_API_KEY"
```

{% endtab %}

{% tab title="Data.txt" %}

````
Limit your responses to 3 sentences unless you are explaining a theoretical concept

When asked about moving, tell some information about being a robot that can move. Maybe a joke.

Yes I can move my arms.
Yes I can move my head.
Yes I can go forward.
Yes I can go backward.
Yes I can lower my volume.
Yes I can higher my volume.
Yes I can change my display
Yes I can change my ChestLED
Yes I can play domo arrigato
Yes I can displayVideo
Yes I can do facerecognition

When not getting any input from user, say the following but do not ask questions about what the user knows or the users capabilities:
Can you say that again?
Please say that again
Sorry, I didnt get that
Are you still there?
Sorry I got distracted, can you say that again?
```
````

{% endtab %}
{% endtabs %}


# Misty Intruder Alert

In this project you can learn how to have Misty trigger an intruder alert if she doesn't recognize a person and send a message to your phone.&#x20;

If Misty detects a face that's not stored in her memory, she will access your WhatsApp web and send a text message to your phone number saying: "Intruder!". If she does recognize the person, she will send the message "person\_name is home!". Before executing the code, open <https://web.whatsapp.com/> and log in with your phone number.&#x20;

We will utilize the Python `pywhatkit` library for this purpose, which enables message sending on WhatsApp in a semi-automatic way. To install pywhatkit, use the following command line in your terminal: "`pip install pywhatkit`". Press enter and wait for the installation of all the required packages.\
\
Another required action prior to using Whatsapp is to train Misty on the faces you want her to recognize as friends (misty studio Explore>Vision>Train faces) and upload all the files you'll run in the code, both images and audio files (misty studio Explore>Expressions>Upload audio files or Upload images or videos).&#x20;

**Note:** Sometimes the WhatsApp message might not be sent before the website closes. In that case, try running the code again.\
\
To reset Misty after running the code, you can access its Misty Studio and in the wizard section, click on the "Body Reset" action.

It can only be used in a desktop environment and is not compatible with the Misty Studio Python Interface, so before initiating, ensure you have all the necessary resources for the task.

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time
import pywhatkit
from datetime import datetime

misty = Robot("YOUR_ROBOT_IP_ADDRESS") #your Robot IP Address

def send_whatsapp_message(MyNumber, MyTextMessage): #function to send a message on WhatsApp
    myobj = datetime.now()
    pywhatkit.sendwhatmsg(MyNumber, MyTextMessage, myobj.hour, myobj.minute + 1, 10, True)

MyNumber = "___" #your phone number (don't forget to include the country code)

misty.start_face_recognition()

def recognized(data):
    print(data)  
    if data["message"]["label"] == 'unknown person': #intruder
        misty.display_image("e_Anger.jpg")
        misty.transition_led(255, 0, 0, 0, 0, 255, "Blink", 100)
        misty.move_arms(-80, -80)
        misty.speak("Intruder! Intruder!")
        misty.play_audio("Police Siren Sound Effect.mp3", 50)
        MyTextMessage = "Intruder!"
        send_whatsapp_message(MyNumber, MyTextMessage)
    else :
        misty.display_image("e_Joy2.jpg") #familiar face
        misty.change_led(0, 255, 0)
        for i in range (2):
            misty.move_arms(80, -80, 50, 50)
            time.sleep(1)
            misty.move_arms(80, 0, 50, 50)
            time.sleep(1)
        MyTextMessage = data["message"]["label"] + " is home!"
        send_whatsapp_message(MyNumber, MyTextMessage)


misty.register_event(event_name='face_recognition_event', event_type=Events.FaceRecognition, callback_function=recognized, keep_alive=False)
misty.keep_alive()
```

\
You can use this link to access the Police Siren Sound Effect.mp3 file <https://youtu.be/HKieGUH9pzg>


# Conference Assistant

This project aims to make Misty a conference assistant.

This is a basic interpretation but her code can be enriched with many actions and sentences that can assist the speaker at a conference. This code will allow the speaker to control the presentation triggering Misty's bump sensors. If the rear right sensor is triggered Misty will move to the next slide, if the rear left sensor is triggered Misty will move to the previous slide. After you run the Python code from your computer open your presentation and wait for the magic to happen.

We have also added another command: if you press the front left sensor all the events will be stopped and your program can finish under your control.&#x20;

We will utilize the Python `pyautogui` library for this purpose, which enables our code to control the keyboard. To install pyautogui, use the following command line in your terminal: "`pip install pyautogui`". Press enter and wait for the installation of all the required packages.

It can only be used in a desktop environment and is not compatible with the Misty Studio Python Interface, so ensure you have all the necessary resources for the task before initiating.

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import pyautogui
import time

misty = Robot("YOUR_ROBOT_IP_ADDRESS") #your Robot IP Address

time.sleep(2)
def bumped(event):
    if(event["message"]["sensorId"] == "brr" and event["message"]["isContacted"] == True):
        misty.change_led(0, 255, 255)
        pyautogui.press("right")
        print("next")

    if(event["message"]["sensorId"] == "brl" and event["message"]["isContacted"] == True):
        misty.change_led(255, 0, 255)
        pyautogui.press('left')
        print("previous")

    if(event["message"]["sensorId"] == "bfl" and event["message"]["isContacted"] == True):
        misty.unregister_all_events()
        print("finished")

misty.register_event(event_name='bump_event', event_type=Events.BumpSensor, callback_function=bumped, keep_alive=True)
misty.keep_alive()

```

{% endcode %}


# QR code detector

With this feature it's possible for Misty to detect and read QR codes. These can be used to tag spaces, for example: labs, classrooms, kitchens or living rooms to make Misty start actions.

We suggest to use a printed QR code, about 15x15 cm. The QR code that we used for this project is the Misty Lessons one.

This project will only work in the Misty Desktop Environment because we will modify the events file.

Open the folder containing the Python-SDK that you use for the [desktop environment](https://github.com/MistyCommunity/Python-SDK).\
The folder should look like this one.

<figure><img src="/files/GrGYa0rpHOz6b2qC1iKC" alt="" width="563"><figcaption></figcaption></figure>

Open the folder MistyPy, open the file [Events.py](http://events.py/) in Visual Studio code and modify the Events class to add the QRdetection Event.<br>

<figure><img src="/files/kkNHzjgDDyg64DGxomx3" alt="" width="563"><figcaption></figcaption></figure>

Now you're ready to use Misty's scanning QR capabilities in Python.

#### Python Code

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events

misty = Robot("YOUR_ROBOT_IP_ADDRESS") #your Robot IP Address

misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)

misty.start_qr_tag_detector()

def code(data):
    print(data) #this shows all the data related to the event
    print(data['message']['decodedInfo'])

misty.register_event(event_name='qrtag_event', event_type=Events.QrTagDetection, callback_function=code, keep_alive=True, debounce = 500)

misty.keep_alive()
```

In the first part of the code, we set the robot by initializing the libraries, calling it and setting it in a standard position.\
Once these actions are done we start the event like any other event in the Misty world by calling the action .*start\_qr\_tag\_detector().*

We create a function that will handle the data read by scanning the QR code. We can then register the event and then keep it alive.

You can insert if statements to handle different QR codes and assign to each a different Misty action.

For example, if we want Misty to turn her RGB LED purple when she recognizes the QR code of the Misty lessons or have the  RGB LED red when she doesn't, our function can look like this one:&#x20;

```python
def code(data):
    print(data['message']['decodedInfo'])
    if data['message']['decodedInfo'] == 'unknown person': #intruder
        misty.change_led(100, 70, 160)
    else :
        misty.change_led(255, 0, 0)
```


# Misty follow human

Misty can track and follow you!

You can use this code to track and follow all [Misty's known objects](/resource-database/known-objects). In this example, Misty will follow just object 0  - "person".

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time
import math

misty = Robot("YOUR_ROBOT_IP_ADDRESS")
misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)

#modes
turn_in_place = True
follow_human = True

#constants
yaw_left = 81.36
yaw_right = -85.37
pitch_up = -40.10
pitch_down = 26.92

#variables
curr_head_pitch = 0
curr_head_yaw = 0

#Event handler for getting the current head position
def curr_position(data):
    global curr_head_pitch, curr_head_yaw
    if data["message"]["sensorId"] == "ahp":
        curr_head_pitch = data["message"]["value"]
        print(curr_head_pitch)
    if data["message"]["sensorId"] == "ahy":
        curr_head_yaw = data["message"]["value"]
        print(curr_head_yaw)

def get_pos():
    misty.register_event(event_name="get_curr_position", event_type= Events.ActuatorPosition, keep_alive= True, callback_function=curr_position)
    time.sleep(0.25)
    misty.unregister_event(event_name="get_curr_position")

# Event handler for person detection
def person_detection(data):
    print(data)
    if data["message"]["confidence"] >= 0.6:
        width_of_human = data["message"]["imageLocationRight"] - data["message"]["imageLocationLeft"]
        x_error = (160.0 - (data["message"]["imageLocationLeft"] + data["message"]["imageLocationRight"]) / 2.0) / 160.0
        # Use this for non-human tracking
        # y_error = (160.0 - ((data["message"]["imageLocationTop"] + data["message"]["imageLocationBottom"]) / 2.0)) / 160.0
        
        #Use this for human tracking
        y_error = (160.0 - 0.8 * data["message"]["imageLocationTop"] - 0.2 * data["message"]["imageLocationBottom"]) / 160.0

        threshold = max((0.3 if turn_in_place or follow_human else 0.2), (321.0 - width_of_human) / 1000.0)
        damper_gain = 5.0 if turn_in_place or follow_human else 7.0

        get_pos()
        actuate_to_yaw = curr_head_yaw + x_error * ((yaw_left - yaw_right) / damper_gain) if abs(x_error) > threshold else None
        actuate_to_pitch = curr_head_pitch - y_error * ((pitch_down - pitch_up) / 3.0) if abs(y_error) > threshold else None

        linear_velocity = 0
        angular_velocity = 0

        if actuate_to_yaw and abs(actuate_to_yaw) > 15 and (turn_in_place or follow_human):
            angular_velocity = math.copysign(min(abs(actuate_to_yaw) * 0.6, 25), actuate_to_yaw)

        if angular_velocity != 0:
            if math.copysign(1, actuate_to_yaw - curr_head_yaw) == math.copysign(1, angular_velocity):
                if abs(actuate_to_yaw) > 40:
                    actuate_to_yaw /= 1.5
            else:
                actuate_to_yaw = 0
                angular_velocity = 0
                if not follow_human:
                    misty.stop()

        if follow_human:
            if angular_velocity == 0:
                linear_velocity = (130 - width_of_human) * 0.5
                linear_velocity = min(abs(linear_velocity), 20) * math.copysign(1, linear_velocity)
                linear_velocity = linear_velocity if abs(linear_velocity) > 5 else 0
                misty.change_led(0, 255, 255)

        misty.move_head(actuate_to_pitch, None, actuate_to_yaw)
        if turn_in_place or follow_human:
            misty.drive(linear_velocity, angular_velocity)

misty.start_object_detector(0.5, 0, 15)
misty.register_event(event_name="personDetection", event_type= Events.ObjectDetection, callback_function=person_detection, debounce=500, keep_alive=True)

misty.keep_alive()
```

#### Modes

There are three different modes that you can use in this Python code (Misty's head will always move):&#x20;

```
Mode 1: No driving ("turnInPlace" : false, "followHuman" : false)
Mode 2: Allow turning in place, but not driving forward/backward ("turnInPlace" : true, "followHuman" : false)
Mode 3: Allow full driving ("turnInPlace" : true, "followHuman" : true)
```

#### Constants

Those constants are used to calibrate the maximum range of movement of Misty's head. \
Those should be equal for all Misty's but it would be preferred to check them before you run your code to have better accuracy.

To do it you can use this code:

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot("YOUR_ROBOT_IP_ADDRESS")
misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)

#constants
yaw_left = 0
yaw_right = 0
pitch_up = 0
pitch_down = 0

print(------)
#get current position functions
def curr_position(data):
    global ------ 
    if data["message"]["sensorId"] == "ahp":
        ------ = data["message"]["value"]
        time.sleep(0.5)
        print(------)

def get_pos():
    misty.move_head(90, 0, 0)
    time.sleep(5)
    misty.register_event(event_name="get_curr_position", event_type= Events.ActuatorPosition, keep_alive= True, callback_function=curr_position)
    time.sleep(0.25)
    misty.unregister_event(event_name="get_curr_position")

get_pos()
```

Replace in the marked space (------) the name of the position that you want to check and adjust the move\_head parameters and the type of data according to the following table

| Position    | move\_head            | SensorID |
| ----------- | --------------------- | -------- |
| yaw\_left   | move\_head(0, 0, 90)  | "ahy"    |
| yaw\_right  | move\_head(0, 0, -90) | "ahy"    |
| pitch\_up   | move\_head(-90, 0, 0) | "ahp"    |
| pitch\_down | move\_head(90, 0, 0)  | "ahp"    |

For example, if you want to check the position of the yaw left your code should look like this one:

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot("YOUR_ROBOT_IP_ADDRESS")
misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)

#constants
yaw_left = 0
yaw_right = 0
pitch_up = 0
pitch_down = 0

print(yaw_left)
#get current position functions
def curr_position(data):
    global yaw_left 
    if data["message"]["sensorId"] == "ahy":
        yaw_left = data["message"]["value"]
        time.sleep(0.5)
        print(yaw_left)

def get_pos():
    misty.move_head(0, 0, 90)
    time.sleep(5)#give Misty enough time to achieve the position
    misty.register_event(event_name="get_curr_position", event_type= Events.ActuatorPosition, keep_alive= True, callback_function=curr_position)
    time.sleep(0.25)
    misty.unregister_event(event_name="get_curr_position")

get_pos()
```

Then you can replace this number in the constants of the main code.

The variables will contain Misty's current head position and the following functions are used:

* Record the position and get it as a number memorized in the variables and follow the human.
* Detect and follow the person.

The math in the code is to better handle the person's position and adapt Misty's movements.


# Misty wave back

Misty can find you and react to your movements!

In this skill, Misty uses two onboard AI Capabilities

1. Object Detection
2. Human Pose Estimation

Object Detection is used to make Misty look at the closest person. For this specific skill, I only wanted Misty to find Human. Hence I specifically look for the 1st human object and ignore the rest.

Human pose estimation is used to detect the \~waving arm gesture. The event provides 16 keypoints per message like nose, eye, ear, shoulder, elbow, wrist, hip, ankle etc.. Using these keypoints, logic can be built to detect specific gestures.&#x20;

In this case, the logic will be:

```javascript
Elbow is lower than Shoulder && Shoulder is lower than Wrist
```

This project will only work in the Misty Desktop Environment because we will modify the events file, exactly like we did for the QR code detector.

Open the folder containing the Python-SDK that you use for the [desktop environment](https://github.com/MistyCommunity/Python-SDK). \
The folder should look like this one.

<figure><img src="/files/gSBqsSf9tT2Dr0MDrC14" alt="" width="563"><figcaption></figcaption></figure>

Open the folder MistyPy, open the file [Events.py](http://events.py/) in Visual Studio code and modify the Events class to add the PoseEstimation Event.

<figure><img src="/files/RJweqNyH4oVQ0e5xzPpa" alt="" width="563"><figcaption></figcaption></figure>

Now you're ready to use Misty's Human Pose Estimation capabilities in Python.

#### Constants

Since Misty will have to track your face, including some constants about Misty's head's maximum range of movements will be necessary.&#x20;

To get those constants you can use the same code as the one in the [Misty follow human](/python-projects/misty-follow-human) project in the [Misty follow human](/python-projects/misty-follow-human#constants)section.&#x20;

### Python code

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time
import math
import random

# Initialize Misty
misty = Robot("YOUR_ROBOT_IP_ADDRESS")
misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)
misty.display_image("e_DefaultContent.jpg")

# Constants
yaw_left = 81.36
yaw_right = -85.37
pitch_up = -40.10
pitch_down = 26.92

# Variables
curr_head_pitch = 0
curr_head_yaw = 0
waving_now = False
person_width_history = [0, 0, 0, 0]

#Event handler for getting the current head position
def curr_position(data):
    global curr_head_pitch, curr_head_yaw
    if data["message"]["sensorId"] == "ahp":
        curr_head_pitch = data["message"]["value"]
    if data["message"]["sensorId"] == "ahy":
        curr_head_yaw = data["message"]["value"]

def get_pos():
    misty.register_event(event_name="get_curr_position", event_type= Events.ActuatorPosition, keep_alive= True, callback_function=curr_position)
    time.sleep(0.25)
    misty.unregister_event(event_name="get_curr_position")

#Event handler for analyzing the human pose
def human_pose(data):
    global waving_now
    print("starting pose estimation")
    keypoints = data["message"]["keypoints"]
    
    # 5,6- Shoulder 7,8- Elbow  9,10- Wrist 

    if (waving_now == False):
        #left hand
        if (confident(keypoints[7]) and confident(keypoints[5]) and confident(keypoints[9])):
            if (pair_correlation(keypoints[7],keypoints[5]) and pair_correlation(keypoints[5],keypoints[9])):
                if (scale_valid(keypoints[7],keypoints[5])):
                    waving_now = True
                    wave_back("left")
                    
        #right hand
        elif (confident(keypoints[8]) and confident(keypoints[6]) and confident(keypoints[10])):
            if (pair_correlation(keypoints[8],keypoints[6]) and pair_correlation(keypoints[6],keypoints[10])):
                if (scale_valid(keypoints[8],keypoints[6])):
                    waving_now = True
                    wave_back("right")       

# Functions helper for the human pose
def scale_valid(keypoint_one, keypoint_two):
    x_offset = keypoint_one["imageX"] - keypoint_two["imageX"]
    y_offset = keypoint_one["imageY"] - keypoint_two["imageY"]
    return math.sqrt(x_offset**2 + y_offset**2) > 60

def confident(data):
    return data["confidence"] >= 0.6

def pair_correlation(keypoint_one, keypoint_two):
    return keypoint_one["imageY"] > keypoint_two["imageY"]

def wave_back(arm):
    global waving_now
    if arm == "left":
        print("Waving back left")
        misty.play_audio("s_Acceptance.wav")
        misty.display_image("e_Joy2.jpg")
        misty.transition_led(0, 90, 0, 0, 255, 0, "Breathe", 800)
        misty.move_arms(80, -89)
        time.sleep(1)
        misty.move_arms(80, 0)
        time.sleep(0.75)
        misty.move_arms(80, -89)
        time.sleep(0.75)
    else :
        print("Waving back right")
        misty.play_audio("s_Awe.wav")
        misty.display_image("e_Love.jpg")
        misty.transition_led(90, 0, 0, 255, 0, 0, "Breathe", 800)
        misty.move_arms(-89, 80)
        time.sleep(1)
        misty.move_arms(0, 80)
        time.sleep(0.75)
        misty.move_arms(-89, 80)
        time.sleep(0.75)

    time.sleep(1.5)
    misty.display_image("e_DefaultContent.jpg")
    misty.transition_led(0, 40, 90, 0, 130, 255, "Breathe", 1200)
    misty.move_arms(random.randint(70, 89), random.randint(70, 89))
    time.sleep(1.5)
    waving_now = False

# Human pose estimation event
def start_human_pose_estimation():
    misty.start_pose_estimation(0.2, 0, 1)
    misty.register_event(event_name="pose_estimation", event_type=Events.PoseEstimation, keep_alive= True, callback_function= human_pose)

#Event handler for analyzing person detection
def person_detection(data):
    if data["message"]["confidence"] >= 0.6:
        print("person detected")
        
        width_of_human = data["message"]["imageLocationRight"] - data["message"]["imageLocationLeft"]
        person_width_history.pop(0)
        person_width_history.append(width_of_human)
        
        # The first part checks if this measurement is the closest person and the second part checks if there is only one person that Misty can see
        if abs(width_of_human - min(person_width_history)) > abs(width_of_human - max(person_width_history)) or std_deviation(person_width_history) <= 40:
            x_error = (160.0 - (data["message"]["imageLocationLeft"] + data["message"]["imageLocationRight"]) / 2.0) / 160.0
            y_error = (160.0 - 1.4 * data["message"]["imageLocationTop"] + 0.2 * data["message"]["imageLocationBottom"]) / 160.0
            threshold = max(0.1, (341.0 - width_of_human) / 1000.0)

            get_pos()
            actuate_to_yaw = curr_head_yaw + x_error * ((yaw_left - yaw_right) / 6.0) if abs(x_error) > threshold else None
            actuate_to_pitch = curr_head_pitch - y_error * ((pitch_down - pitch_up) / 3.0) if abs(y_error) > threshold else None
        
            if abs(curr_head_pitch - round(actuate_to_pitch)) > 11 or abs(curr_head_yaw - round(actuate_to_yaw)) > 11:
                misty.move_head(actuate_to_pitch, None, actuate_to_yaw)

# Functions helper for person detection
def std_deviation(array):
    mean_value = sum(array) / len(array)
    return math.sqrt(sum([(x - mean_value) ** 2 for x in array]) / len(array))

# Person tracking event
def start_person_tracking() :
    misty.start_object_detector(0.5, 0, 15)
    misty.register_event(event_name="person_detection", event_type= Events.ObjectDetection, callback_function=person_detection, keep_alive=True)

# Start program
start_person_tracking()
start_human_pose_estimation()
misty.keep_alive()
```

In this code, every group of functions or variables is explained in their use.

As always the first steps are declaring the libraries, initializing the robot and the constants.

Right after it's used the same couple of functions are used in [Misty follow human](/python-projects/misty-follow-human) to get the current Misty's head's values.

Then you can find the logic behind the Human Pose Estimation event. \
These are the 16 keypoints that Misty can recognize:

* NOSE(0)
* LEFT\_EYE(1)
* RIGHT\_EYE(2)
* LEFT\_EAR(3)
* RIGHT\_EAR(4)
* LEFT\_SHOULDER(5)
* RIGHT\_SHOULDER(6)
* LEFT\_ELBOW(7)
* RIGHT\_ELBOW(8)
* LEFT\_WRIST(9)
* RIGHT\_WRIST(10)
* LEFT\_HIP(11)
* RIGHT\_HIP(12)
* LEFT\_KNEE(13)
* RIGHT\_KNEE(14)
* LEFT\_ANKLE(15)
* RIGHT\_ANKLE(16)

Example of Data Received under each KeyPoint&#x20;

bodyPart: 0 \
confidence: 0.3205725 \
imageX: 191 \
imageY: 253 \
pitch: 0.003858468 \
yaw: -0.126723886

After recognizing the logic it's time to animate Misty and you can modify it in the wave\_back function.

In the person detection function the first part attempts to look just at the closest person when multiple people are in front of Misty, while the second adjusts Misty's head position.&#x20;

The last lines of code start the whole program and keep it alive.

```
There are lots of magic numbers in the data received from these functions!
Play with it!
```


# Misty OA

With the Obstacle Avoidance skill Misty will be able to drive and navigate your space without difficulties!

The idea behind this project is as simple as possible: if Misty detects a shorter distance between her sensors and a flat surface, for example, shorter than 40 cm, she changes direction and tries to avoid the obstacle by turning, going straight forward for a short distance, turning and checking the new distance.

To do so it will be necessary to use the TOF (Time of Flight) event that contains all the measurements of Misty's distance sensors.

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot("YOUR-ROBOT-IP-ADDRESS")
misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)
misty.display_image("e_DefaultContent.jpg")

def TOF(data):
    if data["message"]["sensorPosition"] == "Center": #gets the central sensor distance
        dist = data["message"]["distanceInMeters"]
        print(dist)
        time.sleep(0.5)
        if dist < 0.4: # checks if it is less than 40cm
            misty.display_image("e_Contempt.jpg")
            misty.play_audio("s_PhraseOopsy.wav")
            print("obstacle")
            misty.drive_time(0, 50, 4500)
            time.sleep(5)
            misty.drive_time(20, 0, 2000)
            time.sleep(3)
            misty.drive_time(0, -50, 4500)
            time.sleep(5)
            misty.drive_time(0, 0, 2000)
            time.sleep(2)
            # Unregister the event to clean the data
            misty.unregister_event("TOF_event")
            # Registers the new event that will do the same
            misty.register_event(event_name='TOF_event_obstacle', event_type=Events.TimeOfFlight, callback_function=TOF_obstacle, keep_alive=True, debounce = 100)
            time.sleep(2)
        else :
            misty.drive(10, 0)
            print("free")

def TOF_obstacle(data):
    if data["message"]["sensorPosition"] == "Center":
        dist = data["message"]["distanceInMeters"]
        print(dist)
        time.sleep(0.5)
        if dist < 0.4:
            misty.display_image("e_Contempt.jpg")
            misty.play_audio("s_PhraseOopsy.wav")
            print("obstacle_obst")
            misty.drive_time(0, 50, 4500)
            time.sleep(5)
            misty.drive_time(20, 0, 2000)
            time.sleep(3)
            misty.drive_time(0, -50, 4500)
            time.sleep(5)
            misty.drive_time(0, 0, 2000)
            time.sleep(2)
            misty.unregister_event("TOF_event_obstacle")
            misty.register_event(event_name='TOF_event', event_type=Events.TimeOfFlight, callback_function=TOF, keep_alive=True, debounce = 100)
            time.sleep(2)
        else :
            misty.drive(10, 0)
            print("free_obst")

misty.register_event(event_name='TOF_event', event_type=Events.TimeOfFlight, callback_function=TOF, keep_alive=True, debounce = 100)
misty.keep_alive()
```

The reason why there are two functions that look almost exactly the same is related to an hidden issue in this project: the old distances.

After Misty detects an obstacle a time.sleep() command is sent to allow her to comfortably stop and proceed to the next action. During this time the TOF event is still active and this means that Misty keeps recording the distances, after entering the if statement which corresponds to the obstacle detected gate. When Misty will start recording the new measures, these will be the old ones and the skill will not work according to the reality.  &#x20;

The easiest way to get rid of them is simply to unregister the whole event, so every information about it will be deleted. After we register a completely new event, with a similar callback function, where only the name is different, that will keep Misty's behaviour the same way. Then when Misty meets a new obstacle she'll move and then she will unregister this second event (the first event is called "TOF\_event", the second "TOF\_event\_obstacle") and go back to the previous one.

With this flow, the two functions and events loop without bothering each other.

In the code, after detecting an object, Misty turns to the left, travels for two seconds forward, and then turns to the right to be in the correct position for the next check.

To make Misty turn to the right you can replace the misty.drive\_time() commands with these:

```python
misty.drive_time(0, -50, 4500)
time.sleep(5)
misty.drive_time(20, 0, 2000)
time.sleep(3)
misty.drive_time(0, 50, 4500)
time.sleep(5)
misty.drive_time(0, 0, 2000)
time.sleep(2)
```

Enjoy your Obstacle Avoidance skill with Misty, personalize it and share it to the world!


# Get weather

Learn how Misty can interact with the Web, request data and analyze them!

In this project, Misty will use the command send\_external request to connect with [Weatherstack](https://weatherstack.com/) and request your city's data.

```python
from mistyPy.Robot import Robot
import json

misty = Robot("YOUR_ROBOT_IP_ADDRESS")
misty.set_default_volume(20)
misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)
misty.display_image("e_Joy.jpg")

access_key = "<your-weatherstack-key>" #follow instructions to get your API key
query = "<your_city>" #for example: Stockholm

data = misty.send_external_request("GET", "http://api.weatherstack.com/current?access_key="+ access_key+"&query="+ query)
if data.status_code == 200: # success code
    parsed = json.loads(data.text) # transform the result in json

    #locate your variables
    weather_descriptions = parsed["current"]["weather_descriptions"][0]
    city = parsed["location"]["name"]
    temperature = parsed["current"]["temperature"]

    #create Misty animations with your data!
    print(f"Just letting you know it's {temperature} and {weather_descriptions} in {city}")
    misty.speak(f"Just letting you know it's {temperature} and {weather_descriptions} in {city}")
else:
    print("Failed to get data")
```

In the first lines of Misty's code, we import the usual libraries and the JSON one.

We need to import this library because the data from Weatherstack are sent in that format.

After initializing the Misty robot and the other variables, like the access\_key and the query we run the misty.send\_external\_request command that will use the GET method to request the weather data in your city.

The following statement tests if the request succeeded and if yes proceeds to organize the data and assign the value to the variables that you'll use in your Misty's animations. Otherwise, you'll get an error message and Misty's skill will finish.

### Weatherstack key

Weatherstack is an online weather data API service that provides real-time weather information, historical data, and weather forecasts for various locations around the world. It is commonly used by developers to integrate weather data into applications and websites. The service is known for its reliability, ease of use, and wide coverage, making it popular for various weather-based applications and projects.

* Navigate in [Weatherstack](https://weatherstack.com/)
* Click on "START USING THE API"
* Sign up with your credentials and your plan (we used the free version, which gives only 250 API requests)
* Click on "VISIT DASHBOARD"
* and you'll see your API Key with many other information.

### Weatherstack data

In this example, we requested the weather in Stockholm.

There are three main areas:&#x20;

* request: contains the information about your request
* location: contains the information about your location like the name of the city, latitude, longitude and local time when the request was sent
* current: contains the information about the weather in your city like temperature, weather descriptions, wind details, humidity and many more data

```json
{
  "request": {
    "type": "City",
    "query": "Stockholm, Sweden",
    "language": "en",
    "unit": "m"
  },
  "location": {
    "name": "Stockholm",
    "country": "Sweden",
    "region": "Stockholms Lan",
    "lat": "59.333",
    "lon": "18.050",
    "timezone_id": "Europe/Stockholm",
    "localtime": "2024-10-04 09:52",
    "localtime_epoch": 1728035520,
    "utc_offset": "2.0"
  },
  "current": {
    "observation_time": "07:52 AM",
    "temperature": 10,
    "weather_code": 113,
    "weather_icons": [
      "https://cdn.worldweatheronline.com/images/wsymbols01_png_64/wsymbol_0001_sunny.png"
    ],
    "weather_descriptions": [
      "Sunny"
    ],
    "wind_speed": 6,
    "wind_degree": 297,
    "wind_dir": "WNW",
    "pressure": 1020,
    "precip": 0,
    "humidity": 67,
    "cloudcover": 6,
    "feelslike": 9,
    "uv_index": 3,
    "visibility": 10,
    "is_day": "yes"
  }
}
```

You can refer to the data in Python in this way:

```python
parsed = json.loads(data.text)

variable = parsed["<area_name>"]["detail_name"]
```

If you want to extrapolate the weather description, that's an array, you can refer to the position index for that element. In this case, the weather description has only one element in the array so we'll refer to the first element (position index = 0)

&#x20;`weather_descriptions = parsed["current"]["weather_descriptions"][0]`


# Misty Delivery

Everyone needs a Misty who can deliver fresh soft drinks!

In this project, Misty will be programmed to listen to our favourite drink, and thanks to the NLP understanding it, go straight to the fridge and ask to have it in her amazing[ tin holder arm](/hardware-extension/tin-holder).

The first step to create a customized NLP conversation is to program our very unique context.

For the moment this is possible only with Blockly.

In your Misty Studio, click on Programming and then Blockly. From there open Misty's NLP Blockly section and create this context:

<figure><img src="/files/p6uyGmvosJlFZIqpj1EJ" alt="" width="511"><figcaption></figcaption></figure>

Then click on 'Run' and let Misty learn about your drinks choice.

In the Python code, after initializing the variables it's time to build the conversation, the states, the actions and how the flow will go from one to the other.

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

misty = Robot("YOUR-ROBOT-IP-ADDRESS")
misty.change_led(0, 0, 255)
misty.move_head(0, 0, 0)
misty.display_image("e_DefaultContent.jpg")

#create conversation
misty.create_conversation(name="DeliveryConv", startingState="StartDelivery", overwrite=True)

#create states
misty.create_state(name= "StartDelivery", speak="What can I get you to drink?", contexts="['DeliveryContext']", listen=True, noMatchSpeech= "Could you please repeat that?",  repeatMaxCount= 3, startAction="listen", overwrite=True)
misty.create_state(name= "DeliveryCoca-cola", speak= "Affirmative! One Coca-Cola, delivered with robotic precision.", listen=False, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="ActionCoca-cola", overwrite=True)
misty.create_state(name= "DeliveryFanta", speak= "Excellent choice! Fanta it is! I'll whiz over with that right away.", listen=False, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="ActionFanta", overwrite=True)
misty.create_state(name= "DeliverySprite", speak= "One Sprite, coming up! No lemon, no lime, just Sprite!", listen=False, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="ActionSprite", overwrite=True)
misty.create_state(name= "DeliveryBeer", speak= "You got it! One beer, perfectly chilled and ready to enjoy.", listen=False, noMatchSpeech="could you please repeat that?", repeatMaxCount= 3, startAction="ActionBeer", overwrite=True)

#map states
misty.map_state(conversation= "DeliveryConv", state= "StartDelivery", trigger= "SpeechHeard", nextState= "DeliveryCoca-cola", triggerFilter="coca-cola", reEntry=False, overwrite=True)
misty.map_state(conversation= "DeliveryConv", state= "StartDelivery", trigger= "SpeechHeard", nextState= "DeliveryFanta", triggerFilter="fanta", reEntry=False, overwrite=True)
misty.map_state(conversation= "DeliveryConv", state= "StartDelivery", trigger= "SpeechHeard", nextState= "DeliverySprite", triggerFilter="sprite", reEntry=False, overwrite=True)
misty.map_state(conversation= "DeliveryConv", state= "StartDelivery", trigger= "SpeechHeard", nextState= "DeliveryBeer", triggerFilter="beer", reEntry=False, overwrite=True)

#create actions
misty.create_action(name="ActionCoca-cola", 
                    script= "LED:255,0,0;IMAGE:coca-cola.png;ARMS:80,40,1000;HEAD:0,0,0,1000;",
                    overwrite= True)

misty.create_action(name="ActionFanta", 
                    script= "LED:255,165,0;IMAGE:fanta.jpg;ARMS:80,40,1000;HEAD:0,0,0,1000;",
                    overwrite= True)

misty.create_action(name="ActionSprite", 
                    script= "LED:0,255,0;IMAGE:sprite.jpg;ARMS:80,40,1000;HEAD:0,0,0,1000;",
                    overwrite= True)

misty.create_action(name="ActionBeer", 
                    script= "LED:255,255,0;IMAGE:beer.jpg;ARMS:80,40,1000;HEAD:0,0,0,1000;",
                    overwrite= True)

def path_to_the_kitchen():
    misty.drive_time(0, 50, 4500)
    time.sleep(5)
    misty.drive_time(40, 0, 8000)
    time.sleep(9)
    misty.drive_time(0, 50, 4500)
    time.sleep(5)
    misty.drive_time(40, 0, 10000)
    time.sleep(11)
    misty.drive_time(0, -50, 4500)
    time.sleep(5)
    misty.drive_time(40, 0, 6000)
    time.sleep(7)
    misty.drive_time(0, -50, 4500)
    time.sleep(5)
    misty.drive_time(0, 0, 2000)
    time.sleep(2)
    
def go_back():
    misty.drive_time(0, 50, 4500)
    time.sleep(5)
    misty.drive_time(40, 0, 6000)
    time.sleep(7)
    misty.drive_time(0, 50, 4500)
    time.sleep(5)
    misty.drive_time(40, 0, 10000)
    time.sleep(11)
    misty.drive_time(0, -50, 4500)
    time.sleep(5)
    misty.drive_time(40, 0, 8000)
    time.sleep(9)
    misty.drive_time(0, -50, 4500)
    time.sleep(5)
    misty.drive_time(0, 0, 2000)
    time.sleep(2)


def Key_Phrase_Recognized(data):
    print(data)
    misty.change_led(0, 255, 0)
    misty.play_audio("s_PhraseHello.wav")
    time.sleep(4)
    misty.start_conversation("DeliveryConv")
    time.sleep(12)
    path_to_the_kitchen()
    misty.move_head(-35, 0, 0)
    misty.speak("Hey, is there anyone that can put this drink in my arm?")
    time.sleep(3)
    go_back()             

misty.register_event(event_name='KPR_event', event_type=Events.KeyPhraseRecognized, callback_function=Key_Phrase_Recognized, keep_alive=True)
misty.keep_alive()
```

We have also created two functions that represent the path to the fridge in the kitchen and the path to go back to the desk of the user who first asked for Misty's delivery service.

It's possible to trigger the whole system only after saying: "hey Misty!" and she will start listening to your request.

It's also important to import the images that Misty will display to ask for the soft drink like:

<div><figure><img src="/files/WFj7HqxyvMgRuVQjcjwY" alt=""><figcaption></figcaption></figure> <figure><img src="/files/zacOHN8pPMOG5qZb8sId" alt="" width="441"><figcaption></figcaption></figure></div>

<div><figure><img src="/files/9sveJ7YBG4bwI5YZobIe" alt="" width="378"><figcaption></figcaption></figure> <figure><img src="/files/UYogs2ZhrUT9dzV4MVqt" alt=""><figcaption></figcaption></figure></div>

Have fun with this new Misty project!


# Motivational Misty

Do you ever feel down? From now on there will always be Misty to cheer you up!

In this project, Misty transforms into a motivational companion who delivers encouraging messages, delightful visuals, and engaging animations whenever her head is touched. This project uses Misty’s built-in features to create an uplifting interaction, making her the perfect source of inspiration and motivation.

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import random
import time

misty = Robot("YOUR-ROBOT-IP-ADDRESS")
misty.change_led(0, 0, 255)
misty.move_head(0, 0, 0)
misty.display_image("e_DefaultContent.jpg")

def goofy():
    misty.play_audio("s_Awe2.wav")
    time.sleep(2)
    misty.display_image("e_JoyGoofy.jpg")
    misty.transition_led(0, 255, 0, 0, 0, 255, "breathe", 1200)  # Transition from green to blue
    misty.move_arms(-40, 40)  # Move left arm up, right arm down
    misty.move_head(-5, 0, 5)  # Head tilt
    misty.speak("Keep going! You're doing great!")
    print("goofy")

def love():
    misty.play_audio("s_Love.wav")
    time.sleep(2)
    misty.display_image("e_Love.jpg")
    misty.transition_led(255, 255, 0, 0, 255, 0, "breathe", 1200)  # Transition from yellow to green
    misty.move_arms(20, -20)  # Move right arm up, left arm down
    misty.move_head(0, 0, -5)  # Nod head down
    misty.speak("Believe in yourself, you can achieve anything!")
    print("love")

def joy():
    misty.play_audio("s_Joy.wav")
    time.sleep(2)
    misty.display_image("e_Joy.jpg")
    misty.transition_led(0, 0, 255, 255, 0, 0, "breathe", 1200)  # Transition from blue to red
    misty.move_arms(-20, -20)  # Both arms down
    misty.move_head(5, 5, 0)  # Head nod to the side
    misty.speak("You’ve got this, keep pushing forward!")
    print("joy")

def ecstacy():
    misty.play_audio("s_Ecstacy2.wav")
    time.sleep(2)
    misty.display_image("e_EcstacyHilarious.jpg")
    misty.transition_led(255, 0, 0, 255, 255, 0, "breathe", 1200)  # Transition from red to yellow
    misty.move_arms(40, 40)  # Both arms up
    misty.move_head(-10, 0, 0)  # Shake head
    misty.speak("Every step you take is progress, don’t stop now!")
    print("ecstacy")

def amazement():
    misty.play_audio("s_Amazement.wav")
    time.sleep(2)
    misty.display_image("e_Amazement.jpg")
    misty.transition_led(255, 165, 0, 0, 255, 255, "breathe", 1200)  # Transition from orange to cyan
    misty.move_arms(-40, 20)  # Left arm down, right arm mid-level
    misty.move_head(0, -5, 5)  # Slight tilt and nod
    misty.speak("Great things never come from comfort zones. Keep it up!")
    print("amazement")

# Store animations in a list
animations = [goofy, love, joy, ecstacy, amazement]

# Function to handle touch events
def touched(data):
    print("Head touched!")
    random.choice(animations)()

# Register touch event
misty.register_event(event_name='touch', event_type=Events.TouchSensor, callback_function=touched, keep_alive=False)

# Keep the program running
misty.keep_alive()
```

After including the usual libraries, and setting up your Misty to default condition is time to create different Misty animations.\
Each animation combines a unique blend of Misty's movements, LED colour transitions, audio clips, and expressive face displays, providing a variety of positive and engaging interactions.\
\
Later all the different animations are stored in a list and with&#x20;

`random.choice(animations)()`

you can select a random one.

The last lines of code enable Misty to listen to the TouchSensor Event, make sure to set keep\_alive = False, in this way, Misty will react only once to your touch and will not be triggered several times, which would lead to mixed animations all at once.


# Misty Museum Guide

Misty loves to dive in new jobs! This time she'll be a museum guide!

This is the biggest Python code for Misty so far!\
She'll be a museum guide showing you the history of robotics, from industrial machines to interactive robots of today, like Misty!

Misty will move to different stages, each representing a milestone in the history of robotics.

This is Misty's path for this specific museum but feel free to adjust it according to your preferences:

```
↑
introduction
|
|
1 ----- 2 ----- 3               9 ----- 10 ----- 11
↓       ↓      /|               |\      ↓       /|
                |               |                |
               ←4               8→              ←12
                |               |                | 
                |               |                |
                5 ----- 6 ----- 7               ←13
                ↓       ↓       ↓                |
                                                 |
                        14&conclusion------------⌋
                        ↓
```

The arrows show the direction that Misty is facing when she stops, from the introduction to the conclusion.

Misty will show many pictures on her display, so they must be uploaded to the robot before the skill can be launched.

{% file src="/files/HTZV41bWVeiNAPc1lrry" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import random
import time

misty = Robot("YOUR-ROBOT-IP-ADDRESS")
misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)
misty.move_arms(20, 20)

# stages
def initial_joke():
    misty.set_default_volume(20)
    misty.speak("Can you hear me now?")
    time.sleep(2)
    misty.set_default_volume(100)
    misty.speak("And now?")
    time.sleep(2)
    misty.set_default_volume(70)
    misty.speak("Okay this should work")
    time.sleep(2)
    misty.start_action("Hi")

def introduction():
    misty.change_led(0,0,0)
    misty.display_image("e_Admiration.jpg")
    misty.speak("Hello, everyone! I'm Misty, your friendly museum guide for today.")
    time.sleep(3)
    misty.move_arms(80, -89)
    time.sleep(1)
    misty.move_arms(80, 0)
    time.sleep(0.75)
    misty.move_arms(80, -89)
    time.sleep(0.75)
    misty.speak("We're about to embark on an exciting journey through the history of robotics.")
    random_chest()
    head_side_to_side()
    time.sleep(2)
    misty.speak("From the first examples developed in the 1950s to the modern, interactive robots like me. you’ll see how robots have evolved over time.")
    time.sleep(5)
    misty.speak("Let’s begin!")
    time.sleep(3)

def stage_1_1950s():
    misty.move_head(10, 0, 0)
    misty.display_image("unimate_1.png")
    random_chest()
    misty.speak("Our journey starts in the 1950s. One of the earliest robots was Unimate, created by George Devol in 1954.")
    time.sleep(7)
    misty.move_arms(0, 0)
    misty.display_image("unimate_2.png")
    misty.speak("Unimate was designed to automate factory tasks like moving and stacking heavy metal parts.")
    time.sleep(6)
    random_chest()
    misty.move_arms(30, -30)
    misty.display_image("unimate_3.png")
    misty.speak("In 1961, it became the first robot used in a factory, working on the assembly line at General Motors. This marked the beginning of industrial robotics.")
    time.sleep(10)

def stage_2_1960s():
    misty.display_image("lunokhod_1.png")
    random_chest()
    misty.speak("During the 1960s, robots weren't just working in factories; they were also exploring new frontiers.")
    time.sleep(6)
    misty.display_image("lunokhod_2.png")
    time.sleep(2)
    misty.speak("In 1966, Lunokhod, the first prototype of a lunar rover, was developed in the Soviet Union.")
    misty.start_action("Love")
    time.sleep(6)
    random_chest()
    misty.display_image("lunokhod_3.png")
    misty.speak("Although it didn’t make it to the Moon until 1970, it set the stage for robotics in space exploration.")
    time.sleep(6)

def stage_3_1970s():
    misty.move_head(-10, 0, 0)
    misty.display_image("puma_1.png")
    random_chest()
    misty.speak("Robots in the 1970s continued to grow in sophistication. A great example is the PUMA robot arm, developed in 1978.")
    time.sleep(9)
    misty.display_image("puma_2.png")
    misty.speak("It was designed to handle precise tasks in assembly lines and became one of the most popular robotic arms, setting a standard for robotics in manufacturing.")
    random_chest()
    time.sleep(8)
    i = 0
    while i < 4:
        misty.move_arms(40, -40)
        time.sleep(0.5)
        misty.move_arms(-40, 40)
        time.sleep(0.5)
        i = i + 1

def stage_4_1980s():
    misty.move_head(-10, 0, 0)
    misty.display_image("shakey_1.png")
    random_chest()
    misty.speak("The 1980s saw the rise of robots that could think for themselves. One significant robot was Shakey, developed by Stanford Research Institute.")
    time.sleep(9)
    misty.display_image("shakey_2.png")
    random_chest()
    misty.speak("It was the first robot that could perceive its surroundings, plan its actions, and navigate through its environment, showing how robots could make decisions.")
    time.sleep(9)
    misty.drive_time(0, -50, 10000) # turn 360°
    time.sleep(10)

def stage_5_early_1990s():
    misty.display_image("asimo_1.png")
    random_chest()
    misty.move_arms(80, 80)
    misty.speak("In the early 1990s, robots started to take on more human-like forms. One of the most famous is ASIMO, created by Honda in 1996.")
    time.sleep(9)
    misty.display_image("asimo_2.png")
    random_chest()
    misty.speak("ASIMO was designed to walk, run, and even climb stairs like a human, demonstrating how robots could interact more naturally with their surroundings.")
    time.sleep(10)
    misty.start_action("Walk-fast")
    misty.drive_time(50, 0, 1000)
    time.sleep(1)
    misty.drive_time(-50, 0, 1000)
    time.sleep(1)
    time.sleep(2)

def stage_6_late_1990s():
    misty.move_head(-10, 0, 0)
    misty.display_image("aibo_1.png")
    random_chest()
    misty.speak("In the late 1990s, robots began to make their way into our homes. One of the most memorable was AIBO, the robotic dog developed by Sony in 1999.")
    time.sleep(9)
    misty.display_image("aibo_2.png")
    time.sleep(3)
    random_chest()
    misty.speak("AIBO could move, bark, play, and even learn new tricks, becoming one of the first successful social robots.")
    time.sleep(7)
    misty.start_action("Party")

def stage_7_early_2000s():
    misty.move_head(-10, 0, 0)
    misty.move_arms(0, 0)
    misty.display_image("davinci_1.png")
    random_chest()
    misty.speak("As we moved into the 2000s, robots took on new roles. Da Vinci, a surgical robot, became widely used for performing delicate surgeries with high precision.")
    time.sleep(10)
    misty.display_image("opportunity_1.png")
    time.sleep(3)
    random_chest()
    misty.speak("At the same time, NASA’s Spirit and Opportunity rovers were exploring Mars, proving that robots could work beyond Earth.")
    time.sleep(7)

def stage_8_mid_2000s():
    misty.move_arms(0, 0)
    misty.display_image("roomba_1.png")
    random_chest()
    misty.speak("In the mid-2000s, robots became more present in everyday life. Remember the Roomba vacuum cleaner, launched by iRobot in 2002?")
    time.sleep(7)
    misty.display_image("roomba_2.png")
    random_chest()
    misty.speak("It could automatically navigate a room and clean the floors. Around the same time, Paro, a robotic seal, was used for therapy in hospitals, showing how robots could calm people.")
    time.sleep(10)
    drive_square()

def stage_9_late_2000s():
    misty.display_image("nao_1.png")
    time.sleep(2)
    random_chest()
    misty.start_action("Concerned")
    misty.speak("Towards the end of the 2000s, robots became more interactive. Nao, a small humanoid robot, was released in 2008.")
    time.sleep(7)
    misty.display_image("nao_2.png")
    random_chest()
    misty.speak("Nao could walk, talk, dance, and even recognize faces, becoming a popular platform for research and education.")
    time.sleep(7)
    misty.change_led(255, 0, 0)

def stage_10_early_2010s():
    misty.move_head(-10, 0, 0)
    misty.move_arms(0, 0)
    misty.display_image("pepper_1.png")
    random_chest()
    misty.speak("In the early 2010s, robots began to benefit from AI. Pepper, introduced in 2014, was a social robot that could read emotions and have conversations.")
    time.sleep(12)
    misty.display_image("pepper_2.png")
    random_chest()
    time.sleep(2)
    misty.start_action("Party")
    misty.speak("Meanwhile, AI-based robots like Watson showed how robots could understand language, even winning games like Jeopardy!")
    time.sleep(8)

def stage_11_mid_2010s():
    misty.move_head(-10, 0, 0)
    misty.display_image("jibo_1.png")
    random_chest()
    misty.speak("During the mid-2010s, assistive robots like Jibo became popular. Jibo was designed to be a helpful companion at home.")
    time.sleep(8)
    misty.display_image("tesla_1.png")
    random_chest()
    misty.drive_time(70, 0, 3000) #straight
    time.sleep(3)
    misty.drive_time(-70, 0, 3000) #straight
    time.sleep(3)
    misty.speak("Around the same time, companies like Tesla were developing self-driving cars, expanding what robots could do in our daily lives.")
    time.sleep(11)

def stage_12_late_2010s():
    misty.move_head(-10, 0, 0)
    misty.display_image("vector_1.png")
    random_chest()
    misty.speak("During the late 2010s, there was a surge in home robots designed to make life more fun. One example is Vector, a friendly robot created by Anki in 2018.")
    time.sleep(11)
    misty.display_image("cozmo_1.png")
    random_chest()
    arms_both_up_both_down()
    misty.speak("Another was Cozmo, also by Anki, which taught kids about coding through fun games and activities.")
    time.sleep(7)
    misty.change_led(255, 0, 0)

def stage_13_2020s():
    misty.move_head(10, 0, 0)
    misty.display_image("sophia_1.png")
    random_chest()
    misty.speak("As we move further into the 2020s, humanoid and interactive robots have reached new levels. Sophia by Hanson Robotics is one famous example.")
    time.sleep(10)
    misty.display_image("optimus_1.png")
    time.sleep(4)
    random_chest()
    misty.start_action("Hug2")
    misty.display_image("atlas_1.png")
    misty.speak("There's also Optimus by Tesla, designed to assist with tasks, and Atlas by Boston Dynamics, known for its agility.")
    time.sleep(7)

def stage_14_misty():
    misty.start_action("Love")
    misty.speak("And then there is me, Misty, the world's most lovable robot")
    time.sleep(4)
    misty.speak("I can create every activity with you, as you had the possibility to see I'm able to change my personality very easily, and you can use me to interact with the world around you! I can map, drive, recognize and much more.")
    time.sleep(10)
    misty.speak("Navigate in the Misty lessons to learn all what I can offer you!")
    time.sleep(6)
    misty.drive_time(0, -90, 12500)
    time.sleep(10)

def conclusion():
    misty.move_head(-10, 0, 0)
    random_chest()
    misty.speak("That concludes our journey through the history of robotics! From industrial machines to interactive robots of today, it’s been an incredible journey.")
    time.sleep(12)
    random_chest()
    misty.start_action("Cheers")
    misty.speak("I hope you enjoyed learning about the story of robots. Thank you for joining me!")
    time.sleep(6)

#utilities

def head_side_to_side():
    misty.move_head(0, 0, -60)
    time.sleep(1)
    misty.move_head(10, 0, -60)
    time.sleep(1)
    misty.move_head(-10, 0, -60)
    time.sleep(1)
    misty.move_head(-10, 0, 60)
    time.sleep(1)
    misty.move_head(10, 0, 60)
    time.sleep(1)
    misty.move_head(0, 0, 0)
    time.sleep(1)

def arms_up_and_down():
    i = 0
    while i < 2:
        misty.move_arms(90, -90)
        time.sleep(1)
        misty.move_arms(-90, 90)
        time.sleep(1)
        i = i + 1

def random_chest():
    misty.change_led(random.randrange(0, 255), random.randrange(0, 255), random.randrange(0, 255))

def turn_straight_turn():
    misty.drive_time(0, -50, 4500) #turn left 90°
    time.sleep(5)
    misty.drive_time(70, 0, 3000)# straight
    time.sleep(3)
    misty.drive_time(0, 50, 4500) #turn right 90°
    time.sleep(5)

def drive_square():
    i = 0
    while i < 4:
        misty.drive_time(40, 0, 3000)
        time.sleep(3)
        misty.drive_time(0, 50, 4500)
        time.sleep(5)
        i = i + 1
    
def arms_both_up_both_down():
    i = 0
    while i < 3:
        misty.move_arms(70, 70)
        time.sleep(0.8)
        misty.move_arms(-70, -70)
        time.sleep(0.8)
        i = i + 1

def path():
    introduction()
    time.sleep(3)
    misty.drive_time(0, 50, 6500) #turn right 180°
    time.sleep(7)
    misty.drive_time(70, 0, 3000) #straight
    time.sleep(3)

    stage_1_1950s()
    time.sleep(3)
    turn_straight_turn()

    stage_2_1960s()
    time.sleep(3)
    turn_straight_turn()
    misty.drive_time(0, 50, 3000) #turn right 45° 
    time.sleep(3)

    stage_3_1970s()
    time.sleep(3)
    misty.drive_time(0, -50, 3000) #turn left 45°
    time.sleep(3)
    misty.drive_time(70, 0, 3000)
    time.sleep(3)
    misty.drive_time(0,-50, 4500)
    time.sleep(5)

    stage_4_1980s()
    time.sleep(3)
    misty.drive_time(70, 0, 3000)
    time.sleep(3)

    stage_5_early_1990s()
    time.sleep(3)
    turn_straight_turn()

    stage_6_late_1990s()
    time.sleep(3)
    turn_straight_turn()

    stage_7_early_2000s()
    time.sleep(3)
    misty.drive_time(0, 50, 6500)
    time.sleep(7)
    misty.drive_time(70, 0, 3000)
    time.sleep(3)
    misty.drive_time(0, -50, 4500)
    time.sleep(5)

    stage_8_mid_2000s()
    time.sleep(3)
    turn_straight_turn()
    misty.drive_time(0, 50, 3000) #turn right 45° 
    time.sleep(3)

    stage_9_late_2000s()
    time.sleep(3)
    misty.drive_time(0, -50, 3000) #turn left 45°
    time.sleep(3)
    misty.drive_time(70, 0, 3000)
    time.sleep(3)
    misty.drive_time(0,-50, 4500)
    time.sleep(5)

    stage_10_early_2010s()
    time.sleep(3)
    turn_straight_turn()
    misty.drive_time(0, 50, 3000) #turn right 45° 
    time.sleep(3)

    stage_11_mid_2010s()
    time.sleep(3)
    misty.drive_time(0, -50, 3000) #turn left 45°
    time.sleep(3)
    misty.drive_time(70, 0, 3000)
    time.sleep(3)
    misty.drive_time(0,-50, 4500)
    time.sleep(5)

    stage_12_late_2010s()
    time.sleep(3)
    turn_straight_turn()

    stage_13_2020s()
    time.sleep(3)
    turn_straight_turn()
    misty.drive_time(70, 0, 5000)
    time.sleep(5)
    misty.drive_time(0, 50, 4500)
    time.sleep(5)

    stage_14_misty()
    time.sleep(3)
    conclusion()
    time.sleep(3)

initial_joke()
time.sleep(2)
path()
time.sleep(2)

```

In the first part of the code, you can see the declaration of Misty's libraries and the creation of the robot. In the stages section, you can find all the different stages each with a different story to tell and unique animations. In the utils some useful functions that involve Misty's behaviour such as moving arms or head in specific locations.

In the last function, path, you can see all Misty's driving and the stage's calls.


# Who for Halloween

Misty plays different Halloween characters and you can choose your favourite!

Misty will start selecting a random character after listening to her keyword.&#x20;

You can interact with her by asking something like: "Hey Misty, what are you going to dress up as for Halloween?"&#x20;

And after an evil laugh she'll randomly select one of her uploaded characters: R2D2, Chewbacca or Zombie!&#x20;

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import random
import time

misty = Robot("YOUR-ROBOT-IP-ADDRESS")
misty.change_led(0, 255, 0)
misty.move_head(0, 0, 0)
misty.move_arms(20, 20)

# Define the playCharacter function in Python
def playCharacter(characterName):
    if characterName == "R2D2":
        # R2D2 character with specific movements and sounds
        misty.change_led(0, 0, 255)  # Blue LED
        misty.display_image("e_R2D2_Robot.png")
        time.sleep(0.5)
        misty.play_audio("R2D2-yeah.wav")
        i = 0
        while i < 3:
            misty.change_led(255, 255, 255)  # White LED
            misty.drive_time(1, -70, 500)
            time.sleep(0.6)
            misty.change_led(0, 0, 255)  # Blue LED
            misty.drive_time(-1, 70, 500)
            time.sleep(0.6)
            i = i + 1    
        time.sleep(0.5)
        misty.play_audio("R2D2-yeah.wav")  # Play sound again

    elif characterName == "Chewbacca":
        # Chewbacca character with sound and arm movement
        misty.display_image("e_Chewie.png")
        misty.change_led(128, 64, 0)  # Brown LED
        time.sleep(0.5)
        misty.play_audio("chewy_roar.wav")
        misty.move_arms(-25, -25)
        misty.move_head(-25, -2, 0)
        time.sleep(0.2)
        misty.play_audio("chewy_roar.wav")  # Play again at high volume

    elif characterName == "Zombie":
        # Zombie character with sound and arm movement
        misty.change_led(128, 128, 0)  # Sickly green LED
        misty.display_image("e_Zombie.png")
        time.sleep(0.5)
        misty.play_audio("s_Fear.wav")
        misty.move_arms(0, 0)  # Arms out straight
        misty.move_head(-20, -5, 0)
        time.sleep(0.2)
        
        misty.move_arms(-15, 18)  # Right arm up, left down
        misty.move_head(-20, 2, 0)
        time.sleep(0.2)
        
        misty.move_arms(10, -18)  # Left arm up, right down
        misty.move_head(-20, -5, 0)
        time.sleep(0.2)
        misty.play_audio("s_Fear.wav")  # Play sound again

# Store animations in a list
characters = ["R2D2", "Chewbacca", "Zombie"]

# Function to handle touch events
def Key_Phrase_Recognized(data):
    misty.display_image("e_JoyGoofy2.jpg")
    misty.speak("Sure, here is one of my many characters! I hope you'll enjoy it and don't get too scared")
    time.sleep(5)
    misty.display_image("e_Rage4.jpg")
    misty.play_audio("s_PhraseEvilAhHa.wav")
    time.sleep(3)
    playCharacter(random.choice(characters))

# Register touch event
misty.register_event(event_name='KPR_event', event_type=Events.KeyPhraseRecognized, callback_function=Key_Phrase_Recognized, keep_alive=True)
# Keep the program running
misty.keep_alive()
```

You can also choose to play one of your characters using this format:

`playCharacter("your-character's-name")`

<div><figure><img src="/files/VaQtIELPT7LQXj1OYsGu" alt=""><figcaption><p>chewbacca</p></figcaption></figure> <figure><img src="/files/L9d2fwJL6RBfw8w5n8G5" alt=""><figcaption><p>R2D2</p></figcaption></figure> <figure><img src="/files/9An9tKOHKGCrfDpoQLoe" alt=""><figcaption><p>zombie</p></figcaption></figure></div>

{% file src="/files/UGc9nxUgJYyUB3J0ynK6" %}

{% file src="/files/XsZGj5k2EmXK8Jt6nSS5" %}

Don't forget to upload these files in your Misty to get amazing results!


# Arduino Backpack

You can use the Arduino-compatible backpack with sensors and other hardware to extend the capabilities of Misty's native hardware.

The Misty Backpack for Arduino is a microcontroller embedded in a magnetic mount that communicates with your skills via the UART serial port on Misty's back

### Programming the Misty Backpack for Arduino

The Misty Backpack for Arduino uses an [ATmega328P microcontroller](http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-7810-Automotive-Microcontrollers-ATmega328P_Datasheet.pdf) pre-programmed with a bootloader that allows you to upload code directly from the [Arduino IDE](https://www.arduino.cc/en/Main/Software). When you upload your code, you must target the Arduino Pro or Pro Mini board and the ATmega328P (3.3v, 8MHz) processor.

Follow these steps to configure the Arduino IDE to upload code to Misty's Arduino-compatible backpack:

1. Download and install the [Arduino IDE](https://www.arduino.cc/en/Main/Software).
2. Connect the Misty backpack to your computer via the backpack's USB micro port.
3. Open the IDE and select **Tools** from the top menu.

   1. From the **Board** sub-menu, select **Arduino Pro or Pro Mini**.
   2. From the **Processor** sub-menu, select **ATmega328P (3.3V, 8MHz)**.
   3. From the **Port** sub-menu, select the port that the microcontroller is connected to. (Your port number may be different than what is pictured here).

   <figure><img src="/files/WzZ7ezKbbbyTgPKyy5LG" alt="" width="448"><figcaption></figcaption></figure>

#### Notes:

When using hardware serial to communicate with Misty (pins D0 and D1), you must remove the board from Misty before you can upload a new sketch. If you are using the software serial pins (D8 and D9), you can upload a new sketch without disconnecting the board.

The Misty Backpack for Arduino supports hardware serial communication over pins D0 and D1. These pins are wired by default to interface with the serial port on Misty's back, that is the way we’ll use to allow communication between Misty and Arduino.

If you want more information about the Misty Backpack for Arduino check [this link](https://docs.mistyrobotics.com/misty-ii/robot/misty-backpack-for-arduino/).


# Arduino

You can create amazing project with Misty and Arduino even without the Arduino Backpack!

### Assembling

Instead of the Arduino Backpack, you can use an Arduino UNO as the board to record data and send information to Misty.

<figure><img src="/files/qF61f3fpTpGQnK4OjfOT" alt="" width="375"><figcaption><p>Take off the Misty standard Backpack</p></figcaption></figure>

<figure><img src="/files/dkn2ViP3GHdLFtDHpBhM" alt="" width="563"><figcaption><p>Unscrew the board </p></figcaption></figure>

<figure><img src="/files/FsCSHNuPInrcFmILY7OC" alt="" width="375"><figcaption><p>Insert the empty board in Misty's back</p></figcaption></figure>

**Connections**

To make Arduino UNO work with Misty are necessary two connections: Serial signal and power

For the Serial signal, those are the connections:

* Misty 3.3V to Arduino 3.3V                                     (red)
* Misty GND to Arduino GND                                    (black)
* Misty RX to Arduino Digital pin 1 (tx→)                 (yellow)
* Misty TX to Arduino Digital pin 0 (rx←)                (green)

<div><figure><img src="/files/WWqSsumsD6GWT0JICyXk" alt=""><figcaption></figcaption></figure> <figure><img src="/files/WguIDE8Uy4NwdkiIyvkS" alt=""><figcaption></figcaption></figure></div>

You can make these connections even using a breadboard, it's not mandatory to do them straight to the Arduino UNO pins.

The USB port in the back of Misty will give power to the board, so you can use the cable with which you upload the Arduino code from your computer to Arduino.

<figure><img src="/files/dch9Yd7I0g6wROSHZM3k" alt="" width="563"><figcaption><p>Plug in the arduino in Misty's USB</p></figcaption></figure>

And you're ready to go!

#### Links

* [Arduino IDE](https://www.arduino.cc/en/software)
* [Standard Arduino UNO board](https://store.arduino.cc/products/arduino-uno-rev3)


# Arduino Lessons

The key to communication between Misty and Arduino is to properly control the different messages that they are writing to each other.

Arduino and Misty will communicate with each other through the serial port that both support. In the first case (Arduino to Misty) Arduino will write and Misty will read, in the second one (Misty to Arduino) Misty will write and Arduino will read.&#x20;

We will use Python to program Misty and C++ to program Arduino.&#x20;

Ensure to have all the necessary components before starting your project.

In these two projects, we will do something really simple just to show you how this mechanism works. After, you will be able to create your complex projects including several sensors and communication in both ways.&#x20;

{% content-ref url="/pages/uXTCVCzOLKwxX0IhozKj" %}
[Arduino to Misty](/arduino/arduino-lessons/arduino-to-misty)
{% endcontent-ref %}

{% content-ref url="/pages/7IZ74UhSHdF4mBtNuc45" %}
[Misty to Arduino](/arduino/arduino-lessons/misty-to-arduino)
{% endcontent-ref %}


# Arduino to Misty

In this project, you will use Arduino to trigger Misty to change the colour of her chest LED.

### Materials&#x20;

* Misty’s Arduino backpack
* 2 buttons
* 2 LEDs
* 2 resistance 330 Ω
* 2 resistance 10 kΩ
* Some wires of different lengths.

### Arduino Circuit

<figure><img src="/files/EuJJ9IFFJBr4F5hPtwqT" alt="" width="563"><figcaption><p>It may seem in this scheme that the buttons are not fully connected but in your breadboard, you can close the pins a bit. You can use the same scheme with Arduino UNO or the arduino Backpack</p></figcaption></figure>

<div><figure><img src="/files/bJoohKA687XmVRnOARmY" alt="" width="375"><figcaption></figcaption></figure> <figure><img src="/files/XygCmGa7Q4mAUzaffU4h" alt="" width="375"><figcaption></figcaption></figure></div>

### Arduino Code&#x20;

```cpp
void setup() {
  Serial.begin(9600);
  pinMode(2, INPUT_PULLUP); 	//pin green button
  pinMode(3, INPUT_PULLUP); 	//pin blue button
  pinMode(10, OUTPUT);      	//pin green LED
  pinMode(11, OUTPUT);      	//pin blue LED
}

void loop() {
  int pushedg = digitalRead(2); //green button
  int pushedb = digitalRead(3); // blue button

  if (pushedg == LOW){		// green button pressed condition
    digitalWrite(10, HIGH);	// green LED on
    Serial.println("green");	// write "green" in the serial
  } else {
    digitalWrite(10, LOW);	// green LED off
  }
  
  if (pushedb == LOW){		// blue button pressed condition
    digitalWrite(11, HIGH);	// blue LED on
    Serial.println("blue");	// write "blue" in the serial
  } else {
    digitalWrite(11, LOW);	// blue LED off
  }
  delay(200);			// wait for 200ms
}
```

The Arduino code is composed of two parts: the void setup and the void loop. The first one will run only once at the beginning and here you set all your variables. The second part will run in a loop until stopped.&#x20;

In this void setup, you open the serial and set the pins you’ll use in this project. In the void loop, you associate the pins to our buttons and then we set two conditions:&#x20;

If the green button is pressed you’ll turn on the green LED on the Arduino board and you will write in the serial the word “green” if this doesn’t happen the green LED will be off. It happens the same with the blue button as you can read in the code. At the end, we insert a pause in order not to send too many signals to Misty.

### Misty Code&#x20;

{% code overflow="wrap" %}

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events

misty = Robot("your_robot_IP_address")

def msg(data):
    print(data)
    if data["message"]["message"] == "green":
misty.change_led(0, 255, 0)
         if data["message"]["message"] == "blue":
misty.change_led(0, 0, 255)

misty.register_event(event_name="serial_message_event", event_type=Events.SerialMessage, callback_function=msg, keep_alive=True)
misty.keep_alive()

```

{% endcode %}

To allow Misty to read in the Serial you need to build an event, so you can import the usual libraries that Misty uses to register events. Then we create the robot and the function that will be called with the event.&#x20;

In this function we read the serial message and if the condition is verified Misty will complete the function. If we press the green button, Arduino will write “green” in the serial and Misty will register “green” as message from the serial port, so she’ll act as shown in the function.


# Misty to Arduino

In this project, you’ll use MIsty’s bump sensor to trigger Arduino to change its LEDs.

### Materials&#x20;

* Misty’s Arduino backpack
* 2 LEDs
* 2 resistance 330 Ω
* Some wires of different lengths.

### Arduino Circuit

<figure><img src="/files/3Pcuxf32rdzqD76uOPoy" alt="" width="563"><figcaption><p>You can use the same scheme with Arduino UNO or the arduino Backpack</p></figcaption></figure>

<figure><img src="/files/LDNfOOl1jfTDJj5g7ARD" alt="" width="375"><figcaption></figcaption></figure>

### Arduino Code&#x20;

```cpp
void setup(){
  Serial.begin(9600);
  Serial.setTimeout(100);
  pinMode(10, OUTPUT);      	//pin red LED
  pinMode(11, OUTPUT);      	//pin blue LED
}

void loop() {
  if (Serial.available() > 0){	        // check if something is written in the Serial
    String str = Serial.readStringUntil('\n');  // read the string
    if (str == "red"){			        // compare the string
      digitalWrite(10, HIGH);		        // red LED on
      Serial.println("red on");	                // write in the serial monitor
    }
    if (str == "blue"){
      digitalWrite(11, HIGH);
      Serial.println("blue on");
    }
    if (str == "red off"){
      digitalWrite(10, LOW);
      Serial.println("red off");
    }
    if (str == "blue off"){
      digitalWrite(11, LOW);
      Serial.println("blue off");
    }
    if (str == "all off"){
      digitalWrite(10, LOW);
      digitalWrite(11, LOW);
      Serial.println("all off");
    }
    if (str == "all on"){
      digitalWrite(10, HIGH);
      digitalWrite(11, HIGH);
      Serial.println("all on");
    }
  }
delay(500);
}

```

In this code, you can see the same structure as the previous one: A void setup and a void loop. In the void setup, you open the serial, set the maximum milliseconds to wait for serial data (the default is 1000 ms) and define the pins that you’ll use in the code.&#x20;

In the void loop, the first thing you should do is to check if there is something written in the serial. The second step is to read the string that exists in the serial by defining it. To read strings completely you can use the character “\n” at the end, which represents the new line character/enter. Once you get the string the last part is to define all the conditions you can have.&#x20;

**For example** :&#x20;

```cpp
if (str == "red"){
      digitalWrite(10, HIGH);	
      Serial.println("red on");
    }
```

If the string you get from Misty is “red” turn on the red LED and write in the Arduino’s serial “red on”. The Serial.println at the end of each if condition is not necessary for the code but it’s helpful for the programmer who can see what’s happening in the code from the serial monitor in the Arduino IDE.

### Misty Code &#x20;

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events

misty = Robot("your_robot_IP_address")

def bumped(data):
    print(data)
    if(data["message"]["sensorId"] == "bfr" and data["message"]["isContacted"] == True):
        misty.change_led(255, 0, 0)
        misty.write_serial("red\n")
    if(data["message"]["sensorId"] == "bfl" and data["message"]["isContacted"] == True):
        misty.change_led(0, 0, 255)
        misty.write_serial("blue\n")
    if(data["message"]["sensorId"] == "brr" and data["message"]["isContacted"] == True):
        misty.change_led(0, 255, 0)
        misty.write_serial("red off\n")
    if(data["message"]["sensorId"] == "brl" and data["message"]["isContacted"] == True):
        misty.change_led(255, 127, 0)
        misty.write_serial("blue off\n")

misty.register_event(event_name='bump_event', event_type=Events.BumpSensor, callback_function=bumped, keep_alive=True)
misty.keep_alive()
```

To write in the serial you don’t need an event, but Misty has it as an element in her Python API. This code is a normal Bump sensor event that associates to each bump sensor a string to write in the serial.&#x20;

**For example**:

```python
if(data["message"]["sensorId"] == "bfr" and data["message"]["isContacted"] == True):
        misty.change_led(255, 0, 0)
        misty.write_serial("red\n")
```

If the bump front right sensor is contacted turn Misty’s chest LED red and write “red\n” in the serial. Arduino will receive the string from the serial, read it until the “\n” character and select the function from its list.


# Misty Tracker

This project aims to make Misty a temperature and light tracker in your room!

When you are using Arduino to collect data there is a big obstacle: Arduino can't move alone!&#x20;

Misty will adapt her expression based on the conditions she will be in, for example when it will be dark and cold she will be scared and so on. You can adapt your actions and the path she will drive.&#x20;

<figure><img src="/files/V6QwY49noTPzAiaIrX70" alt="" width="375"><figcaption></figcaption></figure>

### Materials:&#x20;

* Misty's Arduino Backpack
* Thermistor
* Photoresistor
* 3 LEDs
* 3 resistance 330 Ω
* 2 resistance 10 kΩ
* Some wires of different lenght
* Optional [Arduino breadboard support](/hardware-extension/arduino-breadboard-support)

### Arduino Circuit:

<figure><img src="/files/i0Cy0H6LP3Inw0ZuWMET" alt="" width="563"><figcaption><p>Remember to use Misty's Arduino Backpack board, adapt your resistances to your sensors and check your individual connections, for example, our temperature sensor had only two wires. </p></figcaption></figure>

<figure><img src="/files/6qVDEgLzlhLsOAlG1bFj" alt=""><figcaption></figcaption></figure>

### Arduino Code:

```cpp
// light and temperature
int light_lim = 300;
double temp_lim = 26; // temperature limit

void setup() {
  Serial.begin(9600);
  pinMode(A0, INPUT); // light sensor
  pinMode(A2, INPUT); // temperature sensor
  pinMode(11, OUTPUT); // light LED (white)
  pinMode(9, OUTPUT); // temperature LED hot (red)
  pinMode(8, OUTPUT); // temperature LED cold (blue)
}
 
void loop() {
  // read the value
  int light_value = analogRead(A0);
  double temp_value = (analogRead(A2) - 32) /1.8; // the value is originally read in Fahrenheit, so we convert it in Celsius
  
  //compare the value with limits 
  if ((light_value <= light_lim) && (temp_value <= temp_lim)) { // dark and cold
    digitalWrite(11, HIGH);
    digitalWrite(9, LOW);
    digitalWrite(8, HIGH);
    Serial.print("dark and cold|");
  } 
  else if ((light_value >= light_lim) && (temp_value <= temp_lim)){ // illuminated and cold
    digitalWrite (11, LOW);
    digitalWrite(9, LOW);
    digitalWrite(8, HIGH);
    Serial.print("illuminated and cold|");
  }
  else if ((light_value >= light_lim) && (temp_value >= temp_lim)){ // illuminated and hot
    digitalWrite(11, LOW);
    digitalWrite(9, HIGH);
    digitalWrite(8, LOW);
    Serial.print("illuminated and hot|");
  }
  else if ((light_value <= light_lim) && (temp_value >= temp_lim)){ // dark and hot
    digitalWrite(11, HIGH);
    digitalWrite(9, HIGH);
    digitalWrite(8, LOW);
    Serial.print("dark and hot|");
  } 
  else { // in any other case
    digitalWrite(11, LOW);
    digitalWrite(9, LOW);
    digitalWrite(8, LOW);
    Serial.print("searching|");
  }
  String values = "light value: " + String(light_value) + "    temperature value: " + String(temp_value); 
  Serial.println(values); 
  delay(2000); // scan every 2 seconds
}
```

First of all, you can set some values of light and temperature that will be the limits to decide if an environment is dark or illuminated and cold or hot.

In this void setup, you open the serial and set the pins you’ll use in this project. In this void loop, you read the sensors and write a specific serial message based on your conditions:

There will be four possible conditions, and it has been also added an extra case that will save the program in case none of the conditions is selected. \
\
The Serial message, which is our way of transmitting the information to Misty is built in this way:&#x20;

`light condition and temperature condition | light value and temperature value`

We insert the | because in this way, in the Misty python code, we can split the message and use them indipendently. We want it because:&#x20;

* it would be harder to build an if statement with all the different values each time
* you can print the data and only the data that Misty captured in your output

Arduino will scan the different conditions every two seconds.

### Misty Code:

<pre class="language-python"><code class="lang-python">from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time

ROBOT_IP = "&#x3C;your_robot_ip_address>"
misty = Robot(ROBOT_IP)

def msg(data):
    conditions = data["message"]["message"].split("|")[0]
    values = data["message"]["message"].split("|")[1]
    print(values)

    if conditions == "dark and cold":
        misty.set_flashlight(True)
        misty.display_image("e_Fear.jpg")
        misty.move_arms(80, 80)
        misty.change_led(0, 0, 255)
    elif conditions == "illuminated and cold":
        misty.set_flashlight(False)
        misty.move_arms(-80, -80)
        misty.change_led(0, 255, 0)
    elif conditions == "illuminated and hot":
        misty.set_flashlight(False)
        misty.play_audio("s_Joy.wav")
        misty.display_image("e_Admiration.jpg")
        misty.change_led(255, 0, 0)
    elif conditions == "dark and hot":
        misty.set_flashlight(True)
        misty.start_action("love")
        misty.change_led(255, 0, 255)
    elif conditions == "searching":
        misty.set_flashlight(False)
        misty.start_action("look-left")
        time.sleep(0.5)
        misty.start_action("look-right")
    else:
        misty.change_led(100, 70, 160)

misty.register_event(event_name="serial_message_event", event_type=Events.SerialMessage, callback_function=msg, keep_alive=True)
# my tracked space
for i in range(2):
    misty.drive_time(50, 0, 4000)
<strong>    time.sleep(4500)
</strong><strong>    for j in range(4):
</strong>        misty.move_head(-20, 0, 0)
        time.sleep(1)
        misty.move_head(20, 0, 0)
        time.sleep(1)
    misty.drive_time(0, 50, 6500)
    time.sleep(7000)
<strong>    misty.drive_time(50, 0, 4000)
</strong>    time.sleep(4500)
    misty.drive_time(0, 50, 6500)
    time.sleep(7000)
# end my tracked space
misty.start_action("hi")
misty.keep_alive()
</code></pre>

To allow Misty to read in the Serial you need to build an event, so you can import the usual libraries that Misty uses to register events. Then you can create the robot and the function that will be called with the event.

In this function Misty reads the serial message and if the condition that Arduino sent is verified Misty will act according to what you wrote in that case.&#x20;

In the function, you can split values and conditions so you can use them independently as previously said. In the end, you can loop the space that will be tracked several times, in this case 2 and meanwhile Misty will track all your data.&#x20;


# Misty Arm

The Misty telescopic robotic arm is finally here. Find the front part that best suits your needs and share it!

Misty's telescopic arm opens a whole new world of applications. Misty can help you solve many tasks like grabbing and bringing objects around!

<figure><img src="/files/2j8mK1KS1ylxWMwq0pLQ" alt="" width="375"><figcaption></figcaption></figure>

### Materials

* [Arduino UNO](https://store.arduino.cc/products/arduino-uno-rev3)
* Breadboard
* Some wires of different lengths
* [Bluetooth Arduino module HC-05](https://www.amazon.it/dp/B0722MD4FY)
* One resistance 1kΩ and one 2 kΩ &#x20;
* [Servo SG90](https://www.amazon.it/dp/B0CRD8XP71)
* [Arduino breadboard support](/hardware-extension/arduino-breadboard-support)
* [Misty arm](/hardware-extension/misty-arm)
* Powerbank&#x20;
* Alligator pins
* [USB to open-cable adaptor](https://youtu.be/5lnIviZ9Aqs)

For testing the signal between Arduino and Misty

* One LED and one button
* One resistance of 330 Ω and one of 10kΩ

### Circuit

<figure><img src="/files/jz7Ky9lb6RPJAAB9dpDW" alt=""><figcaption><p>Arduino circuit</p></figcaption></figure>

<figure><img src="/files/Gp9qCuBu35t96JPniZoN" alt=""><figcaption><p>Picture of the real circuit</p></figcaption></figure>

Three points have to be clarified in the circuit because the software that creates the circuits doesn't have some elements:&#x20;

#### Misty Serial connections

<figure><img src="/files/gW2FJLr2SYrhBNCR0Yrk" alt=""><figcaption></figcaption></figure>

In the real circuit, [as explained in this section](/arduino/arduino), it's necessary to create these connections to allow communication between Misty and Arduino.

Connect the pins as explained [here](/arduino/arduino#links)

<figure><img src="/files/AO6fnumj8RqdlepZfEIs" alt=""><figcaption><p>Arduino-Misty serial connection</p></figcaption></figure>

#### Power supplies

<figure><img src="/files/sJkX77sminDSDXobKrpe" alt=""><figcaption></figcaption></figure>

In the real circuit, those pins are not connected to the necessary power bank because otherwise the Bluetooth module and the motors couldn't work simultaneously.

For this reason, we need to connect the 5V to the plus and the GND to the ground of the power bank to give power to the circuit.

<figure><img src="/files/22M0l0FU6tlOYhdZ15jZ" alt="" width="375"><figcaption><p>Insert the USB-open wires in the powerbank</p></figcaption></figure>

<figure><img src="/files/0UMBScphmw1y8evTVlQB" alt="" width="375"><figcaption><p>Use alligator clips to connect the open wires to the breadboard + and -.<br>Use the 5V from the arduino in the + and the ground in the -</p></figcaption></figure>

#### Arduino Bluetooth module HC-05

<figure><img src="/files/FMQMPCv2aTCUv1qRSKxK" alt=""><figcaption></figcaption></figure>

Of course what we have there it's not a LED but it's the Arduino Bluetooth Module HC-05.\
The pins of the LED represent the central four pins of the HC-05.

The VCC is connected to the 5V\
The GND is connected to the ground/minus\
The TXD is connected to pin 4\
The RXD is connected to two resistances. Connect the 1kΩ to the pin 3 and the 2kΩ to the ground.  &#x20;

<div><figure><img src="/files/FVHsXN4QWA5itSNpKa7U" alt="" width="188"><figcaption><p>Arduino Bluetooth Module HC-05 <br>board</p></figcaption></figure> <figure><img src="/files/Uu0T32q5Hefrsfh2j2f5" alt="" width="375"><figcaption><p>Real circuit</p></figcaption></figure></div>

&#x20;

### MIT app inventor

App file:

{% file src="/files/WIsPOvozollCZQZNJTMZ" %}

In order to comfortably move the arm from a smartphone application we will create a mobile app with [MIT app inventor](https://ai2.appinventor.mit.edu/).\
Once you open an account you can click on Projects>Import project(.aia) from your computer and select the file above.

Your app will be ready! You can download MIT app inventor from the app store of your phone.\
Once you have the app on your phone you can click on Build>Android app and scan the QR code with your phone. Download the app and use your arm!

If you're interested in building or designing your app you can drag and drop the elements that you need from the left side to the screen and set them in the position you prefer. Don't forget to insert the Bluetooth client (connectivity) and the clock (sensors) as well.\
Once you have the screen done you can build the code by clicking on Blocks.

At the top, we have two buttons for connecting the smartphone to the Arduino Bluetooth HC-05.

Then, with a very simple design you can find three sliders each controlling one motor.

Moving the slider will change the position of the motors. It's better to tap the position we want the motor to be rather than sliding it because it has a higher chance of straight success.

<figure><img src="/files/eS3DLGUmvWTdAJeC2UQJ" alt=""><figcaption><p>Designer area</p></figcaption></figure>

<figure><img src="/files/TkrXmPufQHDT3MaJ6nSZ" alt=""><figcaption><p>Blocks area</p></figcaption></figure>

### Arduino code

```cpp
#include <SoftwareSerial.h>
#include <Servo.h>

Servo servo01;
Servo servo02;

SoftwareSerial Bluetooth(3, 4); // Arduino(RX, TX) - HC-05 Bluetooth (TX, RX)

int servo1Pos, servo2Pos, MaPos; // current position
String dataIn = "";

void setup() {
  Serial.begin(9600);
  servo01.attach(10);
  servo02.attach(11);
  Bluetooth.begin(9600); // Default baud rate of the Bluetooth module
  Bluetooth.setTimeout(1);
  delay(20);
  pinMode(9, OUTPUT);      	//pin green LED
  pinMode(7, INPUT_PULLUP); 	//pin green button
}

void loop() {
  // Check for incoming data
  int pushedg = digitalRead(7); //green button
  if (pushedg == LOW){		// green button pressed condition
    digitalWrite(9, HIGH);	// green LED on
    Serial.println("green");	// write "green" in the serial
  } else {
    digitalWrite(9, LOW);	// green LED off
  }
  if (Bluetooth.available() > 0) {
    dataIn = Bluetooth.readString();  // Read the data as string
    if (dataIn.startsWith("s1")) {
      String dataInS = dataIn.substring(2, dataIn.length()); // Extract only the number. E.g. from "s1120" to "120"
      servo1Pos = dataInS.toInt();  // Convert the string into integer
      servo01.write(servo1Pos);
    }
    
    // Move Servo 2
    if (dataIn.startsWith("s2")) {
      String dataInS = dataIn.substring(2, dataIn.length());
      servo2Pos = dataInS.toInt();
      servo02.write(servo2Pos);
    }

    //Move Misty arm
    if (dataIn.startsWith("ma")) {
      String dataInS = dataIn.substring(2, dataIn.length());
      MaPos = dataInS.toInt();
      Serial.println(MaPos);
    }
  }
}
```

On the top, we define the libraries and the components that we will use in the code such as the servo library, the two servos, the Arduino Bluetooth module HC-05 and some variables.

In the setup it's important to set the serial begin to 9600 (this is the bound rate for the communication between Misty and Arduino) and to set also the Arduino Bluetooth module HC-05 on the same bound rate. \
\
In the loop, before `if (Bluetooth.available() > 0)`  we check if Misty can receive commands from the Arduino by clicking the button as in [this lesson](/arduino/arduino-lessons/arduino-to-misty). \
Later we just detect the message and and assign it to each motor.\
If the message starts with s1 or s2 the servos will move, if the message starts with ma (Misty arm) then Arduino will send a message to Misty to move the arm in that position.

### Misty code

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events

misty = Robot()
misty.change_led(255, 0, 0)

def msg(data):
    print(data)
    if data["message"]["message"] == "green":
        misty.change_led(0, 255, 0)
        print("green")
    else:
        misty.change_led(0, 255, 255)
        string = data["message"]["message"]
        arm_value = int(string)
        misty.move_arm("right", arm_value)
        print("moving arm")

misty.register_event(event_name="serial_message_event", event_type=Events.SerialMessage, callback_function=msg, keep_alive=True)
misty.keep_alive()
```

In this code, we only set a Serial message read event. In the event, if Misty receives the serial message "green", it means that the Arduino button is pushed so Misty will change her chest LED to green, otherwise it's the position to which the right arm has to go so we convert that string to an int variable.\
\
You can use this code in the Misty Studio as well, if you want to get the output when this is released you can use this code in your Desktop Environment and don't forget to add Misty's IP address in\
`misty = Robot().`

### Comments:

#### Arduino Bluetooth module HC-05

Once you plug-in your HC-05 go into the settings of your device and connect it with your phone.\
It will ask you to insert a password and you can use either 1234 or 0000.\
Once your module is connected to your phone it will appear in the list of available devices when you open your app and click on the button connect to connect with your HC-05 module.

#### Button and LED

It's not necessary to have the button and the LED in the circuit and so in the project but I used it to check if the communication with Misty was working so it may be helpful for your project too.

#### Credits

To build this project I took inspiration from[ this project](https://howtomechatronics.com/tutorials/arduino/diy-arduino-robot-arm-with-smartphone-control/). You can click on it and get more information about your new Misty arm.

#### Arm Motors

The slider of the Misty arm has been set in the range of -29 to 25 to prevent the new and longer arm from damaging the original motor that controls the arm movement. \
If the arm is accidentally triggered to go in a lower position then 25 shuts **immediately** Misty from the button in the back (the same one that you use to turn her on).&#x20;


# Misty weather forecaster

Have you ever thought that your Misty could become an excellent weather reporter and forecaster?

The Adafruit BME280 board will give you barometric pressure, temperature and humidity! <br>

<figure><img src="/files/TCKcTabgKmbAuANZW93z" alt=""><figcaption></figcaption></figure>

The humidity level is a nice report to have in your home. \
The barometric pressure data are temperature-dependent. \
This means that if your temperature goes up Misty will read a lower pressure reading.

### Materials:

* Misty's Arduino Backpack
* [Adafruit BME280 board](https://www.adafruit.com/product/2651)
* [Arduino breadboard support](/hardware-extension/arduino-breadboard-support)

### Adafruit BME280 instructions

You can find the instructions for this board in [this link](https://learn.adafruit.com/adafruit-bmp280-barometric-pressure-plus-temperature-sensor-breakout/arduino-test)

If you're using the breadboard you will need to solder the breadboard 1 pins into the circuitboard. \
You can slide the board over the pins and heat the pins, then apply solder. After your board is ready here is the pin layout, this is on an Arduino board but of course, it's the same on Misty backpack. You can use the 3.3volt supply.<br>

<figure><img src="/files/NmQ5thaAKMjNqgPQuTj4" alt="" width="563"><figcaption></figcaption></figure>

### Arduino Code:

```cpp
//Libraries
#include <Wire.h>
#include <SPI.h>
#include <Adafruit_Sensor.h>
#include <Adafruit_BME280.h>
#define SEALEVELPRESSURE_HPA (1013.25)
Adafruit_BME280 bme; // I2C
unsigned long delayTime;

// Void Setup to check the initial condition
void setup() {
  Serial.begin(9600);
  while(!Serial); // time to get serial running
  Serial.println(F("BME280 test"));
  unsigned status = bme.begin();
  if (!status) {
    Serial.println("Could not find a valid BME280 sensor, check wiring, address, sensor ID!");
    while (1) delay(10);
  }
  Serial.println("-- Default Test --");
  delayTime = 1000;
  Serial.println();
}

//Infinite function that reads and sends the data to Misty
void loop() {
  Serial.print("Temperature: ");
  Serial.print(bme.readTemperature());
  Serial.println(" *F");
  Serial.print("Pressure: ");
  Serial.print(bme.readPressure() / 100.0F);
  Serial.println(" hPa");
  Serial.print("Humidity: ");
  Serial.print(bme.readHumidity());
  Serial.println(" %");
  Serial.println();
  delay(delayTime);
}
```

You can follow [these instructions](https://learn.adafruit.com/adafruit-bmp280-barometric-pressure-plus-temperature-sensor-breakout/arduino-test#download-adafruit-bmp280-library-2957956) to complete the setup of the BME280 board before starting it.\
\
In this Arduino code, we declare the libraries that we'll use, void setup and void loop.\
\
The void setup prepares the code and runs only once, in this case, we check the presence of the BME280 sensor. \
\
In the void loop, we read and send the data from the Arduino sensor to Misty via serial port.

### Misty Code:

```python
from mistyPy.Robot import Robot
from mistyPy.Events import Events
import time
import threading

# Connect to Misty
misty = Robot("YOUR MISTY IP")
print("Successfully connected to Misty.")
misty.change_led(0, 255, 0)

#Variables
initial_pressure = None
initial_temperature = None
temperature = None
pressure = None
humidity = None
announce_interval = 360 # 6 minutes in seconds

#Functions
#Normalizing data
misty.speak("Let's see what's going on with your weather")
def parse_data(data):
    message = data["message"]["message"]
    try:
      if "Temperature" in message:
        temp_index = message.index("Temperature: ") + len("Temperature: ")
        temp_end_index = message.index(" *F", temp_index)
        temperature_celsius = float(message[temp_index:temp_end_index].strip())
        temperature_fahrenheit = temperature_celsius * 9.0/5.0 + 32.0
        return ("temperature", round(temperature_fahrenheit, 2))
      elif "Pressure" in message:
        pressure_index = message.index("Pressure: ") + len("Pressure: ")
        pressure_end_index = message.index(" hPa",pressure_index)
        pressure = message[pressure_index:pressure_end_index].strip()
        return ("pressure", pressure)
      elif "Humidity" in message:
        humidity_index = message.index("Humidity: ") + len("Humidity: ")
        humidity_end_index = message.index(" %", humidity_index)
        humidity = message[humidity_index:humidity_end_index].strip()
        return ("humidity", humidity)
    except ValueError:
      return None
 
#Change Misty's screen
def display_data(temperature, pressure, humidity):
    display_text = f"\nTemperature: {temperature} *F\nPressure:{pressure} hPa\nHumidity: {humidity} %"
    misty.display_image("e_SystemBlackScreen.jpg")
    misty.display_text(display_text, "Large")
    print(f"Updated display with: {display_text}")

#Announcement of the weather condition
def announce_weather_condition(pressure):
    if pressure > 974:# change this value according to your experiments
        misty.speak("There is no storm system in the vicinity for sometime and most likely sunny skies.")
    elif pressure < 960:
        misty.speak("We are currently within a precipitation system.")

#Starting readings
def announce_readings():
    global temperature, pressure, humidity
    while True: #infinite loop
        #while reading data
        if temperature and pressure and humidity:
            misty.speak(f"The current temperature is {temperature} degrees Fahrenheit, the pressure is {pressure} hPa, and the humidity is {humidity} percent.")
            announce_weather_condition(pressure)
        time.sleep(announce_interval)

#Pressure variation
def check_pressure_change(new_pressure):
    global initial_pressure
    if initial_pressure is None:
       initial_pressure = new_pressure
       pressure_difference = new_pressure - initial_pressure
       return pressure_difference
    if abs(pressure_difference) >= 5:
       direction = "up" if pressure_difference > 0 else "down"
       misty.speak(f"I am detecting a distinct change in pressure by 5 points {direction}.")

#Read the data
def serial_msg_test(data):
    global temperature, pressure, humidity, initial_temperature, initial_pressure
    print(f"Received data: {data}")
    parsed = parse_data(data)
    if parsed:
        if parsed[0] == "temperature":
            temperature = parsed[1]
            if initial_temperature is None:
                initial_temperature = temperature
                misty.speak(f"The current temperature is {initial_temperature} degrees Fahrenheit.")
        elif parsed[0] == "pressure":
            pressure = float(parsed[1])
            if initial_pressure is None:
                initial_pressure = pressure
                misty.speak(f"The current pressure is {initial_pressure} hPa.")
                announce_weather_condition(pressure)
            check_pressure_change(pressure)
        elif parsed[0] == "humidity":
            humidity = parsed[1]
    
    if temperature and pressure and humidity:
        display_data(temperature, pressure, humidity)

# Start the announcement thread
announcement_thread = threading.Thread(target=announce_readings)
announcement_thread.daemon = True
announcement_thread.start()

# Register serial event
misty.register_event(event_name="serial_message_event", event_type=Events.SerialMessage, callback_function=serial_msg_test, keep_alive=True)
print("Serial event registered. Waiting for data...")

# Keep the program running 
# CHOOSE OF THE FOLLOWING SOLUTION

#you can use this variation suggested by 
try:
    while True:
        pass # No delay to ensure real-time updates
except KeyboardInterrupt:
    print("Program terminated by user.")
except Exception as e:
    print(f"Error in keep_alive: {e}")

#or the usual misty command
misty.keep_alive()
```

This is a very well-built code. On top, we have the libraries that Misty will use. Then we connect with our robot via IP address as usual and declare the variables that we'll use in the code.

Then we have the functions that we'll use to read and normalize the data, check the variation in pressure and change Misty's display and behaviour according to the data.

In the last lines, we find the function that registers the event and two possible solutions to keep Misty alive. Remember to use only one of the two solutions.

#### Notes from the creator:

The barometric values I entered for when Misty says "a storm is en route or not ..." will change depending on where you live and the altitude you are at.&#x20;

The best way to figure out your own is to run the program frequently, on sunny clear days, rainy days and most importantly note the levels during a major storm (snow, thunderstorms etc).&#x20;

Eventually, you will figure out which values are appropriate to your own location.

### Thanks

A special thanks to Scott who shared with us his Misty weather forecaster project!\
You can find his contacts here if you are willing to collaborate with him on new Misty projects!\
<crmfghtr@yahoo.com>&#x20;


# Arduino breadboard support

You can use Misty's Arduino backpack for all your projects but it was missing a fundamental piece: where can I put the breadboard?

On this page, you will be able to download the .stl files containing the models for 3D printing your Arduino breadboard support.&#x20;

#### .stl files:

{% file src="/files/xiM4P80ZX0r9beNWlaiN" %}

{% file src="/files/kZEfds4zmhFePxvrXQRe" %}

{% file src="/files/DTyJlqrZuCmENftTF03M" %}

<div><figure><img src="/files/5Bx0oNEBkwcndFWvgQZP" alt=""><figcaption><p>bottom part</p></figcaption></figure> <figure><img src="/files/lIgVwJQ6wGi4Owc46iUK" alt=""><figcaption><p>middle part</p></figcaption></figure> <figure><img src="/files/1uU6aP69sp69h0qmlyEQ" alt=""><figcaption><p>upper part</p></figcaption></figure></div>

#### Assembly steps:&#x20;

1. **Prep the Parts:** Before glueing, lightly sand with sandpaper the surfaces you'll be joining on the bottom and middle sections. This creates a better bond. Sand the sides of the three middle blocks as well, for a smoother fit in the upper part.
2. **Assemble the Support:** Once the surfaces are done, glue the bottom and middle sections together.
3. **Modify Current Misty's Trailer Hitch:**
   * Remove the trailer hitch on Misty's back bottom.
   * Unscrew the bottom screw to remove the outer part of the trailer hitch.
   * Take your new 3D-printed piece and attach it to the trailer hitch using the same screw, just like the original piece.
4. **Attach:**
   * Now that the support is complete, place the upper part on it.

**Congratulations!** You're all set to start building amazing projects with Misty and her Arduino Backpack. With your sensors connected to the breadboard, you can program Misty to track anything you desire, making her a fun and interactive companion!

<div><figure><img src="/files/I3XQKzPwa5IfjYrLpYC0" alt="" width="188"><figcaption><p>prepare the pieces</p></figcaption></figure> <figure><img src="/files/KMSrjgckqpZ0hhZ9EGih" alt="" width="188"><figcaption><p>glue the pieces</p></figcaption></figure></div>

<div><figure><img src="/files/WQyUJXMrzTjXRhlQrSgo" alt=""><figcaption><p>trailer hitch</p></figcaption></figure> <figure><img src="/files/wMxUNQ9f1j7wpdxioj2P" alt=""><figcaption><p>unscrew</p></figcaption></figure> <figure><img src="/files/DBIfl7AXMswRcJx8Z3Vj" alt=""><figcaption><p>screw</p></figcaption></figure></div>

<div><figure><img src="/files/KkocvIoUhBHABycDAYv0" alt=""><figcaption><p>replace with the new trailer hitch</p></figcaption></figure> <figure><img src="/files/bX47axzSlMlRKfHvRxtL" alt=""><figcaption><p>add the upper part</p></figcaption></figure> <figure><img src="/files/WfGHStmU2hqCAbNvCUMB" alt=""><figcaption><p>insert the board</p></figcaption></figure></div>

***

Developing a second version of the breadboard support to fit the Arduino UNO as well.


# Misty arm

It was a must for Misty to have a telescopic robotic arm, to give her even more degrees of freedom and the ability to grip or bring around several objects!

On this page, you will be able to download the .stl files containing the models for 3D printing your Misty arm.

#### .stl files:&#x20;

{% file src="/files/HRYn8fnsJPVvPdpcE8PH" %}

{% file src="/files/lKM7cPd5tGWO91Qw4Mpb" %}

{% file src="/files/IaPnSvfkXrLMzdZHqgx8" %}

<div><figure><img src="/files/uRE7REqA4VDn2gSgwTMv" alt="" width="375"><figcaption><p>arm</p></figcaption></figure> <figure><img src="/files/Z8XxdPyIByekG100SBOx" alt="" width="200"><figcaption><p>gear</p></figcaption></figure> <figure><img src="/files/yEmDI8OGqNj50O9V1t8x" alt="" width="375"><figcaption><p>mobile</p></figcaption></figure></div>

#### Adjustments

The front part of the arm has been developed to be completely customized.&#x20;

In fact, we want to have the possibility to have different tools but not to print a new arm every time.  To solve this we inserted a negative M - of Misty - where you can put the front part that suits your needs better. For example, we chose to insert this [gripper](https://www.thingiverse.com/thing:5149951) - I modified the size of the two gears of the gripper to 103% to have a better fit.&#x20;

We modified the base of the gripper deleting the behind wall and inserting a positive M that matches with the mobile one.

<figure><img src="/files/Hc91DxOsb9LnlmhNZ38J" alt="" width="375"><figcaption><p>Modified base of the gripper</p></figcaption></figure>

{% file src="/files/vWj3vVPMIQTZ67ncOSAF" %}
M letter for matching pieces
{% endfile %}

{% file src="/files/U9fUeku0pwfQwRcdd5Cq" %}

The servo motors that we'll use in this project are the positional SG90.&#x20;

#### Assembly

1. **Prepare the new arm** \
   \- Screw the SG90 servo motor in the arm with the small gear towards the outside part. \
   \- Slice the mobile part in the arm. \
   \- Insert the gear in the servo motor (the gear will rotate in the middle part of the rail, so you don't need to insert it from the very beginning of the gear)\
   \- Insert your front part in the mobile.&#x20;
2. **Screw the new arm**\
   **-** Unscrew the old arm. \
   \- With the same screw insert the new 3d printed arm.

<div><figure><img src="/files/BQKOlBPvru63lkMm55Eu" alt="" width="188"><figcaption><p>arm</p></figcaption></figure> <figure><img src="/files/5uGxwMT3LsZLfGeUnmAQ" alt="" width="188"><figcaption><p>mobile part and gear</p></figcaption></figure></div>

<figure><img src="/files/IAiARyV7kRFlVdFdizuJ" alt="" width="375"><figcaption><p>complete arm</p></figcaption></figure>

#### Pictures

<div><figure><img src="/files/rNlRm5ffkmfSM7U7ofdx" alt=""><figcaption><p>arm extension</p></figcaption></figure> <figure><img src="/files/3xVFTrTIySlLccFwXsy0" alt=""><figcaption><p>side picture with cup holder</p></figcaption></figure></div>

<div><figure><img src="/files/5duujNZwTeOhVy8lHiDL" alt=""><figcaption><p>arm</p></figcaption></figure> <figure><img src="/files/K6KBfF9LhAmadbvTqSoy" alt=""><figcaption><p>standard position</p></figcaption></figure></div>


# Tin holder

Why Misty can't bring you a soda?

In our [Thingiverse profile](https://www.thingiverse.com/misty_maker/designs), you can find many useful extensions and attachments to Misty's hardware, such as a holder for small tins.\
\
There is no assembly required, print and enjoy!\
&#x20;

{% file src="/files/CDF2qJml6sl4pAaJHSgQ" %}

{% file src="/files/pkZTUEZxAqUohxr5V9na" %}

<div><figure><img src="/files/Z3oViRgMjfS8t0AZPOPU" alt="" width="563"><figcaption></figcaption></figure> <figure><img src="/files/4myP9NckbpgEfbBDjAp2" alt="" width="563"><figcaption></figcaption></figure></div>


# Image files

On this page you can find all Misty's eyes and their reference names.

<table data-view="cards"><thead><tr><th></th><th data-hidden></th><th data-hidden></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td>e_Admiration.jpg</td><td></td><td></td><td><a href="/files/1Yrv1ontoWopmWFmTGc6">/files/1Yrv1ontoWopmWFmTGc6</a></td></tr><tr><td>e_Aggressiveness.jpg </td><td></td><td></td><td><a href="/files/lAkomOpUc56PO66J2ugR">/files/lAkomOpUc56PO66J2ugR</a></td></tr><tr><td>e_Amazement.jpg </td><td></td><td></td><td><a href="/files/I6zoh54iIU3wWYuh37R0">/files/I6zoh54iIU3wWYuh37R0</a></td></tr><tr><td>e_Anger.jpg </td><td></td><td></td><td><a href="/files/pXLGXldPhQeRBn2tNZHw">/files/pXLGXldPhQeRBn2tNZHw</a></td></tr><tr><td>e_ApprehensionConcerned.jpg </td><td></td><td></td><td><a href="/files/VqcPzikFv4AYoawiU4dI">/files/VqcPzikFv4AYoawiU4dI</a></td></tr><tr><td>e_Contempt.jpg </td><td></td><td></td><td><a href="/files/XkKptxaDek2PfGvl6LEu">/files/XkKptxaDek2PfGvl6LEu</a></td></tr><tr><td>e_ContentLeft.jpg</td><td></td><td></td><td><a href="/files/ebWG0vpRSsn67un0QOMx">/files/ebWG0vpRSsn67un0QOMx</a></td></tr><tr><td> e_ContentRight.jpg </td><td></td><td></td><td><a href="/files/OGIBcUx0sCfEG0skucO9">/files/OGIBcUx0sCfEG0skucO9</a></td></tr><tr><td>e_DefaultContent.jpg</td><td></td><td></td><td><a href="/files/NahSv2vTjVRLDLjGwKyQ">/files/NahSv2vTjVRLDLjGwKyQ</a></td></tr><tr><td> e_Disgust.jpg </td><td></td><td></td><td><a href="/files/I90By3eZbeFB3RWYwo0y">/files/I90By3eZbeFB3RWYwo0y</a></td></tr><tr><td>e_Disoriented.jpg </td><td></td><td></td><td><a href="/files/wpIxQM6WgU8cfYPdGGJh">/files/wpIxQM6WgU8cfYPdGGJh</a></td></tr><tr><td>e_EcstacyHilarious.jpg</td><td></td><td></td><td><a href="/files/HD1mjfGsu7OKXHLcMVX6">/files/HD1mjfGsu7OKXHLcMVX6</a></td></tr><tr><td> e_EcstacyStarryEyed.jpg</td><td></td><td></td><td><a href="/files/SEeg2juB4gI5AL9haLJ3">/files/SEeg2juB4gI5AL9haLJ3</a></td></tr><tr><td> e_Fear.jpg </td><td></td><td></td><td><a href="/files/CL53ohetgjcPsoxkCh1f">/files/CL53ohetgjcPsoxkCh1f</a></td></tr><tr><td>e_Grief.jpg</td><td></td><td></td><td><a href="/files/555ctmXPHCOmzS456qFH">/files/555ctmXPHCOmzS456qFH</a></td></tr><tr><td> e_Joy.jpg </td><td></td><td></td><td><a href="/files/k2eJcZq86QOWvbEpsmNz">/files/k2eJcZq86QOWvbEpsmNz</a></td></tr><tr><td>e_Joy2.jpg </td><td></td><td></td><td><a href="/files/aJBvS3m2hc3JHAcEvydm">/files/aJBvS3m2hc3JHAcEvydm</a></td></tr><tr><td>e_JoyGoofy.jpg</td><td></td><td></td><td><a href="/files/0XtmycsCHM2gN5yIuQtk">/files/0XtmycsCHM2gN5yIuQtk</a></td></tr><tr><td>e_JoyGoofy2.jpg </td><td></td><td></td><td><a href="/files/IsCcfl2G289ItbG9dIb4">/files/IsCcfl2G289ItbG9dIb4</a></td></tr><tr><td>e_JoyGoofy3.jpg</td><td></td><td></td><td><a href="/files/B9ysujkfEXzKsyxUBeTB">/files/B9ysujkfEXzKsyxUBeTB</a></td></tr><tr><td>e_Love.jpg </td><td></td><td></td><td><a href="/files/mn9miFP0q8er3E06QbKf">/files/mn9miFP0q8er3E06QbKf</a></td></tr><tr><td>e_Rage.jpg </td><td></td><td></td><td><a href="/files/9oTxd48p1oiwTuis8GdL">/files/9oTxd48p1oiwTuis8GdL</a></td></tr><tr><td>e_Rage2.jpg </td><td></td><td></td><td><a href="/files/VZl3Khys1lBz7Qphg04Q">/files/VZl3Khys1lBz7Qphg04Q</a></td></tr><tr><td>e_Rage3.jpg </td><td></td><td></td><td><a href="/files/8QfKi6XoJsi6WmhTvFa9">/files/8QfKi6XoJsi6WmhTvFa9</a></td></tr><tr><td>e_Rage4.jpg </td><td></td><td></td><td><a href="/files/3uDC0yi0XRNw7rIiBbgt">/files/3uDC0yi0XRNw7rIiBbgt</a></td></tr><tr><td>e_RemorseShame.jpg </td><td></td><td></td><td><a href="/files/HtdhIA6X7FSNizk9V8ms">/files/HtdhIA6X7FSNizk9V8ms</a></td></tr><tr><td>e_Sadness.jpg </td><td></td><td></td><td><a href="/files/5jEL3sjYuOwD0cxqNx3D">/files/5jEL3sjYuOwD0cxqNx3D</a></td></tr><tr><td>e_Sleeping.jpg</td><td></td><td></td><td><a href="/files/X8KMvK04yLjmbCot0TMF">/files/X8KMvK04yLjmbCot0TMF</a></td></tr><tr><td>e_SleepingZZZ.jpg</td><td></td><td></td><td><a href="/files/NnAOkMISugGRNswzTm15">/files/NnAOkMISugGRNswzTm15</a></td></tr><tr><td>e_Sleepy.jpg </td><td></td><td></td><td><a href="/files/drWKqvHJKVc6Zr3o4Wnp">/files/drWKqvHJKVc6Zr3o4Wnp</a></td></tr><tr><td>e_Sleepy2.jpg </td><td></td><td></td><td><a href="/files/XPK62D4cjiinNdtliUzZ">/files/XPK62D4cjiinNdtliUzZ</a></td></tr><tr><td>e_Sleepy3.jpg </td><td></td><td></td><td><a href="/files/gLYeNpbpYlWwfS4DvMIv">/files/gLYeNpbpYlWwfS4DvMIv</a></td></tr><tr><td>e_Sleepy4.jpg </td><td></td><td></td><td><a href="/files/RnFmWRT00vXn0wubMuNR">/files/RnFmWRT00vXn0wubMuNR</a></td></tr><tr><td>e_Surprise.jpg </td><td></td><td></td><td><a href="/files/HyjDb1FH7va94ioVMklr">/files/HyjDb1FH7va94ioVMklr</a></td></tr><tr><td>e_SystemBlinkLarge.jpg </td><td></td><td></td><td><a href="/files/nd4hORz3MXdCMJwiYryu">/files/nd4hORz3MXdCMJwiYryu</a></td></tr><tr><td>e_SystemBlinkStandard.jpg </td><td></td><td></td><td><a href="/files/HhRDSZNaa5LpCwjzTxKC">/files/HhRDSZNaa5LpCwjzTxKC</a></td></tr><tr><td>e_SystemCamera.jpg </td><td></td><td></td><td><a href="/files/oivejpntguPu4jrsAqra">/files/oivejpntguPu4jrsAqra</a></td></tr><tr><td>e_Terror.jpg </td><td></td><td></td><td><a href="/files/iHQyTXF5kkQbj6HvSZuQ">/files/iHQyTXF5kkQbj6HvSZuQ</a></td></tr><tr><td>e_Terror2.jpg </td><td></td><td></td><td><a href="/files/wsPRrsaOWcK6HaLb1d5d">/files/wsPRrsaOWcK6HaLb1d5d</a></td></tr><tr><td>e_TerrorLeft.jpg </td><td></td><td></td><td><a href="/files/AxoYRu1x5QdzKFYmFDeu">/files/AxoYRu1x5QdzKFYmFDeu</a></td></tr><tr><td>e_TerrorRight.jpg </td><td></td><td></td><td><a href="/files/9Oy42cjCZVhPQXlZKvE1">/files/9Oy42cjCZVhPQXlZKvE1</a></td></tr><tr><td>e_SystemBlackScreen.jpg </td><td></td><td></td><td><a href="/files/dw49ZBb4a89OZOONLAER">/files/dw49ZBb4a89OZOONLAER</a></td></tr><tr><td>e_SystemFlash.jpg </td><td></td><td></td><td><a href="/files/lplyjItmYvsKxHxuSQ61">/files/lplyjItmYvsKxHxuSQ61</a></td></tr><tr><td>e_SystemGearPrompt.jpg </td><td></td><td></td><td><a href="/files/rOANKAUEHvdGryngonst">/files/rOANKAUEHvdGryngonst</a></td></tr><tr><td>e_SystemLogoPrompt.jpg </td><td></td><td></td><td><a href="/files/Bti5phFtdC2r2y7VFY8s">/files/Bti5phFtdC2r2y7VFY8s</a></td></tr></tbody></table>


# Audio files

In this page you can find the name of all of Misty's default vocal expressions and audio.

* s\_Acceptance.wav&#x20;
* s\_Amazement.wav
* &#x20;s\_Amazement2.wav
* s\_Anger wav&#x20;
* s\_Anger2.wav&#x20;
* s\_Anger3.wav&#x20;
* s\_Anger4.wav&#x20;
* s\_Annoyance wav
* &#x20;s\_Annoyance2.wav&#x20;
* s\_Annoyance3.wav
* s\_Annoyance4.wav&#x20;
* s\_Awe wav&#x20;
* s\_Awe2.wav&#x20;
* s\_Awe3.wav&#x20;
* s\_Boredom.wav&#x20;
* s\_Disapproval.wav
* &#x20;s\_Disgust.wav
* &#x20;s\_Disgust2.wav&#x20;
* s\_Disgust3.wav&#x20;
* s\_DisorientedConfused.wav
* &#x20;s\_DisorientedConfused2.wav
* s\_DisorientedConfused3.wav
* &#x20;s\_DisorientedConfused4 wav&#x20;
* s\_DisorientedConfused5 wav
* &#x20;s\_DisorientedConfused6 wav
* &#x20;s\_Distraction.wav s\_Ecstacy.wav
* &#x20;s\_Ecstacy2 wav
* s\_Fear.wav&#x20;
* s\_Grief.wav&#x20;
* s\_Grief2.wav&#x20;
* s\_Grief3.wav
* &#x20;s\_Grief4.wav&#x20;
* s\_Joy.wav&#x20;
* s\_Joy2 wav&#x20;
* s\_Joy3.wav&#x20;
* s\_Joy4.wav&#x20;
* s\_Loathing.wav&#x20;
* s\_Love.wav&#x20;
* s\_PhraseByeBye wav&#x20;
* s\_PhraseEvilAhHa.wav&#x20;
* s\_PhraseHello.wav&#x20;
* s\_PhraseNoNoNo.wav&#x20;
* s\_PhraseOopsy.wav&#x20;
* s\_PhraseOwOwOw\.wav&#x20;
* s\_PhraseOwwww\.wav&#x20;
* s\_PhraseUhOh.wav&#x20;
* s\_Rage.wav&#x20;
* s\_Sadness.way&#x20;
* s\_Sadness2.wav&#x20;
* s\_Sadness3.wav&#x20;
* s\_Sadness4 wav&#x20;
* s\_Sadness5.wav&#x20;
* s\_Sadness6.wav&#x20;
* s\_Sadness7.wav
* &#x20;s\_Sleepy.wav&#x20;
* s\_Sleepy2.wav&#x20;
* s\_Sleepy3.wav
* &#x20;s\_Sleepy4.wav&#x20;
* s\_SleepySnore.wav&#x20;
* s\_SystemCameraShutter.wav&#x20;
* s\_SystemFailure wav&#x20;
* s\_SystemSuccess.wav&#x20;
* s\_SystemWakeWord.wav


# Languages

In this page you can find a list of all Misty's languages and their respective reference codes for API calls.

| Language Code        | Language Name                    |
| -------------------- | -------------------------------- |
| "ar-xa-x-arc-local"  | Arabic 1                         |
| "ar-xa-x-ard-local"  | Arabic 2                         |
| "ar-xa-x-are-local"  | Arabic 3                         |
| "ar-xa-x-arz-local"  | Arabic 4                         |
| "bn-bd-x-ban-local"  | Bengali 1 (Bangladesh)           |
| "bn-in-x-bin-local"  | Bengali 2 (India)                |
| "bn-in-x-bnf-local"  | Bengali 3 (India)                |
| "bn-in-x-bnm-local"  | Bengali 4 (India)                |
| "yue-hk-x-jar-local" | Chinese 1 (Hong Kong)            |
| "cs-cz-x-jfs-local"  | Czech 1 (Czech Republic)         |
| "da-dk-x-kfm-local"  | Danish 1 (Denmark)               |
| "da-dk-x-nmm-local"  | Danish 2 (Denmark)               |
| "da-dk-x-sfp-local"  | Danish 3 (Denmark)               |
| "da-dk-x-vfb-local"  | Danish 4 (Denmark)               |
| "de-de-x-deb-local"  | German 1 (Germany)               |
| "de-de-x-deg-local"  | German 2 (Germany)               |
| "de-de-x-nfh-local"  | German 3 (Germany)               |
| "el-gr-x-vfz-local"  | Greek 2 (Greece)                 |
| "en-au-x-afh-local"  | English 1 (Australia)            |
| "en-au-x-aua-local"  | English 2 (Australia)            |
| "en-au-x-aub-local"  | English 3 (Australia)            |
| "en-au-x-auc-local"  | English 4 (Australia)            |
| "en-au-x-aud-local"  | English 5 (Australia)            |
| "en-gb-x-fis-local"  | English 6 (UK)                   |
| "en-gb-x-gba-local"  | English 7 (UK)                   |
| "en-gb-x-gbb-local"  | English 8 (UK)                   |
| "en-gb-x-gbc-local"  | English 9 (UK)                   |
| "en-gb-x-gbd-local"  | English 10 (UK)                  |
| "en-gb-x-rjs-local"  | English 11 (UK)                  |
| "en-in-x-ahp-local"  | English 12 (India)               |
| "en-in-x-cxx-local"  | English 13 (India)               |
| "en-in-x-ena-local"  | English 14 (India)               |
| "en-in-x-enc-local"  | English 15 (India)               |
| "en-in-x-end-local"  | English 16 (India)               |
| "en-in-x-ene-local"  | English 17 (India)               |
| "en-ng-x-tfn-local"  | English 18 (Nigerian)            |
| "en-us-x-sfg-local"  | English 19 (US) \[Misty Default] |
| "en-us-x-tpd-local"  | English 20 (US)                  |
| "es-es-x-ana-local"  | Spanish 1 (Spain)                |
| "es-us-x-sfb-local"  | Spanish 2 (US)                   |
| "et-ee-x-tms-local"  | Estonian 1                       |
| "fi-fi-x-afi-local"  | Finnish 1 (Finland)              |
| "fil-ph-x-cfc-local" | Filipino 1 (Philippines)         |
| "fil-ph-x-fic-local" | Filipino 2 (Philippines)         |
| "fil-ph-x-fid-local" | Filipino 3 (Philippines)         |
| "fil-ph-x-fie-local" | Filipino 4 (Philippines)         |
| "fr-ca-x-caa-local"  | French 1 (Canada)                |
| "fr-ca-x-cab-local"  | French 2 (Canada)                |
| "fr-ca-x-cav-local"  | French 3 (Canada)                |
| "fr-ca-x-cad-local"  | French 4 (Canada)                |
| "fr-fr-x-fra-local"  | French 5 (France)                |
| "fr-fr-x-frb-local"  | French 6 (France)                |
| "fr-fr-x-frc-local"  | French 7 (France)                |
| "fr-fr-x-frd-local"  | French 8 (France)                |
| "fr-fr-x-vlf-local"  | French 9 (France)                |
| "gu-in-x-guf-local"  | Gujarati 1 (India)               |
| "gu-in-x-gum-local"  | Gujarati 2 (India)               |
| "hi-in-x-cfn-local"  | Hindi 1 (India)                  |
| "hi-in-x-hia-local"  | Hindi 2 (India)                  |
| "hi-in-x-hic-local"  | Hindi 3 (India)                  |
| "hi-in-x-hid-local"  | Hindi 4 (India)                  |
| "hi-in-x-hie-local"  | Hindi 5 (India)                  |
| "hu-hu-x-kfl-local"  | Hungarian 1 (Hungary)            |
| "id-id-x-dfz-local"  | Indonesian 1 (Indonesia)         |
| "id-id-x-idc-local"  | Indonesian 2 (Indonesia)         |
| "id-id-x-idd-local"  | Indonesian 3 (Indonesia)         |
| "id-id-x-ide-local"  | Indonesian 4 (Indonesia)         |
| "it-it-x-itb-local"  | Italian 1 (Italy)                |
| "it-it-x-itc-local"  | Italian 2 (Italy)                |
| "it-it-x-itd-local"  | Italian 3 (Italy)                |
| "it-it-x-kda-local"  | Italian 4 (Italy)                |
| "ja-jp-x-htm-local"  | Japanese 1 (Japan)               |
| "ja-jp-x-jab-local"  | Japanese 2 (Japan)               |
| "ja-jp-x-jac-local"  | Japanese 3 (Japan)               |
| "ja-jp-x-jad-local"  | Japanese 4 (Japan)               |
| "jv-id-x-jvf-local"  | Javanese 1                       |
| "km-kh-x-khm-local"  | Khmer 1                          |
| "kn-in-x-knf-local"  | Kannada 1 (India)                |
| "kn-in-x-knm-local"  | Kannada 2 (India)                |
| "ko-kr-x-ism-local"  | Korean 1 (South Korea)           |
| "ko-kr-x-kob-local"  | Korean 2 (South Korea)           |
| "ko-kr-x-koc-local"  | Korean 3 (South Korea)           |
| "ko-kr-x-kod-local"  | Korean 4 (South Korea)           |
| "ml-in-x-mlf-local"  | Malayalam 1 (India)              |
| "ml-in-x-mlm-local"  | Malayalam 2 (India)              |
| "ml-in-x-mrf-local"  | Malayalam 3 (India)              |
| "cmn-cn-x-ccc-local" | Mandarin Chinese 1               |
| "cmn-cn-x-ccd-local" | Mandarin Chinese 2               |
| "cmn-cn-x-cce-local" | Mandarin Chinese 3               |
| "cmn-cn-x-ssa-local" | Mandarin Chinese 4               |
| "cmn-tw-x-ctc-local" | Mandarin Chinese 5               |
| "cmn-tw-x-ctd-local" | Mandarin Chinese 6               |
| "cmn-tw-x-cte-local" | Mandarin Chinese 7               |
| "nb-no-x-cfl-local"  | Norwegian 1 (Norway)             |
| "nb-no-x-cmj-local"  | Norwegian 2 (Norway)             |
| "nb-no-x-rfj-local"  | Norwegian 3 (Norway)             |
| "nb-no-x-tfs-local"  | Norwegian 4 (Norway)             |
| "nb-no-x-tmg-local"  | Norwegian 5 (Norway)             |
| "ne-np-x-nep-local"  | Nepali 1                         |
| "nl-nl-x-bmh-local"  | Dutch 1 (Netherlands)            |
| "nl-nl-x-dma-local"  | Dutch 2 (Netherlands)            |
| "nl-nl-x-lfc-local"  | Dutch 3 (Netherlands)            |
| "nl-nl-x-tfb-local"  | Dutch 4 (Netherlands)            |
| "nl-nl-x-yfr-local"  | Dutch 5 (Netherlands)            |
| "pl-pl-x-afb-local"  | Polish 1 (Poland)                |
| "pl-pl-x-bmg-local"  | Polish 2 (Poland)                |
| "pl-pl-x-jmk-local"  | Polish 3 (Poland)                |
| "pl-pl-x-oda-local"  | Polish 4 (Poland)                |
| "pl-pl-x-zfg-local"  | Polish 5 (Poland)                |
| "pt-br-x-afs-local"  | Portuguese 1 (Brazil)            |
| "pt-pt-x-jfb-local"  | Portuguese 2 (Portugal)          |
| "pt-pt-x-jmn-local"  | Portuguese 3 (Portugal)          |
| "pt-pt-x-pmj-local"  | Portuguese 4 (Portugal)          |
| "pt-pt-x-sfs-local"  | Portuguese 5 (Portugal)          |
| "ro-ro-x-vfv-local"  | Romanian 1 (Romania)             |
| "ru-ru-x-dfc-local"  | Russian 1 (Russia)               |
| "ru-ru-x-ruc-local"  | Russian 2 (Russia)               |
| "ru-ru-x-rud-local"  | Russian 3 (Russia)               |
| "ru-ru-x-rue-local"  | Russian 4 (Russia)               |
| "ru-ru-x-ruf-local"  | Russian 5 (Russia)               |
| "si-lk-x-sin-local"  | Sinhalese 1                      |
| "sk-sk-x-sfk-local"  | Slovak 1 (Slovakia)              |
| "su-id-x-suf-local"  | Sundanese 2                      |
| "sv-se-x-lfs-local"  | Swedish 1 (Sweden)               |
| "ta-in-x-taf-local"  | Tamil 1 (India)                  |
| "ta-in-x-tag-local"  | Tamil 2 (India)                  |
| "ta-in-x-tef-local"  | Tamil 3 (India)                  |
| "ta-in-x-tem-local"  | Tamil 4 (India)                  |
| "th-th-x-mol-local"  | Thai 1 (Thailand)                |
| "tr-tr-x-ama-local"  | Turkish 1 (Turkey)               |
| "tr-tr-x-cfs-local"  | Turkish 2 (Turkey)               |
| "tr-tr-x-efu-local"  | Turkish 3 (Turkey)               |
| "tr-tr-x-mfm-local"  | Turkish 4 (Turkey)               |
| "tr-tr-x-tmc-local"  | Turkish 5 (Turkey)               |
| "uk-ua-x-hfd-local"  | Ukrainian 1 (Ukraine)            |
| "ur-pk-x-cfn-local"  | Urdu 1 (Pakistan)                |
| "ur-pk-x-urm-local"  | Urdu 2 (Pakistan)                |
| "vi-vn-x-gft-local"  | Vietnamese 1 (Vietnam)           |
| "vi-vn-x-vic-local"  | Vietnamese 2 (Vietnam)           |
| "vi-vn-x-vid-local"  | Vietnamese 3 (Vietnam)           |
| "vi-vn-x-vie-local"  | Vietnamese 4 (Vietnam)           |
| "vi-vn-x-vif-local"  | Vietnamese 5 (Vietnam)           |


# Known objects

In this page, you'll be able to find all Misty's default known objects.

In this page, you'll be able to find all Misty's default known objects and their labels.

## Humans and Animals

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">Person</td><td><a href="/files/9xjF8aNnStrktUEbfFIb">/files/9xjF8aNnStrktUEbfFIb</a></td></tr><tr><td align="center">Bird</td><td><a href="/files/VbOSmfJ85mYLhSmeoFxv">/files/VbOSmfJ85mYLhSmeoFxv</a></td></tr><tr><td align="center">Cat</td><td><a href="/files/PDPN4cn4MlgQlC3jz0H9">/files/PDPN4cn4MlgQlC3jz0H9</a></td></tr><tr><td align="center">Dog</td><td><a href="/files/T6xWXv6YGs0c1qSWsGFW">/files/T6xWXv6YGs0c1qSWsGFW</a></td></tr><tr><td align="center">Sheep</td><td><a href="/files/RI3fKPkdH2yGeylYfpaf">/files/RI3fKPkdH2yGeylYfpaf</a></td></tr><tr><td align="center">Cow</td><td><a href="/files/ZbEAesyGZP3Mxk22YsYZ">/files/ZbEAesyGZP3Mxk22YsYZ</a></td></tr><tr><td align="center">Elephant</td><td><a href="/files/UkylvnCb31uiazynjJkJ">/files/UkylvnCb31uiazynjJkJ</a></td></tr><tr><td align="center">Bear</td><td><a href="/files/AKnLCBEzAu0pj4rmnSJz">/files/AKnLCBEzAu0pj4rmnSJz</a></td></tr><tr><td align="center">Zebra</td><td><a href="/files/cjZpsKvnCXtn7eGJszX0">/files/cjZpsKvnCXtn7eGJszX0</a></td></tr></tbody></table>

## Vehicles and Infrastructure

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">Bicycle</td><td><a href="/files/OgSilGFIkt6PUdYLZDFm">/files/OgSilGFIkt6PUdYLZDFm</a></td></tr><tr><td align="center">Car</td><td><a href="/files/f8LdAEVFlETsXU6pKeD8">/files/f8LdAEVFlETsXU6pKeD8</a></td></tr><tr><td align="center">Motorcycle</td><td><a href="/files/IYWJubLXSknzzbPmj25c">/files/IYWJubLXSknzzbPmj25c</a></td></tr><tr><td align="center">Airplane</td><td><a href="/files/HY42vMubnGs8TWI4BGZR">/files/HY42vMubnGs8TWI4BGZR</a></td></tr><tr><td align="center">Train</td><td><a href="/files/i82mewsmga66DBzROg5B">/files/i82mewsmga66DBzROg5B</a></td></tr><tr><td align="center">Boat</td><td><a href="/files/M9vZyBZLnbI9aXjHRUcS">/files/M9vZyBZLnbI9aXjHRUcS</a></td></tr><tr><td align="center">Fire_hydrant</td><td><a href="/files/wXQRN1XwhuNPWbvwvxD9">/files/wXQRN1XwhuNPWbvwvxD9</a></td></tr></tbody></table>

## Personal Items

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">Backpack</td><td><a href="/files/ueRZRATzmvaaqcfDMkWg">/files/ueRZRATzmvaaqcfDMkWg</a></td></tr><tr><td align="center">Umbrella</td><td><a href="/files/PyKDPEVxHylNggSiFRA1">/files/PyKDPEVxHylNggSiFRA1</a></td></tr><tr><td align="center">Handbag</td><td><a href="/files/el9kgdymSvvUd4gbneiN">/files/el9kgdymSvvUd4gbneiN</a></td></tr><tr><td align="center">Tie</td><td><a href="/files/qYfIpyA5J8kJVYNfMZUN">/files/qYfIpyA5J8kJVYNfMZUN</a></td></tr><tr><td align="center">Suitcase</td><td><a href="/files/3wJSbH4IoyFORu3oL3ul">/files/3wJSbH4IoyFORu3oL3ul</a></td></tr><tr><td align="center">Teddy_bear</td><td><a href="/files/JqUC2sjL8EvceplZoGte">/files/JqUC2sjL8EvceplZoGte</a></td></tr><tr><td align="center">Toothbrush</td><td><a href="/files/eEvkNPYhoAqP5himPsPh">/files/eEvkNPYhoAqP5himPsPh</a></td></tr><tr><td align="center">Hair_drier</td><td><a href="/files/vMVtHI6EuMbozIRKsp4D">/files/vMVtHI6EuMbozIRKsp4D</a></td></tr><tr><td align="center">Book</td><td><a href="/files/P2ut1eTVcuyA0eUWz6ao">/files/P2ut1eTVcuyA0eUWz6ao</a></td></tr></tbody></table>

## Sports Items

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">Frisbees</td><td><a href="/files/haxgRrWDyfzxc3ZcTs77">/files/haxgRrWDyfzxc3ZcTs77</a></td></tr><tr><td align="center">Skis</td><td><a href="/files/I48NbAK6mb9qr3IOOSCg">/files/I48NbAK6mb9qr3IOOSCg</a></td></tr><tr><td align="center">Snowboard</td><td><a href="/files/l6aUygLaKyINgkBmu4Wd">/files/l6aUygLaKyINgkBmu4Wd</a></td></tr><tr><td align="center">Sports_ball</td><td><a href="/files/bE1xirpGX7iGuygoeoDy">/files/bE1xirpGX7iGuygoeoDy</a></td></tr><tr><td align="center">Kite</td><td><a href="/files/wSWjx1BoCkIuuGd4KyxK">/files/wSWjx1BoCkIuuGd4KyxK</a></td></tr><tr><td align="center">Baseball_bat</td><td><a href="/files/C4egBBaHnStw0bsKOh9t">/files/C4egBBaHnStw0bsKOh9t</a></td></tr><tr><td align="center">Baseball_glove</td><td><a href="/files/Fahayzu4psicuceqiyFD">/files/Fahayzu4psicuceqiyFD</a></td></tr><tr><td align="center">Skateboard</td><td><a href="/files/h5FJdCpCkanLXywTa7l6">/files/h5FJdCpCkanLXywTa7l6</a></td></tr><tr><td align="center">Surfboard</td><td><a href="/files/WVADmMcNTccocZR7M41O">/files/WVADmMcNTccocZR7M41O</a></td></tr><tr><td align="center">Tennis_racket</td><td><a href="/files/SGm0mulqq1UChsR2XWuU">/files/SGm0mulqq1UChsR2XWuU</a></td></tr></tbody></table>

## Household items

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">Bottle</td><td><a href="/files/765Z4jm9WjRXMjSJWrkr">/files/765Z4jm9WjRXMjSJWrkr</a></td></tr><tr><td align="center">Wine_glass</td><td><a href="/files/rLOGzAtXJuf1Zo13wVDJ">/files/rLOGzAtXJuf1Zo13wVDJ</a></td></tr><tr><td align="center">Cup</td><td><a href="/files/DCTSYodC52eZOMbNm20c">/files/DCTSYodC52eZOMbNm20c</a></td></tr><tr><td align="center">Fork</td><td><a href="/files/faOXguO1AEou9mgykBvl">/files/faOXguO1AEou9mgykBvl</a></td></tr><tr><td align="center">Knife</td><td><a href="/files/wAI8noHoBGVjm1gHbyBD">/files/wAI8noHoBGVjm1gHbyBD</a></td></tr><tr><td align="center">Spoon</td><td><a href="/files/zplUlPC3eDJXoYmdJlyZ">/files/zplUlPC3eDJXoYmdJlyZ</a></td></tr><tr><td align="center">Bowl</td><td><a href="/files/zyyTHPoNOkXfmEIwznop">/files/zyyTHPoNOkXfmEIwznop</a></td></tr><tr><td align="center">Potted_plant</td><td><a href="/files/YuI1OOdMgqLch0l39EJQ">/files/YuI1OOdMgqLch0l39EJQ</a></td></tr><tr><td align="center">Vase</td><td><a href="/files/9LgciXOXKncu3YfdxvDR">/files/9LgciXOXKncu3YfdxvDR</a></td></tr><tr><td align="center">Clock</td><td><a href="/files/gq32FFF3j34yajgnBkmB">/files/gq32FFF3j34yajgnBkmB</a></td></tr><tr><td align="center">Scissors</td><td><a href="/files/iCgwPGePux2caX1UcnrJ">/files/iCgwPGePux2caX1UcnrJ</a></td></tr></tbody></table>

## Fruits and Vegetables

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">Banana</td><td><a href="/files/MGUFnorH1dj7YwTffeZe">/files/MGUFnorH1dj7YwTffeZe</a></td></tr><tr><td align="center">Apple</td><td><a href="/files/ltFGWRiHfVAhF3YaxsbG">/files/ltFGWRiHfVAhF3YaxsbG</a></td></tr><tr><td align="center">Broccoli</td><td><a href="/files/UUsdd1qP2jEJrefLR3f8">/files/UUsdd1qP2jEJrefLR3f8</a></td></tr><tr><td align="center">Carrot</td><td><a href="/files/yx9CKTOSN3NQtPqhZhC1">/files/yx9CKTOSN3NQtPqhZhC1</a></td></tr><tr><td align="center">Orange</td><td><a href="/files/QCGAOGqo4t3X2hFqnGy8">/files/QCGAOGqo4t3X2hFqnGy8</a></td></tr></tbody></table>

## Fast Food and Desserts

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">Hot_dog</td><td><a href="/files/SAaV6ckihee4Ekw9gLGu">/files/SAaV6ckihee4Ekw9gLGu</a></td></tr><tr><td align="center">Pizza</td><td><a href="/files/sYXw33gMZkNAonxCqXK9">/files/sYXw33gMZkNAonxCqXK9</a></td></tr><tr><td align="center">Donut</td><td><a href="/files/8ZUi77OnUAzcF74rGKav">/files/8ZUi77OnUAzcF74rGKav</a></td></tr><tr><td align="center">Cake</td><td><a href="/files/Wib1BHCtUIgXyOLnO0he">/files/Wib1BHCtUIgXyOLnO0he</a></td></tr><tr><td align="center">Sandwich</td><td><a href="/files/9KKHc6Rs8PSwNZeOL2x2">/files/9KKHc6Rs8PSwNZeOL2x2</a></td></tr></tbody></table>

## Furniture

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">Chair</td><td><a href="/files/TcAmaoUCTwNiffJWINIS">/files/TcAmaoUCTwNiffJWINIS</a></td></tr><tr><td align="center">Couch</td><td><a href="/files/cz2xxQ04TeY9FlUHwen1">/files/cz2xxQ04TeY9FlUHwen1</a></td></tr><tr><td align="center">Bed</td><td><a href="/files/5kjfpGHzawV8zjWs7eWb">/files/5kjfpGHzawV8zjWs7eWb</a></td></tr><tr><td align="center">Dining_table</td><td><a href="/files/7el7UgSjPtalQKSYaTFL">/files/7el7UgSjPtalQKSYaTFL</a></td></tr><tr><td align="center">Toilet</td><td><a href="/files/KqFUW6MUkAzREl9pxISo">/files/KqFUW6MUkAzREl9pxISo</a></td></tr><tr><td align="center">Sink</td><td><a href="/files/aFe7trjDG7CaoBIZ8JWv">/files/aFe7trjDG7CaoBIZ8JWv</a></td></tr></tbody></table>

## Electronics

<table data-view="cards"><thead><tr><th align="center"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td align="center">TV</td><td><a href="/files/R5PeMw0fmxqOT3ePMrV8">/files/R5PeMw0fmxqOT3ePMrV8</a></td></tr><tr><td align="center">Laptop</td><td><a href="/files/6f3v95T6YL3jThU8GW5T">/files/6f3v95T6YL3jThU8GW5T</a></td></tr><tr><td align="center">Mouse</td><td><a href="/files/WF6rbDGcWbqphGrWVzS4">/files/WF6rbDGcWbqphGrWVzS4</a></td></tr><tr><td align="center">Remote</td><td><a href="/files/nYHQtfryDE6vHUEZze2T">/files/nYHQtfryDE6vHUEZze2T</a></td></tr><tr><td align="center">Keyboard</td><td><a href="/files/U8B7TmsKQymOdCmFIB88">/files/U8B7TmsKQymOdCmFIB88</a></td></tr><tr><td align="center">Cell_phone</td><td><a href="/files/wUsetcsKPmwe4aEwvHGP">/files/wUsetcsKPmwe4aEwvHGP</a></td></tr><tr><td align="center">Microwave</td><td><a href="/files/RdpReglGuci6u8ZNaUmQ">/files/RdpReglGuci6u8ZNaUmQ</a></td></tr><tr><td align="center">Oven</td><td><a href="/files/XGl5OxAoS1FqWgSyDbn1">/files/XGl5OxAoS1FqWgSyDbn1</a></td></tr><tr><td align="center">Toaster</td><td><a href="/files/bfZEvhJ8K929I3mpydmB">/files/bfZEvhJ8K929I3mpydmB</a></td></tr><tr><td align="center">Refrigerator</td><td><a href="/files/dNCDvB4fIKlJIXz8QFYX">/files/dNCDvB4fIKlJIXz8QFYX</a></td></tr></tbody></table>


# NLP Actions

In this page, you'll be able to find all Misty's default actions and the default known objects

**Actions in NLP state blocks**

* Admire
* Admire2
* Angry
* Blink
* Body-reset
* Check-it-out
* Check-surroundings-fast
* Check-surroundings-slow
* Cheers
* Concerned
* Confused
* Correct
* Cry-fast
* Cry-slow
* Cute default
* Dizzy fear
* Fear2
* Freefall
* Grief
* Head-down-up-nod
* Head-nod-slow head-up-down-nod
* Hi
* Hug
* Hug2
* Jump
* Listen
* Look-at-hands
* Look-left
* Look-right
* Look-right-then-left
* Look-up-left
* Love mad
* Mad2
* Mad3
* Mad4
* Oh-wow
* Oops
* Party
* Question
* Sad
* Sad2
* Sad3
* Scold
* Self-destruct
* Sleep
* Surprise
* Surprise2
* Surprise3
* Suspicious
* Terror
* Think
* Told-you
* Tutorial1-e\_EcstacyStarryEyed
* Tutorial1-e\_JoyGoofy2
* Tutorial1-e\_Love
* Tutorial1-e\_SystemCamera
* Tutorial1-ecstacy
* Tutorial1-go
* Tutorial1-leftArm
* Tutorial1-rightArm
* Tutorial1-s\_Fear
* Tutorial3-headStroked
* Wake-up
* Walk-angry
* Walk-fast
* Walk-happy
* Walk-slow
* Warn
* Worry
* Yes
* Yes2


# Action Commands

Here you can find all the commands to custom your Misty's reactions!

To get a list of Actions currently available on your robot write this link in your browser :&#x20;

```
http://{ROBOT-IP-ADDRESS}/api/actions
```

In the list that returns, the two most important elements are "name" and "script". The name is arbitrary, and represents the piece of data you'll use to invoke the Action later. The script attribute represents a simple script of commands to be sent executed on Misty. There are a few things to note about these scripts:

* Each command is uppercase
* Command parameters are specified after the command name, following a colon
* Command parameters are delimited by a comma
* Each command line is terminated with a semi-colon
* Actions prefixed with an octothorpe are treated as cleanup commands executed at the end of the animation (commonly things like RESET-EYES or RESET-LAYERS)

Here's a complete list of commands available to the script.

* ARMS:leftDegrees,rightDegrees,timeMs;
* ARMS-V:leftDegrees,rightDegrees,velocity;
* ARMS-OFFSET:leftDegrees,rightDegrees,timeMs; //offset commands are based off current actuator values
* ARMS-OFFSET-V:leftDegrees,rightDegrees,velocity;
* ARM:left/right,degrees,timeMs;
* ARM-V:left/right,degrees,velocity;
* ARM-OFFSET:left/right,degrees,timeMs;
* ARM-OFFSET-V:left/right,degrees,velocity;
* HEAD:pitch,roll,yaw,timeMs; //use null to not change a degree in head commands
* HEAD-OFFSET:pitch,roll,yaw,timeMs; //use 0 to not change a degree in head commands
* HEAD-V:pitch,roll,yaw,velocity; //use null to not change a degree in head commands
* HEAD-OFFSET-V:pitch,roll,yaw,velocity; //use 0 to not change a degree in head commands
* PAUSE:timeMs;
* VOLUME:newDefaultVolume;
* DEBUG: User websocket message to send if skill is debug level;
* PUBLISH: User websocket message to send;
* LIGHT:true/false/on/off;
* PICTURE:image-name-to-save-to,display-on-screen\[,width,height]; optional width and height resize
* SERIAL:write to the serial stream;
* STOP;
* RESET-LAYERS; //clear user defined web, video, text and image layers
* RESET-EYES; //reset eyes and blinking to system defaults
* HALT;
* IMAGE:imageNameToDisplay.jpg; //displays on default eye layer
* IMAGE-URL:[http://URL-to-display.jpg](http://url-to-display.jpg/); //displays on default eye layer
* TEXT:text to display on the screen;
* CLEAR-TEXT;
* SPEAK:What to say; //can use generic data and inline speech, like 'Speak' in animations
* AUDIO:audio-file-name.wav;
* VIDEO:videoName.mp4;
* VIDEO-URL:[http://videoName-to-play.mp4](http://videoname-to-play.mp4/);
* CLEAR-VIDEO;
* WEB:[http://site-name](http://site-name/);
* CLEAR-WEB;
* LED:red,green,blue;
* LED-PATTERN:red1,green1,blue1,red2,green2,blue2,durationMs,blink/breathe/transit;
* START-LISTEN; //starts trying to capture speech
* SPEAK-AND-LISTEN;
* ALLOW-KEYPHRASE; //"Allows" keyphrase to work, but won't start if Misty is speaking or already listening and will wait until she can to allow keyphrase for the interaction
* CANCEL-KEYPHRASE; //turn off keyphrase rec
* SPEAK-AND-WAIT:What to say, timeoutMs;
* SPEAK-AND-EVENT:What to say,trigger,triggerFilter,text;
* SPEAK-AND-LISTEN:What to say; //starts listening after speaking the text
* FOLLOW-FACE;
* FOLLOW-OBJECT:objectName;
* STOP-FOLLOW;
* DRIVE:distanceMeters,timeMs,true/false(reverse);
* HEADING:heading,distanceMeters,timeMs,true/false(reverse);
* TURN:degrees,timeMs,right/left;
* ARC:heading,radius,timeMs,true/false(reverse);
* TURN-HEADING:heading,timeMs,right/left;
* RESPONSIVE-STATE:true/on/false/off; //if true, this interaction will respond to external bot events and commands, defaults to on
* HAZARDS-OFF;
* HAZARDS-ON;
* START-SKILL: skillId;
* STOP-SKILL: skillId;
* EVENT:trigger,triggerFilter,text;// send an event


# ChatGPT PDF files

Upload these files in your chatGPT and make it help you in programming with your Misty!

You can upload these files in your chatGPT and then ask to create code that you can upload in Misty.

This is the prompt that we suggest to use when uploading the files:

```
I am uploading three PDF files that provide comprehensive guidance on programming the Misty robot using Python. These files contain the following:

Misty_Python_API.pdf: This file includes all the API instructions available in Misty's Python SDK, offering detailed information on various functions and commands that can be used to control Misty.
Misty_Python_lessons.pdf: This document provides a series of lessons on how to build code for Misty, covering fundamental programming concepts and practical examples.
Misty_Python_projects.pdf: This file features advanced projects that demonstrate sophisticated use cases for Misty. You can refer to these examples to handle more complex tasks and inspire your responses to advanced requests.

Please utilize the information in these files to generate Python code for the Misty robot based on the queries provided.
```

{% file src="/files/LjiUQkTT40kBFy6ijrO4" %}

{% file src="/files/XsPfDYSivys4SpWiBz0V" %}

{% file src="/files/tOmZxbBF2MEjBx6jVNwu" %}


# AR Tag Dictionary

In this page, you'll be able to find all Misty's avaiable dictionary codes.

The available dictionary codes are as follows:

* 4X4\_50 = 0
* 4X4\_100 = 1
* 4X4\_250 = 2
* 4X4\_1000 = 3
* 5X5\_50 = 4
* 5X5\_100 = 5
* 5X5\_250 = 6
* 5X5\_1000 = 7
* 6X6\_50 = 8
* 6X6\_100 = 9
* 6X6\_250 = 10
* 6X6\_1000 = 11
* 7X7\_50 = 12
* 7X7\_100 = 13
* 7X7\_250 = 14
* 7X7\_1000 = 15
* ARUCO\_ORIGINAL = 16
* APRILTAG\_16h5 = 17
* APRILTAG\_25h9 = 18
* APRILTAG\_36h10 = 19
* APRILTAG\_36h11 = 20


# Technical Specifications

### Processors

QC8016 ARM 4xA53, 1GB – Windows Iot Core

QC8096 ARM 4xA72, 3GB – Android 8.1

### Camera

13 MP Sony IMX 214

### Speakers

3 watt stereo speakers with a custom engineered ported enclosure

### Microphones

3 microphones

Recording is performed in 16 bit, single channel, 16 khz

### Capactive Touch

6 panels, 4 of which are on the top of the head, one in the chin, and one in the handle. Contact measurements are binary

### Obstacle Sensors

Time of flight: 3 forward facing, 1 aft facing

Measures from 10 mm to 1200 mm distance at a 1 mm resolution

### Cliff Sensors

Time of flight: 4 downward facing

Measures from 10 mm to 150 mm at a 1 mm resolution

### Collision Sensors

10 contact switches at the front and rear bumpers

### IMU

Yaw, pitch, roll measure in 360 degrees at a resolution of 1 degree

Gyroscope measures +/- 2000 degrees/sec, 0.1 degree/sec resolution

Accelerometer measures +/- 4 g-forces at 1milli-g-force resolution

### Degrees of Freedom

3 degrees of Neck Freedom and 6 degress of Movement Freedom

### Head Range

Yaw: 170 degrees

Pitch: 90 degrees

Roll: 45 degrees

Encoder measurement is accurate within 0.1 degrees

### Arm Range

330 degree rotational

Encoder measurement is accurate within 0.1 degrees

### Reach

79 mm from axis of rotation

-8 mm from chest

### Arm Payload

2100 grams @ 2.5 cm from axis of rotation

### Track speed

Max: 450 mm/s (limited)

Min: 5 mm/s

### Locomotion Encoders

Optical quadrature, 74.7 µm per tick

Drive system precision is 1 mm

### Screen

480×272, 16 bit

### USB Port

USB 2.0 specification port

480 MB/s max data rate

500 milliamps provided

### Backpack UART

3.3v logic, 9600 baud configured as 8-N-1

### Battery

Capacity: 10.2 amp hours

Runtime at idle: 10 hours

Runtime at max driving speed: 2.2 hours

### Dimensions

Height: 35.56 cm /14 inches

Width: 20.23 cm / 8 inches

Depth: 25.4 cm / 10 inches

Weight: 5 kg / 11 lbs