🧠Lesson 3: Create memories

Aim

The aim of this lesson is to learn how to manage content on Misty and access external media online. By the end of the lesson you will be able to capture content, upload custom images, video and audio as well as stream online content directly on your Misty's display. If you need detailed information about the API calls for this lesson you can check out Record Assetsand Display and LED.

Capturing content

Challenge 1: Snap a photo with Misty

Misty is a master at capturing information about her surroundings, especially with her visor camera. With the misty.take_picture API you can have Misty take a photo of you and objects in your room. Let's try to snap a picture with Misty and display it. In the parameters you can define the image name, choose to display it automatically, set its resolution and even overwrite it if you want snap a new version of the same image. 

from mistyPy.Robot import Robot
misty = Robot()

misty.take_picture(base65=True, fileName="myportrait", width=3200,height= 2400,displayOnScreen=True,overwriteExisting=False)

#short version 
# misty.take_picture(True,"myportrait",3200,2400,True,False)

Upload content

Challenge 2: A new pair of eyes

As you've probably discovered in Misty Studio, one of Misty's greatest talents is display customizability, allowing you to create your very own robot characters by uploading custom eyes and expressions. Typically local files on your computer or tablet can be uploaded in the Expressions tab under Explore in Misty Studio or by running the Upload Block in Blockly. But what if you want to give Misty your own customized eyes using Python code?

To upload an new pair of eyes directly from your computer in Python you would need to create a new programming environment outside of Misty Studio, since this is a more complicated process that we will explore in later lessons, the quickest way to upload images is using HTTP endpoints from your cloud such as Google Drive or directly from a website. To achieve this we will need to rely once more on the power of libraries. For Misty to read and encode data from an HTTP endpoint you will need to import the base64 library and use the misty.save_image and misty.display_imageAPI calls. Let's try it out!

from mistyPy.Robot import Robot
import base64
import requests

#binary-to-text encoding schemes
misty = Robot()

# Encode the image as base64 and UTF-8 so it's safe for the API, 
encoded = result = base64.b64encode(requests.get("https://static.wixstatic.com/media/d181e9_b56a1c7cd8104742af8b00cc716caacd~mv2.gif").content).decode("utf8")

# Send it to Misty with matching dimensions. In this case, displaying immediately and overwriting any existing file
misty.save_image("customeyes.gif", encoded, 640, 480, False, True)
time.sleep(2)
misty.display_image("customeyes.gif")

Challenge 3: Create a robot movie theater

As you continue to design your robot character and interactions with Misty, don't forget to document all the fun moments. You can do this by letting her record them or by uploading your own footage of your interactions. You can also fill her memory banks with your favorite short movies and create a robot cinema. The API calls for uploading and displaying videos work in the same way as for images.

Note: Accepted video file types are .mp4 and .wmv. Maximum file size is 6 MB.

from mistyPy.Robot import Robot
import base64
import requests
import time

#binary-to-text encoding schemes
misty = Robot()

# Encode the video as base64 and UTF-8 so it's safe for the API, 
encoded = result = base64.b64encode(requests.get("https://video.wixstatic.com/video/d181e9_837968f95e3f4cd2a6210ec36ff87f58/360p/mp4/file.mp4").content).decode("utf8")

# Send it to Misty with matching dimensions. In this case, displaying immediately and overwriting any existing file
misty.save_video("mistywow2.mp4", encoded, False, True)
time.sleep(2)
misty.display_video("mistywow2.mp4")
time.sleep(2)
misty.stop(1)

Challenge 4: Add a new vocal expression

In some cases you maybe want to have a Misty play your music or use custom vocal expressions for you robot character. In that case you can download an audio file to Misty's storage from a cloud drive and try out playing it. 

from mistyPy.Robot import Robot
import base64
import requests
import time

#binary-to-text encoding schemes
misty = Robot()

# Encode the image as base64 and UTF-8 so it's safe for the API, 
encoded = result = base64.b64encode(requests.get("https://assets.mixkit.co/active_storage/sfx/93/93-preview.mp3").content).decode("utf8")

# Send it to Misty with matching dimensions. In this case, displaying immediately and overwriting any existing file
misty.save_audio("newvoice1.mp3", encoded, False, True)
time.sleep(1)
misty.play_audio("newvoice1.mp3",30)

Stream content

Challenge 5: Stream you favorite content to Misty

Now that you've mastered how to upload new content to Misty, let's learn how to program Misty to stream online content directly on her display. To achieve this you can use the misty.display_web_view API, this is a great feature if you want Misty to show you the latest news, weather, statistics or other live information. What would you choose to stream?

Misty uses the default webview layer settings the first time she draws content with the DisplayWebView command. 

from mistyPy.Robot import Robot
import time
misty = Robot()

misty.speak("Good morning my human friend, do you wanna know how cold it is outside today? Take a look!")
time.sleep(2)
misty.display_web_view("https://www.yr.no/en/forecast/daily-table/2-5574991/United%20States/Colorado/Boulder/Boulder")
time.sleep(5)
misty.stop(1)

Important! Displaying webviews can consume a lot of computational resources. If you notice Misty's performance decrease while multiple webviews layers are active, you may consider deleting one or more webview layers. You can use the SetWebViewDisplaySettings command to adjust the settings and change the appearance for a specific webview layer. Issuing a SetWebViewDisplaySettings command redraws the updated webview layer on Misty's display.

Up Next 👇

Last updated