1.0 Quick Start
Welcome to CAIT. This quick start guide will help you set up your device and get you started on writing your first A.I. program in the next few minutes.
If you have not already connected your raspberry pi to the local WIFI, it will switch itself into Access Point (AP) mode and the SSID name is cait. The reason for doing this is that it makes headless (without connecting to a monitor) WIFI setup easier by allowing the user's computer to connect directly to the raspberry pi for WIFI configuration.
Make sure to connect your computer's WIFI to the SSID advertised by the raspberry pi, and then go to http://<hostname>.local/setup on your browser. If you are using a windows machine to connect to the raspberry pi, you may experience problems resolving its hostname. In this case, you can install Bonjour and restart the computer. The raspberry pi's hostname should resolve after that.
Please follow the screencast below to complete the setup process:
Because the Raspberry Pi is very computationally constrained, on-device speech recognition and speech generation performance will not be as good as their cloud based counterparts. If you want to use Google Cloud based speech recognition and generation, you can following the following screencast to set it up.
Our visual programming interface is an extension of Google's Blockly visual programming editor with additional A.I. blocks. For a complete reference of the Blockly visual programming language, please refer to Blockly Wiki page. Visual programs written for CAIT are very similar to regular Blockly programs with one exception: each CAIT visual program must have a setup and a main block.
Drag and drop the setup and main block from the Basics category into the main workspace like the diagram shown below to begin a program.
There are five A.I. modules in CAIT: vision, speech, NLP, control, and smart home. Before using any block in each module, you will need to first put the corresponding initialization block into the setup block.
For example, drag and drop the initialize vision block from the Vision category under the Artificial Intelligence section into the setup block. Then, drag and drop the add person with name block from the Vision category into the main block. Within the add person with name block, drag and drop a create text block into it. You can enter your name in the create text block. You should have a program similar to the one below.
If you look at the camera and click the run button, you will see images captured by the camera on the workspace's right-hand side. You will probably also notice that your face is now tagged with your name. Congratulations if you got to this point. You just finished writing your first A.I. program. This program allows CAIT to recognize your face with only a few simple blocks.
Now, you can stop running this program by clicking the stop button. This program adds your face into the face database to allow CAIT to recognize you in the future. You can run this program for each person that you wish CAIT to recognize. You only have to run it once per person. If you rerun this program for the same name, the previous face database record will be replaced by the new one.
You can store the result of face recognition in a variable and use it in different parts of your program. Remove the add person with name block from the main block. Then create a variable in the Variables category. Let's name it person in this program. Drag and drop the set variable block from the Variable category into the main block. Then drag and drop the recognize face block from the Vision category into the set variable block. This program is shown in the diagram below.
Now, click the run button. You will your face is recognized in the camera stream shown on the right. The result of the recognition process is stored in the person variable.
You can extract information from the person variable using the Dictionaries and Lists blocks. Create a variable to store the name of the recognized person. Let's call this variable name in this program, then drag and drop the set variables block of the name variable into the main block. Drag and drop a get value with key block from the Dictionaries category into the set variable block of name. In the get value with key block, drag and drop the person variable block from the Variables category into the get value with key block's dictionary input field. Then drag and drop a create text block into the get value with key block's key input field. In the create text block, enter "name". Finally, drag and drop a print block from the Text category to print out the recognized face's name. This program is shown below.
Click the run button while your face is in front of the camera. You should see a pop-up with your name in it.
With the name extracted from the face recognition result, you can combine it with other A.I. modules such as the Speech blocks. Drag and drop the initialize voice block from the Speech category into the setup block. You can choose to use either online or on device mode. To use online mode, please follow this screencast to set up your Google cloud account. Then drag and drop the say block from the Speech category into the main block. You can remove the print block. Drag and drop the name variable into the say block like the diagram below.
Click the run button while your face is in front of the camera. You should hear your name from the speaker while your face is recognized in the camera stream.
That's it. You have just created a smart greeting program with only a few blocks. You can make much more sophisticated programs by combining different A.I. and programming blocks. If you need to learn more about using the other blocks, please refer to our Tutorial in this Wiki.
You can control your smart home devices with the Smart Home blocks. If you haven't done so, please follow this Wiki Tutorial to set up CAIT with your smart home devices first.
Once you have the smart home devices integrated, you can follow this tutorial to expand your program with smart home capabilities.
You may choose to convert your visual programs into Python code for more power and flexibility. Click the Generate Jupyter Notebook button and enter the Jupyter notebook file's name to save. We will be using the program you just created as an example and named it "greeting.ipynb". Note that for all Jupyter notebook files, you need to enter a ".ipynb" filename extension.
Then in your browser, go to http://<hostname>.local:8000, log in with the same username and password you used to log in to CAIT's visual programming interface.
Click into the cait_workspace folder in the list.
Then click into the python_notebooks folder.
Inside the folder, you will see the "greeting.ipynb" file you just saved. Click to open it.
This is the equivalent Python version of the smart greeting program you created in the Visual Programming Interface. The code is organized into different cells for readability and modifiability. You can execute the code block by block by pressing "Shift + Enter ." If you are already familiar with Python, you can use this as a starting point to create advanced programs. Please refer to the Python API Doc for CAIT's Python API.