Skip to content

Commit

Permalink
Update code sample
Browse files Browse the repository at this point in the history
  • Loading branch information
Yegor Alexandrov committed Feb 10, 2024
1 parent f2ffb33 commit 770184c
Show file tree
Hide file tree
Showing 3 changed files with 135 additions and 40 deletions.
12 changes: 8 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,14 @@ ARG Version
ARG GitCommit
RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM"

ENV HEADID=""
# HEADTOKEN Must be empty (Please check a docs https://doc.clickup.com/6656751/p/h/6b4qf-27775/012c44047324368)
ENV HEADTOKEN=""
ENV HEADURI="test-avatar-srv-2-ule2kkd6ca-wl.a.run.app"
WORKDIR /code

# Please check a docs https://doc.clickup.com/6656751/p/h/6b4qf-27775/012c44047324368
ENV HEADTOKEN="YOUR_TOKEN_HERE"

ENV HEADURI="https://sio-experiment-2-ule2kkd6ca-wl.a.run.app"

RUN python3.8 -m pip install git+https://github.com/zerodistanceinc/wehead_hack_sdk.git

COPY requirements.txt requirements.txt
RUN python3.8 -m pip install -r requirements.txt
Expand Down
110 changes: 109 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,109 @@
# wehead-controller
# Wehead SDK Readme
Welcome to the Wehead SDK! This SDK provides a straightforward and powerful way to control and interact with your Wehead device through WebSockets. With this toolkit, you can command the Wehead to move, speak, process video, and recognize spoken phrases, all programmatically.

## Features
- __Move:__ Command the Wehead to adjust its camera angle with precision.
- __Speak:__ Make the Wehead articulate text in a variety of voices.
- __Video Processing:__ Receive and process video frames from the Wehead in real-time.
- __Speech Recognition Callbacks:__ Respond to specific phrases that the Wehead detects.

## Installation
First, you'll need a valid token. __As part of a hackathon, you can use any string as a token.__

Before diving into the code, install `wehead_hack_sdk`:

```
pip install git+https://github.com/zerodistanceinc/wehead_hack_sdk.git
```

### Debugging with the 3D Simulation
While developing with the Wehead SDK, you might not always have physical access to a Wehead device. Don't worry; we've got you covered with a lifelike 3D simulation environment. This simulation allows you to test and debug your code as if you were interacting with a real Wehead, providing a seamless development experience.

Accessing the 3D Simulation
The 3D simulation can be accessed via any modern web browser. Simply navigate to the following URL, replacing YOUR_TOKEN_HERE with your actual authentication token:

```
https://wehead-landing--yuy2-wp1w4g77.web.app/wehead_call/?dev=1&loc=full&main=1&token=YOUR_TOKEN_HERE
```

Make sure to use the correct token for authentication. Don't forget to grant access to the camera and microphone!


## Getting Started
### Initialization

```Python
from wehead_hack_sdk import Wehead

token = "YOUR_TOKEN_HERE"
wehead = Wehead(token)
```

### Moving the Wehead
To move the Wehead to a specific pitch and yaw, simply call:

```Python
wehead.move(pitch=0.3, yaw=0.3)
```

### Speaking
To make the Wehead speak, use:

```Python
wehead.say(text="Hello, world!", voice="shimmer")
```

### Processing Video
Define a callback function that processes video frames. Here's an example that simply displays the video frames using OpenCV:

```Python
@wehead.on_video
def process_frame(img):
cv2.imshow('Wehead Video', img)
cv2.waitKey(1)
```


### Responding to Spoken Phrases
Similarly, define a callback to handle spoken phrases recognized by the Wehead:

```Python
@wehead.on_phrase
def handle_phrase(text):
print(f"Wehead heard: {text}")
```

## Example
Below is a simple example that integrates everything mentioned above:

```Python
import cv2
import time

from wehead_hack_sdk import Wehead

token = "YOUR_TOKEN_HERE"
wehead = Wehead(token)

wehead.move(pitch=0, yaw=0)
wehead.say("Initializing Wehead SDK Example")

@wehead.on_video
def process_frame(img):
cv2.waitKey(30)
cv2.imshow('Wehead Video', img)
cv2.waitKey(30)

@wehead.on_phrase
def handle_phrase(text):
if text == "Exit.":
wehead.say("Goodbye")
time.sleep(1)
exit()

while True:
pass # Keep the script running
```

## Conclusion
This SDK offers a flexible and easy way to interact with your Wehead. Whether you're creating interactive installations, developing a surveillance system, or simply having fun, the Wehead SDK empowers you to bring your creative vision to life. Enjoy exploring the capabilities of your Wehead!
53 changes: 18 additions & 35 deletions main.py
Original file line number Diff line number Diff line change
@@ -1,42 +1,25 @@
import cv2
import time
import os
import asyncio
import json
from websockets import connect
from time import sleep

from wehead_hack_sdk import Wehead

HEADID = os.environ["HEADID"]
HEADTOKEN = os.environ["HEADTOKEN"]
HEADURI = os.environ["HEADURI"]
token = os.environ.get("HEADTOKEN", "YOUR_TOKEN_HERE")
wehead = Wehead(token)

async def interact_with_websocket():
uri = "wss://" + HEADURI + "/control/user/" + HEADID + "?token=" + HEADTOKEN
i = 0
while i<=10:
async with connect(uri) as websocket:
# send message type 'tts'
tts_message = {
"type": "tts",
"data": {
"text": "Hello Robotics World!"
}
}
await websocket.send(json.dumps(tts_message))
wehead.move(pitch=0.5, yaw=0.5)
wehead.say("Hello robotics world!")

# send message type 'move'
move_message = {
"type": "move",
"data": {
"position": "velocity",
"pitch": 0.1,
"yaw": 0.2
}
}
await websocket.send(json.dumps(move_message))
@wehead.on_video
def process_frame(img):
pass

# receiving messages
async for message in websocket:
print(f"Received message: {message}")
sleep(5)
@wehead.on_phrase
def handle_phrase(text):
if text == "Exit.":
wehead.say("Goodbye")
time.sleep(1)
exit()

asyncio.run(interact_with_websocket())
while True:
pass # Keep the script running

0 comments on commit 770184c

Please sign in to comment.