198 lines
No EOL
6.3 KiB
Markdown
198 lines
No EOL
6.3 KiB
Markdown
# Hugvey / Pillow Talk
|
|
|
|
- Panpoticon
|
|
+ Fancy nickname for the web interface that allows altering the story and running the individual Hugveys
|
|
- Voice
|
|
+ Lyrebird voice syntehsis API wrapper. Set the oAuth token using a token generated [here](http://hugvey.rubenvandeven.com/oauth.php)
|
|
- Client
|
|
+ Individual Hugveys that stream their mic output and play audiofiles trough the Panopticon. Communication with the server is done through zmq
|
|
+ Connect with them trough hugvey1.local etc (1-25).
|
|
- Central Command/server
|
|
+ One server to rule them all. Start individual threads/subprocesses for the individual Hugveys. The Panopticon is started when starting the server.
|
|
|
|
## Server
|
|
|
|
Run the server: `python hugvey_server.py --config server_config.yml`
|
|
|
|
### Panopticon
|
|
|
|
The server also integrates the _panopticon_, the monitoring & administration interface to Hugvey.
|
|
|
|
|
|
## Client
|
|
|
|
To run it: `python hugvey_client.py -c client_config.yml`
|
|
|
|
## Development
|
|
|
|
The Panopticon uses gulp to compile SASS into CSS, and to set up browser-sync for css & js. For now, no js user facing dependencies are managed trough node/npm.
|
|
|
|
After starting the server:
|
|
|
|
```bash
|
|
cd www
|
|
gulp
|
|
```
|
|
|
|
To run a command on all hugveys:
|
|
```
|
|
fab -H rubenvandeven.com,saclab@projects.rubenvandeven.com -- uname
|
|
```
|
|
|
|
## Installation
|
|
|
|
create and load Python virtualenv
|
|
|
|
```bash
|
|
virtualenv -p python3 --system-site-packages venv
|
|
source venv/bin/activate
|
|
```
|
|
|
|
Install requirements
|
|
|
|
```bash
|
|
apt install libasound-dev python3-pyaudio git-lfs
|
|
pip install -r requirements.server.txt
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
Install soft requirements (These are hard requirements for the clients!)
|
|
|
|
```
|
|
apt install sox rsync
|
|
```
|
|
|
|
Don't forget to init git submodules
|
|
|
|
```
|
|
git submodule init
|
|
git sumodule update
|
|
```
|
|
|
|
|
|
copy autossh-hugvey.service to `/etc/systemd/system/`:
|
|
|
|
```bash
|
|
cp autossh-hugvey.service /etc/systemd/system/autossh-hugvey.service
|
|
sudo systemctl enable autossh-hugvey.service
|
|
```
|
|
|
|
### clients
|
|
|
|
```
|
|
apt install supervisor
|
|
```
|
|
|
|
Add the following to the first section of `/etc/supervisor/supervisord.conf`
|
|
|
|
```
|
|
chown=pi:pi
|
|
```
|
|
|
|
|
|
## Deploy / usefull commands
|
|
|
|
```bash
|
|
for i in {1..6}; do rsync -av ~/hugvey/ pi@hugvey$i.local:/home/pi/hugvey/ --exclude=www --exclude=venv --exclude=local --exclude=*.pyc --exclude=.git; done
|
|
```
|
|
|
|
```bash
|
|
for i in {1..6}; do ssh pi@hugvey$i.local "sudo shutdown -h now"; done
|
|
```
|
|
|
|
```bash
|
|
for i in {1..6}; do ssh pi@hugvey$i.local "supervisorctl restart hugvey_client"; done
|
|
```
|
|
|
|
|
|
### Monitoring the server processes
|
|
|
|
To make sure it will not die with 'Too many files open'
|
|
|
|
```bash
|
|
lsof -p $(ps aux|grep "[h]ugvey_server.py" |awk '{print $2}')| awk '{print $9}'|sort -rn|uniq -c|sort -rn|head -20
|
|
```
|
|
|
|
or
|
|
|
|
```bash
|
|
lsof | grep $(ps aux|grep "[h]ugvey_server.py" |awk '{print $2}')| awk '{print $11}'|sort -rn|uniq -c|sort -rn|head -20
|
|
```
|
|
|
|
|
|
## Branches
|
|
|
|
multichannel
|
|
: Experiment with using alsa+sox to stream multiple channels to google.
|
|
|
|
|
|
|
|
# Create Story
|
|
|
|
## Messages
|
|
|
|
Things that can/will be said by Hugvey
|
|
|
|
Text
|
|
: The text that will be said. Or just a description if custom audio is uploaded. Variables can be entered by predending them with a $dollar_sign.
|
|
|
|
Start
|
|
: Mark message as being the start of a strand/tree of messages. Used for eg. diversions and formatting of the editor
|
|
|
|
Beginning
|
|
: There can only be one beginning. This is the message that the Hugvey will start with when starting the story.
|
|
|
|
Chapter Start
|
|
: A chapter can be marked. This is used by the timeout diversions, as it only returns to the next chapter marker after diversion. Furthermore, it is used by diversions to prevent them from triggering if specific sections (chapter markers) have been played.
|
|
|
|
Audio
|
|
: Upload a custom audio file to override the auto generated file
|
|
|
|
Afterrun time
|
|
: _deprecated_ Was the time the microphone kept listening after triggering this message. It was used to have the Hugvey reconsider its direction. Not used anymore
|
|
|
|
Volume factor
|
|
: Parameter send to the `play` command to increase/decrease the playback volume
|
|
|
|
Tempo factor
|
|
: Parameter send to the `play` command to increase/decrease the playback speed
|
|
|
|
Pitch factor
|
|
: Parameter send to the `play` command to increase/decrease the playback pitch (minus values for lower pitch)
|
|
|
|
Color
|
|
: Color the message aids in finding it in the editor window. Also is used by _replycontains_ diversions to prevent from running in specific moments
|
|
|
|
## Directions
|
|
|
|
Directions connect messages from one to the other. Can be created in the editor by selecting a message, and shift+click on its follow-up message. Another way is to select a message and press 'create message' from the right bar. This new message will automatically be connected and inherit the same color.
|
|
|
|
## Condition
|
|
|
|
Messages only head to a specific following message when one of the direction's conditions is matched (OR-condition). First come, first served.
|
|
|
|
The Conditions Description field allows for giving some info on the condition's reasons.
|
|
|
|
Types of conditions:
|
|
|
|
- timeout: timing finishing the playback of the message's audio
|
|
+ seconds: the duration
|
|
+ Only if no reply: timeout will be disabled after the person has spoken anything
|
|
+ Reply needed: If checked, the timeout is counted when it is met. This counter is used by the consecutive-timeouts diversions.
|
|
- replyContains: Match the contents of the speech using a regex. Or just any speech. Used to capture variables.
|
|
+ regex: The regex to match on. Variables can be matched using the python syntax to give the variable a name (?P<variable_name>\w+)
|
|
+ three consecutive timings can be given:
|
|
+ delay reply duration: the duration of the reply since hugvey stopped speaking. If it is more than this value (but less than the larger) it will use the given timing
|
|
+ delay wait time: The time to wait after a person speaks. It doesn't wait for Google's `is_finished` parameter, but rather checks from Google's last response. This way, also short utterances sutch as 'hey' or 'ok' are also properly timed, as these often don't get an is_finished by Google.
|
|
+ instant match: don't use any timings. the moment the regex matches on the speech in progress, the condition is met.
|
|
- variable: returns True if variable is set
|
|
+ TODO
|
|
- diversion: returns True if diversion has ben taken.
|
|
+ TODO
|
|
|
|
## Diversions
|
|
|
|
TODO
|
|
|
|
|
|
times occured/only on n-th instance: determines the order of diversions of the same type (for Timeout and no_response). Starting at 1, as a diversion with value of 0 can occur always |