Go to file
Ruben van de Ven 6dd6bb8444 Bypass lyrebird expired certificate... what's going on there? 2020-04-14 21:49:13 +02:00
Lanbox Lanbox files 2019-06-15 15:26:44 +02:00
hugvey Bypass lyrebird expired certificate... what's going on there? 2020-04-14 21:49:13 +02:00
installation Small & large fixes to various server bits 2019-05-14 17:56:43 +02:00
local@7b7f5c6874 Strip less punctuation and clearer message on broken direction when scanning story 2020-03-12 09:36:48 +01:00
pd Strip less punctuation and clearer message on broken direction when scanning story 2020-03-12 09:36:48 +01:00
recordings Fix #27 and fix #17 - Transcriptions and recordings - two birds, one stone 2019-04-10 10:13:35 +02:00
state State dir 2019-11-13 17:03:21 +01:00
www search field in editor 2020-02-23 22:48:16 +03:00
.gitignore Webdev workflow 2019-01-23 15:26:44 +01:00
.gitmodules Housekeeping with repos 2019-01-25 17:18:50 +01:00
README.md Add socks tunnel description 2020-02-18 14:29:30 +01:00
autossh-hugvey.service change service files 2019-04-11 12:50:51 +02:00
client_config.yml Static ip for clients and separate file server 2019-11-15 23:39:22 +01:00
hugvey_client.py Change playback code to not buffer 2019-11-13 11:19:21 +01:00
hugvey_server.py hugvey 2020-02-22 21:02:48 +01:00
install_server.sh Small & large fixes to various server bits 2019-05-14 17:56:43 +02:00
requirements.server.txt added sox to requirements 2019-05-11 20:21:42 +02:00
requirements.txt Changes and fixes on the spot 2019-11-12 09:54:55 +01:00
server_config.yml Changes to timer 2020-02-15 16:14:25 +03:00
supervisor.conf Enable logging param for supervisor 2019-04-11 12:00:38 +02:00
test_pub.py Client now publishes itself 2019-01-16 09:00:49 +01:00
test_sub.py Streaming audio to central command when on verbose 2019-01-17 17:39:52 +01:00
tools.py Check for missing conditions like in moscow 2020-02-23 21:11:34 +03:00


Hugvey / Pillow Talk

  • Panpoticon
    • Fancy nickname for the web interface that allows altering the story and running the individual Hugveys
  • Voice
    • Lyrebird voice syntehsis API wrapper. Set the oAuth token using a token generated here
  • Client
    • Individual Hugveys that stream their mic output and play audiofiles trough the Panopticon. Communication with the server is done through zmq
    • Connect with them trough hugvey1.local etc (1-25).
  • Central Command/server
    • One server to rule them all. Start individual threads/subprocesses for the individual Hugveys. The Panopticon is started when starting the server.


Run the server: python hugvey_server.py --config server_config.yml

To make sure: don't run the server in a screen subterminal, as the amount of output can clog the server

Screen commands:

Set scrollbuffer in ~/.screenrc: as described here

ctrl+a d: detach, ctrl+a esc: scroll the buffer, ctrl+a i: current buffer settings


The server also integrates the panopticon, the monitoring & administration interface to Hugvey.


To run it: python hugvey_client.py -c client_config.yml


The Panopticon uses gulp to compile SASS into CSS, and to set up browser-sync for css & js. For now, no js user facing dependencies are managed trough node/npm.

After starting the server:

cd www

To run a command on all hugveys:

fab -H rubenvandeven.com,saclab@projects.rubenvandeven.com -- uname


create and load Python virtualenv

virtualenv -p python3 --system-site-packages venv
source venv/bin/activate

Install requirements

apt install libasound-dev python3-pyaudio git-lfs
pip install -r requirements.server.txt
pip install -r requirements.txt

Install soft requirements (These are hard requirements for the clients!)

apt install sox rsync

Don't forget to init git submodules

git submodule init
git sumodule update

copy autossh-hugvey.service to /etc/systemd/system/:

cp autossh-hugvey.service /etc/systemd/system/autossh-hugvey.service
sudo systemctl enable autossh-hugvey.service 


apt install supervisor

Add the following to the first section of /etc/supervisor/supervisord.conf


Deploy / usefull commands

rsync them all

for i in {1..26}; do echo $i; rsync -av ~/hugvey/ pi@hugvey$i.local:/home/pi/hugvey/ --exclude=www --exclude=venv --exclude=local --exclude=*.pyc --exclude=.git --exclude=recordings --exclude=/voice* --exclude=/pd --exclude=Lanbox; done

shut all of them down

for i in {1..26}; do ssh pi@hugvey$i.local "sudo shutdown -h now"; done

restart supervisor on all of them

for i in {1..26}; do ssh pi@hugvey$i.local "supervisorctl restart hugvey_client"; done

install bash on all of them

for i in {1..26}; do echo $i;ssh pi@hugvey$i.local "cd hugvey && sudo bash install_server.sh"; done

make them say their number

for i in {1..26}; do echo $i;ssh pi@hugvey$i.local "espeak -f /etc/hostname"; done

Select wifi network:

for i in {1,2,3,6,7,8,12,13,17,18,20,22}; do echo $i;ssh pi@hugvey$i.local "/sbin/wpa_cli -i wlan0 select_network 0"; done

Monitoring the server processes

To make sure it will not die with 'Too many files open'

examine open file limit with ulimit -a and set it with ulimit -n NUMBER

lsof -p $(ps aux|grep "[h]ugvey_server.py" |awk '{print $2}')| awk '{print $9}'|sort -rn|uniq -c|sort -rn|head -20


lsof | grep $(ps aux|grep "[h]ugvey_server.py" |awk '{print $2}')| awk '{print $11}'|sort -rn|uniq -c|sort -rn|head -20


Experiment with using alsa+sox to stream multiple channels to google.

Create Story


Things that can/will be said by Hugvey

The text that will be said. Or just a description if custom audio is uploaded. Variables can be entered by predending them with a $dollar_sign.
Mark message as being the start of a strand/tree of messages. Used for eg. diversions and formatting of the editor
There can only be one beginning. This is the message that the Hugvey will start with when starting the story.
Chapter Start
A chapter can be marked. This is used by the timeout diversions, as it only returns to the next chapter marker after diversion. Furthermore, it is used by diversions to prevent them from triggering if specific sections (chapter markers) have been played.
Upload a custom audio file to override the auto generated file
Afterrun time
deprecated Was the time the microphone kept listening after triggering this message. It was used to have the Hugvey reconsider its direction. Not used anymore
Volume factor
Parameter send to the play command to increase/decrease the playback volume
Tempo factor
Parameter send to the play command to increase/decrease the playback speed
Pitch factor
Parameter send to the play command to increase/decrease the playback pitch (minus values for lower pitch)
Color the message aids in finding it in the editor window. Also is used by replycontains diversions to prevent from running in specific moments


Directions connect messages from one to the other. Can be created in the editor by selecting a message, and shift+click on its follow-up message. Another way is to select a message and press 'create message' from the right bar. This new message will automatically be connected and inherit the same color.


Messages only head to a specific following message when one of the direction's conditions is matched (OR-condition). First come, first served.

The Conditions Description field allows for giving some info on the condition's reasons.

Types of conditions:

  • timeout: timing finishing the playback of the message's audio
    • seconds: the duration
    • Only if no reply: timeout will be disabled after the person has spoken anything
    • Reply needed: If checked, the timeout is counted when it is met. This counter is used by the consecutive-timeouts diversions.
  • replyContains: Match the contents of the speech using a regex. Or just any speech. Used to capture variables.
    • regex: The regex to match on. Variables can be matched using the python syntax to give the variable a name (?P<variable_name>\w+)
    • three consecutive timings can be given:
      • delay reply duration: the duration of the reply since hugvey stopped speaking. If it is more than this value (but less than the larger) it will use the given timing
      • delay wait time: The time to wait after a person speaks. It doesn't wait for Google's is_finished parameter, but rather checks from Google's last response. This way, also short utterances sutch as 'hey' or 'ok' are also properly timed, as these often don't get an is_finished by Google.
    • instant match: don't use any timings. the moment the regex matches on the speech in progress, the condition is met.
  • variable: returns True if variable is set
    • TODO
  • diversion: returns True if diversion has ben taken.
    • TODO


  • ReplyContains: Match the contents of speech using regex.
    • works only if the current message has one, and only one, direction.

times occured/only on n-th instance: determines the order of diversions of the same type (for Timeout and no_response). Starting at 1, as a diversion with value of 0 can occur always


4G Modem

Visit The password is at the bottom of the device.

Restoring a hugvey

  • Write image /mnt/stash/hugvey.img to the microSD card.
    • You can use gnome-disks 'restore image' for that
  • Open partition called rootfs, and sudo nano etc/hostname
    • Change hugvey20 into hugveyX (the number you need)
  • Start it, rsync, install_server.sh as mentioned above.

Pure Data patch

The pure data patch controls the loop for both the python script and the max patch.

If there's a click in the audio (which randomly occurs), change audio settings. For some reason PD cannot remember/store these. So after each start of PD set Audio Settings:

Sample rate
44100 Block size
1024 Delay, msec

Static file server

To prevent the hugveys from waiting for a suboptimal tornado server, static files can be fetched from a serparate lighttpd server.

Installed with apt install lighttpd.

In /etc/lighttpd/lighttpd.conf set port to 8889 and add an alias for the local dir:

alias.url = ("/local/" => "/home/hugvey/hugvey/local/")

When Telegram is blocked

In Russia, Telegram is blocked, to circumvent this, we set it to run over a SOCKS proxy:

autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -N -D 9090 rubenvandeven.com