Google Assistant on an Orange Pi Zero running DietPi
The novelty of talking with Mme Google wears out. It would be nice to do something practical with voice recognition. Furthermore, who wants an open microphone streaming all sounds in the house to the outside world? It could be paranoia; but I would prefer so called "hotword" recognition to be done locally.
The preferred method seemed to be snowboy from KITT.AI. But I had problems. Fortunately, some clever people had already found solutions, all I had to do was find their site.
Table of Contents
- Audio Hardware
- Installing Python 3
- Installing Python Audio Prerequisites
- Installing snowboy
- snowboy with Domoticz
- Google Assistant with snowboy Hotword Detection
In a previous post about Google Assistant, I showed how to setup the audio hardware on the Orange Pi Zero with an expansion board. If this is already done, skip to the next section.
Power down the OPiZ, plug in the expansion board and then connect powered speakers using the 3.5mm jack on to the expansion board. When the small cube like case is use, it makes for a small neat package which is smaller than the wallwart powering the speakers as can be seen below.
After opening an ssh session as user dietpi, I
made sure the analogue audio was enabled. This can be done in
dietpi-config.
Select Audo Options in the main menu.
If the Soundcard is not set to
Analogue then click on OK.
Then select default 3.5mm Analogue.
A number of packages will be installed including Alsa-utils
which contains arecord and aplay used to test the
audio. It will be necessary to reboot.
Ensure that dietpi is part of the audio group.
Had dietpi not been a member of audio, it would
have been a simple matter to join the group:
The first step in testing the audio hardware is to record sounds through
the microphone to a temporary file sample.wav.
Then it's playback time to ensure that the speakers are getting the audio output.
It works! And I did not have to adjust playback and recording volumes.
All that remains is to create the asound configuration
file.
pcm.!default { type asym capture.pcm "mic" playback.pcm "speaker" } pcm.mic { type plug slave { pcm "hw:0,0" format S16_LE } } pcm.speaker { type plug slave { pcm "hw:0,0" } }
If you are following along from my previous post, then Python 3 is already installed. However the Google Assistant service should be disabled. Hotword detection is to be performed by snowboy, not Google.
If you are starting off with a fresh copy of Armbian from DietPi then the first step is to install Python version 3.
Next a Python 3 virtual environment is created.
The virtual environment env is a directory in the
dietpi home directory. It is activated by the
source /home/dietpi/env/bin/activate and deactivated by the
deactivate command. As the dialogue shows, activating the
virtual environment means that the command python will
now invoke python3. This is done with symbolic links and
modification of the search path and probably more tricks. That would
make it possible to have other versions of Python installed in other virtual
environments and using any version without interference.
The last step is to upgrade the installed versions of pip and
setuptools.
Of course, if the SD card contains a fresh copy of the operating system
the host name will not be domopiz unless you changed it to that
particular name in the Security Options of DietPi-Config.
The next step is to get all the packages needed by Python to access the audio hardware. Some of these may have been installed previously; it does not matter, no harm will occur if an unnecessary supplementary installation is attempted.
It is time to test Python audio by recording some sound using the
microphone on the expansion board, by using a Python script rec
to record an audio file which is then played back.
I remain amazed at the quality of the little electret microphone on the OrangePi Zero expansion board. I recorded my voice by talking towards the screen at a normal conversational level while sitting at my desk. The OPiz was on the floor about a half meter behind me, yet the sound gets recorded albeit the playback level is not very high. The OPiZ was not in its case because of the heat dissipation problem; that may have an effect on sound recording.
The precompiled
Unfortunately, this is where I hit a wall. I could not get the demo script to run properly.
The Python 2.7 library that could not be found should have been a loud
signal, but I did not see it. Fortunately, I found a repository on github by Mihail Burduja who with the help of António Pereira had
the solution. The _snowboydetect.so in the package from KITT.AI
is for Python 2.7. The developper provides the snowboy
source code, so presumably, it should be possible to recompile the library.
But it is not necessary to do so, we can use the Python 3 library in their
repositories. DietPi did not include git in
its distribution. It is easy enough to install but I got lazy and
just got the zip file of the latest version from github.
Then it was just a matter of copying the Python 3 library into the
snowboy directory and running the demo script again
IT WORKS!
Time to move on to the second demo script using "snowboy" and "alexa" as two hot words.
As can be seen, this worked also. Many thanks to Mihail Burduja (alias warchildmd) aided by António Pereira (alias Shaxine) for the information that finally helped me install snowboy on an Orange Pi Zero.
To continue with the LED examples in the documentation, a modified GPIO library for the OrangePi Zero, called
OPi.GPIO will have to be installed. Since the
Python 3 virtual environment is used, substitution of pip3
for pip as suggested in the library documentation is not done.
OPi.GPIO is a drop-in replacement for the classic
Raspberry Pi GPIO Python library RPi.GPIO. So all
that needs to be done to use it is to edit change one letter in the first line
of the light.pyscript.
import OPi.GPIO as GPIO
Again, because the
Python 3 virtual environment is used, the command to
invoke the light.py script is not exactly what is shown in the
KITT.AI documentation. The script just blinks an
LED that is connected to GPIO 17 and ground.
There is more information about using sudo in a Python
virtual environment on the ask ubuntu forum.
There will be a warning on subsequent runs of the script:
GPIO.setwarnings(False) can be added
to the script before GPIO.setmode(GPIO.BCM) lin in the
__init__ definition. I do not like doing that, it seems to me that
control of the GPIO channels should be relinquished (using GPIO.cleanup()?)
when the object is destroyed. I know next to nothing about Python so I could
not do that correctly.
To control the LED with a spoken keyword, I copied the demo.py
script to modify it as instructed. Changes are shown on a white background.
import snowboydecoder import sys import signalfrom light import Lightinterrupted = False def signal_handler(signal, frame): global interrupted interrupted = True def interrupt_callback(): global interrupted return interrupted if len(sys.argv) == 1: print("Error: need to specify model name") print("Usage: python demo.py your.model") sys.exit(-1) model = sys.argv[1] signal.signal(signal.SIGINT, signal_handler) detector = snowboydecoder.HotwordDetector(model, sensitivity=0.5) print('Listening... Press Ctrl+C to exit')led = Light(17) detector.start(detected_callback=led.blink, interrupt_check=interrupt_callback, sleep_time=0.03)
This script turns the LED on for a short duration each time the keyword is is detected.
With two hot words, it becomes possible to turn devices on and off.
Domoticz can be invoked with three relatively small
modifications to the demo2.py script. The changes are shown with
a white background.
import snowboydecoder import sys import signalimport urllib.request# Demo code for listening two hotwords at the same time interrupted = False def signal_handler(signal, frame): global interrupted interrupted = True def interrupt_callback(): global interrupted return interrupted if len(sys.argv) != 3: print("Error: need to specify 2 model names") print("Usage: python demo.py 1st.model 2nd.model") sys.exit(-1) models = sys.argv[1:] # capture SIGINT signal, e.g., Ctrl+C signal.signal(signal.SIGINT, signal_handler)def detect_on(): urllib.request.urlopen('http://192.168.0.45:9071/json.htm?type=command¶m=udevice&idx=52&nvalue=1') snowboydecoder.play_audio_file(snowboydecoder.DETECT_DING) print('Device turned on.\nListening... Press Ctrl+C to exit') def detect_off(): urllib.request.urlopen('http://192.168.0.45:9071/json.htm?type=command¶m=udevice&idx=52&nvalue=0') snowboydecoder.play_audio_file(snowboydecoder.DETECT_DONG) print('Device turned off.\nListening... Press Ctrl+C to exit')sensitivity = [0.5]*len(models) detector = snowboydecoder.HotwordDetector(models, sensitivity=sensitivity)callbacks = [detect_on, detect_off]print('Listening... Press Ctrl+C to exit') # main loop # make sure you have the same numbers of callbacks and models detector.start(detected_callback=callbacks, interrupt_check=interrupt_callback, sleep_time=0.03) detector.terminate()
I saved this modified script under the name demo3.py
(download here. From now on, the lamp with
idx 52 can be turned on with the "snowboy" hot word, and turned off with
the "alexa" hot word.
Almost there, almost at the point of instructing Domoticz to turn devices on and off with voice commands using Google Assistant and hotword detection by snowboy. But... I ran into a glitch and this post is getting long anyway... and I am not sure I want to use this approach.
Granted that sounds like an excuse. So until I figure out a solution, if
I decide it is worthwhile to persevere, here is a quick overview of what I
did. I installed the Google Assistant SDK
following the instructions on the repository. Then as a first test I ran the
pushtotalk.py script. A warning appeared about an overflow and
my query was not understood. I had to repeat but with just the right timing
to get a response.
There was a response if I waited until the overflow warning was displayed and then vocalized a query
I also found that if I let the first attempt at recording a query time out and wait for the second prompt to press the Enter key, before making my first query, there was no problem at all. But the overflow warning was still displayed.
Ignoring this problem, I copied the assistant.py and
gassistant.py from Mihail Burduja repository into the
snowboy directory because the scripts were newer in that
directory.
Now I will look into the work of Mihail Burduja which the full Google Assistant SDK to extend vocal recognition. I should mention another project forked from that depository, Google Assistant for all RPi Boards which has more examples of activating device by voice commands.