2021-10-11
md
Home Automation System on a Raspberry Pi
Additional Servers on a Raspberry Pi Based Home Automation System-> <-Setting up a Raspberry Pi as a Headless Computer in September 2021
<-HA Bridge and Alexa Device Discovery
 Draft of the second part of a projected multi-part guide 

This is part 2 of a series of posts about installing a home automation system on a Raspberry Pi. It covers installing the major services that are needed for the home automation system:

I recommend looking at the Andreas Spiess video , Pi Server based on Docker, with VPN remote access, Dropbox backup, Influx, Grafana, etc. on YouTube. While the intent is quite similar, the approach is rather different relying on Docker which is yet another level ofabstraction.

Table of Contents

  1. Overview of the Home Automation System
  2. Installing the Domoticz Home Automation Server
    1. Unit File to Start and Stop Domoticz
    2. Watchdog
  3. Installing the mosquitto MQTT Broker
  4. Installing the NGINX Web Server
  5. Installing HA Bridge
    1. Installing the Java Runtime Environment
    2. Installing the ha-bridge Script
  6. Configuring the Home Automation System
    1. Adding a Virtual Switch in Domoticz
    2. Configuring a Tasmota Device
    3. Setting up HA Bridge
  7. More ...

Overview of the Home Automation System toc

It is probably best to start by providing a description of the home automation system that has been slowly built over the last eight years since moving into our current house. The following diagram is a simplified representation of the system as it is now.

Most Internet of Things (IoT) devices are either sensors or actuators. Sensors sample some aspect of the environment such as the temperature, humidity level, the light level, or the presence of someone, and so on. Actuators are really just on off switches that control lights, TVs, the garage door, or an irrigation system (an upcoming project). To be honest, sensors and actuators are rather stupid devices that need to be controlled by intelligent entities. This means there has to be an exchange of messages which in turn requires a data transport mechanism and a communication protocol. As can be seen data travels through the wireless local area networking (Wi-Fi) which was already in use for other purposes.

Why Wi-Fi? Show

Having settled on the type of sensors and actuators and the wireless technology to use, I had to choose the home automation software and the hardware platform on which to run it. I chose to run Domoticz on a Raspberry Pi. While Domoticz might not have the popularity Home Assistant, it is, in my opinion, a very powerful yet lightweight application compatible with Linux, Windows and macOS although the later may require more effort. I have installed it without problems on many single-board computers such as a single core Raspberry Pi 1, an Orange Pi Zero, a La Frite AML-S805X-AC and a four coreRaspberry Pi 3 B running the latest version of Raspberrry Pi OS Lite used here.

Why Domoticz? Show

It should be safe to claim that my spouse finds that this hobby of mine has improved some things about the house. Before, to turn the outside lights on when guests were leaving the house in the dark, two switches had to be activated, one by the front door and then another in the attached garage. Now when the balcony lights are turned on or off, the outside garage lights which illuminate the driveway are simultaneously turned off or on. It is very practical that the garage door is automatically shut after a few minutes if one of us forgets to do it when driving away. However there was some pushback when it came to some of the other devices. We all agreed that the light switch for the kitchen light was much too far away - one of the consequences of an open floor plan and bad planning. But no one, including me, thought that the solution was to open a tablet or intelligent phone, get to the Domoticz web interface, and finally click on the virtual kitchen light to control the real light. It was faster to walk to the other end of the kitchen to turn the light on or off. Voice control changed all that. Now everyone is at ease with given oral commands to "she who must not be named" to turn lights on or off, or to set their dimmer values. Domoticz scenes to turn the lights on temporarily as we make our way from the TV room in the basement to the bedroom on the top floor are turned on with voice commands every evening. The value of voice control in increasing the value and acceptability of the home automation system cannot be overstated.

Why Amazon Smart Speakers? Show

There is a self-imposed constraint on our home automation system. It must be an independent local system with minimal reliance on remote servers. Obviously, smart speakers break that rule. They introduce risks which are not negligible. In addition to breaking updates mentioned in the previous sidebar, our experience has been that winter storms that knock out power in a wide area for more than a day or two are often accompanied by longer periods without Internet service. Every two or three years this has happened and it does look like the frequency can only increase. I wanted some mechanism to mitigate the problems. An old X10 wireless remote with 16 buttons is still available in the living room for that purpose. I also added yet another IR remote by the TV much to the chagrin of my spouse. I think I now have a better solution which I labelled Smart Button in the graph. So far the breadboard prototype sitting by my desktop keyboard has been working very well. By the way, this device relies entirely on MQTT to communicate with the home automation system. The hope is that it will grow into something even more practical, when these buttons advantageously replace seven Sonoff Basic providing local control for eight lights as well as remote control for other devices, including scenes and groups, set up in Domoticz.

Installing the Domoticz Home Automation Server toc

Before anything else, I would like to reiterate my gratitude to the Domoticz development team as well as to the community.

Before installing a major program, it is always a good idea to upgrade the system if it has not been done lately.

nostra@damus:~ $ sudo apt update && sudo apt upgrade -y ... 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Need to get 141 MB of archives. After this operation, 154 kB of additional disk space will be used. ...

Even if Raspberry Pi OS had been installed and upgraded just a couple of weeks previously, this operation took a few minutes and upgraded no less than 20 packages. Of course, just what will be done on another Pi will depend on which packages were installed on the machine and the time elapsed since the last upgrade.

Installing Domoticz is truly simple when following The "easy" way instructions. It is a one line command which will pipe an installation script file to bash which is a command line interpreter.

nostra@damus:~ $ curl -L install.domoticz.com | sudo bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 16257 100 16257 0 0 12007 0 0:00:01 0:00:01 --:--:-- 12015 ::: ::: You are root. ::: Verifying free disk space... ...
Welcome ┌──────────────────┤ Domoticz automated installer ├──────────────────┐ │ │ │ │ │ │ │ This installer will transform your device into a Home Automation │ │ System! │ │ │ │ │ │ Domoticz is free, but powered by your donations at: │ │ http://www.domoticz.com │ │ │ │ │ │ Domoticz is a SERVER so it needs a STATIC IP ADDRESS to function │ │ properly. │ │ │ │ │ │ │ │ <Ok> │ │ │ └────────────────────────────────────────────────────────────────────┘

After answering a couple of questions about ports and directories the process will be completed.

Ready... ┌─────────────────────┤ Installation Complete! ├─────────────────────┐ │ │ │ Point your browser to either: │ │ │ │ HTTP: 192.168.1.139:8080 │ │ HTPS: 192.168.1.139:443 │ │ │ │ Wiki: https://www.domoticz.com/wiki │ │ Forum: https://www.domoticz.com/forum │ │ │ │ The install log is in /etc/domoticz. │ │ │ │ │ │ <Ok> │ │ │ └────────────────────────────────────────────────────────────────────┘
... ::: Enabling domoticz.sh service to start on reboot... done. ::: ::: Starting domoticz.sh service... done. ::: done. ::: ::: Installation Complete! Configure your browser to use the Domoticz using: ::: 192.168.1.139:8080 ::: 192.168.1.139:443

You will have noticed that the installation script required root privileges. As your mom taught you as a youngster, this is dangerous akin to accepting candy from a stranger. You could download the script, examine it and then execute it only after you are satisfied that it does nothing nefarious.

nostra@damus:~ $ mkdir -p downloads/domoticz nostra@damus:~ $ cd downloads/domoticz nostra@damus:~/downloads/domoticz $ curl -L install.domoticz.com -o install_domoticz.sh % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 178 100 178 0 0 288 0 --:--:-- --:--:-- --:--:-- 288 100 16582 100 16582 0 0 10880 0 0:00:01 0:00:01 --:--:-- 22168 nostra@damus:~/downloads/domoticz $ less install_domoticz.sh

First time using less? Show

If happy with what has been found, then run the script to proceed with the installation.

nostra@damus:~/downloads/domoticz $ sudo bash install_domoticz.sh

The installation script will check for a number of needed packages and install them if absent.

If truly paranoid, then Domoticz should be compiled directly from the thoroughly audited source code. It does seem possible for mere mortals with patience and time on their hands to do that, but I have never undertaken that task and cannot comment on its difficulty.

No matter how it was installed, the service should be running. This is easily checked.

nostra@damus:~ $ sudo systemctl status domoticz.service ● domoticz.service - LSB: Home Automation System Loaded: loaded (/etc/init.d/domoticz.sh; generated) Active: active (running) since Thu 2021-09-16 00:50:39 ADT; 1h 8min ago Docs: man:systemd-sysv-generator(8) Process: 518 ExecStart=/etc/init.d/domoticz.sh start (code=exited, status=0/SUCCESS) Tasks: 18 (limit: 2178) CGroup: /system.slice/domoticz.service └─534 /home/nostra/domoticz/domoticz -daemon -www 8080 -sslwww 443 Sep 16 00:50:38 damus systemd[1]: Starting LSB: Home Automation System... Sep 16 00:50:38 damus domoticz.sh[518]: 2021-09-16 00:50:38.851 Status: Domoticz V2021.1 (c)2012-2021 GizMoCuz Sep 16 00:50:38 damus domoticz.sh[518]: 2021-09-16 00:50:38.851 Status: Build Hash: 8547c5b7e, Date: 2021-04-17 12:29:11 Sep 16 00:50:38 damus domoticz.sh[518]: 2021-09-16 00:50:38.851 Status: Startup Path: /home/nostra/domoticz/ Sep 16 00:50:38 damus domoticz.sh[518]: domoticz: Domoticz is starting up.... Sep 16 00:50:38 damus domoticz[532]: Domoticz is starting up.... Sep 16 00:50:38 damus domoticz[534]: Domoticz running... Sep 16 00:50:39 damus systemd[1]: Started LSB: Home Automation System.

If an error about a missing Python library appears,

Oct 10 10:37:45 damus domoticz[2010]: 2019-12-08 18:37:45.907 Status: EventSystem - Python: Failed dynamic library load, install the latest libpython3.x library that is available for your platform.

then install the python3-dev package. That package would have been installed if a virtual Python environment had been created in the first post of this series. See Virtual Python Environments in Setting up a Raspberry Pi as a Headless Computer in September 2021.

As a final check start a browser and go to the address specified. The home automation system will display the following page.

Of course, there will be no switches, scenes, temperature sensors and so on. When it is time, I will copy the database, various bash and python scripts from the Domoticz server that is currently running my home automation system and then restore the database to this new server. I have done this a couple of times in the past and it worked flawlessly.

Unit File to Start and Stop Domoticz toc

As described back in late 2019 in Home Automation Servers on Raspbian Buster Lite, I chose to use a systemd unit file to start and stop the domoticz.service instead of the domoticz.sh startup script installed in etc/init.d because there was a problem if a real time clock (RTC) had not been added to the Raspberry Pi (or in my case, when the RTC battery died). This problem no longer occurs in version 2021.1 (build 8547c5b7e dated 2021-04-17 12:29:11) of Domoticz. Unless you are interested in avoiding older init scripts for controlling services this subsection can be skipped as there is no longer a pressing reason to use a unit file. Nevertheless, the Linux page on the Domoticz Wiki does recommend using a unit file when the service is run on Linux distributions with systemd as the init program.

The first step is to create the unit file with nano or an editor of choice.

nostra@damus:~$ sudo nano /etc/systemd/system/domoticz.service

Here is an example unit file. The condition After=time-sync.target will ensure that the service is installed only once the system time has been synchronized. The parameter AmbientCapabilities=CAP_NET_BIND_SERVICE is needed if a TCP port less than 1024 is specified which is the case below where port 443 will be used for secured HTTP connections. See the Wiki page for more details, especially if an older version of Raspbian is being used.

[Unit] Description=domoticz_service After=time-sync.target [Service] User=nostra Group=users ExecStart=/home/nostra/domoticz/domoticz -www 8080 -sslwww 443 WorkingDirectory=/home/nostra/domoticz AmbientCapabilities=CAP_NET_BIND_SERVICE Restart=on-failure RestartSec=1m [Install] WantedBy=multi-user.target

Of course, the user name nostra, which appears three times, needs to be adjusted.

Now stop Domoticz if it is running and remove the old domoticz.sh script (saving it in case something goes wrong). Reload all service (daemon) unit files, start the Domoticz service and check its status to ensure that everything is correct.

nostra@damus:~$ sudo /etc/init.d/domoticz.sh stop [ ok ] Stopping domoticz.sh (via systemctl): domoticz.service. nostra@damus:~$ sudo mv /etc/init.d/domoticz.sh domoticz-sh nostra@damus:~$ sudo systemctl daemon-reload nostra@damus:~$ sudo systemctl start domoticz.service nostra@damus:~$ sudo systemctl status domoticz.service ● domoticz.service - domoticz_service Loaded: loaded (/etc/systemd/system/domoticz.service; disabled; vendor preset Active: active (running) since Mon 2019-12-09 15:38:10 AST; 17s ago Main PID: 2235 (domoticz) Tasks: 16 (limit: 1072) Memory: 11.8M CGroup: /system.slice/domoticz.service └─2235 /home/domoticz/domoticz/domoticz -www 8080 -sslwww 443 ...

The following command will instruct systemd to automatically start the service at boot time.

nostra@damus:~$ sudo systemctl enable domoticz.service Created symlink /etc/systemd/system/multi-user.target.wants/domoticz.service → /etc/systemd/system/domoticz.service.

This command, which creates a symbolic link to the newly created Domoticz unit file, needs to be launched only once.

When booting up after power has been off for more than 10 minutes or so, do not worry if initially the domoticz service does not appear to be functioning. It will continue to repeat its start-up routine until the time is correctly synchronized. This works no matter if the injunction to wait until time synchronization with a SMTP server is active is in place or not. In other words, the unit file line shown in red, which was an important addition back in 2019, is no longer necessary.

Watchdog toc

Since home automation is the main task to be performed by the system, it made sense to add a watchdog that would restart the system should the home automation software stop functioning correctly. I have already discussed this in the post entitled Raspberry Pi and Domoticz Watchdog where more details can be found.

The first step is to create a Lua script that Domoticz will execute every minute. All it does is update the time stamp of a file named /tmp/domoticz.alive every time it is run.

nostra@damus:~ $ nano domoticz/scripts/lua/script_time_domotizAlive.lua

-- Updates the access time of file /tmp/domoticz.alive -- once every minute. The watchdog service will reboot -- the machine if the time stamp of the file does not -- change over 5 minutes. commandArray = {} os.execute('sudo touch /tmp/domoticz.alive') return commandArray

Check the file time on a regular basis to ensure that it is updated every minute.

nostra@damus:~ $ ls -l /tmp total 4 -rw-r----- 1 root root 0 Sep 21 16:36 domoticz.alive ... nostra@damus:~ $ ls -l /tmp total 4 -rw-r----- 1 root root 0 Sep 21 16:37 domoticz.alive ...

Next install the watchdog package.

nostra@damus:~ $ sudo apt install watchdog ... Need to get 82.5 kB of archives. After this operation, 232 kB of additional disk space will be used. ... Processing triggers for man-db (2.8.5-2) ... Processing triggers for systemd (241-7~deb10u8+rpi1) ...

The watchdog configuration file has to be modified. As usual, I used nano to do this.

nostra@damus:~ $ sudo nano /etc/watchdog.conf

... file = /tmp/domoticz.alive change = 300 ... max-load-1 = 24 ... watchdog-device = /dev/watchdog watchdog-timeout = 15

As far as I can ascertain, the timeout value has to be 15 seconds, the default 60 seconds does not work.

Start the watchdog service and then wait over five minutes (300 seconds) to ensure that the system is not rebooted. Then stop the Domoticz service and the Raspberry Pi should be rebooted in about five minutes.

nostra@damus:~ $ sudo systemctl start watchdog.service nostra@damus:~ $ sudo systemctl status watchdog.service ● watchdog.service - watchdog daemon Loaded: loaded (/lib/systemd/system/watchdog.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2019-10-27 01:27:08 ADT; 19s ago Process: 28652 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/ Process: 28653 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS) Main PID: 28655 (watchdog) Tasks: 1 (limit: 2319) Memory: 548.0K CGroup: /system.slice/watchdog.service └─28655 /usr/sbin/watchdog Oct 27 01:27:08 damus watchdog[28655]: interface: no interface to check Oct 27 01:27:08 damus watchdog[28655]: temperature: no sensors to check Oct 27 01:27:08 damus watchdog[28655]: no test binary files Oct 27 01:27:08 damus watchdog[28655]: no repair binary files Oct 27 01:27:08 damus watchdog[28655]: error retry time-out = 60 seconds Oct 27 01:27:08 damus watchdog[28655]: repair attempts = 1 Oct 27 01:27:08 damus watchdog[28655]: alive=/dev/watchdog heartbeat=[none] to=root no_act=no force=no Oct 27 01:27:08 damus watchdog[28655]: watchdog now set to 15 seconds Oct 27 01:27:08 damus systemd[1]: Started watchdog daemon. Oct 27 01:27:08 damus watchdog[28655]: hardware watchdog identity: Broadcom BCM2835 Watchdog timer ... wait 10 minutes - nothing should happen ... nostra@damus:~ $ sudo systemctl stop domoticz.service ... wait at most 6 minutes, the system should reboot

As this example shows, it will be necessary to stop the watchdog if Domoticz is suspended for any length of time otherwise the Raspberry Pi will reboot.

Installing the mosquitto MQTT Broker toc

An MQTT broker is a necessary part of my home automation system. It acts as the relay between the domoticz service and the IoT devices. Most of the latter are ESP828x devices running on firmware by Theo Arends: Tasmota. The mosquitto broker is available in the Rasperry Pi OS repository as can be seen here.

nostra@damus:~ $ apt-cache policy mosquitto mosquitto: Installed: (none) Candidate: 1.5.7-1+deb10u1 Version table: 1.5.7-1+deb10u1 500 500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

The latest version of the Eclipse Mosquitto package is 2.0.12, but version 1.5.7 is recent enough and it is much easier to install mosquitto from the repository than to try to install from the source or from an alternate repository. Indeed, [version 2.0].. is a big change with breaking behaviour changes in the broker so if it is installed be prepared to fiddle with the configuration to make it work with Domoticz (see the Mosquitto MQTT broker v2.0.0 topic in the Domoticz forum). A simple installation of the older broker and the optional utilities (to get mosquitto_sub and mosquitto_pub) can be done without a problem.

nostra@damus: $ sudo apt-get install mosquitto mosquitto-clients -y ... Need to get 485 kB of archives. After this operation, 1,055 kB of additional disk space will be used. ...

After I checked and found that the broker was running automatically.

nostra@damus:~ $ sudo systemctl status mosquitto.service ● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2021-09-22 21:02:18 ADT; 36s ago Docs: man:mosquitto.conf(5) man:mosquitto(8) Main PID: 744 (mosquitto) Tasks: 1 (limit: 2178) CGroup: /system.slice/mosquitto.service └─744 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf Sep 22 21:02:18 damus systemd[1]: Starting Mosquitto MQTT v3.1/v3.1.1 Broker... Sep 22 21:02:18 damus systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker.

To make sure that everything was installed properly, I subscribed to all topics in a terminal on the Raspberry Pi.

nostra@damus:~ $ mosquitto_sub -h 127.0.0.1 -v -t "#"

Then I sent a message to the broker from a terminal on my desktop computer.

michel@hp:~$ mosquitto_pub -h damus.local -t "home" -m "hello" or michel@hp:~$ mosquitto_pub -h 192.168.1.139 -t "home" -m "hello"

If mosquitto is not installed on the desktop, the message could be published from a second terminal on the Raspberry Pi.

michel@hp:~$ ssh nostra@damus.local ... nostra@damus:~$ mosquitto_pub -h damus.local -t "home" -m "hello"

In either case, the message should show up in the first Raspberry Pi terminal.

nostra@damus:~ $ mosquitto_sub -h 127.0.0.1 -v -t "#" home hello

Instead of monitoring exchanges with the mosquitto-clients, a chat type client which can publish as well as subscribe to multiple messages could be useful. While about to shamelessly plug my own lazmqttc which runs on Linux and Windows 10 (and probably macOS though I can't be sure), I ran into MQTT X which looks to be good. I will have to try it.

Installing the NGINX Web Server toc

While there is a stand-alone web server on the Raspberry Pi it is not a very busy because the two most used services, Domoticz and HA Bridge, run their own web server. Essentially, all the web server does is upload firmware to IoT devices that are updated directly from their own web interface, something that is rarely done. This is a Tasmota feature althouh I prefer uploading new firmware to Tasmota devices directly from my desktop. So the uploading firmware with the home automation system is mostly used for IoT devices running firmware that I have written and there is only three of those.

While Apache may be the "most popular" web server, I have used the LIGHTTPD and NGINX Open Source servers on small single-board computers because they are reputed to be "lighter" and more than adequate given the minimal requirements. In this section, I will show how to set up nginx, but there's no doubt something similar could be done with LIGHTTPD and Apache.

Installing the lightest version of NGINX in the Raspberry Pi OS repository is very simple. But before doing that, I recommend to once again upgrade the system if it has not been done lately.

Because the default HTTP port of the builtin web server started by HA Bridge is 80 and nginx uses the same default port, it is best to intall the web server before installing HA bridge. If HA bridge is already installed, just suspend it until the default HTTP port used by nginx is changed.

nostra@damus:~ $ sudo apt install nginx-light -y ... The following NEW packages will be installed: libnginx-mod-HTTP-echo nginx-common nginx-light 0 upgraded, 3 newly installed, 0 to remove and 18 not upgraded. Need to get 645 kB of archives. Need to get 644 kB of archives. After this operation, 1,467 kB of additional disk space will be used. ... nostra@damus:~ $ sudo systemctl status nginx ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2021-09-23 10:27:19 ADT; 43s ago Docs: man:nginx(8) Main PID: 5598 (nginx) Tasks: 5 (limit: 2178) CGroup: /system.slice/nginx.service ├─5598 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; ├─5599 nginx: worker process ├─5600 nginx: worker process ├─5601 nginx: worker process └─5602 nginx: worker process Sep 23 10:27:19 damus systemd[1]: Starting A high performance web server and a reverse proxy server... Sep 23 10:27:19 damus systemd[1]: Started A high performance web server and a reverse proxy server.

Opening the page in a web browser verified that the server is working.

Of course it may be necessary to use the IP address of the Raspberry Pi in Windows or in some other operating system.

No matter which Web server is installed, it will be quite useful to change the owner of the Web directory.

nostra@damus:~ $ cd /var/www nostra@damus:/var/www $ sudo chown -R nostra: html

This way, the default user (nostra in this case) to will be able to add, delete and edit any file in the directory or any subdirectories that will be created.

nostra@damus:~ $ ls -l total 4 drwxr-xr-x 2 nostra nostra 4096 Sep 23 10:32:15 html

Some may wonder why nostra is not a member of the www-data group. Quoting jojopi, [t]here appears to be a common misconception that everything to do with the web should be owned by www-data. Actually it is quite the opposite. Read the complete answer in the Raspberry Forum on the question of Re: /var/www/html permissions which I found quite cogent.

I then went on to create a directory to contain the Tasmota firmware to download to various ESP8266 IoT devices if needed.

nostra@damus:/var/www $ mkdir html/tasmota

While not needed at all, I decided to create a custom 404 error page. This page will be shown when a URL links to a non-existing file.

nostra@damus:/var/www $ nano html/404.html

<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>404</title> </head> <body> <p style="font-size: 8em; text-align: center">404</p> </body> </html>

Then I created a similar 403 error page. This is what will be shown when a URL points to a subdirectory in /var/www/html that does not contain an index.html file or the equivalent, otherwise the content of the directory could be listed as a web page. Again this is not needed at all in this context, but I would suggest to do this with any outward facing web site.

Two entries must be added to the web server configuration file to display these pages when required. Here are the details for nginx.

nostra@damus:~ $ sudo nano /etc/nginx/nginx.conf

Add the entries near bottom of the HTTP block.

HTTP { ... ## # Basic Settings ## ## # Custom error messages ## error_page 403 /403.html; error_page 404 /404.html; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; }

While HA Bridge (the next server to be installed) runs very well once set up, it is a bit difficult to get it working at the start. For that reason, I much prefer not to change the default TCP port of its web interface. Consequently NGINX must listen to a TCP port other than the default 80. As usual this involves a small change to a configuration file.

nostra@damus:~ $ sudo nano /etc/nginx/sites-available/default

Change port 80 at the start of the file

# Default server configuration # server { listen 80 default_server; listen [::]:80 default_server; # SSL configuration ...

to something else. I chose 8888 (remember that Domoticz uses 8080 by default).

# Default server configuration # server { listen 8888 default_server; listen [::]:8888 default_server; # SSL configuration ...

Restart the server for this to take effect.

nostra@damus:~ $ sudo systemctl restart nginx

Installing HA Bridge toc

As discussed above, HA Bridge is a shim that presents many of the virtual devices, scenes and groups in Domoticz as hardware devices that Amazon Dot smart speakers can control locally. Previously, the bridge between Alexa and Domoticz was handled by another single-board computer, but the Raspberry Pi 3 is quite capable of handling HA Bridge as well as all the other services mentioned in this series of posts.

Since HA Bridge is a Java script, the first step is to install the Java run-time environment.

Installing the Java Runtime Environment toc

The Raspberry Pi OS repository contains many Java packages. First there are two implementations: OpenJDK and Oracle Java (remember that Oracle purchased Java from Sun). Unless there is another pressing need, I would stay away from the Oracle implementation because of its more restrictive license. Even when looking only at the "free" OpenJDK implementation, there are no less than 8 available packages.

Development KitRuntime EnvironmentVersion
openjdk-8-jdkopenjdk-8-jre8u212-b01-1+rpi1
openjdk-9-jdkopenjdk-9-jre9.0.4+12-4
openjdk-10-jdkopenjdk-10-jre10.0.2+13-2
openjdk-11-jdkopenjdk-11-jre11.0.12+7-2~deb10u1

The jdk suffix identifies the Java Development Kit while jre is the Java Runtime Environment. The JDK contains the JRE and other components needed to create Java applications (more on the difference here). I decided to install the lightest runtime version, openjdk-8-jre. Presumably, if HA Bridge works in that environment then it would work in any of the other ones. As usual, installing a package from the repository is straightforward.

nostra@damus:~ $ sudo apt install openjdk-8-jre ... The following NEW packages will be installed: openjdk-8-jre 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 61.8 kB of archives. After this operation, 234 kB of additional disk space will be used. ...

Never say Never Show

Installing the ha-bridge Script toc

Since Ha Bridge is a Java script, installation is not too complicated:

  1. Create a directory to contain the script.
  2. Copy the script to the directory.
  3. Setup a symbolic link to the script.
nostra@damus:~ $ mkdir ha-bridge nostra@damus:~ $ cd ha-bridge nostra@damus:~/ha-bridge $ wget https://github.com/bwssytems/ha-bridge/releases/download/v5.4.1RC1/ha-bridge-5.4.1RC1.jar --2021-09-23 16:12:35-- https://github.com/bwssytems/ha-bridge/releases/download/v5.4.1RC1/ha-bridge-5.4.1RC1.jar Resolving github.com (github.com)... 140.82.112.4 Connecting to github.com (github.com)|140.82.112.4|:443... connected. ... 2021-09-23 16:12:38 (5.60 MB/s) - ‘ha-bridge-5.4.1RC1.jar’ saved [9672480/9672480] nostra@damus:~/ha-bridge $ ln -s ha-bridge-5.4.1RC1.jar ha-bridge.jar checking that the link works: nostra@damus:~/ha-bridge $ ls -l total 9448 -rw-r--r-- 1 nostra nostra 9672480 Mar 24 2021 ha-bridge-5.4.1RC1.jar lrwxrwxrwx 1 nostra nostra 22 Sep 23 16:13 ha-bridge.jar -> ha-bridge-5.4.1RC1.jar

Download the latest version of the script. It should be available in the releases page of the project github account. As can be seen, I am using a release candidate for the upcoming version 5.4.1. It added the capability to assign 9 octets ID numbers to the virtual Hue devices shown to Alexa. With the latest update to the Dot firmware, this was a necessary addition otherwise the devices are not discovered by Alexa.

The symbolic link is not absolutely required, but it will make it easier to update the script without having to change the unit file used to integrate the script into the systemd startup sequence. Create the unit file with an editor. As usual I use nano.

nostra@damus:~/ha-bridge $ sudo nano /etc/systemd/system/ha-bridge.service

[Unit] Description=HA Bridge Wants=network.target After=network.target [Service] Type=simple WorkingDirectory=/home/nostra/ha-bridge ExecStart=/usr/bin/java -jar -Dconfig.file=/home/nostra/ha-bridge/data/habridge.config /home/nostra/ha-bridge/ha-bridge.jar [Install] WantedBy=multi-user.target

Do not forget to modify the user name, which in my case was nostra. It appears three times in the unit file.

All that remains now is to start the service.

nostra@damus:~/ha-bridge $ sudo systemctl daemon-reload nostra@damus:~/ha-bridge $ sudo systemctl start ha-bridge.service ● ha-bridge.service - HA Bridge Loaded: loaded (/etc/systemd/system/ha-bridge.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2021-09-23 16:18:53 ADT; 3s ago Main PID: 15282 (java) Tasks: 22 (limit: 2178) CGroup: /system.slice/ha-bridge.service └─15282 /usr/bin/java -jar -Dconfig.file=/home/nostra/ha-bridge/data/habridge.config /home/nostra/ha-bridge/ha-bridge.jar Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55.623:INFO:oejs.session:Thread-0: DefaultSessionIdManager workerName=node0 Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55.623:INFO:oejs.session:Thread-0: No SessionScavenger set, using defaults Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55.634:INFO:oejs.session:Thread-0: Scavenging every 660000ms Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55,677 [main] INFO com.bwssystems.HABridge.hue.HueMulator - Hue emulator servic Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55.710:INFO:oejs.AbstractConnector:Thread-0: Started ServerConnector@184f9e3{HTTP/1.1,[HTTP/1.1]}{0.0.0.0:80} Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55.712:INFO:oejs.Server:Thread-0: Started @2050ms Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55,796 [main] INFO com.bwssystems.HABridge.upnp.UpnpSettingsResource - Description xml service started.... Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55,815 [main] INFO com.bwssystems.HABridge.upnp.UpnpListener - UPNP Discovery Listener starting.... Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55,824 [main] INFO com.bwssystems.HABridge.upnp.UpnpListener - Create and run mDNS service. Sep 23 16:18:55 damus java[15282]: 2021-09-23 16:18:55,916 [main] INFO com.bwssystems.HABridge.upnp.UpnpListener - UPNP Discovery Listener running and ready....

It should now be possible to log into the web interface.

And, of course, it is empty.

Configuring the Home Automation System toc

All the essential services for a home automation system similar to ours are in place, but so far the system is an empty shell. Our HA system has been built over the course of many months and there is no practical way to describe it all in detail here. Instead, I will describe the installation of a virtual switch in Domoticz, which is the first step in adding a Tasmota IoT device into the HA system. In the following subsection, the virtual Domoticz sensor is added to the list of emulated devices in HA Bridge which is the first step in adding a Smart Home connected device in an Alexa account.

Adding a Virtual Switch in Domoticz toc

I have broken down the process of installing the first virtual switch in Domoticz, into 18 steps. Do not despair, the first 16 steps are required for setting the server location's latitude and longitude and for the one-time installation of the MQTT and Dummy "hardware". The geographical coordinates are used by Domoticz to calculate sunrise and sunset times. MQTT is needed to communicate with Tasmota enabled IoT devices and the Dummy hardware is a generic holder for virtual sensors, which can be many types of things including switches.

Most administrative tasks in Domoticz are done in its web interface. As already seen this is reached at http://damus.local:8080 or http://192.168.1.139:8080 (or whatever static IP address was assigned to the Raspberry Pi).

That is it. Now the Domoticz database contains one virtual device named First Switch which can be seen in the Switches tab.

Remember only these last two steps have to be repeated for each additional switch or virtual sensor that needs to be added. Again, this is done in the Setup / Hardware configuration page.

Configuring a Tasmota Device toc

Just to verify that everything works, the newly created virtual Domoticz switch is linked to a CE LA-WF3 Wi-Fi wall plug on which Tasmota 9.1.0 is installed. It was last used to control some lamps in the TV room so that it was possible to open its web interface. Settings need to be changed in order to bind the plug to the virtual device in the newly created Domoticz instance. These settings are the MQTT broker address, the Domoticz device number called its Idx and, optionally, the Syslog server address, log level and the device name. Since only one device has been added to the new HA database, it is not entirely surprising that the device number is 1. No matter, one should check just to be sure. The Idx can be found in the Devices table found in the Domoticz web interface: Setup / Devices.

The settings can be changed one at a time in various menus of the Configuration screen of the wall plug web interface. That gets tedious very quickly and is prone to errors; usually forgetting to set something. I prefer to use the Console which makes it possible to enter a single, multi-command string, exploiting the Backlog function.

Backlog Hostname first-switch; Topic first-switch; FriendlyName first-switch; deviceName first-switch; mqtthost damus.local; loghost damus.local; syslog 1; DzIdx1 1;

The sharp-eyed reader will have noticed that the device name is already changed to first-switch. The reason is that I forgot to do a screen capture when first executing the Backlog so I returned to the console to capture the screen after the fact.

When entering the Backlog command, do not forget the trailing semicolon. It must be there otherwise the last part, which in this case specifies the device index number in Domoticz, will not be executed.

As shown in the Backlog command, the local Zeroconf host name for the Pi was used both for the address of the MQTT broker (parameter mqtthost) and the Syslog host (parameter loghost). The static IP address of the Pi could have been used instead. Indeed I would suggest to do that because I am not positive that mDNS is included by default in more recent versions of Domoticz. I know that mDNS advertising was removed after version 9.1.0.

To test this configuration, it is probably easier to plug a light into the wall plug at this point. Toggle the state of the virtual switch by clicking on the light bulb icon in the web interface.

Make sure that the physical wall plug (i.e. the light) follows suite. Then try in the other direction. Turn the light on and off using the wall plug switch (not the light switch if there is one) and check that the virtual device status is updated. Be aware that the update may not be instantaneous. The impatient can refresh the web page; F5 works in many web browsers.

Setting up HA Bridge toc

The first part of the following procedure needs to be done once only. It sets the host name or IP address of the server on which Domoticz runs. There is no requirement for Ha Bridge to be run on the same computer as the rest of the home automation system. Indeed, I have already described its installation on a separated Orange Pi Zero single board computer in HA Bridge on Armbian Working with Domoticz and Alexa.

The second part of the procedure adds all the virtual devices in Domoticz that can be presented as Hue bulbs to the Ha Bridge database of emulated devices. In practice, not all virtual Domoticz devices will be added, only those where it makes sense to use voice commands.

More ... toc

The reader will have noticed that there’s no real discussion of security in this configuration of the home automation system. The justification for this is that it is meant to be on a local area network that will never be exposed to the Internet. It could be on its own virtual LAN for example. Obviously, strict enforcement of this embargo would defeat part of the purpose of a HA system. Many consider remote access to devices about the house to be very useful. In the next instalment of this guide, the installation of a virtual private network server will be covered. That offers a way of securely accessing the HA system which is not itself secure.

Additional Servers on a Raspberry Pi Based Home Automation System-> <-Setting up a Raspberry Pi as a Headless Computer in September 2021
<-HA Bridge and Alexa Device Discovery