This is part 2 of the series of posts about installing a home automation system around Domoticz on a Raspberry Pi with Raspbian Buster Lite. It covers installing the major services that are needed for the home automation system: the home automation server itself, an MQTT broker, a Web server, and other services that I find quite useful.

Just a few days ago, Andreas Spiess published a video on YouTube, Pi Server based on Docker, with VPN remote access, Dropbox backup, Influx, Grafana, etc. While the intent is quite similar, the approach rather different. I recommend looking at that video, perhaps using Docker (yet another level of abstraction) would be preferable for some.

Table of Contents

1. Installing Domoticz
1. Use a Unit File to Start Domoticz
2. Watchdog
2. Installing mosquitto
3. Installing a Web Server
   1. Lighttpd
   2. nginx
   3. Configuration
4. Installing WireGuard
5. Installing Syncthing

Installing Domoticz

Installing Domoticz is as simple as always when following The "easy" way instructions. I would like to reiterate my gratitude to the Domoticz development team as well as to the community.

The installation script will install the git and libudev-dev packages.

Start a browser and go to the address specified. The home automation system will display the following page.

Check that everything is functioning correctly. Something like the following should be displayed.

If an error about a missing Python library appears,

then install the python3-dev package. That package would have been installed if a virtual Python environment had been created in the first post of this series. See Working Directories in Installation and Configuration of Raspbian Buster Lite.

When it is time, I will copy the database, various bash and python scripts from the Domoticz server that is currently running my home automation system and then restore the database to this new server. I have done this a couple of times in the past and it works flawlessly too.

Use Unit File to Start Domoticz

While everything looks to be in order, it is not the case if a real time clock (RTC) has not been installed on the Raspberry Pi or if the RTC battery has given up the ghost. To reproduced the problem, remove any RTC, power down the Raspberry Pi, wait six minutes or more and then turn the power on making sure that the device has access to the Internet. One can see that the domoticz.service started and then mysteriously stopped.

This is a well known problem, which I have discussed at length here: The Domoticz Time Synchronization Problem. The best solution around this problem is to start Domoticz with a systemd unit file with appropriate parameters instead of the init script created by the installation script. Indeed the Linux page on the Domoticz Wiki recommends using the appropriated method for starting a service on Linux distributions with systemd as the init program.

The first step is to create the unit file with nano or an editor of choice.

Here is an example unit file. The condition After=time-sync.target will ensure that the service is installed only once the system time has been synchronized. The parameter AmbientCapabilities=CAP_NET_BIND_SERVICE is needed if a TCP port less than 1024 is specified which is the case below where port 443 will be used for secured HTTP connections. See the Wiki page for more details, especially if an older version of Raspbian is being used.

Of course, the user name woopi needs to be adjusted and note that it appears 3 times in the unit file.

Now stop Domoticz if it is running and remove the script (saving it in case something goes wrong). Reload daemons, start the Domoticz service and check its status to ensure everything is correct.

The following command will instruct systemd to automatically start the service at boot time.

This command, which creates a symbolic link to the newly created Domoticz unit file, needs to be launched only once.

When booting up after power has been off for more than 10 minutes or so, do not worry if initially the domoticz service does not appear to be functioning:

It will be restarted in about one minute by which time the time-sync.target will have been reached.

Check that system time synchronization is active.

There will be more on this topic in the next instalment of this guide when a real time clock is added.

Watchdog

Since home automation is the main task to be performed by the system, it made sense to add a watchdog that would restart the system should the home automation software stop functioning correctly. I have already discussed this Raspberry Pi and Domoticz Watchdog where more details can be found.

The first step is to create a Lua script that Domoticz will execute every minute. All it does is change the time stamp of a file.

Check the file time on a regular basis to ensure that it is updated every minute.

Next install the watchdog package.

Next the configuration file has to be modified. As usual, I used nano to do this.

As far as I can ascertain, the timeout value has to be 15 seconds, the default 60 seconds does not work.

Start the watchdog service and then wait over five minutes (300 seconds) to ensure that the system is not rebooted. Then stop the Domoticz service and the Raspberry Pi should be rebooted in about five minutes.

As this example shows, it will be necessary to stop the watchdog if Domoticz is suspended for any length of time otherwise the Raspberry Pi will reboot.

Installing mosquitto

An MQTT broker is a necessary part of my home automation system. The mosquitto broker is available in the Rasbian Buster repository as can be seen here.

The latest version of the package is 1.6.7, but version 1.5.7 is recent enough and it is much easier to install mosquitto from the repository than to try to install from the source or from an alternate repository. A simple installation of the broker and the optional utilities (to get mosquitto_sub and mosquitto_pub) went ahead without a problem.

After I checked and found that the broker was running automatically.

To make sure that everything was installed properly, I subscribed to all topics in a terminal on the Raspbery Pi.

Then I sent a message to the broker from a terminal on my desktop computer.

If mosquitto is not installed on the desktop, the message could be published from a second terminal on the Raspberry Pi.

In either case, the message should show up in the first Raspberry Pi terminal.

Installing a Web Server

In the past I used a very specific version of Lighttpd in order to implement a reverse proxy as done in Secure Webcam streaming with MJPG-Streamer on a Raspberry Pi. This is no longer required as I use a virtual private network (see the next section) to access the MJPG-Streamer web page. All I need is a simple installation of the current version of the Web server.

Lighttpd

This installs the latest version of Lighttpd available in the Raspbian depository.

Success, lighttpd was indeed installed and running as confirmed with systemctl.

I ignored the request to update the unit file; restarting the service took care of care of that problem.

For some reason, the default web page was not created. But it was available elsewhere, so there was no problem copying it over to its proper location.

I opened the page in a web browser to check that the server is installed.

Of course it may be necessary to use the IP address of the Raspberry Pi in Windows or in some other operating system.

nginx

Installing the lightest version of nginx is just as simple.

The last command verified that a default index page was written to the Web root directory. Opening the page in a web browser verified that the server is working.

Of course it may be necessary to use the IP address of the Raspberry Pi in Windows or in some other operating system.

Configuration

No matter which Web server is installed, it will be quite useful to change the owner of the Web directory.

This way it will be possible for woopi to add, delete and edit any file in the directory or any sub directories that will be created.

Some may wonder why woopi is not a member of the www-data group. Quoting jojopi, [t]here appears to be a common misconception that everything to do with the web should be owned by www-data. Actually it is quite the opposite. Read the complete answer in the Raspberry Forum on the question of Re: /var/www/html permissions which I found quite cogent.

I then went on to create a directory to contain the Tasmota firmware to download to various ESP8266 IoT devices if needed.

I decided to create a custom 404 error page. This page will be shown when a URL links to a non existing file.

Then I created a similar 403 error page. This is what will be shown when a URL points to a sub directory in /var/www/html that does not contain an index.html file or the equivalent. The content of the directory will therefore not be shown.

Two entries must be added to the web server configuration file. Here are the details for Lighttpd.

Add the entries at the end of the server... block near the top of the file.

If nginx is used, follow these steps.

Add the entries near bottom of the http block.

Restart the server for this to take effect.

Installing WireGuard

One of the perks of having a home automation system is that it allows one to control devices in the house from outside. Perhaps the simplest way of going about that is to use port forwarding which involves opening a TCP port of the local area network to the outside world. Usually, that also requires setting up a dynamic DNS address for the local area network. Always access the Domoticz web server using the secure HTTPS protocol if you do that.

There are a couple coping mechanisms. One could install a Tor server on the Raspberry Pi and use a Tor browser to access the Domoticz server on the Pi. While I did use this approach in the past, I no longer do because it was too slow and cumbersome. One could try MyDomoticz which according to the Domoticz Wiki "enables Domoticz users to access their Domoticz appliance (sometimes called an "instance") from outside their local network without having to forward a http port on their router or even knowing their IP address. All communication occurs via a secure webpage and there is also no need to fiddle with certificates locally. Using this service is purely optional and can be enabled in the settings menu of Domoticz." I have not used this service and cannot comment on it.

For the last few months, I have been using a virtual private network to access not only my Domoticz server but all the resources on the local area network. A VPN does require an opened TCP port and a dynamic DNS host name. Initially, I did install an OpenVPN server but I switched to WireGuard and I am pleased with the results.

Unfortunately, WireGuard is not available in the official depository.

So I installed the package from the unstable Debian repository following the instructions provided by Adrian Mihalko on GitHub. There are more details in my post Installing WireGuard on Raspbian Stretch and Buster.

I found that dirmangr was already installed, so installation of the package could be started.

There is not much to check until the server and clients have been configured. At a minimum I verified that the wg-quick utility was installed and that is configuration directory was created although it is empty.

Then I copied the configuration file /etc/wireguard/wg0.conf from the SD card containing the previous installation Raspbian. This is a bit complicated because of the restrictive attributes of the directory and file. First, I copied the file from the SD card reader on the desktop to my home directory.

Then I copied the wg0.conf file to the default home directory on the Raspberry Pi using Filezilla. Finally, I copied the configuration file from the home directory on the Pi to the WireGuard configuration directory and then launched the VPN server.

While everything looks OK, it will not work until the Raspberry Pi 3 B is given the static IP address of the old Raspberry Pi 1 B. But I am not going to do this until everything is mostly ready.

Enabling automatic start of the wg0 interface at boot time makes sense on a server.

I installed two Android clients using the Adrian Mihalko user management script. The same script was used to generate keys and modify the configuration file on the Raspberry Pi to be able to communicate with another Wireguard peer on a portable computer running Linux. Again, there are more details in Installing WireGuard on Raspbian Stretch and Buster.

Installing Syncthing

The latest addition to the home automation machine is Syncthing a decentralized file synchronization program. With this system, the Domoticz database is synchronized with copies on other computers. I also use Syncthing to synchronize directories that containt scripts. It is all still rather new and I have not yet written a post on this subject, nevertheless here are the steps taken to install the service on the Raspberry Pi.

Using apt-cache policy, it became clear that Syncthing in the Debian and Raspbian repository are rather out of date.

So I followed the instructions at Syncthing, Debian/Ubuntu Packages to get the latest version 1.3.0 (to be supplanted by v1.3.1 on Nov. 5, 2019).

Now start the program manually to verify that it functions.

Since I want to control the application throught its Web interface from my desktop computer, I need to modify the configuration as explained in the FAQ.

Locate the <gui> entry and change the <address> value from 127.0.0.1:8384 to 0.0.0.0:8384.

On opening the Syncthing interface from the desktop using a Web browser pointed to 192.168.1.22:8384, there was a warning about the lack of security, so I followed the instruction to add a user and password to the GUI interface.

I wanted syncthing to start automatically when the Raspberry Pi is booted. Here are the steps I followed.

With more experience I may write up a post on using this service. As an indication of what I am doing, here is a screenshot of the opening page of the Web interface.

A directory named ~/goldserver_syspy on the desktop (hp) is shared with directory ~/.syspy on the Raspberry Pi. This shared directory is named syspy in syncthing. Any changes made to a file in ~/golserver_syspy on the desktop are copied to the corresponding file in .syspy of Raspberry Pi. Conversely, any changes made on the Raspberry Pi are copied to the desktop.

For information on setting up Syncthing on other computers and Android tablets and sharing directories see the documentation.