md
Home Automation Servers on Raspbian Buster Lite
November 2, 2019
<-Installation and Configuration of Raspbian Buster Lite Various Hardware with Raspbian Buster Lite->

This is part 2 of the series of posts about installing a home automation system around Domoticz on a Raspberry Pi with Raspbian Buster Lite. It covers installing the major services that are needed for the home automation system: the home automation server itself, an MQTT broker, a Web server, and other services that I find quite useful.

Just a few days ago, Andreas Spiess published a video on YouTube, Pi Server based on Docker, with VPN remote access, Dropbox backup, Influx, Grafana, etc. While the intent is quite similar, the approach rather different. I recommend looking at that video, perhaps using Docker (yet another level of abstraction) would be preferable for some.

Table of Contents

  1. Installing Domoticz
    1. Use a Unit File to Start Domoticz
    2. Watchdog
  2. Installing mosquitto
  3. Installing a Web Server
    1. Lighttpd
    2. nginx
    3. Configuration
  4. Installing WireGuard
  5. Installing Syncthing

Installing Domoticz toc

Installing Domoticz is as simple as always when following The "easy" way instructions. I would like to reiterate my gratitude to the Domoticz development team as well as to the community.

woopi@goldserver:~ $ curl -L install.domoticz.com | sudo bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 16257 100 16257 0 0 12007 0 0:00:01 0:00:01 --:--:-- 12015 ::: ::: You are root. ::: Verifying free disk space... ... after answering a couple of questions about ports and directories:
Ready... ┌─────────────────────┤ Installation Complete! ├─────────────────────┐ │ │ │ Point your browser to either: │ │ │ │ HTTP: 192.168.1.101:8080 │ │ HTPS: 192.168.1.101:443 │ │ │ │ Wiki: https://www.domoticz.com/wiki │ │ Forum: https://www.domoticz.com/forum │ │ │ │ The install log is in /etc/domoticz. │ │ │ │ │ │ <Ok> │ │ │ └────────────────────────────────────────────────────────────────────┘
... ::: Installation Complete! Configure your browser to use the Domoticz using: ::: 192.168.1.101:8080 ::: 192.168.1.101:443

The installation script will install the git and libudev-dev packages.

Start a browser and go to the address specified. The home automation system will display the following page.

Check that everything is functioning correctly. Something like the following should be displayed.

woopi@goldserver:~$ sudo systemctl status domoticz.service ● domoticz.service - domoticz_service Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-10-17 10:37:43 AST; 6min ago Main PID: 2010 (domoticz) Tasks: 14 (limit: 1072) Memory: 5.3M CGroup: /system.slice/domoticz.service └─2010 /home/domoticz/domoticz/domoticz -www 8080 -sslwww 443 Oct 10 10:37:43 goldserver domoticz[2010]: 2019-12-08 18:37:43.891 Status: WebServer(HTTP) started on address: :: with port 8080 Oct 10 10:37:43 goldserver domoticz[2010]: 2019-12-08 18:37:43.897 Status: WebServer(SSL) started on address: :: with port 443 Oct 10 10:37:43 goldserver domoticz[2010]: 2019-12-08 18:37:43.902 Starting shared server on: :::6144 Oct 10 10:37:43 goldserver domoticz[2010]: 2019-12-08 18:37:43.902 Status: TCPServer: shared server started... Oct 10 10:37:43 goldserver domoticz[2010]: 2019-12-08 18:37:43.903 Status: RxQueue: queue worker started... Oct 10 10:37:45 goldserver domoticz[2010]: 2019-12-08 18:37:45.904 Status: EventSystem: reset all events... Oct 10 10:37:45 goldserver domoticz[2010]: 2019-12-08 18:37:45.906 Status: EventSystem: reset all device statuses... Oct 10 10:37:45 goldserver domoticz[2010]: 2019-12-08 18:37:45.908 Status: EventSystem: Started Oct 10 10:37:45 goldserver domoticz[2010]: 2019-12-08 18:37:45.908 Status: EventSystem: Queue thread started...

If an error about a missing Python library appears,

Oct 10 10:37:45 goldserver domoticz[2010]: 2019-12-08 18:37:45.907 Status: EventSystem - Python: Failed dynamic library load, install the latest libpython3.x library that is available for your platform.

then install the python3-dev package. That package would have been installed if a virtual Python environment had been created in the first post of this series. See Working Directories in Installation and Configuration of Raspbian Buster Lite.

When it is time, I will copy the database, various bash and python scripts from the Domoticz server that is currently running my home automation system and then restore the database to this new server. I have done this a couple of times in the past and it works flawlessly too.

Use Unit File to Start Domoticz toc

While everything looks to be in order, it is not the case if a real time clock (RTC) has not been installed on the Raspberry Pi or if the RTC battery has given up the ghost. To reproduced the problem, remove any RTC, power down the Raspberry Pi, wait six minutes or more and then turn the power on making sure that the device has access to the Internet. One can see that the domoticz.service started and then mysteriously stopped.

woopi@goldserver:~ $ sudo systemctl status domoticz.service ● domoticz.service - LSB: Home Automation System Loaded: loaded (/etc/init.d/domoticz.sh; generated) Active: active (exited) since Thu 2019-10-17 11:11:32 ADT; 41min ago Docs: man:systemd-sysv-generator(8) Process: 474 ExecStart=/etc/init.d/domoticz.sh start (code=exited, status=0/SUCCESS) Oct 17 11:11:30 goldserver systemd[1]: Starting LSB: Home Automation System... Oct 17 11:11:32 goldserver domoticz.sh[474]: 2019-10-17 11:11:32.643 Status: Domoticz V4.10717 (c)2012-2019 GizMoCuz Oct 17 11:11:32 goldserver domoticz.sh[474]: 2019-10-17 11:11:32.662 Status: Build Hash: b38b49e5, Date: 2019-05-09 08:04:08 Oct 17 11:11:32 goldserver domoticz.sh[474]: 2019-10-17 11:11:32.663 Status: Startup Path: /home/woopi/domoticz/ Oct 17 11:11:32 goldserver domoticz.sh[474]: domoticz: Domoticz is starting up.... Oct 17 11:11:32 goldserver domoticz[484]: Domoticz is starting up.... Oct 17 11:11:32 goldserver domoticz[627]: Domoticz running... Oct 17 11:11:32 goldserver systemd[1]: Started LSB: Home Automation System. Oct 17 11:17:37 goldserver domoticz[627]: Domoticz stopped...

This is a well known problem, which I have discussed at length here: The Domoticz Time Synchronization Problem. The best solution around this problem is to start Domoticz with a systemd unit file with appropriate parameters instead of the init script created by the installation script. Indeed the Linux page on the Domoticz Wiki recommends using the appropriated method for starting a service on Linux distributions with systemd as the init program.

The first step is to create the unit file with nano or an editor of choice.

woopi@goldserver:~$ sudo nano /etc/systemd/system/domoticz.service

Here is an example unit file. The condition After=time-sync.target will ensure that the service is installed only once the system time has been synchronized. The parameter AmbientCapabilities=CAP_NET_BIND_SERVICE is needed if a TCP port less than 1024 is specified which is the case below where port 443 will be used for secured HTTP connections. See the Wiki page for more details, especially if an older version of Raspbian is being used.

[Unit] Description=domoticz_service After=time-sync.target [Service] User=woopi Group=users ExecStart=/home/woopi/domoticz/domoticz -www 8080 -sslwww 443 WorkingDirectory=/home/woopi/domoticz AmbientCapabilities=CAP_NET_BIND_SERVICE Restart=on-failure RestartSec=1m [Install] WantedBy=multi-user.target

Of course, the user name woopi needs to be adjusted and note that it appears 3 times in the unit file.

Now stop Domoticz if it is running and remove the script (saving it in case something goes wrong). Reload daemons, start the Domoticz service and check its status to ensure everything is correct.

woopi@goldserver:~$ sudo /etc/init.d/domoticz.sh stop [ ok ] Stopping domoticz.sh (via systemctl): domoticz.service. woopi@goldserver:~$ sudo mv /etc/init.d/domoticz.sh domoticz-sh woopi@goldserver:~$ sudo systemctl daemon-reload woopi@goldserver:~$ sudo systemctl start domoticz.service woopi@goldserver:~$ sudo systemctl status domoticz.service ● domoticz.service - domoticz_service Loaded: loaded (/etc/systemd/system/domoticz.service; disabled; vendor preset Active: active (running) since Mon 2019-12-09 15:38:10 AST; 17s ago Main PID: 2235 (domoticz) Tasks: 16 (limit: 1072) Memory: 11.8M CGroup: /system.slice/domoticz.service └─2235 /home/domoticz/domoticz/domoticz -www 8080 -sslwww 443 ...

The following command will instruct systemd to automatically start the service at boot time.

woopi@goldserver:~$ sudo systemctl enable domoticz.service Created symlink /etc/systemd/system/multi-user.target.wants/domoticz.service → /etc/systemd/system/domoticz.service.

This command, which creates a symbolic link to the newly created Domoticz unit file, needs to be launched only once.

When booting up after power has been off for more than 10 minutes or so, do not worry if initially the domoticz service does not appear to be functioning:

woopi@goldserver:~$ sudo systemctl status domoticz.service ● domoticz.service - domoticz_service Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled) Active: activating (auto-restart) (Result: signal) since Thu 2019-10-17 19:20:00 AST; 53s ago Process: 881 ExecStart=/home/domoticz/domoticz/domoticz -www 8080 -sslwww 443 (code=killed, signal=USR1) Main PID: 881 (code=killed, signal=USR1)

It will be restarted in about one minute by which time the time-sync.target will have been reached.

woopi@goldserver:~$ sudo systemctl status domoticz.service ● domoticz.service - domoticz_service Loaded: loaded (/etc/systemd/system/domoticz.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2019-10-17 19:21:29 AST; 2h 35min ago Main PID: 503 (domoticz) Memory: 4.1M CGroup: /system.slice/domoticz.service └─503 /home/sancho/domoticz/domoticz -www 8080 -sslwww 443 ...

Check that system time synchronization is active.

woopi@goldserver:~$ timedatectl status Local time: Thu 2019-10-17 11:00:37 AST Universal time: Thu 2019-10-17 15:30:37 UTC RTC time: Thu 2019-10-17 15:30:37 Time zone: America/Moncton (AST, -0400) System clock synchronized: yes systemd-timesyncd.service active: active RTC in local TZ: no

There will be more on this topic in the next instalment of this guide when a real time clock is added.

Watchdog toc

Since home automation is the main task to be performed by the system, it made sense to add a watchdog that would restart the system should the home automation software stop functioning correctly. I have already discussed this Raspberry Pi and Domoticz Watchdog where more details can be found.

The first step is to create a Lua script that Domoticz will execute every minute. All it does is change the time stamp of a file.

woopi@goldserver:~ $ nano domoticz/scripts/lua/script_time_domotizAlive.lua

-- Updates the access time of file /tmp/domoticz.alive -- once every minute. The watchdog service will reboot -- the machine if the time stamp of the file does not -- change over 5 minutes. commandArray = {} os.execute('sudo touch /tmp/domoticz.alive') return commandArray

Check the file time on a regular basis to ensure that it is updated every minute.

woopi@goldserver:~ $ ls -l /tmp total 4 -rw-r----- 1 root root 0 Oct 9 16:36 domoticz.alive ... woopi@goldserver:~ $ ls -l /tmp total 4 -rw-r----- 1 root root 0 Oct 9 16:37 domoticz.alive ...

Next install the watchdog package.

woopi@goldserver:~ $ sudo apt install watchdog ... Need to get 82.5 kB of archives. After this operation, 232 kB of additional disk space will be used. ... Processing triggers for man-db (2.8.5-2) ... Processing triggers for systemd (241-7~deb10u1+rpi1) ...

Next the configuration file has to be modified. As usual, I used nano to do this.

woopi@goldserver:~ $ sudo nano /etc/watchdog.conf

... file = /tmp/domoticz.alive change = 300 ... max-load-1 = 24 ... watchdog-device = /dev/watchdog watchdog-timeout = 15

As far as I can ascertain, the timeout value has to be 15 seconds, the default 60 seconds does not work.

Start the watchdog service and then wait over five minutes (300 seconds) to ensure that the system is not rebooted. Then stop the Domoticz service and the Raspberry Pi should be rebooted in about five minutes.

woopi@goldserver:~ $ sudo systemctl start watchdog.service woopi@goldserver:~ $ sudo cat /var/log/mosquitto/mosquitto.log 1572481138: mosquitto version 1.5.7 starting woopi@goldserver:~ $ sudo systemctl status watchdog.service ● watchdog.service - watchdog daemon Loaded: loaded (/lib/systemd/system/watchdog.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2019-10-27 01:27:08 ADT; 19s ago Process: 28652 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/ Process: 28653 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS) Main PID: 28655 (watchdog) Tasks: 1 (limit: 2319) Memory: 548.0K CGroup: /system.slice/watchdog.service └─28655 /usr/sbin/watchdog Oct 27 01:27:08 goldserver watchdog[28655]: interface: no interface to check Oct 27 01:27:08 goldserver watchdog[28655]: temperature: no sensors to check Oct 27 01:27:08 goldserver watchdog[28655]: no test binary files Oct 27 01:27:08 goldserver watchdog[28655]: no repair binary files Oct 27 01:27:08 goldserver watchdog[28655]: error retry time-out = 60 seconds Oct 27 01:27:08 goldserver watchdog[28655]: repair attempts = 1 Oct 27 01:27:08 goldserver watchdog[28655]: alive=/dev/watchdog heartbeat=[none] to=root no_act=no force=no Oct 27 01:27:08 goldserver watchdog[28655]: watchdog now set to 15 seconds Oct 27 01:27:08 goldserver systemd[1]: Started watchdog daemon. Oct 27 01:27:08 goldserver watchdog[28655]: hardware watchdog identity: Broadcom BCM2835 Watchdog timer ... wait 10 minutes - nothing should happen ... woopi@raspberry:~ $ sudo systemctl stop domoticz.service ... wait at most 6 minutes, the system should reboot

As this example shows, it will be necessary to stop the watchdog if Domoticz is suspended for any length of time otherwise the Raspberry Pi will reboot.

Installing mosquitto toc

An MQTT broker is a necessary part of my home automation system. The mosquitto broker is available in the Rasbian Buster repository as can be seen here.

woopi@goldserver:~ $ sudo apt-cache policy mosquitto mosquitto: Installed: (none) Candidate: 1.5.7-1 Version table: 1.5.7-1 500 500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

The latest version of the package is 1.6.7, but version 1.5.7 is recent enough and it is much easier to install mosquitto from the repository than to try to install from the source or from an alternate repository. A simple installation of the broker and the optional utilities (to get mosquitto_sub and mosquitto_pub) went ahead without a problem.

woopi@goldserver: $ sudo apt-get install mosquitto mosquitto-clients -y ... Need to get 484 kB of archives. After this operation, 1,054 kB of additional disk space will be used. ...

After I checked and found that the broker was running automatically.

woopi@goldserver:~ $ sudo systemctl status mosquitto.service ● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-10-17 02:49:20 BST; 6min ago Docs: man:mosquitto.conf(5) man:mosquitto(8) Main PID: 2180 (mosquitto) Tasks: 1 (limit: 2200) Memory: 656.0K CGroup: /system.slice/mosquitto.service └─2180 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf Oct 17 02:49:20 goldserver systemd[1]: Starting Mosquitto MQTT v3.1/v3.1.1 Broker... Oct 17 02:49:20 goldserver systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker.

To make sure that everything was installed properly, I subscribed to all topics in a terminal on the Raspbery Pi.

woopi@goldserver:~ $ mosquitto_sub -h 127.0.0.1 -v -t "#"

Then I sent a message to the broker from a terminal on my desktop computer.

michel@hp:~$ mosquitto_pub -h goldserver.local -t "home" -m "hello" or michel@hp:~$ mosquitto_pub -h 192.168.1.22 -t "home" -m "hello"

If mosquitto is not installed on the desktop, the message could be published from a second terminal on the Raspberry Pi.

michel@hp:~$ ssh woopi@goldserver.local ... woopi@goldserver:~$ mosquitto_pub -h goldserver.local -t "home" -m "hello"

In either case, the message should show up in the first Raspberry Pi terminal.

woopi@goldserver:~ $ mosquitto_sub -h 127.0.0.1 -v -t "#" home hello

Installing a Web Server toc

In the past I used a very specific version of Lighttpd in order to implement a reverse proxy as done in Secure Webcam streaming with MJPG-Streamer on a Raspberry Pi. This is no longer required as I use a virtual private network (see the next section) to access the MJPG-Streamer web page. All I need is a simple installation of the current version of the Web server.

Lighttpd toc

This installs the latest version of Lighttpd available in the Raspbian depository.

woopi@goldserver:~ $ sudo apt install lighttpd -y ... The following NEW packages will be installed: libfam0 libmariadb3 lighttpd lighttpd-modules-ldap lighttpd-modules-mysql mariadb-common mysql-common spawn-fcgi 0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded. Need to get 545 kB of archives. After this operation, 1,659 kB of additional disk space will be used. ...

Success, lighttpd was indeed installed and running as confirmed with systemctl.

woopi@goldserver:~ $ sudo systemctl status lighttpd.service ● lighttpd.service - Lighttpd Daemon Loaded: loaded (/lib/systemd/system/lighttpd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-10-17 03:12:01 AST; 2h 43min ago Main PID: 3444 (lighttpd) Tasks: 1 (limit: 2200) Memory: 1.6M CGroup: /system.slice/lighttpd.service └─3444 /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf Oct 17 03:12:01 goldserver systemd[1]: Starting Lighttpd Daemon... Oct 17 03:12:01 goldserver systemd[1]: Started Lighttpd Daemon. Oct 17 03:12:01 goldserver systemd[1]: /lib/systemd/system/lighttpd.service:6: PIDFile= references path below legacy directory /var/run/, updating /var/run/lighttpd.pid → /run/lighttpd.pid; please update the unit file accordingly.

I ignored the request to update the unit file; restarting the service took care of care of that problem.

woopi@goldserver:~ $ sudo systemctl restart lighttpd.service woopi@goldserver:~ $ sudo systemctl status lighttpd.service ● lighttpd.service - Lighttpd Daemon Loaded: loaded (/lib/systemd/system/lighttpd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-10-07 03:18:05:05 AST; 8s ago Process: 1095 ExecStartPre=/usr/sbin/lighttpd -tt -f /etc/lighttpd/lighttpd.conf (code=exited, status=0/SUCCESS) Main PID: 1100 (lighttpd) Tasks: 1 (limit: 2319) Memory: 1.3M CGroup: /system.slice/lighttpd.service └─1100 /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf Nov 01 21:05:05 goldserver systemd[1]: Starting Lighttpd Daemon... Nov 01 21:05:05 goldserver systemd[1]: Started Lighttpd Daemon.

For some reason, the default web page was not created. But it was available elsewhere, so there was no problem copying it over to its proper location.

woopi@goldserver:~ $ sudo cp /usr/share/lighttpd/index.html /var/www/html/

I opened the page in a web browser to check that the server is installed.

Of course it may be necessary to use the IP address of the Raspberry Pi in Windows or in some other operating system.

nginx toc

Installing the lightest version of nginx is just as simple.

woopi@goldserver:~ $ sudo apt install nginx-light -y ... Need to get 644 kB of archives. After this operation, 1,467 kB of additional disk space will be used. ... woopi@goldserver:~ $ sudo systemctl status nginx ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2019-10-21 21:44:21 ADT; 3h 59min ago Docs: man:nginx(8) Main PID: 1571 (nginx) Memory: 1.1M CGroup: /system.slice/nginx.service ├─1571 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; └─1572 nginx: worker process Oct 21 21:44:21 panza systemd[1]: Starting A high performance web server and a reverse proxy server... Oct 21 21:44:21 panza systemd[1]: Started A high performance web server and a reverse proxy server. woopi@goldserver:~ $ ls -l /var/www/html total 4 -rw-r--r-- 1 root root 612 Oct 21 21:44 index.nginx-debian.html

The last command verified that a default index page was written to the Web root directory. Opening the page in a web browser verified that the server is working.

Of course it may be necessary to use the IP address of the Raspberry Pi in Windows or in some other operating system.

Configuration toc

No matter which Web server is installed, it will be quite useful to change the owner of the Web directory.

woopi@goldserver:~ $ cd /var/www woopi@goldserver:/var/www $ sudo chown -R woopi: html

This way it will be possible for woopi to add, delete and edit any file in the directory or any sub directories that will be created.

woopi@goldserver:~ $ ls -l total 4 drwxr-xr-x 2 woopi woopi 4096 Oct 21 22:15 html

Some may wonder why woopi is not a member of the www-data group. Quoting jojopi, [t]here appears to be a common misconception that everything to do with the web should be owned by www-data. Actually it is quite the opposite. Read the complete answer in the Raspberry Forum on the question of Re: /var/www/html permissions which I found quite cogent.

I then went on to create a directory to contain the Tasmota firmware to download to various ESP8266 IoT devices if needed.

woopi@goldserver:/var/www $ mkdir html/sonoff

I decided to create a custom 404 error page. This page will be shown when a URL links to a non existing file.

woopi@goldserver:/var/www $ nano html/404.html

<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>404</title> </head> <body> <p style="font-size: 8em; text-align: center">404</p> </body> </html>

Then I created a similar 403 error page. This is what will be shown when a URL points to a sub directory in /var/www/html that does not contain an index.html file or the equivalent. The content of the directory will therefore not be shown.

Two entries must be added to the web server configuration file. Here are the details for Lighttpd.

woopi@goldserver:~ $ sudo nano /etc/lighttpd/lighttpd.conf

Add the entries at the end of the server... block near the top of the file.

... server.port = 80 server.error-handler-403 = "403.html" server.error-handler-404 = "404.html" ...

If nginx is used, follow these steps.

woopi@goldserver:~ $ sudo nano /etc/nginx/nginx.conf

Add the entries near bottom of the http block.

http { ... ## # Basic Settings ## ## # Custom error messages ## error_page 403 /403.html; error_page 404 /404.html; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; }

Restart the server for this to take effect.

woopi@goldserver:~ $ sudo systemctl restart lighttpd or woopi@goldserver:~ $ sudo systemctl restart nginx

Installing WireGuard toc

One of the perks of having a home automation system is that it allows one to control devices in the house from outside. Perhaps the simplest way of going about that is to use port forwarding which involves opening a TCP port of the local area network to the outside world. Usually, that also requires setting up a dynamic DNS address for the local area network. Always access the Domoticz web server using the secure HTTPS protocol if you do that.

There are a couple coping mechanisms. One could install a Tor server on the Raspberry Pi and use a Tor browser to access the Domoticz server on the Pi. While I did use this approach in the past, I no longer do because it was too slow and cumbersome. One could try MyDomoticz which according to the Domoticz Wiki "enables Domoticz users to access their Domoticz appliance (sometimes called an "instance") from outside their local network without having to forward a http port on their router or even knowing their IP address. All communication occurs via a secure webpage and there is also no need to fiddle with certificates locally. Using this service is purely optional and can be enabled in the settings menu of Domoticz." I have not used this service and cannot comment on it.

For the last few months, I have been using a virtual private network to access not only my Domoticz server but all the resources on the local area network. A VPN does require an opened TCP port and a dynamic DNS host name. Initially, I did install an OpenVPN server but I switched to WireGuard and I am pleased with the results.

Unfortunately, WireGuard is not available in the official depository.

woopi@goldserver:~ $ sudo apt-cache policy wireguard N: Unable to locate package wireguard

So I installed the package from the unstable Debian repository following the instructions provided by Adrian Mihalko on GitHub. There are more details in my post Installing WireGuard on Raspbian Stretch and Buster.

woopi@goldserver:~ $ sudo apt-get install raspberrypi-kernel-headers ... Need to get 24.9 MB of archives. After this operation, 163 MB of additional disk space will be used. ...

I found that dirmangr was already installed, so installation of the package could be started.

woopi@goldserver:~ $ echo "deb http://deb.debian.org/debian/ unstable main" | sudo tee --append /etc/apt/sources.list.d/unstable.list deb http://deb.debian.org/debian/ unstable main woopi@goldserver:~ $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8B48AD6246925553 ... woopi@goldserver:~ $ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC 648ACFD622F3D138 ... woopi@goldserver:~ $ printf 'Package: *\nPin: release a=unstable\nPin-Priority: 150\n' | sudo tee --append /etc/apt/preferences.d/limit-unstable ... woopi@goldserver:~ $ sudo apt update ... Reading state information... Done All packages are up to date. woopi@goldserver:~ $ sudo apt install wireguard -y ... Need to get 451 kB of archives. After this operation, 2,397 kB of additional disk space will be used. ... Module build for kernel 4.19.75-v8+ was skipped since the kernel headers for this kernel does not seem to be installed. Setting up wireguard-tools (0.0.20191012-1) ... Setting up wireguard (0.0.20191012-1) ... Processing triggers for man-db (2.8.5-2) ...

There is not much to check until the server and clients have been configured. At a minimum I verified that the wg-quick utility was installed and that is configuration directory was created although it is empty.

woopi@goldserver:~ $ which wg-quick /usr/bin/wg-quick woopi@goldserver:~ $ sudo ls -l /etc/wireguard total 0

Then I copied the configuration file /etc/wireguard/wg0.conf from the SD card containing the previous installation Raspbian. This is a bit complicated because of the restrictive attributes of the directory and file. First, I copied the file from the SD card reader on the desktop to my home directory.

michel@hp:~$ sudo cp /media/michel/rootfs/etc/wireguard/wg0.conf .

Then I copied the wg0.conf file to the default home directory on the Raspberry Pi using Filezilla. Finally, I copied the configuration file from the home directory on the Pi to the WireGuard configuration directory and then launched the VPN server.

woopi@goldserver:~ $ sudo cp wg0.conf /etc/wireguard/ woopi@goldserver:~ $ wg-quick up wg0 [#] ip link add wg0 type wireguard [#] wg setconf wg0 /dev/fd/63 [#] ip -4 address add 192.168.99.1/24 dev wg0 [#] ip link set mtu 1420 up dev wg0 [#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

While everything looks OK, it will not work until the Raspberry Pi 3 B is given the static IP address of the old Raspberry Pi 1 B. But I am not going to do this until everything is mostly ready.

woopi@goldserver:~ $ wg-quick down wg0 [#] ip link delete dev wg0 [#] iptables -D FORWARD -i wg0 -j ACCEPT; iptables -D FORWARD -o wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o wlan0 -j MASQUERADE

Enabling automatic start of the wg0 interface at boot time makes sense on a server.

pi@raspberrypi:~ $ sudo systemctl enable wg-quick@wg0 Created symlink /etc/systemd/system/multi-user.target.wants/wg-quick@wg0.service → /lib/systemd/system/wg-quick@.service.

I installed two Android clients using the Adrian Mihalko user management script. The same script was used to generate keys and modify the configuration file on the Raspberry Pi to be able to communicate with another Wireguard peer on a portable computer running Linux. Again, there are more details in Installing WireGuard on Raspbian Stretch and Buster.

Installing Syncthing toc

The latest addition to the home automation machine is Syncthing a decentralized file synchronization program. With this system, the Domoticz database is synchronized with copies on other computers. I also use Syncthing to synchronize directories that containt scripts. It is all still rather new and I have not yet written a post on this subject, nevertheless here are the steps taken to install the service on the Raspberry Pi.

Using apt-cache policy, it became clear that Syncthing in the Debian and Raspbian repository are rather out of date.

woopi@goldserver:~ $ apt-cache policy syncthing syncthing: Installed: (none) Candidate: 1.0.0~ds1-1 Version table: 1.1.4~ds1-4 150 150 http://deb.debian.org/debian unstable/main armhf Packages 1.0.0~ds1-1 500 500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

So I followed the instructions at Syncthing, Debian/Ubuntu Packages to get the latest version 1.3.0 (to be supplanted by v1.3.1 on Nov. 5, 2019).

woopi@goldserver:~ $ curl -s https://syncthing.net/release-key.txt | sudo apt-key add - OK woopi@goldserver:~ $ echo "deb https://apt.syncthing.net/ syncthing candidate" | sudo tee /etc/apt/sources.list.d/syncthing.list deb https://apt.syncthing.net/ syncthing candidate woopi@goldserver:~ $ sudo apt-get update Get:1 http://deb.debian.org/debian unstable InRelease [139 kB] ... Fetched 14.4 MB in 19s (747 kB/s) Reading package lists... Done woopi@goldserver:~ $ sudo apt install syncthing ... Need to get 9,076 kB of archives. After this operation, 19.7 MB of additional disk space will be used. ... Processing triggers for mime-support (3.62) ... woopi@goldserver:~ $

Now start the program manually to verify that it functions.

woopi@goldserver:~ $ syncthing [monitor] 12:05:25 INFO: Default folder created and/or linked to new config [monitor] 12:05:25 INFO: Starting syncthing [start] 12:05:26 INFO: syncthing v1.3.1-rc.2 "Fermium Flea" (go1.13.3 linux-arm) deb@build.syncthing.net 2019-10-07 11:30:25 UTC [start] 12:05:26 INFO: Generating ECDSA key and certificate for syncthing... [start] 12:05:26 INFO: Default folder created and/or linked to new config [start] 12:05:26 INFO: Default config saved. Edit /home/woopi/.config/syncthing/config.xml to taste (with Syncthing stopped) or use the GUI [KJ3K7] 12:05:26 INFO: My ID: KJ3K7WW-KN63VNC-ZZ77WOR-6SMQQEX-CA5GLVP-B3MXVIR-KNKPJ4J-E527OAU [KJ3K7] 12:05:27 INFO: Single thread SHA256 performance is 15 MB/s using crypto/sha256 (15 MB/s using minio/sha256-simd). [KJ3K7] 12:05:28 INFO: Hashing performance is 14.59 MB/s [KJ3K7] 12:05:28 INFO: Starting deadlock detector with 20m0s timeout [KJ3K7] 12:05:28 INFO: No stored folder metadata for "default": recalculating [KJ3K7] 12:05:28 INFO: Ready to synchronize "Default Folder" (default) (sendreceive) [KJ3K7] 12:05:28 INFO: Overall send rate is unlimited, receive rate is unlimited [KJ3K7] 12:05:28 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW [KJ3K7] 12:05:28 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW [KJ3K7] 12:05:28 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW [KJ3K7] 12:05:28 INFO: Anonymous usage reporting is always enabled for candidate releases. [KJ3K7] 12:05:28 INFO: QUIC listener ([::]:22000) starting [KJ3K7] 12:05:28 INFO: TCP listener ([::]:22000) starting [KJ3K7] 12:05:28 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting [KJ3K7] 12:05:28 INFO: Completed initial scan of sendreceive folder "Default Folder" (default) [KJ3K7] 12:05:28 INFO: Loading HTTPS certificate: open /home/woopi/.config/syncthing/https-cert.pem: no such file or directory [KJ3K7] 12:05:28 INFO: Creating new HTTPS certificate [KJ3K7] 12:05:29 INFO: GUI and API listening on 127.0.0.1:8384 [KJ3K7] 12:05:29 INFO: Access the GUI via the following URL: http://127.0.0.1:8384/ [KJ3K7] 12:05:29 INFO: My name is "goldserver" [KJ3K7] 12:05:47 INFO: quic://0.0.0.0:22000 detected NAT type: Port restricted NAT [KJ3K7] 12:05:47 INFO: quic://0.0.0.0:22000 resolved external address quic://74.61.102.29:22000 (via stun.syncthing.net:3478) [KJ3K7] 12:05:48 INFO: Detected 1 NAT service [KJ3K7] 12:06:09 INFO: Joined relay relay... CtrlC shut down application [monitor] 12:07:15 INFO: Signal 2 received; exiting [KJ3K7] 12:07:15 INFO: QUIC listener ([::]:22000) shutting down [KJ3K7] 12:07:15 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down [KJ3K7] 12:07:15 INFO: Disconnected from relay relay: [KJ3K7] 12:07:16 INFO: TCP listener ([::]:22000) shutting down [KJ3K7] 12:07:16 INFO: Exiting

Since I want to control the application throught its Web interface from my desktop computer, I need to modify the configuration as explained in the FAQ.

woopi@goldserver:~ $ nano .config/syncthing/config.xml

Locate the <gui> entry and change the <address> value from 127.0.0.1:8384 to 0.0.0.0:8384.

<gui enabled="true" tls="false" debugging="false"> <address>0.0.0.0:8384</address> <apikey>...

On opening the Syncthing interface from the desktop using a Web browser pointed to 192.168.1.22:8384, there was a warning about the lack of security, so I followed the instruction to add a user and password to the GUI interface.

I wanted syncthing to start automatically when the Raspberry Pi is booted. Here are the steps I followed.

woopi@goldserver:~ $ sudo wget https://raw.githubusercontent.com/syncthing/syncthing/master/etc/linux-systemd/system/syncthing%40.service \ > -O /etc/systemd/user/syncthing@.service --2019-10-22 13:30:40-- https://github.com/syncthing/syncthing/blob/master/etc/linux-systemd/system/syncthing%40.service Resolving github.com (github.com)... 140.82.113.4 Connecting to github.com (github.com)|140.82.113.4|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: ‘/etc/systemd/user/syncthing@.service’ /etc/systemd/user/syncthing@. [ <=> ] 75.77K 493KB/s in 0.2s 2019-10-22 13:30:40 (493 KB/s) - ‘/etc/systemd/user/syncthing@.service’ saved [77590] woopi@goldserver:~ $ sudo systemctl enable syncthing@woopi.service Created symlink /etc/systemd/system/multi-user.target.wants/syncthing@woopi.service → /lib/systemd/system/syncthing@.service. woopi@goldserver:~ $ sudo systemctl start syncthing@woopi.service woopi@goldserver:~ $ sudo systemctl status syncthing@* ● syncthing@woopi.service - Syncthing - Open Source Continuous File Synchronization for woopi Loaded: loaded (/lib/systemd/system/syncthing@.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2019-10-22 13:33:30 ADT; 18s ago Docs: man:syncthing(1) Main PID: 2876 (syncthing) Tasks: 15 (limit: 2319) Memory: 20.5M CGroup: /system.slice/system-syncthing.slice/syncthing@woopi.service └─2876 /usr/bin/syncthing -no-browser -no-restart -logflags=0 Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: Overall send rate is unlimited, receive rate is unlimited Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: Anonymous usage reporting is always enabled for candidate releases. Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: Completed initial scan of sendreceive folder "Default Folder" (defa Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: QUIC listener ([::]:22000) starting Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: TCP listener ([::]:22000) starting Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) star ...

With more experience I may write up a post on using this service. As an indication of what I am doing, here is a screenshot of the opening page of the Web interface.

A directory named ~/goldserver_syspy on the desktop (hp) is shared with directory ~/.syspy on the Raspberry Pi. This shared directory is named syspy in syncthing. Any changes made to a file in ~/golserver_syspy on the desktop are copied to the corresponding file in .syspy of Raspberry Pi. Conversely, any changes made on the Raspberry Pi are copied to the desktop.

For information on setting up Syncthing on other computers and Android tablets and sharing directories see the documentation.

<-Installation and Configuration of Raspbian Buster Lite Various Hardware with Raspbian Buster Lite->