md
Home Automation Servers on Raspbian Buster Lite
November 2, 2019
<-Installation and Configuration of Raspbian Buster Lite Various Hardware with Raspbian Buster Lite->

This is part 2 of the series of posts about installing a home automation system around Domoticz on a Raspberry Pi with Raspbian Buster Lite. It covers installing the major services that are needed for the home automation system: the home automation server itself, an MQTT broker, a Web server, and other services that I find quite useful.

Just a few days ago, Andreas Spiess published a video on YouTube, Pi Server based on Docker, with VPN remote access, Dropbox backup, Influx, Grafana, etc. While the intent is quite similar, the approach rather different. I recommend looking at that video, perhaps using Docker (yet another level of abstraction) would be preferable for some.

Table of Contents

  1. Installing Domoticz
    1. Time Problem with Domoticz
    2. Auto Restart Domoticz Service
    3. Wait for systemd-timesync
    4. Watchdog
  2. Installing mosquitto
  3. Installing a Web Server
    1. Lighttpd
    2. nginx
    3. Configuration
  4. Installing WireGuard
  5. Installing Syncthing

Installing Domoticz toc

Installing Domoticz is as simple as always when following The "easy" way instructions.

woopi@goldserver:~ $ curl -L install.domoticz.com | sudo bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 16257 100 16257 0 0 12007 0 0:00:01 0:00:01 --:--:-- 12015 ::: ::: You are root. ::: Verifying free disk space... ... after answering a couple of questions about ports and directories:
Ready... ┌─────────────────────┤ Installation Complete! ├─────────────────────┐ │ │ │ Point your browser to either: │ │ │ │ HTTP: 192.168.1.101:8080 │ │ HTPS: 192.168.1.101:443 │ │ │ │ Wiki: https://www.domoticz.com/wiki │ │ Forum: https://www.domoticz.com/forum │ │ │ │ The install log is in /etc/domoticz. │ │ │ │ │ │ <Ok> │ │ │ └────────────────────────────────────────────────────────────────────┘
... ::: Installation Complete! Configure your browser to use the Domoticz using: ::: 192.168.1.101:8080 ::: 192.168.1.101:443

The installation script will install the git and libudev-dev packages.

Start a browser and go to the address specified. The home automation system will display the following page.

This is a good place to express all my gratitude to the Domoticz development team as well as to the community.

When it is time, I will copy the database, various bash and python scripts from the Domoticz server that is currently running my home automation system and then restore the database to this new server. I have done this a couple of times in the past and it works flawlessly too.

Time Problem with Domoticz toc

Everything looks to be in order but this may not the case. Power down the Raspberry Pi, wait six minutes or more and then turn the power on and try to open the Domoticz web page. It may be that the web server cannot be reached because the domoticz.service started and then mysteriously stopped.

woopi@goldserver:~ $ sudo systemctl status domoticz.service ● domoticz.service - LSB: Home Automation System Loaded: loaded (/etc/init.d/domoticz.sh; generated) Active: active (exited) since Thu 2019-10-17 11:11:32 ADT; 41min ago Docs: man:systemd-sysv-generator(8) Process: 474 ExecStart=/etc/init.d/domoticz.sh start (code=exited, status=0/SUCCESS) Oct 17 11:11:30 goldserver systemd[1]: Starting LSB: Home Automation System... Oct 17 11:11:32 goldserver domoticz.sh[474]: 2019-10-17 11:11:32.643 Status: Domoticz V4.10717 (c)2012-2019 GizMoCuz Oct 17 11:11:32 goldserver domoticz.sh[474]: 2019-10-17 11:11:32.662 Status: Build Hash: b38b49e5, Date: 2019-05-09 08:04:08 Oct 17 11:11:32 goldserver domoticz.sh[474]: 2019-10-17 11:11:32.663 Status: Startup Path: /home/woopi/domoticz/ Oct 17 11:11:32 goldserver domoticz.sh[474]: domoticz: Domoticz is starting up.... Oct 17 11:11:32 goldserver domoticz[484]: Domoticz is starting up.... Oct 17 11:11:32 goldserver domoticz[627]: Domoticz running... Oct 17 11:11:32 goldserver systemd[1]: Started LSB: Home Automation System. Oct 17 11:17:37 goldserver domoticz[627]: Domoticz stopped...

Even if you do not have this known problem, please read on because it could present itself later on.

Note the 6-minute gap between the last two entries in the log. As waaren explains in a Domoticz Forum entry Crashes on cold start on Raspberry and other systems without RTC, the server "interprets this time [gap] as a hang of some critical internal processes" and shuts down. The Domoticz server will work after a reboot or after restarting the service manually, which is not an acceptable solution.

As waaren says, I should not have encountered the problem because there was a real-time clock (RTC) (DS3231, see part 3 of this series of posts) on the Raspberry Pi. Fortunately, its battery was dead so it was not doing anything. I changed the RTC battery and the service no longer crashes on cold starts. Clearly, this could be a recurring problem when the battery expires again, so the Domoticz installation needs to be improved to handle the situation when the RTC is no longer functioning.

On the older Raspberry Pi there will not be a real-time clock. So I tried using the startupdelay option when starting Domoticz, by editing the start up script.

pi@raspberrypi:~ $ sudo nano /etc/init.d/domoticz.sh

I added a 30 second delay to the DAEMON_ARGS parameter.

DAEMON_ARGS="$DAEMON_ARGS -startupdelay 30"

The result was disappointing.

pi@raspberrypi:~ $ sudo systemctl status domoticz.service ● domoticz.service - LSB: Home Automation System Loaded: loaded (/etc/init.d/domoticz.sh; generated) Active: active (exited) since Fri 2019-10-18 12:32:28 AST; 3min 27s ago Docs: man:systemd-sysv-generator(8) Process: 472 ExecStart=/etc/init.d/domoticz.sh start (code=exited, status=0/SUCCESS) Oct 17 22:23:54 raspberrypi systemd[1]: Starting LSB: Home Automation System... Oct 17 22:23:55 raspberrypi domoticz.sh[472]: 2019-10-17 22:23:55.961 Status: Domoticz V4.10717 (c)2012-2019 GizMoCuz Oct 17 22:23:55 raspberrypi domoticz.sh[472]: 2019-10-17 22:23:55.970 Status: Build Hash: b38b49e5, Date: 2019-05-09 08:04:08 Oct 17 22:23:55 raspberrypi domoticz.sh[472]: 2019-10-17 22:23:55.971 Status: Startup Path: /home/pi/domoticz/ Oct 17 22:23:55 raspberrypi domoticz.sh[472]: 2019-10-17 22:23:55.971 Status: Startup delay... waiting 30 seconds... Oct 18 12:32:28 raspberrypi domoticz.sh[472]: domoticz: Domoticz is starting up.... Oct 18 12:32:28 raspberrypi domoticz[481]: Domoticz is starting up.... Oct 18 12:32:28 raspberrypi domoticz[640]: Domoticz running... Oct 18 12:32:28 raspberrypi systemd[1]: Started LSB: Home Automation System. Oct 18 12:32:36 raspberrypi domoticz[640]: Domoticz stopped...

Clearly the time had not been updated when the script was started but after the 30 second delay when domoticz was starting up the time was correct. Nevertheless, the service was stopped.

In his forum post, waaren suggests ways of working around the problem. The most elegant may very well be Alexandre Gambier's Domoticz restart service which is discussed next. In the end I devised a different solution based on the built-in smtp client systemd-timesync.

Auto Restart Domoticz Service toc

Installing the service is a simple affair: create a directory and then add a short unit file in that directory.

woopi@goldserver:~ $ sudo mkdir /etc/systemd/system/domoticz.service.d woopi@goldserver:~ $ sudo nano /etc/systemd/system/domoticz.service.d/restart.conf

Copy the content of Alexandre Gambier's unit file:

[Service]
Type=forking
PIDFile=/run/domoticz.pid
RemainAfterExit=no
Restart=on-failure
RestartSec=5s

into the text editor and then exit nano (CtrlX), saving the file when prompted. Reload all unit files and check that the restart configuration service has been installed.

woopi@goldserver:~ $ sudo systemctl daemon-reload woopi@goldserver:~ $ sudo systemctl status domoticz ● domoticz.service - LSB: Home Automation System Loaded: loaded (/etc/init.d/domoticz.sh; generated) Drop-In: /etc/systemd/system/domoticz.service.d └─restart.conf Active: active (running) since Thu 2019-10-17 12:22:36 ADT; 13min ago ...

Check by powering down the Pi for at least five minutes.

woopi@goldserver:~ $ sudo shutdown now Connection to goldserver.local closed by remote host. Connection to goldserver.local closed.
woopi@goldserver:~ $ journalctl -q | grep goldserver Oct 17 15:29:09 goldserver domoticz.sh[478]: 2019-10-17 15:29:09.629 Status: Domoticz V4.10717 (c)2012-2019 GizMoCuz Oct 17 15:29:09 goldserver domoticz.sh[478]: 2019-10-17 15:29:09.638 Status: Build Hash: b38b49e5, Date: 2019-05-09 08:04:08 Oct 17 15:29:09 goldserver domoticz.sh[478]: 2019-10-17 15:29:09.638 Status: Startup Path: /home/woopi/domoticz/ Oct 17 15:29:09 goldserver domoticz.sh[478]: domoticz: Domoticz is starting up.... Oct 17 15:29:09 goldserver domoticz[490]: Domoticz is starting up.... Oct 17 15:29:09 goldserver domoticz[545]: Domoticz running... Oct 17 15:55:56 goldserver domoticz[545]: Domoticz stopped... Oct 17 15:55:59 goldserver systemd[1]: domoticz.service: Main process exited, code=killed, status=10/USR1 Oct 17 15:55:59 goldserver systemd[1]: domoticz.service: Failed with result 'signal'. Oct 17 15:56:04 goldserver systemd[1]: domoticz.service: Service RestartSec=5s expired, scheduling restart. Oct 17 15:56:04 goldserver systemd[1]: domoticz.service: Scheduled restart job, restart counter is at 1. Oct 17 15:56:04 goldserver domoticz.sh[658]: 2019-10-17 15:56:04.897 Status: Domoticz V4.10717 (c)2012-2019 GizMoCuz Oct 17 15:56:04 goldserver domoticz.sh[658]: 2019-10-17 15:56:04.897 Status: Build Hash: b38b49e5, Date: 2019-05-09 08:04:08 Oct 17 15:56:04 goldserver domoticz.sh[658]: 2019-10-17 15:56:04.897 Status: Startup Path: /home/woopi/domoticz/ Oct 17 15:56:04 goldserver domoticz.sh[658]: domoticz: Domoticz is starting up.... Oct 17 15:56:04 goldserver domoticz[663]: Domoticz is starting up.... Oct 17 15:56:04 goldserver domoticz[664]: Domoticz running.

That works very well, the service was stopped as before but then the restart service kicked in and Domoticz was restarted and remained running.

woopi@goldserver:~ $ sudo reboot Connection to goldserver.local closed by remote host. Connection to goldserver.local closed. after waiting a minute or two michel@hp:~$ ssh woopi@goldserver.local or michel@hp:~$ ssh pi@192.168.1.101 woopi@goldserver:~ $ journalctl -q | grep goldserver Oct 17 16:06:24 goldserver domoticz.sh[429]: 2019-10-17 16:06:24.121 Status: Domoticz V4.10717 (c)2012-2019 GizMoCuz Oct 17 16:06:24 goldserver domoticz.sh[429]: 2019-10-17 16:06:24.129 Status: Build Hash: b38b49e5, Date: 2019-05-09 08:04:08 Oct 17 16:06:24 goldserver domoticz.sh[429]: 2019-10-17 16:06:24.129 Status: Startup Path: /home/woopi/domoticz/ Oct 17 16:06:24 goldserver domoticz.sh[429]: domoticz: Domoticz is starting up.... Oct 17 16:06:24 goldserver domoticz[439]: Domoticz is starting up.... Oct 17 16:06:24 goldserver domoticz[449]: Domoticz running...

As promised, the drop-in restart service did not restart Domoticz this time because it was not stopped. The only niggling problem is a warning shown in the service status report or when stop or restarting the service.

woopi@goldserver:~ $ sudo systemctl status domoticz Warning: The unit file, source configuration file or drop-ins of domoticz.service changed on disk. Run 'systemctl daemon-reload' to reload units. ● domoticz.service - LSB: Home Automation System Loaded: loaded (/etc/init.d/domoticz.sh; generated) Drop-In: /etc/systemd/system/domoticz.service.d └─restart.conf

To get rid of the restart drop-in service, just delete the domoticz.service.d directory in /etc/systemd/system and reload the service unit files.

woopi@goldserver:~ $ sudo rm -r /etc/systemd/system/domoticz.service.d woopi@goldserver:~ $ sudo systemctl daemon-reload

Wait for systemd-timesync toc

I decided to use another approach based on the one proposed by waaren. It's interest for me lies in the fact that it works with the Raspberry Pi 3 with a working or non-working RTC and with the Raspberry Pi 1 which does not have an RTC and where ntp will not be installed. All that is required is a few extra lines in the Domoticz start script.

woopi@goldserver:~ sudo nano /etc/init.d/domoticz.sh

# # Function that starts the daemon/service # do_start() { # Wait at most count seconds for time synchronization # See http://sigmdel.ca/guide_buster02_en.html#syncdomoticz count=30 while [ ! -f "/run/systemd/timesync/synchronized" ] do count=$((count-1)) if [ $((count)) -lt 1 ] then echo "Could not sync" return 2 fi echo -n "." sleep 1 done echo "Sync'd" # Return # 0 if daemon has been started # 1 if daemon was already running # 2 if daemon could not be started ... }

Insert a while loop at the beginning of the do_start() function. It will wait until systemd-timesync has created a file named synchronized to indicate that it has obtained a valid date and time. If systemd-timesync, an sntp client, has not synchronized with a server within n seconds, the do_start() will be exited returning a value of 2 meaning the daemon could not be started. I take no credit for this, I just adapted the script by jcyr found on his reply in the Raspberry Forum. There are other approaches that could be of interest in that discussion. The following shows the the change in action when an ssh session was opened just after the Raspberry Pi without a hardware real time clock was rebooted.

michel@hp:~$ ssh woopi@goldserver.local woopi@goldserver.local's password: xxxxxx not echoed to the screen ... woopi@goldserver:~ $ sudo systemctl status domoticz ● domoticz.service - LSB: Home Automation System Loaded: loaded (/etc/init.d/domoticz.sh; generated) Active: activating (start) since Mon 2019-10-21 20:00:04 ADT; 16s ago Docs: man:systemd-sysv-generator(8) Cntrl PID: 520 (domoticz.sh) Tasks: 2 (limit: 2319) Memory: 884.0K CGroup: /system.slice/domoticz.service ├─520 /bin/sh /etc/init.d/domoticz.sh start └─735 sleep 1 Oct 21 20:00:04 goldserver systemd[1]: Starting LSB: Home Automation System... woopi@goldserver:~ $ sudo systemctl status domoticz ● domoticz.service - LSB: Home Automation System Loaded: loaded (/etc/init.d/domoticz.sh; generated) Active: active (running) since Mon 2019-10-21 20:00:38 ADT; 7s ago Docs: man:systemd-sysv-generator(8) Process: 520 ExecStart=/etc/init.d/domoticz.sh start (code=exited, status=0/SUCCESS) Tasks: 22 (limit: 2319) Memory: 23.2M CGroup: /system.slice/domoticz.service └─756 /home/woopi/domoticz/domoticz -daemon -www 8080 -sslwww 447 Oct 21 20:00:04 goldserver systemd[1]: Starting LSB: Home Automation System... Oct 21 20:00:37 goldserver domoticz.sh[520]: ........................Sync'd Oct 21 20:00:38 goldserver domoticz.sh[520]: 2019-10-21 20:00:38.115 Status: Domoticz V4.10717 (c)2012-2019 GizMoCuz Oct 21 20:00:38 goldserver domoticz.sh[520]: 2019-10-21 20:00:38.122 Status: Build Hash: b38b49e5, Date: 2019-05-09 08:04:08 Oct 21 20:00:38 goldserver domoticz.sh[520]: 2019-10-21 20:00:38.123 Status: Startup Path: /home/woopi/domoticz/ Oct 21 20:00:38 goldserver domoticz.sh[520]: domoticz: Domoticz is starting up.... Oct 21 20:00:38 goldserver domoticz[755]: Domoticz is starting up.... Oct 21 20:00:38 goldserver domoticz[756]: Domoticz running... Oct 21 20:00:38 goldserver systemd[1]: Started LSB: Home Automation System.

The status of the domoticz.service was checked just after booting and as can be seen, the install script was still waiting for systemd-timesync. The second time around, the sntp client had finally got the time from an Internet time server after about 20 seconds. If the Domoticz service were to be restarted, it would be "Sync'd" right away.

Here is the unformatted while loop to add to the start script.

Watchdog toc

Since home automation is the main task to be performed by the system, it made sense to add a watchdog that would restart the system should the home automation software stop functioning correctly. I have already discussed this Raspberry Pi and Domoticz Watchdog where more details can be found.

The first step is to create a Lua script that Domoticz will execute every minute. All it does is change the time stamp of a file.

woopi@goldserver:~ $ nano domoticz/scripts/lua/script_time_domotizAlive.lua

-- Updates the access time of file /tmp/domoticz.alive -- once every minute. The watchdog service will reboot -- the machine if the time stamp of the file does not -- change over 5 minutes. commandArray = {} os.execute('sudo touch /tmp/domoticz.alive') return commandArray

Check the file time on a regular basis to ensure that it is updated every minute.

woopi@goldserver:~ $ ls -l /tmp total 4 -rw-r----- 1 root root 0 Oct 9 16:36 domoticz.alive ... woopi@goldserver:~ $ ls -l /tmp total 4 -rw-r----- 1 root root 0 Oct 9 16:37 domoticz.alive ...

Next install the watchdog package.

woopi@goldserver:~ $ sudo apt install watchdog ... Need to get 82.5 kB of archives. After this operation, 232 kB of additional disk space will be used. ... Processing triggers for man-db (2.8.5-2) ... Processing triggers for systemd (241-7~deb10u1+rpi1) ...

Next the configuration file has to be modified. As usual, I used nano to do this.

woopi@goldserver:~ $ sudo nano /etc/watchdog.conf

... file = /tmp/domoticz.alive change = 300 ... max-load-1 = 24 ... watchdog-device = /dev/watchdog watchdog-timeout = 15

As far as I can ascertain, the timeout value has to be 15 seconds, the default 60 seconds does not work.

Start the watchdog service and then wait over five minutes (300 seconds) to ensure that the system is not rebooted. Then stop the Domoticz service and the Raspberry Pi should be rebooted in about five minutes.

woopi@goldserver:~ $ sudo systemctl start watchdog.service woopi@goldserver:~ $ sudo cat /var/log/mosquitto/mosquitto.log 1572481138: mosquitto version 1.5.7 starting woopi@goldserver:~ $ sudo systemctl status watchdog.service ● watchdog.service - watchdog daemon Loaded: loaded (/lib/systemd/system/watchdog.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2019-10-27 01:27:08 ADT; 19s ago Process: 28652 ExecStartPre=/bin/sh -c [ -z "${watchdog_module}" ] || [ "${watchdog_module}" = "none" ] || /sbin/modprobe $watchdog_module (code=exited, status=0/ Process: 28653 ExecStart=/bin/sh -c [ $run_watchdog != 1 ] || exec /usr/sbin/watchdog $watchdog_options (code=exited, status=0/SUCCESS) Main PID: 28655 (watchdog) Tasks: 1 (limit: 2319) Memory: 548.0K CGroup: /system.slice/watchdog.service └─28655 /usr/sbin/watchdog Oct 27 01:27:08 goldserver watchdog[28655]: interface: no interface to check Oct 27 01:27:08 goldserver watchdog[28655]: temperature: no sensors to check Oct 27 01:27:08 goldserver watchdog[28655]: no test binary files Oct 27 01:27:08 goldserver watchdog[28655]: no repair binary files Oct 27 01:27:08 goldserver watchdog[28655]: error retry time-out = 60 seconds Oct 27 01:27:08 goldserver watchdog[28655]: repair attempts = 1 Oct 27 01:27:08 goldserver watchdog[28655]: alive=/dev/watchdog heartbeat=[none] to=root no_act=no force=no Oct 27 01:27:08 goldserver watchdog[28655]: watchdog now set to 15 seconds Oct 27 01:27:08 goldserver systemd[1]: Started watchdog daemon. Oct 27 01:27:08 goldserver watchdog[28655]: hardware watchdog identity: Broadcom BCM2835 Watchdog timer ... wait 10 minutes - nothing should happen ... woopi@raspberry:~ $ sudo systemctl stop domoticz.service ... wait at most 6 minutes, the system should reboot

As this example shows, it will be necessary to stop the watchdog if Domoticz is suspended for any length of time otherwise the Raspberry Pi will reboot.

Installing mosquitto toc

An MQTT broker is a necessary part of my home automation system. The mosquitto broker is available in the Rasbian Buster repository as can be seen here.

woopi@goldserver:~ $ sudo apt-cache policy mosquitto mosquitto: Installed: (none) Candidate: 1.5.7-1 Version table: 1.5.7-1 500 500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

The latest version of the package is 1.6.7, but version 1.5.7 is recent enough and it is much easier to install mosquitto from the repository than to try to install from the source or from an alternate repository. A simple installation of the broker and the optional utilities (to get mosquitto_sub and mosquitto_pub) went ahead without a problem.

woopi@goldserver: $ sudo apt-get install mosquitto mosquitto-clients -y ... Need to get 484 kB of archives. After this operation, 1,054 kB of additional disk space will be used. ...

After I checked and found that the broker was running automatically.

woopi@goldserver:~ $ sudo systemctl status mosquitto.service ● mosquitto.service - Mosquitto MQTT v3.1/v3.1.1 Broker Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-10-17 02:49:20 BST; 6min ago Docs: man:mosquitto.conf(5) man:mosquitto(8) Main PID: 2180 (mosquitto) Tasks: 1 (limit: 2200) Memory: 656.0K CGroup: /system.slice/mosquitto.service └─2180 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf Oct 17 02:49:20 goldserver systemd[1]: Starting Mosquitto MQTT v3.1/v3.1.1 Broker... Oct 17 02:49:20 goldserver systemd[1]: Started Mosquitto MQTT v3.1/v3.1.1 Broker.

To make sure that everything was installed properly, I subscribed to all topics in a terminal on the Raspbery Pi.

woopi@goldserver:~ $ mosquitto_sub -h 127.0.0.1 -v -t "#"

Then I sent a message to the broker from a terminal on my desktop computer.

michel@hp:~$ mosquitto_pub -h goldserver.local -t "home" -m "hello" or michel@hp:~$ mosquitto_pub -h 192.168.1.22 -t "home" -m "hello"

If mosquitto is not installed on the desktop, the message could be published from a second terminal on the Raspberry Pi.

michel@hp:~$ ssh woopi@goldserver.local ... woopi@goldserver:~$ mosquitto_pub -h goldserver.local -t "home" -m "hello"

In either case, the message should show up in the first Raspberry Pi terminal.

woopi@goldserver:~ $ mosquitto_sub -h 127.0.0.1 -v -t "#" home hello

Installing a Web Server toc

In the past I used a very specific version of Lighttpd in order to implement a reverse proxy as done in Secure Webcam streaming with MJPG-Streamer on a Raspberry Pi. This is no longer required as I use a virtual private network (see the next section) to access the MJPG-Streamer web page. All I need is a simple installation of the current version of the Web server.

Lighttpd toc

This installs the latest version of Lighttpd available in the Raspbian depository.

woopi@goldserver:~ $ sudo apt install lighttpd -y ... The following NEW packages will be installed: libfam0 libmariadb3 lighttpd lighttpd-modules-ldap lighttpd-modules-mysql mariadb-common mysql-common spawn-fcgi 0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded. Need to get 545 kB of archives. After this operation, 1,659 kB of additional disk space will be used. ...

Success, lighttpd was indeed installed and running as confirmed with systemctl.

woopi@goldserver:~ $ sudo systemctl status lighttpd.service ● lighttpd.service - Lighttpd Daemon Loaded: loaded (/lib/systemd/system/lighttpd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-10-17 03:12:01 AST; 2h 43min ago Main PID: 3444 (lighttpd) Tasks: 1 (limit: 2200) Memory: 1.6M CGroup: /system.slice/lighttpd.service └─3444 /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf Oct 17 03:12:01 goldserver systemd[1]: Starting Lighttpd Daemon... Oct 17 03:12:01 goldserver systemd[1]: Started Lighttpd Daemon. Oct 17 03:12:01 goldserver systemd[1]: /lib/systemd/system/lighttpd.service:6: PIDFile= references path below legacy directory /var/run/, updating /var/run/lighttpd.pid → /run/lighttpd.pid; please update the unit file accordingly.

I ignored the request to update the unit file; restarting the service took care of care of that problem.

woopi@goldserver:~ $ sudo systemctl restart lighttpd.service woopi@goldserver:~ $ sudo systemctl status lighttpd.service ● lighttpd.service - Lighttpd Daemon Loaded: loaded (/lib/systemd/system/lighttpd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-10-07 03:18:05:05 AST; 8s ago Process: 1095 ExecStartPre=/usr/sbin/lighttpd -tt -f /etc/lighttpd/lighttpd.conf (code=exited, status=0/SUCCESS) Main PID: 1100 (lighttpd) Tasks: 1 (limit: 2319) Memory: 1.3M CGroup: /system.slice/lighttpd.service └─1100 /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf Nov 01 21:05:05 goldserver systemd[1]: Starting Lighttpd Daemon... Nov 01 21:05:05 goldserver systemd[1]: Started Lighttpd Daemon.

For some reason, the default web page was not created. But it was available elsewhere, so there was no problem copying it over to its proper location.

woopi@goldserver:~ $ sudo cp /usr/share/lighttpd/index.html /var/www/html/

I opented the page in a web browser to check that the server is installed.

Of course it may be necessary to use the IP address of the Raspberry Pi in Windows or in some other operating system.

nginx toc

Installing nginx is just as simple.

sancho@panza:~ $ sudo apt install nginx-light -y ... Need to get 644 kB of archives. After this operation, 1,467 kB of additional disk space will be used. ... sancho@panza:~ $ sudo systemctl status nginx ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2019-10-21 21:44:21 ADT; 3h 59min ago Docs: man:nginx(8) Main PID: 1571 (nginx) Memory: 1.1M CGroup: /system.slice/nginx.service ├─1571 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; └─1572 nginx: worker process Oct 21 21:44:21 panza systemd[1]: Starting A high performance web server and a reverse proxy server... Oct 21 21:44:21 panza systemd[1]: Started A high performance web server and a reverse proxy server. sancho@panza:~ $ ls -l /var/www/html total 4 -rw-r--r-- 1 root root 612 Oct 21 21:44 index.nginx-debian.html

The last command verified that a default index page was written to the Web root directory.

Configuration toc

No matter which Web server is installed, it will be quite useful to change the owner of the Web directory.

woopi@goldserver:/var/www $ sudo chown -R woopi: html

This way it will be possible for woopi to add, delete and edit any file in the directory or any sub directories that will be created.

Some may wonder why woopi is not a member of the www-data group. Quoting jojopi, [t]here appears to be a common misconception that everything to do with the web should be owned by www-data. Actually it is quite the opposite. Read the complete answer in the Raspberry Forum on the question of Re: /var/www/html permissions which I found quite cogent.

I then went on to create a directory to contain the Tasmota firmware to download to various ESP8266 IoT devices if needed.

woopi@goldserver:/var/www $ mkdir html/sonoff

I decided to create a custom 404 error page.

woopi@goldserver:/var/www $ mkdir nano html/404.html

<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>404</title> </head> <body> <p style="font-size: 8em; text-align: center">404</p> </body> </html>

An etnry must be added to the web server configuration file. Here are the details for Lighttpd.

woopi@goldserver:~ $ sudo nano /etc/lighttpd/lighttpd.conf

Add an entry at the end of the server... block near the top of the file.

... server.port = 80 server.error-handler-404 = "404.html" ...

If nginx is used, follow these steps.

woopi@goldserver:~ $ sudo nano /etc/nginx/nginx.conf

Add the entry near bottom of http block.

http { ... ## # Basic Settings ## ## # Custom error messages ## error_page 404 /404.html; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; }

Restart the server for this to take effect.

woopi@goldserver:~ $ sudo systemctl restart lighttpd or woopi@goldserver:~ $ sudo systemctl restart nginx

Installing WireGuard toc

One of the perks of having a home automation system is that it allows one to control devices in the house from outside. Perhaps the simplest way of going about that is to use port forwarding which involves opening a TCP port of the local area network to the outside world. Usually, that also requires setting up a dynamic DNS address for the local area network. Always access the Domoticz web server using the secure HTTPS protocol if you do that.

There are a couple coping mechanisms. One could install a Tor server on the Raspberry Pi and use a Tor browser to access the Domoticz server on the Pi. While I did use this approach in the past, I no longer do because it was too slow and cumbersome. One could try MyDomoticz which according to the Domoticz Wiki "enables Domoticz users to access their Domoticz appliance (sometimes called an "instance") from outside their local network without having to forward a http port on their router or even knowing their IP address. All communication occurs via a secure webpage and there is also no need to fiddle with certificates locally. Using this service is purely optional and can be enabled in the settings menu of Domoticz." I have not used this service and cannot comment on it.

For the last few months, I have been using a virtual private network to access not only my Domoticz server but all the resources on the local area network. A VPN does require an opened TCP port and a dynamic DNS host name. Initially, I did install an OpenVPN server but I switched to WireGuard and I am pleased with the results.

Unfortunately, WireGuard is not available in the official depository.

woopi@goldserver:~ $ sudo apt-cache policy wireguard N: Unable to locate package wireguard

So I installed the package from the unstable Debian repository following the instructions provided by Adrian Mihalko on GitHub. There are more details in my post Installing WireGuard on Raspbian Stretch and Buster.

woopi@goldserver:~ $ sudo apt-get install raspberrypi-kernel-headers ... Need to get 24.9 MB of archives. After this operation, 163 MB of additional disk space will be used. ...

I found that dirmangr was already installed, so installation of the package could be started.

woopi@goldserver:~ $ echo "deb http://deb.debian.org/debian/ unstable main" | sudo tee --append /etc/apt/sources.list.d/unstable.list deb http://deb.debian.org/debian/ unstable main woopi@goldserver:~ $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8B48AD6246925553 ... woopi@goldserver:~ $ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC 648ACFD622F3D138 ... woopi@goldserver:~ $ printf 'Package: *\nPin: release a=unstable\nPin-Priority: 150\n' | sudo tee --append /etc/apt/preferences.d/limit-unstable ... woopi@goldserver:~ $ sudo apt update ... Reading state information... Done All packages are up to date. woopi@goldserver:~ $ sudo apt install wireguard -y ... Need to get 451 kB of archives. After this operation, 2,397 kB of additional disk space will be used. ... Module build for kernel 4.19.75-v8+ was skipped since the kernel headers for this kernel does not seem to be installed. Setting up wireguard-tools (0.0.20191012-1) ... Setting up wireguard (0.0.20191012-1) ... Processing triggers for man-db (2.8.5-2) ...

There is not much to check until the server and clients have been configured. At a minimum I verified that the wg-quick utility was installed and that is configuration directory was created although it is empty.

woopi@goldserver:~ $ which wg-quick /usr/bin/wg-quick woopi@goldserver:~ $ sudo ls -l /etc/wireguard total 0

Then I copied the configuration file /etc/wireguard/wg0.conf from the SD card containing the previous installation Raspbian. This is a bit complicated because of the restrictive attributes of the directory and file. First, I copied the file from the SD card reader on the desktop to my home directory.

michel@hp:~$ sudo cp /media/michel/rootfs/etc/wireguard/wg0.conf .

Then I copied the wg0.conf file to the default home directory on the Raspberry Pi using Filezilla. Finally, I copied the configuration file from the home directory on the Pi to the WireGuard configuration directory and then launched the VPN server.

woopi@goldserver:~ $ sudo cp wg0.conf /etc/wireguard/ woopi@goldserver:~ $ wg-quick up wg0 [#] ip link add wg0 type wireguard [#] wg setconf wg0 /dev/fd/63 [#] ip -4 address add 192.168.99.1/24 dev wg0 [#] ip link set mtu 1420 up dev wg0 [#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

While everything looks OK, it will not work until the Raspberry Pi 3 B is given the static IP address of the old Raspberry Pi 1 B. But I am not going to do this until everything is mostly ready.

woopi@goldserver:~ $ wg-quick down wg0 [#] ip link delete dev wg0 [#] iptables -D FORWARD -i wg0 -j ACCEPT; iptables -D FORWARD -o wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o wlan0 -j MASQUERADE

Enabling automatic start of the wg0 interface at boot time makes sense on a server.

pi@raspberrypi:~ $ sudo systemctl enable wg-quick@wg0 Created symlink /etc/systemd/system/multi-user.target.wants/wg-quick@wg0.service → /lib/systemd/system/wg-quick@.service.

I installed two Android clients using the Adrian Mihalko user management script. The same script was used to generate keys and modify the configuration file on the Raspberry Pi to be able to communicate with another Wireguard peer on a portable computer running Linux. Again, there are more details in Installing WireGuard on Raspbian Stretch and Buster.

Installing Syncthing toc

The latest addition to the home automation machine is Syncthing a decentralized file synchronization program. With this system, the Domoticz database is synchronized with copies on other computers. I also use Syncthing to synchronize directories that containt scripts. It is all still rather new and I have not yet written a post on this subject, nevertheless here are the steps taken to install the service on the Raspberry Pi.

Using apt-cache policy, it became clear that Syncthing in the Debian and Raspbian repository are rather out of date.

woopi@goldserver:~ $ apt-cache policy syncthing syncthing: Installed: (none) Candidate: 1.0.0~ds1-1 Version table: 1.1.4~ds1-4 150 150 http://deb.debian.org/debian unstable/main armhf Packages 1.0.0~ds1-1 500 500 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages

So I followed the instructions at Syncthing, Debian/Ubuntu Packages to get the latest version 1.3.0 (to be supplanted by v1.3.1 on Nov. 5, 2019).

woopi@goldserver:~ $ curl -s https://syncthing.net/release-key.txt | sudo apt-key add - OK woopi@goldserver:~ $ echo "deb https://apt.syncthing.net/ syncthing candidate" | sudo tee /etc/apt/sources.list.d/syncthing.list deb https://apt.syncthing.net/ syncthing candidate woopi@goldserver:~ $ sudo apt-get update Get:1 http://deb.debian.org/debian unstable InRelease [139 kB] ... Fetched 14.4 MB in 19s (747 kB/s) Reading package lists... Done woopi@goldserver:~ $ sudo apt install syncthing ... Need to get 9,076 kB of archives. After this operation, 19.7 MB of additional disk space will be used. ... Processing triggers for mime-support (3.62) ... woopi@goldserver:~ $

Now start the program manually to verify that it functions.

woopi@goldserver:~ $ syncthing [monitor] 12:05:25 INFO: Default folder created and/or linked to new config [monitor] 12:05:25 INFO: Starting syncthing [start] 12:05:26 INFO: syncthing v1.3.1-rc.2 "Fermium Flea" (go1.13.3 linux-arm) deb@build.syncthing.net 2019-10-07 11:30:25 UTC [start] 12:05:26 INFO: Generating ECDSA key and certificate for syncthing... [start] 12:05:26 INFO: Default folder created and/or linked to new config [start] 12:05:26 INFO: Default config saved. Edit /home/woopi/.config/syncthing/config.xml to taste (with Syncthing stopped) or use the GUI [KJ3K7] 12:05:26 INFO: My ID: KJ3K7WW-KN63VNC-ZZ77WOR-6SMQQEX-CA5GLVP-B3MXVIR-KNKPJ4J-E527OAU [KJ3K7] 12:05:27 INFO: Single thread SHA256 performance is 15 MB/s using crypto/sha256 (15 MB/s using minio/sha256-simd). [KJ3K7] 12:05:28 INFO: Hashing performance is 14.59 MB/s [KJ3K7] 12:05:28 INFO: Starting deadlock detector with 20m0s timeout [KJ3K7] 12:05:28 INFO: No stored folder metadata for "default": recalculating [KJ3K7] 12:05:28 INFO: Ready to synchronize "Default Folder" (default) (sendreceive) [KJ3K7] 12:05:28 INFO: Overall send rate is unlimited, receive rate is unlimited [KJ3K7] 12:05:28 INFO: Using discovery server https://discovery.syncthing.net/v2/?noannounce&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW [KJ3K7] 12:05:28 INFO: Using discovery server https://discovery-v4.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW [KJ3K7] 12:05:28 INFO: Using discovery server https://discovery-v6.syncthing.net/v2/?nolookup&id=LYXKCHX-VI3NYZR-ALCJBHF-WMZYSPK-QG6QJA3-MPFYMSO-U56GTUK-NA2MIAW [KJ3K7] 12:05:28 INFO: Anonymous usage reporting is always enabled for candidate releases. [KJ3K7] 12:05:28 INFO: QUIC listener (:22000) starting [KJ3K7] 12:05:28 INFO: TCP listener (:22000) starting [KJ3K7] 12:05:28 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) starting [KJ3K7] 12:05:28 INFO: Completed initial scan of sendreceive folder "Default Folder" (default) [KJ3K7] 12:05:28 INFO: Loading HTTPS certificate: open /home/woopi/.config/syncthing/https-cert.pem: no such file or directory [KJ3K7] 12:05:28 INFO: Creating new HTTPS certificate [KJ3K7] 12:05:29 INFO: GUI and API listening on 127.0.0.1:8384 [KJ3K7] 12:05:29 INFO: Access the GUI via the following URL: http://127.0.0.1:8384/ [KJ3K7] 12:05:29 INFO: My name is "goldserver" [KJ3K7] 12:05:47 INFO: quic://0.0.0.0:22000 detected NAT type: Port restricted NAT [KJ3K7] 12:05:47 INFO: quic://0.0.0.0:22000 resolved external address quic://74.61.102.29:22000 (via stun.syncthing.net:3478) [KJ3K7] 12:05:48 INFO: Detected 1 NAT service [KJ3K7] 12:06:09 INFO: Joined relay relay... CtrlC shut down application [monitor] 12:07:15 INFO: Signal 2 received; exiting [KJ3K7] 12:07:15 INFO: QUIC listener (:22000) shutting down [KJ3K7] 12:07:15 INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) shutting down [KJ3K7] 12:07:15 INFO: Disconnected from relay relay: [KJ3K7] 12:07:16 INFO: TCP listener (:22000) shutting down [KJ3K7] 12:07:16 INFO: Exiting

Since I want to control the application throught its Web interface from my desktop computer, I need to modify the configuration as explained in the FAQ.

woopi@goldserver:~ $ nano .config/syncthing/config.xml

Locate the <gui> entry and change the <address> value from 127.0.0.1:8384 to 0.0.0.0:8384.

<gui enabled="true" tls="false" debugging="false"> <address>0.0.0.0:8384</address> <apikey>...

On opening the Syncthing interface from the desktop using a Web browser pointed to 192.168.1.22:8384, there was a warning about the lack of security, so I followed the instruction to add a user and password to the GUI interface.

I wanted syncthing to start automatically when the Raspberry Pi is booted. Here are the steps I followed.

woopi@goldserver:~ $ sudo wget https://raw.githubusercontent.com/syncthing/syncthing/master/etc/linux-systemd/system/syncthing%40.service \ > -O /etc/systemd/user/syncthing@.service --2019-10-22 13:30:40-- https://github.com/syncthing/syncthing/blob/master/etc/linux-systemd/system/syncthing%40.service Resolving github.com (github.com)... 140.82.113.4 Connecting to github.com (github.com)|140.82.113.4|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: ‘/etc/systemd/user/syncthing@.service’ /etc/systemd/user/syncthing@. [ <=> ] 75.77K 493KB/s in 0.2s 2019-10-22 13:30:40 (493 KB/s) - ‘/etc/systemd/user/syncthing@.service’ saved [77590] woopi@goldserver:~ $ sudo systemctl enable syncthing@woopi.service Created symlink /etc/systemd/system/multi-user.target.wants/syncthing@woopi.service → /lib/systemd/system/syncthing@.service. woopi@goldserver:~ $ sudo systemctl start syncthing@woopi.service woopi@goldserver:~ $ sudo systemctl status syncthing@* ● syncthing@woopi.service - Syncthing - Open Source Continuous File Synchronization for woopi Loaded: loaded (/lib/systemd/system/syncthing@.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2019-10-22 13:33:30 ADT; 18s ago Docs: man:syncthing(1) Main PID: 2876 (syncthing) Tasks: 15 (limit: 2319) Memory: 20.5M CGroup: /system.slice/system-syncthing.slice/syncthing@woopi.service └─2876 /usr/bin/syncthing -no-browser -no-restart -logflags=0 Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: Overall send rate is unlimited, receive rate is unlimited Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: Anonymous usage reporting is always enabled for candidate releases. Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: Completed initial scan of sendreceive folder "Default Folder" (defa Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: QUIC listener (:22000) starting Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: TCP listener (:22000) starting Oct 22 13:33:32 goldserver syncthing[2876]: [KJ3K7] INFO: Relay listener (dynamic+https://relays.syncthing.net/endpoint) star ...

With more experience I may write up a post on using this service. As an indication of what I am doing, here is a screenshot of the opening page of the Web interface.

A directory named ~/goldserver_syspy on the desktop (hp) is shared with directory ~/.syspy on the Raspberry Pi. This shared directory is named syspy in syncthing. Any changes made to a file in ~/golserver_syspy on the desktop are copied to the corresponding file in .syspy of Raspberry Pi. Conversely, any changes made on the Raspberry Pi are copied to the desktop.

For information on setting up Syncthing on other computers and Android tablets and sharing directories see the documentation.

<-Installation and Configuration of Raspbian Buster Lite Various Hardware with Raspbian Buster Lite->