md
Updating Raspbian to Stretch
January 10, 2019

With the New Year, I decided it was time to tackle once again the task of updating the Raspbian operating system on my Raspberry Pi (Model B Plus Rev 1.2) to the latest version of Stretch and to update the home automation system Domoticz at the same time. I had been holding back because of problems using the newest version of Lighttpd found in Raspbian Stretch to stream the output from Mpjg-Streamer over a secure connection. I have not solved that problem yet, but I have been experimenting with running a VPN server on the Raspberry Pi and that would be a secure work around.

I expected that updating to Raspbian Stretch from Jessie would be without problems. Especially since Stretch has been out for a while now and the Raspberry Pi Foundation has issued a number of updates. And I have tested it, with Bluetooth notably and managed to do better with the newer version. But I did run into little problems that nevertheless took some time to figure out. Also, I have gotten better at setting up the Raspberry Pi so I decided that this recounting of my experience might be of use to others.

To be clear then, this post is just another one of my series on setting up a Raspberry Pi. I suspect that it would be of interest mostly to new fans of small single-board computers, in this case the Raspberry Pi 3 Model B. Assuming that many of these would be new to Linux, I have added sidebars entitled Information for neophytes in which I ramble about what that I think could be of help to beginners.

Table of Contents

  1. Getting Stretch on an SD card
  2. Headless Set Up
  3. Initial Boot
  4. Change the Default User
  5. Updating the Operating System
  6. Installing Domoticz
  7. Installing mosquitto
  8. Installing MJPG-Streamer
  9. Odds and Ends
  10. Conclusion

Getting Stretch on an SD card toc

The home automation system cannot be offline while I am experimenting with updates. Instead everything is being done on a Raspberry Pi Model 3 B. Once that new system is running, I hope to do the same thing on the old Model B which I have found more than adequate as a home automation server.

As usual, I downloaded the latest Lite version of Raspbian:

  Raspbian Stretch Lite
  Minimal image based on Debian Stretch
  Version: November 2018
  Release date: 2018-11-13
  Kernel version: 4.14

from https://www.raspberrypi.org/downloads/raspbian/. I followed the instructions to copy the image onto a micro-SD card in Installing operating system images but there was a minor glitch.

An older version of Etcher was causing an error similar to that reported in Ubuntu 18.04 cannot launch etcher 1.4.6. Getting a newer version of the program from balena at https://www.balena.io/etcher/ resolved the issue. I removed the older version, by deleting the application etcher-electron-1.4.6-x86_64.AppImage, which I had installed in a directory named ~/.local/bin (short form for /home/michel/.local/bin). I also removed the configuration directory /home/michel/.config/etcher-electron and the application desktop file: /home/michel/.local/share/applications/appimagekit-Etcher.desktop.

Installing the current version of Etcher in a Linux system is quite simple. I extracted the application, balena-etcher-electron-1.4.9-x86_64.AppImage from the downloaded archive and copied it to ~/.local/bin (it could be just about anywhere else). I then started the application and let it install a desktop file. From then on I could start Etcher in the usual way. And, as before, it was a breeze to copy the operating system image onto the SD card with the application. It is not even necessary to decompress the downloaded archive containing the image file.

All these file management chores, copying, deleting, extracting from archives and so on, can all be done with "point and click" ease using Double Commander. It is one of the first utilities I install in all the Linux distributions I use. It even works in Windows, macOS and FreeBSD. Highly recommended.

Headless Set Up toc

Do something often enough, you should get better at it.

The Raspberry Pi is used as a headless server with no monitor and no keyboard. It is a hassle to dig out a monitor and keyboard just for the initial configuration of such a system. Since connectivity with the local network will be by Wi-Fi, it would be great if that could also be set up before the first boot. Well, it can; even more can be achieved if the desktop computer used to create the SD image is running Linux.

I followed the instructions from Raspian France: Mettre en place une Raspberry Pi sans écran ni clavier, but similar information is available in many places. For example, Brian Lough has a YouTube video, A Truly Headless Setup for your Pi.

The setup is in two or three parts. I am assuming that the (micro) SD card containing the Raspbian image is still connected to the desktop computer and its boot partition is showing up as drive connected to the system. If not, remove the card from the reader, wait awhile and insert it in the reader again. The partition should show up as its file system is the ubiquitous FAT32 which most operating system can read. If the desktop is running Linux the second, much bigger partition, rootfs will also show up. Windows will not read a Ext4 partition. I don't know about macOS.

  1. Enable ssh. All that is needed for this is to create a file of any size, any content, named ssh in the boot partition. It can even be an empty file.
    michel@hp:~$ cd /media/michel/boot michel@hp:/media/michel/boot$ touch ssh michel@hp:/media/michel/boot$ ls -l ssh just checking -rw-r--r-- 1 michel michel 0 jan 3 11:04 ssh

    Information for neophytes

    These commands, shown on a greenish background, are typed out in a virtual console or a terminal which is a wrapper program that runs a shell or command processor on my desktop Ubuntu system. The terminal is easily started with the simultaneous three key press: AltCtrlT.

    The desktop computer name is hp and my user name is michel, hence the shell starts each command line with michel@hp:. After the colon, comes the present working directory (pwd in the jargon). In this case it is initially ~ which is short for the current user home directory which here is /home/michel. Finally the shell writes the prompt which here is $. It can be different dependant on the shell being used. That's Linux for you, there is more than one shell.

    The first command, cd (for change directory), changes the present working directory. The second command, touch, is used to change the time stamp of a file. But here we are exploiting a side effect of the command; if the named file does not exist, it is created. The third command, ls, lists the content of the present working directory if no parameters follow it. In this instance, the -l parameter asks for a detailed listing. The added filename ssh will limit the listing to matching files. We can then see that the file ssh has indeed been created and that its size is 0.

    Instead of using touch, a file with minimal content named ssh could have been created with a text editor such as nano (see below). It would not matter what the file contained.

    Do not forget to press the Enter key after typing each command. That is how the shell knows that you are done and that it is time for it to do its job. A long time ago, back when most computer programs were punched on cards made of light cardboard, I caused the reboot of a laboratory system when I first used an interactive terminal because I hadn't realized the need for the Enter. I will not go into the details, suffice it to say I am still red faced about ruining someone's experiment. We all have to learn.

    Because Double Commander allows me to launch a terminal in a specific directory, I did not even have to do the three-finger salute mentioned, nor issue the change directory command cd shown above.

  2. Create the wpa_supplicant configuration file with Wi-Fi credentials. I used nano from the command line, but any text editor such as Geany could probably be used. The file, wpa_supplicant.conf, goes in the boot partition alongside the ssh file created in the previous state. You have to adjust the network credentials of course.
    michel@hp:/media/michel/boot$ nano wpa_supplicant.conf
    country=ca <<-- 2 letter country code update_config=1 ctrl_interface=/var/run/wpa_supplicant network={ scan_ssid=1 ssid="wifi_network_name" psk="wifi_password" }

    Information for neophytes

    GNU nano is a small text editor found on most Linux distributions including Ubuntu and, as we shall see later, Raspbian. It is a relatively simple to use; all keystrokes, with the exception of control and meta sequences, enter text into the file being edited at the current cursor position. The most important control sequences are displayed at the bottom. There is an on-line manual.

  3. Add static IP address for Ethernet and wireless network interfaces. This requires access to the rootfs partition. As far as I know it cannot be done in the Windows. The fixed IP address is added to the dhcpcd configuration file. I prefer to back up that file found in the rootfs/etc directory before modifying it.
    michel@hp:/media/michel/rootfs/etc$ sudo cp dhcpcd.conf dhcpcd.bak michel@hp:/media/michel/rootfs/etc$ sudo nano dhcpcd.conf

    Alternatively, nano can be made to backup the original when saving a changed version of the file.

    michel@hp:/media/michel/rootfs/etc$ sudo nano -B dhcpcd.conf
    ... interface eth0 static ip_address=192.168.1.100/24 static routers=192.168.1.1 static domain_name_servers=192.168.1.1 interface wlan0 static ip_address=192.168.1.101/24 static routers=192.168.1.1 static domain_name_servers=192.168.1.1

    Information for neophytes

    All the IP addresses shown above need to be adjusted to take into account the parameters of the local area network. The router IP address (often called gateway elsewhere) and the DNS server IP address are usually the same and often, but not always, similar to the one shown above. But some local area networks (LAN) use the so-called 24-bit address block starting at 10.0.0.0 and others use the 20-bit block starting at 172.16.0.0. All the LANs I have used were in the 16-bit block starting at 192.168.0.0, with the only exceptions being a virtual private network and the virtual machines in Oracle VM VirtualBox. If you want to learn more about these private networks look at the Wikipedia article.

    I think it is best to assign different IP addresses to the two interfaces, unless it is certain that the Ethernet and wireless interfaces will never be simultaneously connected to the same local network.

Of course, if Wi-Fi is not needed then there is no need to create the wpa_supplicant.conf file and there is no need to set up the wlan0 network interface in dhcpcd.conf. Similarly, if the Ethernet interface will not be used, if using a Raspberry Pi Zero for example, then there is no need to define a static address for it in the configuration file. If physically possible, I would still recommend setting up the Ethernet interface, just in case the Wi-Fi connection initially fails?

Initial Boot toc

It is now time for the initial boot. All that needs to be done is to safely eject the SD card from the desktop reader, then place it in the SD slot of the Raspberry Pi and, lastly turn the power on. You may have to wait until the LED activity stops and the red LED is steadily on to open an ssh session.

Since I was able to set up a static IP, it was very easy to open a remote session with ssh.

michel@hp:/media/michel/rootfs/etc$ cd $home michel@hp:~$ ssh pi@192.168.1.101 The authenticity of host '192.168.1.101 (192.168.1.101)' can't be established. ECDSA key fingerprint is SHA256:h8AxJbL5t4WltEu415klkv7LS9px2CFfdorz74Dc0JM. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.1.101' (ECDSA) to the list of known hosts. pi@192.168.1.101's password: raspberry not echoed on screen! Linux raspberrypi 4.14.79-v7+ #1159 SMP Sun Nov 4 17:50:20 GMT 2018 armv7l ... SSH is enabled and the default password for the 'pi' user has not been changed. This is a security risk - please login as the 'pi' user and type 'passwd' to set a new password. pi@raspberrypi:~ $

Information for neophytes

Opening a remote session on the Raspberry Pi means that shell commands (things like cd, ls and cat) can be executed on the Rapsberry Pi from another computer on the same network as the Raspberry Pi. It comes down to using the desktop keyboard and monitor as if they were connected to the Raspberry Pi.

Of course you have to use a program on the desktop to do this. It is a command line utility called ssh on Linux systems which is part of the OpenSSH suite of tools which implement the SSH (Secure Shell) protocol. Most Windows users would probably use PuTTY instead of ssh to open a remote session on the Raspberry Pi. From what I can make out, Mac OS X users can use the built-in ssh client in a terminal window as shown above but there are alternatives.

It is necessary to log into an existing user account and provide the correct password when starting a remote session or a local session with a keyboard and monitor connected to the Raspberry Pi. Initially there is only one user account: pi and the password is raspberry. Note how ssh doesn't recognize the secure key returned by the Raspberry Pi (ECSDA is the cryptography algorithm used to generate the key) so it asks if it is alright to connect. Once you have answered yes, in full, it will store the key and will not ask again.

ssh is a secure replacement for telnet. Anyone else remember using NCSA telnet over a Crynwr packet driver on PC running MS-DOS? I would open sessions on the IT department's mini-computer a PDP-11 if I remember. I forget exactly why? To send e-mails? It was a complicated thing to do in the early 80's. I do remember accessing gopher sites that way in the early 90's and even building a demonstration gopher site for our department trying to get the institution to get on with it. It never did, but then HTTP took over.

If it had not been possible to set a static IP address, then try to connect to raspberrypi.local as shown here.

michel@hp:~$ ssh pi@raspberrypi.local

This will work in a Linux system where the Avahi utility is installed which is the case for most major distributions. It should also work in MAC OS (where Bonjour takes care of the zeroconf protocols). It probably will not work in a fresh install of Windows, but it might if Bonjour was added to connect to a networked printer, for example.

If another Raspberry Pi with the default raspberrypi hostname was already connected to the network, then the newer Pi's hostname would be mangled to avoid conflicts. Its qualified host name address could be something like raspberrypi-1.local. You could use a zeroconf service browser to try to find it.

It would be very awkward to run a home automation server such as Domoticz on a computer with a dynamically assigned IP address. Accordingly, if it had not been possible to set a fixed IP address previously, it should be done at this point.

pi@raspberrypi:~ $ nano /etc/dhcpcd.conf

You just need to add the static addresses as shown in part 3 of the previous section.

As suggested by the log on screen, you really should change the password and I strongly suggest that you change the hostname to avoid collisions with other Raspberry Pi you may connect to the network. For illustrative purposes, I set the hostname to goldserver. There are a couple of other changes that I always do. I set the correct timezone, but I do not bother changing the locale. Minimizing the amount of memory allocated to the GPU is a good idea. Use the configuration utility raspi-config to change these settings.

pi@raspberrypi:~ $ sudo raspi-config

Reboot as suggested, and then after a while reconnect with ssh.

michel@hp:~$ ssh pi@goldserver.local or michel@hp:~$ ssh pi@192.168.1.1

To finish the initial set up, I changed the configuration to save wear on the SD card by keeping the system journal in a file in RAM .

pi@goldserver:~ $ sudo nano /etc/systemd/journald.conf
# This file is part of systemd. # [Journal] Storage=volatile Compress=yes RuntimeMaxUse=48M

Change the Default User toc

For the first time, I decided to change the default user as an added security measure in a systematic way following the instructions provided by Jim McDonnell at Unix etc., How to Rename the Default Raspberry Pi User. In the past, I had simply added a new user and eventually removed pi. Following the instructions is much cleaner.

There were a couple of error messages.

sed: can't read lightdm/lightdm.conf: No such file or directory sed: can't read polkit-1/localauthority.conf.d/60-desktop-policy.con: No such file or directory

Not to worry, these were because the configuration files are not present in the Lite version of Raspbian as mentioned in the article.

I did run into another little problem: sudo would complain that it could not resolve host goldserver. It turned out that I had to edit the hosts file.

woopi@goldserver:~ $ sudo nano /etc/hosts sudo: unable to resolve host goldserver nevertheless nano started and hosts could be edited
127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback ff02::1 ip6-allnodes ff02::2 ip6-allrouters 127.0.1.1 goldserver <<--- change to new hostname

This was probably because of an error on my part. I think I changed the host name by directly editing the hostname file instead of using the raspi-config utility.

Updating the Operating System toc

Before installing any major new piece of software, it is always important to update the operating system. This is something that is done more or less automatically in full blown desktop systems such as Ubuntu. On the Lite version of Raspbian, it has to be done manually.

woopi:goldserver~ $ sudo apt-get update && sudo apt-get upgrade ... The following packages will be upgraded: base-files libperl5.24 libpolkit-agent-1-0 libpolkit-backend-1-0 libpolkit-gobject-1-0 libssl1.0.2 libssl1.1 libwbclient0 openssl perl perl-base perl-modules-5.24 policykit-1 python-rpi.gpio raspberrypi-sys-mods raspi-config samba-common 17 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Need to get 10.3 MB of archives. After this operation, 120 kB of additional disk space will be used. Do you want to continue? [Y/n] y ...

Information for neophytes

Newcomers to Debian Linux might wonder what these commands do. The first, apt-get update, scans the default Debian repository and any other repositories that you may have added (see below for an example) to update the list of currently available software packages in the repositories. The second, apt-get upgrade will replace currently installed packages with any newer version if available. It is important to realize that apt-get upgrade uses to local list of available packages to decide if a newer version is available, it does not scan the repositories itself. That why it is important to execute apt-get update first.

If the -y option were to be added to the sudo apt-get upgrade command, it would not be necessary to confirm, with a y, that the operation is to be completed. I prefer not to do in order to see clearly how much disk space will be occupied by the upgrade.

Other Linux distributions have different package managers: Red Hat and CentOS uses yum, Arch Linux uses pacman and so on.

Even within Debian the story is rather more complex. As far as I know, APT (Advanced Packaging Tool) is actually a "front end" for DPKG the Debian Package Management System. There are other front ends, such as Synaptic, which I occasionally use on my Ubuntu desktop, and Aptitude.

With its inclusion in Ubuntu 16.04 in 2016, a new front end called simply apt started to become popular. Basically, it aggregates the commands spread across the apt-get and apt-cache utilities. You will find in my posts that I sometimes use apt update and apt upgrade instead of the apt-get versions shown above. They are basically interchangeable. The same is true for apt show and apt-get show, but I prefer to use apt-cache policy to get pithier information about available versions of a package as well as the installed version when a package is already installed. For more information see Difference Between apt and apt-get Explained.

Here are three commands that could prove useful to new users of the Linux shell. First, to end a session, use the exit command. If you were running a remote session with ssh then that will bring you back into the desktop terminal program.

woopi@goldserver:~ $ exit logout Connection to goldserver.local closed. michel@hp:~$

The Raspberry Pi will continue to chug along when you logout. Processes including servers such as a Web server and a home automation server that were running will continue to function.

To reboot the Raspberry Pi, issue the reboot command which is not too surprising. But this has to be done as a superuser (with adminitrator priviliges in Windows jargon), which requires using the sudo prefix as done on a number of occasions above.

woopi@goldserver:~ $ sudo reboot Connection to goldserver.local closed by remote host. Connection to goldserver.local closed. michel@hp:~$

You may not see the connection closed messages. The other command is used to shut down the Raspberry Pi in an orderly fashion. It is always best to issue that command and waiting for a little while before removing power to a Rasbperry Pi.

woopi@goldserver:~ $ sudo shutdown now Connection to goldserver.local closed by remote host. Connection to goldserver.local closed. michel@hp:~$

Be careful with these commands. I have lost a lot of work when I carelessly typed the command without looking at the prompt. Instead of shutting down a Raspberry Pi, I entered the command in a Ubuntu shell with the obvious consequence that my desktop computer became very quiet very quickly. You have been warned.


Installing Domoticz toc

Installing Domoticz has always been a very simple matter in the past when following The "easy" way instructions. But for some reason, I was getting an error.

woopi@goldserver:~ $ curl -L install.domoticz.com | sudo bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0bash: line 1: syntax error near unexpected token `newline' bash: line 1: `<!DOCTYPE html>' 100 11643 100 11643 0 0 11699 0 --:--:-- --:--:-- --:--:-- 11701

That means that the shell, bash was choking on the first line of the installation script which was <!DOCTYPE html>. That was strange but after some thought, and running the curl command without the pipe to bash, I realized that the default index page was being returned from domoticz.com and not the script which was presumably at install.domoticz.com. Then it dawned on me that this was probably a consequence of a broken or missing redirection of secure HTTP requests. Something which I have struggled with on my site. So the following worked flawlessly as before.

woopi@goldserver:~ $ curl -L https://install.domoticz.com | sudo bash % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 16257 100 16257 0 0 12007 0 0:00:01 0:00:01 --:--:-- 12015 ::: ::: You are root. ::: Verifying free disk space... ... after answering a couple of questions about ports and directories: Ready... ┌─────────────────────┤ Installation Complete! ├─────────────────────┐ │ │ │ Point your browser to either: │ │ │ │ HTTP: 192.168.1.101:8080 │ │ HTPS: 192.168.1.101:443 │ │ │ │ Wiki: https://www.domoticz.com/wiki │ │ Forum: https://www.domoticz.com/forum │ │ │ │ The install log is in /etc/domoticz. │ │ │ │ │ │ <Ok> │ │ │ └────────────────────────────────────────────────────────────────────┘

When it is time, I will copy the database, various bash and python scripts and floorplan images from the Domoticz server that is currently running my home automation system and then restore the database to this new server. I have done this a couple of times in the past and it works flawlessly too.

This is a good place to express all my gratitude to the Domoticz development team as well as to the community.

Installing mosquitto toc

An MQTT broker is a necessary part of my home automation system. The mosquitto broker is available in the Rasbian Stretch repository but it is version 1.4.10 as can be seen here.

woopi@goldserver:~ $ sudo apt-cache policy mosquitto mosquitto: Installed: (none) Candidate: 1.4.10-3+deb9u2 Version table: 1.4.10-3+deb9u2 500 500 http://raspbian.raspberrypi.org/raspbian stretch/main armhf Packages

Version 1.5.5 of the broker is installed in my freshly upgraded Raspbian Jessie distribution. Looking into http://repo.mosquitto.org/debian/, I found a file named mosquitto-stretch.list and decided to follow the instructions in the accompanying readme.txt which are the same as those on the mosquitto.org blog which I had followed to install Mosquitto on Jessie. That was a mistake. The usual sudo apt-get update issued after adding a repository failed. It took a little while to fix the problem (see the following insert).

Apparently others did not have this problem. See for example, Install Mosquitto on a Raspberry Pi by Alex written just over a year ago. Obviously, I have a newer version of Stretch and the broker installed would have been an older version, probably 1.4.14. Could these differences explain why my results were so dismal?

Looking further into the question, I found a discussion on the Raspberry Foundation forum on this topic: MQTT installation on Stretch fixed?. It was a bit daunting, with the latest posts in October of last year talking of persistent problems and reverting to Jessie. The two 2018 updates by Dan K. at XPERIMENTIA make it look like installation in Stretch was an on again, off again proposition. On the other hand, Dougie Lawson reported that a simple installation from the Stretch repository would work without a problem. Since I value his opinion, I decided to just try and install version 1.4.10.

Here is the output of the update utility after trying to add the mosquitto repository.

woopi@goldserver: $ apt-get update Reading package lists... Done W: chmod 0700 of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (1: Operation not permitted) E: Could not open lock file /var/lib/apt/lists/lock - open (13: Permission denied) E: Unable to lock directory /var/lib/apt/lists/ W: Problem unlinking the file /var/cache/apt/pkgcache.bin - RemoveCaches (13: Permission denied) W: Problem unlinking the file /var/cache/apt/srcpkgcache.bin - RemoveCaches (13: Permission denied)

To help in removing the added gpg key, I found a somewhat outdated reference How can I remove gpg key that I added using apt-key add -? which showed that I could do the following.

woopi@goldserver:~ $ sudo apt-key list /etc/apt/trusted.gpg -------------------- pub rsa2048 2012-04-01 [SC] A0DA 38D0 D76E 8B5D 6388 7281 9165 938D 90FD DD2E uid [ unknown] Mike Thompson (Raspberry Pi Debian armhf ARMv6+VFP) <mpthompson@gmail.com> sub rsa2048 2012-04-01 [E] pub rsa2048 2012-06-17 [SC] CF8A 1AF5 02A2 AA2D 763B AE7E 82B1 2992 7FA3 303E uid [ unknown] Raspberry Pi Archive Signing Key sub rsa2048 2012-06-17 [E] pub rsa4096 2013-01-04 [SC] [expires: 2019-12-31] 8277 CCB4 9EC5 B595 F2D2 C713 6161 1AE4 3099 3623 uid [ unknown] Mosquitto Apt Repository <repo@mosquitto.org> sub rsa4096 2013-01-04 [E] [expires: 2019-12-31]

The last entry is the culprit to be deleted.

woopi@goldserver:~ $ sudo apt-key del "8277 CCB4 9EC5 B595 F2D2 C713 6161 1AE4 3099 3623" OK woopi@goldserver:~ $ sudo apt-key list /etc/apt/trusted.gpg -------------------- pub rsa2048 2012-04-01 [SC] A0DA 38D0 D76E 8B5D 6388 7281 9165 938D 90FD DD2E uid [ unknown] Mike Thompson (Raspberry Pi Debian armhf ARMv6+VFP) <mpthompson@gmail.com> sub rsa2048 2012-04-01 [E] pub rsa2048 2012-06-17 [SC] CF8A 1AF5 02A2 AA2D 763B AE7E 82B1 2992 7FA3 303E uid [ unknown] Raspberry Pi Archive Signing Key sub rsa2048 2012-06-17 [E]

The key is gone, but apt-get update would still report an error. Obviously, the mosquitto.org repository is still being searched.

woopi@goldserver:~ $ sudo apt-get update Hit:1 http://archive.raspberrypi.org/debian stretch InRelease Get:2 http://raspbian.raspberrypi.org/raspbian stretch InRelease [15.0 kB] Get:3 https://repo.mosquitto.org/debian stretch InRelease [11.0 kB] Ign:3 https://repo.mosquitto.org/debian stretch InRelease Get:4 https://repo.mosquitto.org/debian stretch/main all Packages [1,616 B] Get:5 https://repo.mosquitto.org/debian stretch/main armhf Packages [10.7 kB] Fetched 38.2 kB in 2s (18.7 kB/s) Reading package lists... Done W: GPG error: https://repo.mosquitto.org/debian stretch InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 61611AE430993623 W: The repository 'https://repo.mosquitto.org/debian stretch InRelease' is not signed. N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use. N: See apt-secure(8) manpage for repository creation and user configuration details.

Removing the downloaded /mosquitto-stretch.list from the list of repositories solved the problem

woopi@goldserver:~ $ ls -l /etc/apt/sources.list.d total 8 -rw-r--r-- 1 root root 51 Aug 8 16:39 mosquitto-stretch.list -rw-r--r-- 1 root root 195 Nov 13 10:02 raspi.list woopi@goldserver:~ $ sudo rm /etc/apt/sources.list.d/mosquitto-stretch.list woopi@goldserver:~ $ ls -l /etc/apt/sources.list.d/ total 4 -rw-r--r-- 1 root root 195 Nov 13 10:02 raspi.list woopi@goldserver:~ $ sudo apt-get update Hit:1 http://archive.raspberrypi.org/debian stretch InRelease Hit:2 http://raspbian.raspberrypi.org/raspbian stretch InRelease Reading package lists... Done

I hope I will remember these steps the next time I want to remove a manually added package repository and its security key.

So I proceeded with a simple installation of the broker and the optional utilities (to get mosquitto_sub and mosquitto_pub) without a problem.

woopi@goldserver: $ sudo apt-get install mosquitto mosquitto-clients

After rebooting I checked and found that the broker was running automatically.

woopi@goldserver:~ $ sudo systemctl status mosquitto.service ● mosquitto.service - LSB: mosquitto MQTT v3.1 message broker Loaded: loaded (/etc/init.d/mosquitto; generated; vendor preset: enabled) Active: active (running) since Tue 2019-01-08 01:46:18 AST; 11h ago Docs: man:systemd-sysv-generator(8) Process: 316 ExecStart=/etc/init.d/mosquitto start (code=exited, status=0/SUCC CGroup: /system.slice/mosquitto.service └─382 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf Jan 08 01:46:18 goldserver systemd[1]: Starting LSB: mosquitto MQTT v3.1 message Jan 08 01:46:18 goldserver mosquitto[316]: Starting network daemon:: mosquitto. Jan 08 01:46:18 goldserver systemd[1]: Started LSB: mosquitto MQTT v3.1 message

Cursory verification indicated that the newest version of Domoticz and Mosquitto version 1.4.0 work together without a problem. The ultimate verification will come when I install all this on the home automation system.

Installing MJPG-Streamer toc

I have already documented how to use MJPG-Streamer in conjunction with Domoticz to stream video from a USB webcam. Everything in part 2. Software works without change in Rasbian Stretch. There is no point going into detail again, here is the sequence of commands needed to install the video streaming software with its prerequisites.

pi@woopi:~ $ sudo apt-get install -y cmake libjpeg8-dev pi@woopi:~ $ wget https://github.com/jacksonliam/mjpg-streamer/archive/master.zip pi@woopi:~ $ unzip master.zip pi@woopi:~ $ cd mjpg-streamer-master/mjpg-streamer-experimental pi@woopi:~/mjpg-streamer-master/mjpg-streamer-experimental $ make pi@woopi:~/mjpg-streamer-master/mjpg-streamer-experimental $ sudo make install

The executable binary file is placed in the /usr/local/bin directory, the input and output libraries in /usr/local/lib/mjpg-streamer/ and the served web pages in /usr/local/share/mjpg-streamer/www. Here is the command to stream the video from a webcam (Logitech HD Webcam C270) running at 10 frames per second at a 1280x720 pixel resolution.

pi@woopi:~ $ mjpg_streamer -i "/usr/local/lib/mjpg-streamer/input_uvc.so -n -f 10 -r 1280x720" \ > -o "/usr/local/lib/mjpg-streamer/output_http.so -p 8085 -w /usr/local/share/mjpg-streamer/www" MJPG Streamer Version.: 2.0 i: Using V4L2 device.: /dev/video0 i: Desired Resolution: 1280 x 720 i: Frames Per Second.: 10 i: Format............: JPEG i: TV-Norm...........: DEFAULT o: www-folder-path......: /usr/local/share/mjpg-streamer/www/ o: HTTP TCP port........: 8085 o: HTTP Listen Address..: (null) o: username:password....: disabled o: commands.............: enabled

I also have a V.1 Raspberry Pi camera module (Rev 1.3). To test it with MJPG-Streamer, I powered down the Raspberry Pi, removed the USB webcam, and connected the ribbon cable as shown in How to install / use the Raspberry Pi Camera. Use the connector between the 3.5mm RCA jack and the HDMI connector. Pull up the black tab on the connector about 2 mm, slide the ribbon into the slot with the silver or gold connector facing the HDMI connector. Press down the black tab to lock the ribbon cable.

After powering up, use the configuration utility raspi-config to enable the camera.

woopi@goldserver:~ $ sudo raspi-config

It is the first option in the Interfacing Options screen. It will be necessary to reboot for the change to take effect and then to open a new session on the Raspberry Pi. Using the default values of the input library for the camera module provided with MJPG-Streamer is very simple.

woopi@goldserver:~ $ mjpg_streamer -i "/usr/local/lib/mjpg-streamer/input_raspicam.so" \ > -o "/usr/local/lib/mjpg-streamer/output_http.so -p 8085 -w /usr/local/share/mjpg-streamer/www" MJPG Streamer Version.: 2.0 i: fps.............: 5 i: resolution........: 640 x 480 i: camera parameters..............: Sharpness 0, Contrast 0, Brightness 50 Saturation 0, ISO 0, Video Stabilisation No, Exposure compensation 0 Exposure Mode 'auto', AWB Mode 'auto', Image Effect 'none' Metering Mode 'average', Colour Effect Enabled No with U = 128, V = 128 Rotation 0, hflip No, vflip No ROI x 0.000000, y 0.000000, w 1.000000 h 1.000000 o: www-folder-path......: /usr/local/share/mjpg-streamer/www/ o: HTTP TCP port........: 8085 o: HTTP Listen Address..: (null) o: username:password....: disabled o: commands.............: enabled i: Starting Camera Encoder Buffer Size 81920

I wanted to use the same parameters as used with the Webcam. It is possible but options are defined differently with this input library.

woopi@goldserver:~ $ mjpg_streamer -i "/usr/local/lib/mjpg-streamer/input_raspicam.so -x 1280 -y 720 -fps 10" \ > -o "/usr/local/lib/mjpg-streamer/output_http.so -p 8085 -w /usr/local/share/mjpg-streamer/www" MJPG Streamer Version.: 2.0 i: fps.............: 10 i: resolution........: 1280 x 720 i: camera parameters..............: ...

More information about the input module can be found on the mjpg-streamer github.

I found the results acceptable, but the camera module seemed more sluggish than the webcam. Something to investigate later.

Odds and Ends toc

Some hardware is hanging off my home automation system: a real-time clock and a temperature and humidity sensor. The real time clock is based on the DS3231. The instructions about installing it that I wrote back in March 2017 (with an update in May of the same year), Real Time Clock, DS3231, for Domoticz on the Raspberry Pi, work in the newer version of Raspbian.

As for reading the temperature and humidity sensor, a DHT11, things appear to be simpler than what I wrote in Temperature Sensors on a Raspberry Pi hosting Domoticz back in June of 2017. It looks like it is not necessary to edit the /boot/config.text file. On the other hand, I had to install python pip which was not mentioned in that post. Perhaps, I had already installed that package before connecting the temperature sensor.

I also enabled the UDP module of the log server on the Raspberry Pi. The configuration file is new slightly different from what I wrote in my previous post: Syslog Server on Raspbian.

woopi@goldberg:~/pythons $ sudo nano /etc/rsyslog.conf
# /etc/rsyslog.conf Configuration file for rsyslog. # # For more information see # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html ################# #### MODULES #### ################# module(load="imuxsock") # provides support for local system logging module(load="imklog") # provides kernel logging support #module(load="immark") # provides --MARK-- message capability # provides UDP syslog reception module(load="imudp") <-- enabled by removing leading '#' input(type="imudp" port="514") <-- enabled by removing leading '#' # provides TCP syslog reception #module(load="imtcp") #input(type="imtcp" port="514")

Conclusion toc

The last major piece of software to install is a web server. As explained before, I used Lighttpd but I am having some problems with it. If I cannot get the latest version to act as a secure reverse proxy for the MJPG-Streamer web server, I could try installing another package or I could try to revert to the older version that worked in Jessie.

To be honest, this is just being pigheaded. I just hate being stumped by something that worked previously, so I would really like to solve the problem. However, I am pretty sure I will not use the reverse proxy. I will install the latest version of Lighttpd, mostly to support OTA updates of IoT devices on the local network. But I will also install a VPN server on the Raspberry Pi to secure access to the system from outside the local network (i.e. over the Internet).

OpenVpn is presently operating rather well on the older Raspberry Pi running Jessie. It might be a good time to check out WireGuard. There is a project on Github by Arian Mihalko to facilitate its installation onto a Rapsberry Pi and Dr Zzs has a video on the subject.

So this is a good point to back up the SD card. As I experiment with all the above, I could very well want or have to come back to this point which will be easy with the backup. Here is how I do it on my Ubuntu desktop.

michel@hp:~/Documents/domotique$ df -h Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur udev 5,9G 0 5,9G 0% /dev tmpfs 1,2G 2,1M 1,2G 1% /run ... tmpfs 1,2G 16K 1,2G 1% /run/user/121 tmpfs 1,2G 52K 1,2G 1% /run/user/1000 /dev/sde1 44M 23M 22M 51% /media/michel/boot /dev/sde2 7,4G 1,2G 5,9G 17% /media/michel/rootfs michel@hp:~/Documents/domotique$ sudo umount /dev/sde2 michel@hp:~/Documents/domotique$ sudo umount /dev/sde1 michel@hp:~/Documents/domotique$ sudo dd bs=4M if=/dev/sde of=backup_2019_01_08_strech.img 1920+0 enregistrements lus 1920+0 enregistrements écrits 8053063680 bytes (8,1 GB, 7,5 GiB) copied, 474,673 s, 17,0 MB/s