md
No More Tasmota/ESP8266 Mosquitto MQTT Disconnects
November 8, 2019
<-Home Automation Servers on Raspbian Buster Lite --

Many have noticed numerous unexplained disconnections with the MQTT broker. To make a long story short, the source of the problem is not in Tasmota, it is not in the Arduino Client for MQTT (PubSubClient), and it is not in the Eclipse Mosquitto MQTT broker. As far as I have been able to determine, the problem is in some of the Wi-Fi code of the ESP8266 Arduino core. If you are using Tasmota, use a version from release 6.7.1 and the client unwarranted disconnections from the MQTT broker should disappear. If compiling your own firmware, use version 2.5.3 of the ESP8266 Arduino core. To get it, use the following URL

https://github.com/Jason2866/Arduino/releases/download/2.5.3/package_esp8266com_index.json

in the Additional Board Manager URL's (Menu: File/Preferences/Settings or shortcut Ctrl,).

What follows is the recounting of how I encountered the problem and stumbled on the simple solution. I am sure that many will find this absolutely riveting.

After installing the Mosquitto MQTT broker on the Raspberry Pi 3 B that will now host our home automation system, I had a look at the log file. It contained numerous messages about socket errors and disconnections. Here is a short sample of what I found.

woopi@goldserver:~ $ sudo tail -f /var/log/mosquitto/mosquitto.log ... 1572128560: New client connected from 192.168.1.108 as DVES_6F028F (c1, k10, u'DVES_USER'). 1572128624: Socket error on client DVES_6F028F, disconnecting. 1572128625: New connection from 192.168.1.108 on port 1883. 1572128625: New client connected from 192.168.1.108 as DVES_6F028F (c1, k10, u'DVES_USER'). 1572128641: New connection from 192.168.1.45 on port 1883. 1572128641: New client connected from 192.168.1.22 as mosqpub|16971-goldserver (c1, k60). 1572128641: Client mosqpub|16971-goldserver disconnected. ...

There is a cron job on the server that publishes a short MQTT message every two minutes. An MQTT connection with a client that is publishing a message only lasts for the time needed to send the message to the broker. Accordingly, the last pair of connect and disconnect operations in the log is not a problem.

The other disconnections were unexpected. These were obviously from ESP8266 based devices, mostly Itead Sonoff Basic Wi-Fi switches. All these are running a fairly recent version of the excellent Tasmota firmware. I must admit that this flurry of disconnections was rather disconcerting, but it did not seem to have any impact on the performance of the home automation system. Nevertheless, I wanted to find the cause and a solution if possible.

So for the last couple of weeks I have been searching the Web for information on this topic. It turned out that this was another well-known problem that I was not aware even existed. I should look at the logs more often.

Some in the Home Assistant community seemed to blame Tasmota and decided to use different firmware and do away with MQTT altogether. I did not think that was probable because the biggest culprit was my automatic garage door closer built on an Itead SV with my own firmware. Was the SV in play? Probably not because just as many disconnections when running the firmware on a Wemos (now Lolin) D1 mini. While my firmware is very much "borrowed" from an older version of Tasmota, the MQTT connect and reconnect parts are almost a copy and paste of the example code included in the PubSubClient library. To add to the mystery, my router monitor which is a repurposed Itead Sonoff Basic only rarely disconnected from the MQTT broker even though it has the same MQTT connnect and reconnect functions used in the garage door closer.

Some blamed the PubSubClient library, but discussions in the issues section of the Tasmota GitHub seemed to reject that hypothesis. There were mentions of the MQTT KeepAlive time, something that Tasmota handled, but that my firmware did not. So I set up a second mosquitto MQTT broker on another Raspberry Pi and started to experiment with that variable using the D1 mini. I did not make any progress.

There were discussions about versions of the ESP8266 Arduino core. I remembered skipping version 2.4.x, going from 2.3.0 directly to 2.5.2 because of problems with the intermediary versions. Perhaps I should try the old core again? When looking at the Tasmota Wiki for more information, I noticed that version 6.7.1 of Tasmota is available and it will be supported from ESP8266/Arduino library Core version pre-2.6.0 due to reported security and stability issues on previous Core version. Also I could no longer find the Wiki page that had information about MQTT problems. This is where the penny dropped.

I modified my experiment yesterday. I had a spare Sonoff Basic R3 so I flashed Tasmota 6.7.1 on it. I recompiled my automatic garage door closer firmware using the 2.5.3 Esp8266 Arduio core and flashed that on a Lolin (Wemos) D1 mini. Both were then started and connected to the second MQTT broker running on a Raspberry Pi 3 B. The operating system is the newest edition of Rasbian Buster Lite (Sept. 2019). The MQTT broker is mosquitto 1.5.7-1 from the Raspbian repository. There has not been a single socket error and disconnect from the MQTT broker since this experiment was started over 12 hours ago.

Those solutions to a problem that just require an upgrade are just the best.

<-Home Automation Servers on Raspbian Buster Lite --