Since the start of 2019, voice control of IoT devices at home is done with the combination of Amazon smart speakers and HA Bridge as a shim between Alexa and the home automation software I have been using for years Domoticz. Replacing a less than optimal combination of Google Home Assistant and IFTTT, which I am not even sure would work now, this new system has proven to be quick, reliable and useful. I have no doubt this aspect of the home automation system is in no small part responsible for the acceptance of the whole idea of home automation in this household.

That is all good, but I have encountered a problem which has plagued others. Adding devices has been a problem. In the last two or three times this was done, it felt like there was a likelihood that the process would fail and voice control of all IoT devices would be lost. While the good news is that it was ultimately possible to include new devices, the bad news is that it could not be done incrementally and it was necessary to remove all devices and then have Alexa discover all devices again. Most often, this did not work on the first try, or even after several attempts. Only yesterday did I manage to add two devices after many attempts over a period of days during which all voice commands failed. It was stressful to fear that the entire subsystem of 30 plus virtual devices in the Alexa -- HA Bridge -- Domoticz chain on which others rely could have been broken for the sake of adding two new devices. Below I will describe what seems to have worked.

Before continuing, it must be said that this is not a description of the experience one would encounter using Alexa with Amazon devices only or with third-party devices through their proprietary platforms such as the ubiquitous Tuya Smart application. Hopefully when a subscription service such as Home Assistant Cloud (from Nabu Casa Inc.) within Home Assistant is used to integrate Alexa and Tuya devices then discovery is all taken care of. However, I can't say as I have no experience with such systems.

An important feature of my small home automation system is its minimal reliance on outside resources such as cloud services. Almost all physical IoT devices in the house use ESP microcontrollers (ESP8266, ESP8285, ESP32) that are connected to the local area network by Wi-Fi. Remote control is seamless using the MQTT messaging protocol. As far as I know, Alexa, Home Assistant, and et al do not support MQTT directly. This is where HA Bridge plays a role. Amazon Echo smart speakers support Philips Hue Bridges locally. A Philips Hue bridge or hub is a device that is connected to the local area network over an wired Ethernet link that communicates with Hue bulbs using the wireless Zigbee protocol. An Echo, if connected to the same local area network (albeit by Wi-Fi) can instruct a Hue bridge to turn a Hue bulb on or off, set it brightness to a given level or the colour, etc. Of course, being software, HA Bridge cannot control Hue bulbs with the required Zigbee commands. What it does is present itself on the LAN as a Philips hub that controls a number of devices. When the Echo instructs it to say turn on one of these devices, HA Bridge passes the appropriate command to a home automation server (Domoticz in my case, but it could be Home Assistant, HomeWizard, OpenHab, etc.) that then sends an MQTT message to the real device.

The top part of the image shows the connected devices as Alexa sees them, while the bottom part is a truer representation of my home automation system. Alexa believes it is dealing with Zigbee devices through a Philips Bridge, but in fact it is dealing with an emulator, HA Bridge which translates some commands for Hue bulbs into http requests instructing Domoticz to change the state of smart Wi-Fi switches.

For this to work, IoT devices need to be registered with the Alexa service. This is done during "device discovery" when the Echo Dot finds Hue hubs on the local network with the Universal Plug and Plan (UPnP) protocol designed precisely to enable mutual discovery of networked devices on a LAN. After finding a Hue hub, the Echo queries its capabilities and from there all the devices returned by the hub are registered into the smart home database housed in my Alexa account. Here is part of the HA Bridge (@192.168.1.115) log showing part of its exchange with the Echo Dot (@192.168.1.108) while masquerading as a Hue Hub.

Device discovery is where I run into problems. Below is the procedure that finally worked, as I best remember what I did. Its crucial elements are to first clear all devices from Alexa and to enable 9 byte device identity numbers in the very latest version of the HA Bridge script before the Echo Dot is told to discover devices.

1. Enable UPnP in the router/gateway. Because this protocol is associated with security problems in home networks, it is normally disabled it. However it is needed by the Amazon Dot when it looks for devices because as explained above that is how smart speaker finds HA Bridge masquarading as a Philips Hue hub.
2. Unplug all Amazon Echo devices except for one. According to some reports, there may be some sort of interdepence if multiple Echos are on the network during discovery. I used a single 3rd generation Echo Dot to discover devices.
3. Using a web browser on the desktop, log into Alexa at alexa.amazon.com. Actually, the address will be different depending on the country of residence, I use alexa.amazon.ca. If there are more than a very few IoT devices, be sure to use a desktop web browser and not a web browser on a telephone or tablet (running Android or IOS) because deleting devices in the Alexa application can only be done one device at a time and each deletion requires a number of steps to complete.
4. Courageously, remove all devices. Click on Smart Home listed among other topics along left side of the page. Then scroll down to the end of the list of devices and click on the Forget All button at the bottom.

   

   On the French version of the application, this choice is Maison connectée about half way down while the button is named Tout supprimer as shown above.

   All devices will be removed from the list. Opening the Amazon application on an Android tablet confirmed that all devices are gone. However in my experience, if detection of devices were to be launched at this point to hopefully refresh the list of devices, there is a high probability that the supposedly delete devices will reappear and a second copy of many devices will be added. That is how the above image says that there were 44 devices when in fact there should have approximately 30.

   The goal is to definitely clear all devices and to be able to perform a discovery with Alexa reporting that it found none. Here is more or less how I managed this.

   • Unplug the Echo - no Echo devices should be active.
   • Open a session on the machine on which HA Brige is running and move the HA Bridge device database, device.db, to another directory and restart the service.
     zero@opi:~$ ls ha-bridge/data device.db habridge.config zero@opi:~$ sudo mkdir ha-bridge/data-bak zero@opi:~$ sudo mv ha-bridge/data/device.db ha-bridge/data-bak/ zero@opi:~$ ls ha-bridge/data habridge.config zero@opi:~$ sudo systemctl restart ha-bridge.service
   • After some time (I don't really know if it is really necessary to wait a very long time such as overnight, but it may help), plug a single Echo device back on. Once is has completed its initialisation procedure, check using a Web browser or the Alexa application that the list of connected devices is empty.
   • Instruct the Echo dot to find all smart devices. I prefer to give this command orally after closing any web browser and Alexa application that might have been opened previously. It might help to cross some fingers or mutter some incantation to the tech gods at this point.
   • If no devices are found, then chances are the rest will work correctly, go on to the next point. Otherwise repeat the above, changing something and hope to get to the point where Alexa has forgotten all devices and the Echo dot cannot find any device. Patience and luck might be the order of the day.
5. Stop the ha-bridge.service with systemctl and replace the Java script with the latest release candidate at https://github.com/bwssytems/ha-bridge/releases which is currently ha-bridge-5.4.1RC1.jar, Fixes for v5.4.0 @bwssytems bwssytems released this on 24 Mar #1293, Added capability to renumber with 9 octets. Note that there is a seperate release candidate for those using Java version 11.
6. Update the ExecStart line of the ha-bridge.service unit file in directory /etc/systemd/system to use the the newly downloaded Java script file.

   I do something else. The ExcStart line will start ~/ha-bridge/ha-bridge.jar without any version number. That file is a symbolic link to the latest jar file, so I updated the symbolic link instead of modifying the unit file.

   zero@opi:~/ha-bridge$ ls -l total 37492 drwxr-xr-x 2 zero zero 4096 Aug 13 02:19 data -rw-r--r-- 1 zero zero 9670679 Dec 14 2020 ha-bridge-5.4.0.old -rw-r--r-- 1 zero zero 9672480 Mar 24 15:35 ha-bridge-5.4.1RC1.jar lrwxrwxrwx 1 zero zero 22 Jul 6 20:56 ha-bridge.jar -> ha-bridge-5.4.1RC1.jar

   Note how I changed the extension of the previous version of the script just to ensure it will not be used.

7. Copy the device.db file back into the ~./ha-bridge/data/ directory if it was removed in step 4.
8. Start the ha-bridge service with systemctl. Check the status of the service, just to ensure it is running.
9. Go to the HA Bridge Web interface with a browser. Click on the About menu to verify that the correct version of the bridge is running.

   Since no changes were made to the configuration in the ~./ha-bridge/data directory, all the devices in the database should be visible in the Bridge Devices tab and it should be possible to turn devices on or off with the test buttons.

10. Go to the Domoticz Devices tab and go down the list of all Domoticz virtual devices HA Bridge found. Click on the Build Item button to add the new devices to be installed in Alexa.
11. Go back to the Bridge Devices tab and carefuly examine the list of devices known to HA Bridge that will be exposed to Alexa. Make sure that the on/off commands work as well as the dim command if appropriate. I have gotten so paranoid that I remove any non-ASCII characters in the device names. Device names can be edited in the Alexa application so that it will be possible to restore Unicode characters if necessary.
12. Modify or verify the following settings in Bridge Control:
    • UPNP IP Address: set to the IP address of the server on which HA Bridge is running.
    • Domoticz Names and IP Addresses: these settings must be correct otherwise the test buttons would not have worked in the previous section.

      Of course if another gateway to the IoT devices is used instead of Domoticz then its settings must be correct. Again, if the on/off test buttons in the previous page worked, then this must mean the settings are correct.

    • Unique ID to use 9 Octets: set to true. Issue #1325 5.4.0 devices not being discovered by dbulling opened June 30 makes the point about the need for 9 byte id numbers and goes on about changing the values manually. However this was fixed on March 24 with version 5.4.0 release candidate 1 installed in step 5.
      Fixes for v5.4.0 @bwssytems bwssytems released this on 24 Mar #1293 Added capability to renumber with 9 octets
      All that is needed is to enable this feature as shown here.
    • UPNP Send Delay: 1275. I increased this value from the default 650 ms because during some discovery attempts only 10 of the 30 odd devices were being loaded. Your mileage may vary.
    • ID Seed (start numbering from this value): I changed the seed number from 100 to 150 so that all devices would have new identity numbers.
    • When finished, scroll back to the top of the Bridge Control page and click on the Save button.
    • Wait for the bridge to reinitialize.
13. Back in the Bridge Devices, ensure that no ID number in the first column is bold (which would prevent renumbering that device)
14. Click on the Renumber Devices button.
15. At this point, I prefer to reboot the system, and access the HA Bridge web interface again to check that everything is as required: the Bridge Devices page contains all the devices to be imported into Alexa and none other, and the settings in Bridge Control are correct. Click on the Test button at the top of that page and check that the "uniqueid" field of the devices contains an 9 octet number similar to a MAC address except for the "-" separator for the last byte. Here is an example, albeit nicely formatted.
    [ ... {"id":"150", "uniqueid":"00:9d:fc:d5:06:39:a0:ce-d1", "name":"Marches sous-sol", "mapId":"Marches sous-sol-undefined", "mapType":"domoticzDevice", "deviceType":"switch", "offUrl":"[{\"item\":\"http://192.168.1.22:8080/json.htm?type\u003dcommand\u0026param\u003dswitchlight\u0026idx\u003d438\u0026switchcmd\u003dOff\",\"type\":\"domoticzDevice\"}]", "onUrl":"[{\"item\":\"http://192.168.1.22:8080/json.htm?type\u003dcommand\u0026param\u003dswitchlight\u0026idx\u003d438\u0026switchcmd\u003dOn\",\"type\":\"domoticzDevice\"}]", "inactive":false, "noState":false, "offState":false, "deviceState":{"on":false,"bri":0,"alert":"none","reachable":true}, "onFirstDim":false, "onWhenDimPresent":false, "lockDeviceId":false, "dimNoOn":false, "dimOnColor":false}, {"id":"151", "uniqueid":"00:eb:61:df:e5:58:12:a0-92", "name":"Entree", "mapId":"Entrée-undefined", "mapType":"domoticzDevice", "deviceType":"switch", "offUrl":"[{\"item\":\"http://192.168.1.22:8080/json.htm?type\u003dcommand\u0026param\u003dswitchlight\u0026idx\u003d537\u0026switchcmd\u003dOff\","type\":\"domoticzDevice\"}]", "dimUrl":"[{\"item\":\"http://192.168.1.22:8080/json.htm?type\u003dcommand\u0026param\u003dswitchlight\u0026idx\u003d537\u0026switchcmd\u003dSet%20Level\u0026level\u003d${intensity.percent}\",\"type\":\"domoticzDevice\"}]", "onUrl":"[{\"item\":\"http://192.168.1.22:8080/json.htm?type\u003dcommand\u0026param\u003dswitchlight\u0026idx\u003d113\u0026switchcmd\u003dOn\",\"type\":\"domoticzDevice\"}]", "inactive":false, "noState":false, "offState":false, "deviceState":{"on":false,"bri":0,"alert":"none","reachable":true}, "onFirstDim":false, "onWhenDimPresent":false, "lockDeviceId":false, "dimNoOn":false, "dimOnColor":false}, ... ]
16. Tell the Echo Dot to discover devices. This should work. Perhaps you will need to cross your legs at the ankles, face North-East and launch the discovery on an even minute to get it to work, but as explained at the start, with enough patience and experimentation, Alexa has so far managed to work with my home automation system.

I am not sure just how useful this post will be. In the past, notes such as these were kept but it never seemed to help in subsequent attempts to add devices in Alexa. So be careful, and weigh the cost of possibly losing all voice commands versus the gain from adding a single device. Be aware that all scenes previously set up in Alexa will be lost as well as all settings for Amazon or third party IoT devices.

Don't forget to

• save the HA Bridge device database,
• save the HA Bridge configuration,
• disable UPnP on the local area network.

Hope this can be of help to anyone facing similar problems even if the instructions are not as explicit as is usual on this site. I'll end here reminding everyone that this post contains many warnings about the possibility that something can go wrong and that the ability to control home automation devices with Alexa might be lost. You have been warned.