The USB-TTL (3.3V) cable ordered from China in December has still not arrived. Being rather impatient, I decided to use a Raspberry Pi to program some Sonoff switches. The Raspberry Pi works at 3.3V and has a hardware serial output that can be connected directly to the Sonoff UART. For the most part, I will do with a Raspberry Pi what others do with a desktop computer and a USB-TTL cable.

Warnings

While I do sympathize if you ruin a Raspberry Pi or a Sonoff by following the instructions that follow, I will not be held responsible for the consequences and damages. I do not know of any way to reinstall the original ITEAD firmware on the Sonoff. So if you damage or replace this firmware, it will not be possible to use the EWeLink software to control the switch. In addition, any warranty that might exist on the Sonoff (is there one?) will be lost by breaking the seal when the case is open. Most importantly, NEVER program a Sonoff connected to the mains.

Firmware for the Sonoff

The ESP8266, which is the micro-controller used in the ITEAD Sonoff, can be driven with any one of a number of available firmware packages. I decided to try three of them which are capable of communicating with an MQTT broker:

1. Sonoff-MQTT-OTA-Arduino

Written by Theo Arends, this firmware garners well deserved praises from many. It seems to me to be the most complete and at the same time the simplest way to control a Sonoff (or other similar device of ITEAD) with the Domoticz home automation program. As the name suggests, the firmware is designed for use with Arduino's integrated development environment (IDE), which allows you to modify the code, compile it, and then upload it to the Sonoff.

2. Sonoff-MQTT-OTA

This firmware, also by Theo Arends, could be of interest to anyone using a home automation program that handles the MQTT protocol.

3. NodeMCU

Because I use NodeMCU with the Wemos D1 mini, I programmed a Sonoff with this firmware.

There is no need to know a programming language and no need to install the Arduino IDE to use Theo Arends firmware. In what follows, I explain how without the IDE. However, there are advantages to changing the basic configuration about the LAN, especially if you plan to program several Sonoffs. On the other hand, NodeMCU is actually a Lua interpreter and you need to add a script in that language to control the Sonoff. Fortunately there is one that can serve as a basis.

Obtaining the firmware packages

In all three cases, the firmware is obtained from the Web. In two cases, the binary files have to be extacted from ZIP archives. While the instructions below pertain to a Ubuntu box, I assume much the same can be done in Windows or Mac OS/X.

1. Sonoff-MQTT-OTA-Arduino

Click on the green button Clone or download at the GitHub site, and then click on Download ZIP and save the archive Sonoff-MQTT-OTA-Arduino-master.zip in a folder. Extract the content of the archive which contains the single binary file that will be used.

2.Sonoff-MQTT-OTA

Click on the green button Clone or download at the GitHub site, and then click on Download ZIP and save the archive Sonoff-MQTT-OTA-master.zip in a folder. Extract the content of the archive which contains one of the binary files to be used. We also need three files from the Espressif ESP8266 SDK. Get the current version of the SDK that does not contain an operating system. Currently this is SP8266 NONOS SDK V2.0.0 20160810. Again extract the content of that archive.

Theo Arends has stopped improving both of the abover packages which have been replaced with Sonoff-Tasmota. This newest version is what I have been using for the last few weeks. I could not find a binary file in this new repository. That means that Arduino will need to be installed to compile the source code. Theo Arends provides a number of precompiled binaries for version 5.9.1 2017-11-07. It should be possible to install the new binary file with the method outlined below if there is no USB-TTL converter available, but I have not verified that because I updated the switches already using Sonoff-MQTT-OTA-Arduino firmware to the new version over the air (OTA). Note that it is not possible to update Sonoff-MQTT-OTA this way.

Update: November 17, 2017
November 22, 2017

3. NodeMCU

The NoceMCU firmware must be build with the desired modules. The simplest is to do a "cloud build" which means getting the firmware compiled on the NodeMCU custom builds site created by Marcel Stör. I chose the following modules CJSON, DHT, file, GPIO, HTTP, MQTT, node, timer, UART, WiFi. After a few minutes, an email is received with links to two binary files. I downloaded the integer version nodemcu-master-10-<date>-integer.bin.

Preparing the Raspberry Pi

An older, circa end of 2011, Raspberry Pi, model B (rev. 2.0, with 512MB of memory) proved suffisant. I did install the newest version of Raspbian, Jessie Lite dated January 11, 2017.

I basically followed my own instructions for installing Domoticz on a Raspberry Pi. Be sure to allocate a static IP address different from that given to all devices on the local area network.

The Sonoff will be programmed across a serial connection with the Raspberry Pi. Raspbian connects a console to the UART during the boot process, which prevents subsequent connection to the /dev/ttyAMA0 which is attached to the UART. Stopping and deactivating the serial-getty service frees the port permanently. You also need to modify the /boot/cmdline.txt file. I preferred to make a backup copy before editing this text.

In the last step, use the editor to remove the console=serial0,115200 option from the file.

As I have already explained, when talking about the Wemos D1 mini, we use the esptool.py script to flash the Sonoff ESP8266. Except that now it is the Raspberry Pi that will upload the binary files. It is therefore necessary to install this script on the small computer. While Python version 2.7 is included in Jessie Lite, the necessary script is not. It is easier to use pip to install esptool.py because there are missing dependencies. So you have to install two packages. If you want to use NodeMCU then you have to install a third package.

Now you have to copy to the Raspberry Pi the firmware images previousl downloaded to the desktop. I created three directories on the SD card of the Raspberry Pi for each of the firmware. Then I copied the required binary file(s) into each folder. All this was done with Filezilla.

1. Sonoff-MQTT-OTA-Arduino

Source: .../Sonoff-MQTT-OTA-Arduino-master/api/arduino/sonoff-ino.bin
Target: /home/pi/sonoff-ino/sonoff-ta-ino.bin

2. Sonoff-MQTT-OTA

Source: .../Sonoff-MQTT-OTA-master/api/sonoff/user1.bin
Target: /home/pi/sonoff/user1.bin

Source: .../esp8266sdk/sdk/bin/boot_v1.6bin
Target: /home/pi/sonoff/boot_v1.6bin

Source: .../esp8266sdk/sdk/bin/esp_init_data_default.bin
Target: /home/pi/sonoff/esp_init_data_default.bin

Source: .../esp8266sdk/sdk/bin/blank.bin
Target: /home/pi/sonoff/blank.bin

3. NodeMCU

Source: .../nodemcu/nodemcu-master-10-modules-2017-01-18-01-27-00-integer.bin
Target: /home/pi/nodemcu/nodemcu-master-10-modules-2017-01-18-01-27-00-integer.bin

The exact path of the source files obviously depends on the location chosen when they were obtained from the Web.

The next step is to use the Raspberry Pi to flash a Sonoff with one of these choices.

Connecting a Sonoff and a Raspberry Pi

It is not too difficult to prepare the Sonoff. Open the housing, a finger nail is the only tool needed. Solder a four or five pin header as shown below. In addition there numerous instructions on the Web and You Tube (search for "Sonoff flashing"). There are four connections to be made between this connector and that of Raspberry Pi.

Sonoff		Raspberry Pi
GND	4	GND	25
TxD	3	RxD	10
RxD	2	TxD	8
Vcc	1	3.3V	1

The pins on the Sonoff header are numbered 1 to 5 starting at the pin nearest the tactile push button.
With a more recent Raspberry Pi, the header has 40 pins. Ground will be found on the last pin on the left, the 39th pin.

The fifth pin on the Sonoff is not needed to flash it but it is of interest as it is connected to GPIO14 of the ESP8266 (see Local Control of Sonoff Switches).

The switch on power wire going to the Sonoff is virtual, I use a breadboard to plug or unplug the Vcc connection as needed. However, you can find on the Web an illustration a real switch on the Vcc wire connecting the Sonoff to a USB-TTL cable. It's a good tip if you need to program several Sonoffs.

I think the 4 connections mentioned above are valid for all Raspberry Pi models. However, before you connect the Sonoff to the UART of the Raspberry Pi, you should make sure that there is no error, because the assignment of some of the Raspberry Pi GPIO pins differs depending on the model. To know which model we have:

The model is identified from the Revision number using this list. That is how I confirmed that Revision 000e meant that I had a Raspberry Pi Model B (Rev 2.0, 512Mb).

Initialement, connect GND, TxD-RxD, RxD-TxD between the two devices, but not Vcc. If the 3.3V wire is already connected, that is alright, just disconnect it.

Flashing the Firmware

The photograph shows the devices before flashing. The white wire the connection Raspberry Pi TxD <--> ESP8266 RxD which is represented by a blue wire in the above diagram where it would not have been practical to use white as a colour.

The Raspberry Pi is powered up, an SSH session is opened between my desktop and the Pi and the Sonoff is not powered; the 3.3V wire is not connected.

To set the Sonoff in flashing mode, keep its tactile switch depressed as it is powered up by connecting the red wire.

Once the Sonoff is powered up, release the tactile button. Nothing happens, the LED remains off which is the signal that the Sonoff is in flashing mode, but which is a bit disconcerting the first few times.

There are two optional steps that I recommend doing.

1. Backup the firmware currently in flash memory

pi@raspberrypi:~$ esptool.py --port /dev/ttyAMA0 read_flash 0x00000 0x100000 Sonoff_backup_01.bin

Perhaps it would be possible to restore the factory installed firmware from this file. I have not tested that, so do not count on it. Once this is done, the Sonoff may be put in normal mode. You will have to power cycle it while holding the tactile button down routine again.

2. Erase the content of the flash memory

pi@raspberrypi:~$ esptool.py --port /dev/ttyAMA0 erase_flash

I strongly recommend doing this. As I found out the hard way, you can get strange errors if the Sonoff firmware starts interpreting remaining bits in the configuration area as valid data. Once this is done, the Sonoff may be put in normal mode. You will have to power cycle it while holding the tactile button down routine again.

The Python script esptool is used to flash the binary file(s). Here is the command according to the chosen firmware.

1. Sonoff-MQTT-OTA-Arduino

pi@raspberrypi:~$ cd sonoff-ino pi@raspberrypi:~/sonoff-ino $ esptool.py --port /dev/ttyAMA0 write_flash -fs 8m \ > 0x00000 sonoff-ta.ino.bin

2. Sonoff-MQTT-OTA

pi@raspberrypi:~$ cd sonoff pi@raspberrypi:~/sonoff $ esptool.py --port /dev/ttyAMA0 write_flash -fs 8m \ > 0x00000 boot_v1.6.bin \ > 0x01000 user1.bin \ > 0xFC000 esp_init_data_default.bin \ > 0xFE000 blank.bin

3. NodeMCU

pi@raspberrypi:~$ cd nodemcu pi@raspberrypi:~/nodemcu $ esptool.py --port /dev/ttyAMA0 write_flash -fs 8m \ > 0x00000 nodemcu-master-10-modules-2017-01-18-01-27-00-integer.bin

NOTES
1. If the device did not work once flashed, try using DOUT flash mode:

pi@raspberrypi:~/nodemcu $ esptool.py --port /dev/ttyAMA0 write_flash -fm dout -fs 8m \ > ... as before

It appears that newer versions of ESP8285 and ESP8286 devices, including the Sonoff, contain flash memory with DOUT acces mode. I followed Theo Arends July 14th tip about using this mode when I flashed some newly acquired switches. Thanks to Sébastien Giroux for reminding me to update these older instructions.

2. if there was a warning about an incorrectly set flash size, use the option "-fs 1MB" instead of "-fs 8m" as required by newer versions of esptool.py.

Update: November 22, 2017

Once flashing is completed (esptool will signal the end), power cycle the Sonoff. No matter what is shown on the screen, esptool cannot perform a "hard reset". According to the esptool documentation "The DTR serial control line is used to reset the chip into a normal boot sequence." But there is no DTR connection between the Raspberry Pi and the Sonoff. So, turn off the 3.3 V going to the Sonoff, then connect it back on. From then on, serial communication between the devices will be possible if the firmware supports it.

As it happens, each of the three firmware does provide a serial link. In principle, it is entirely possible to configure the Sonoff with a program like miniterm.py, but in practice we do the bare minimum with Theo Arendt firmware, just enough to be able to complete the configuration with either the Sonoff Web server (Sonoff-MQTT-OTA -Arduino only), or by publishing MQTT messages (both firmwares).

1. Sonoff-MQTT-OTA-Arduino

pi@raspberrypi:~/sonoff $ miniterm.py /dev/ttyAMA0 115200 -e

2. Sonoff-MQTT-OTA

pi@raspberrypi:~/sonoff $ miniterm.py /dev/ttyAMA0 115200

3. NodeMCU

pi@raspberrypi:~/sonoff $ miniterm.py /dev/ttyAMA0 115200

It will probably be necessary to press the Enter key a few times to finally see a menu, in the case of Theo Arends firmware, or the Lua prompt ">" in the case of NodeMCU.

Configuring the Sonoff

At the very least the Sonoff, or more exactly the ESP8266, must know the name of the wireless network, its password and the name or IP address of the MQTT broker. There is probably more information needed to work with a home automation program. Just how the information is fed to the Sonoff depends on the firmware.

1. Sonoff-MQTT-OTA-Arduino

The firmware will dump a lot of information across the serial link. To the point where it is initially a bit confusing. But reading the feed, it looks like the Sonoff could actually connect to the WiFi network with the WPS protocol... which would be great if my arms could reach the router two floors down.

The simplest, at this point, is to specify the password and name of the local WiFi netword. After that, the firmware has a Web server that can be used to configure the Sonoff.

The format is: parameter name then parameter value separated by a space. Do not include quote characters, equal sign or whatever. This means that it is preferable to not have spaces in the network name or password.

The last two lines are repeated continously while the Sonoff tries to connect to an unknow MQTT broker. Pick up the line in bold, it shows the address of the Sonoff Web server to be used to complete the configuration.

The left panel is the home page of the Web server. We can see, in very large letter, the state of the relay. It can be toggled by clicking on Toggle. Of course, the Configuration button needs to be clicked to continue with the configuration of the switch. The second panel shows the configuration web page.

WiFi configuration is partially done otherwise we would not be here. We must configure the MQTT service. Actually, the only thing that needs to be done, is entering the name or IP address of the MQTT broker as can be seen on the right panel.

Controlling the Sonoff with Domoticz

Sonoff-MQTT-OTA-Arduino takes care of details integrating a Sonoff switch with the home automation program Domoticz. It really could not be simpler, but it still requires three steps. First, create a virtual switch on the hardware nammed Dummy (Does nothing, use with virtual switches only)in Domoticz. This is the switch that will remotely turn the Sonoff on or off. Here is a quick reminder of the procedure.

1. Click on the Setup tab.
2. Click on the Hardware item in the menu.
3. If the virtual hardware Dummy (Does nothing, ...) already exists move on to step 4.
   • Complete the fields
     Name: = Virtuel, and
     Type: = Dummy (Does nothing... that is picked from the drop down list.
   • Click on the Add button at the bottom of the page. The virtual hardware will now show up in the table at the top of the page.
4. Click on Create Virtual Sensors in the virtual (Dummy) hardware.
5. Name the sensor, I used TestSonoff and set the Sensor Type: to Switch.
6. Click on the OK button. A message saying where the switch was installed will briefly flash on the window.
7. Click on the Setup tab again.
8. Click on the Devices item in the menu.
9. Lookup the idx number of the newly created switch in the index column (idx).

The second step is to go back to the Sonoff's configuration on its Web server. Click the Configuration button and then click the Configure Domoticz button. Set the firmware index number Idx to the index value Domoticz assigned to the virtual switch just created.

These first two steps will have to be repeated for every additional Sonoff. The third step needs to be done only once in Domoticz.

1. Click on the Setup tab.
2. Click on the Hardware item in the menu.
3. If the MQTT Client Gateway with LAN interface hardware already exist go on to the next numbered step.
   • To add the MQTT hardware complete the fields
     Name: = MQTT, et
     Type: = MQTT Client Gateway with LAN interface which is picked from the dropdown list.
   • Click the Add button at the bottom of the page. The MQTT hardware should now be in the table at the top of the page.
4. Click on the MQTT Client Gateway with LAN interface row.
5. Select out from the Publish Topic: drop down list. This corresponds to the Flat explanation under the field.
6. Click on the Update button at the left under the hardware table.

All that remains to be done is that all works. Click on the bulb icon of the newly created virtual switch in the Switches tab. The Sonnof LED should follow the state of the virtual device. Press on the Sonoff tactile button, the virual switch icon should reflect the state of the Sonoff. The feedback in that direction is slower, it could take a second or two before the icon changes value.

2. Sonoff-MQTT-OTA

When a Sonoff, flashed with Sonoff-MQTT-OTA is powered up, the LED will immediately flash three times and then at regular intervals until it is connected to a WiFi network.

So that is our first task; connecting it to the network. If it is not already the case, open a terminal in the Raspberry Pi to communicate with the Sonoff. It will likely be necessary to press on the Enter a few times until the menu is visible.

A minimum of three parameters must be specified: the name of the WiFi network (SSID), its password, and the name or IP address of the MQTT (Host) agent. Only the name of the parameter and its value separated from a space should be entered. Do not add quotation marks or other punctuation marks around the value of the parameter. It would therefore be preferable not to have any space in any of these parameters.

It may well be that what we see on the screen will not be so clear. The firmware tries to establish a WiFi connection regularly and displays all sorts of information that must be ignored by continuing to write the commands as if nothing had happened.

Controlling the Sonoff with MQTT

It's time to check that everything works. Theo Arends firmware publishes its MQTT messages under the subject stat/sonoff and subscribes to messages whose subject is cmnd/sonoff. So let's open a terminal to subscribe to messages from Sonoff.

Another terminal can be used to publish commandments:

The Sonoff LED should blink twice as it receives each commandment.

The terminal subscribing to stat/sonof will show replies to the above sent by the Sonoff.

Pressing on the Sonoff tactile switch will toggle the state of its relay. At the same time, the firmware will send a message to the MQTT broker giving the state of the relay.

There is an obvious problem, all Sonoff switches with the stock firmware will respond to the same message cmnd/sonoff/ and it is impossible to distinguish which switch sent a particular stat/sonoff/ message. But Theo Arends anticipated that and created an MQTT command message to change the MQTT topic. I use the last 3 hexadecimal digits of the MAC address of the ESP8266 to define a unique MQTT topic for each Sonoff. Those numbers are shown in the status report seed above DVES_B01C19.

To follow what happens, I stopped the subscription to the MQTT broker by pressing the combination of keys CtrlC in the terminal. Then, I subscribed to all stat topics.

Then I published a message changing the topic:

and this is what appeared in the terminal listenting to the MQTT broker: From now on, that Sonoff will use topics stat/sonoffB01C19 et cmnd/sonoffB01C19 to send its responses and get its commands respectively.

For this to work properly, there should be only one Sonoff device on the network that still uses the generic cmnd/sonoff/ and stat/sonoff/ topics. Otherwise, they will all use the new topic and we will be back to square one.

In passing, there is an easy way to recover the topic sent by a modified Sonoff. Subscribe to all "stat/#" topics with the MQTT broker and click on the tactile button on the Sonoff and see the response.

3. NodeMCU

After flashing the NodeMCU firmware, the Sonoff will sit there doing nothing since the automatically executed init.lua script is not present. It is possible to ensure that the Lua interpreter is functionning correctly. Assuming that miniterm.py is connected, the interprenter prompt, '>', should be visible, albeit after pressing Enter a few times if necessary.

What, 4/3 is equal to 1? Of course it is... in integer arithmetic.

We need a Lua script to deal with the Sonoff. Luckily, there is one by Michael Andresen at captain-slow.dk. I copied it, and modified ever so slightly to make it easier to setup the local area network parameters. You can download it by clicking on sonoff_01.lua.

Adjust the parameters at the beginning of the file

Copy the modified script to the Raspberry Pi and use nodemcu-uploader to upload (what else?) the script to the Sonoff.

Before executing the script, you should subscribe to all MQTT topics in another terminal.

While nodemcu-uploader can launch the uploaded script (replace upload with exec) it is more useful to open a serial session with the Sonoff to get more information specially in case of a crash (called panic in NodeMCU).

The last two lines will take a little bit of time to appear.

Clicking on the Sonoff tactile button will cause three events: the LED will flash, the terminal subscribint to all MQTT topics will display a message

and the Sonoff will also display a message to the connected serial link

Sending an "OFF" message on the topic "/home/1/testNodeMCU" to the MQTT broker

will toggle the state of the Sonoff switch (assuming it was ON).

References

Two articles on the flashing and programming an ESP8266 with a Raspberry Pi served as my starting point. Here are the references: Connect ESP8266 to Raspberry Pi et Use ESP8266 module as a wireless switcher de Razvan Dubau.

The page Identifying Your Model of Raspberry Pi explains how to identify the Raspberry Pi model. See the section named Identifying Your Board Once Booted.

As for the UART port of the Raspberry Pi, I consulted Configuring The GPIO Serial Port On Raspbian Jessie Including Pi 3 de Jon Watkin.