Domespic [4] Domoticz
December 13, 2016
Previous: Part 3 - Gateways Next: Part 5 – Extension

Install Domoticz

As the download page at Domoticz says, a one-line script will install the program.

pi@rpi2b:~ $ sudo curl -L install.domoticz.com | bash

Start a web browser on your desktop and connect to Domoticz' built-in web server on the Raspberry Pi at http://192.168.0.22:8080. You will not see much, because nothing is installed, but at least you are now assured that everything is working.

Its time to define the system settings. Click on the Setup button at the top right and then select Settings. By default the System tab is opened. At a minimum, the Location (longitude and lattitude) should be defined. I used my car's GPS to find the exact values. Perhaps a GPS equipped smart phone or tablet could be used. If none of these possibilities are available, Domoticz can provide reasonably accurate coordinates given a street address. Presumably, the information is used to calculate sunset and sunrise.

It is a good idea to password protect the website. Provide a username and a password in the Website Protection section. It does become cumbersome to fill in that information. Thankfully, it is possible to specify "login free" local networks. Using a browser on a machine located on any one of those specified networks will log in automatically, bypassing the login requirement.

Consult the Domoticz wiki page Application Settings for more information about other settings.

Consult the page dedicated to installation on the Raspberry Pi. In particular you should look at the section entitled Raspberry Pi additional software installations. You will notice that I shamelessly copied the "Setting up a RAM drive on Raspberry Pi" from that page.

Add X10/Mochad to Domoticz

My first attempt at adding Mochad was based on a Domoticz wiki page: X10 devices, CM15Pro. It certainly worked but it was not optimal. I have since added a warning in the wiki to explain that native support for the CM15A and CM19A interfaces using Mochad is available. Adding switches is now a snap that does not involve writing scripts. As a bonus, there is also real time updating of the status of devices. Unfortunately, the dimming function of lamp modules is not supported; they must be installed as on/off switches.

Before adding X10 lights, switches and other devices, the "hardware" (the Mochad bridge) needs to be installed. This is done only once. After that X10 hardware modules are added one by one. All this is done using Domoticz web interface, the same one that you can later use to turn a light on or off.

Log on to Domoticz web page at http://192.168.0.22:8080. Click on the Setup tab and then, once the setup menu is showing, click on the Hardware submenu button.
Add hardware

Fill the fields to define the hardware. Change the Type: to Mochad CM15Pro/CM19A bridge with LAN interface, give the hardware a name, Mochad seems like an obvious choice, and specify the Remote Address and Port used by the Mochad daemon. In this case, this is 192.168.0.22:1099 but instead I used the localhost address 127.0.0.1:1099. If a different Ethernet address is assigned to the Raspberry Pi, it will not be necessary to adjust the Remote Address field. Click on the Add button.
Add Mochad bridge
The added hardware should then be visible in the table at the top of the page.

Individual X10 devices can now be added. Turn the light or device off. Then click on the Switches tab and then on the Learn Light/Switch button at the top right of the page to automagically install a device. Domoticz will display a prompt saying to Press button on Remote... which, of course, must be a button that turns on the device to be added. Be quick about it, you will be given only a couple of seconds to press that button. Of course you can click on Lean Light/Switch again and retry as often as you want.

If there is no remote handy, then netcat into Mochad and prepare the command to turn on the device but wait to press the Return or Enter key until prompted to press the button on the remote:

pi@rpi2b:~ $ nc localhost 1099 rf d1 on Press Enter key when prompted

Domoticz will display a prompt asking for a unique name for the device and the type of switch it is:
Add Light/Switch Device
Leave the Switch Type set at On/Off, dimmers do not work in the Mochad bridge eventhough Mochad does handle them.

Whether the remote or a Mochad command was used, make sure the light or device was turned on. If it is, then installation is complete. Not much to it!

Debugging

If something does not work, there are a few things that can be done to pinpoint the error. First, open a new ssh session on the Raspberry Pi and issue the following commands which should connect to Mochad and toggle the dresser light on and off:

pi@rpi2b:~ $ nc localhost 1099 rf d1 on or perhaps "pl d1 on" if using a CM15A 11/26 11:26:13 Tx RF HouseUnit: D1 Func: On rf d1 off 11/26 11:26:17 Tx RF HouseUnit: D1 Func: Off

This must work. If it does not, go back to the debugging instructions concerning Mochad above.

If Mochad is working correctly, then continue investigating by turning the dresser light on and off using Domoticz. Check the Domoticz log (click on the Setup tab and then on the Log button). When things work normally you should see something like:

2016-11-26 13:23:38.892 User: Admin initiated a switch command (1/Dresser lamp/On)
2016-11-26 13:23:41.745 (Mochad) Lighting 1 (Dresser lamp)
2016-11-26 13:23:47.131 (Mochad) Lighting 1 (Dresser lamp)
2016-11-26 13:24:05.892 User: Admin initiated a switch command (1/Dresser lamp/Off)
2016-11-26 13:24:39.374 (Mochad) Lighting 1 (Dresser lamp)
2016-11-26 13:24:46.101 (Mochad) Lighting 1 (Dresser lamp)
in the log and the Mochad deamon should display
11/26 13:23:41 Tx RF HouseUnit: D1 Func: On
11/26 13:24:39 Tx RF HouseUnit: D1 Func: Off
The latter will show if the wrong X10 address was given which can happen if a netcad command was used to setup the device. It will be necessary to delete the switch and install it again but with the correct address of course. If the log is empty, then something weird is going on. Maybe deleting the switches and the Mochad bridge and then installing everything again one switch at a time could help.

Add X10/Heyu to Domoticz

Reference: Domoticz wiki: X10 with CM11a

There is no native (hardware) support for the Heyu in Domoticz so it will be necessary to add devices "manually" and to handle them with scripts.

The first step is to create a "dummy" hardware representing the gateway. Again click on the Setup tab and then select Hardware. For a name, I chose Heyu. For the Type click on Dummy (Does nothing, use for virtual switches only) in the drop down list.
Add dummy hardware
Don't forget to click on the Add button. This step is done only once. The following two steps are repeated for each device to be added.

Before proceeding with the installation of X10 modules (lamp, appliance, receptacles etc.), it is best to plan ahead. Namely, each device must be given a name in the database. This is the name that will be used in the script. Names should not be too long, especially if you will be using an intelligent telephone to log into Domoticz and interact with the device. Addresses given to virtual devices do not have to correspond to the corresponding X10 address of the physical devices, but its probably less confusing that they match.

Here is an example setup:

Device X10 address Type Name
Floor lamp B1 lamp module (dimmable) Floor lamp
North wall sconce B2 wall receptacle (on/off) North sconce
South wall sconce B3 wall receptacle (on/off) South sconce
Table lamp B4 lamp module (dimmable) Table lamp

Since the Heyu hardware is virtual, those four X10 devices will be controlled with virtual sensors. Click on the Setup tab, then select Hardware and then click on Create Virtual Sensors in Heyu hardware.
Add virtual sensors

The Create Virtual Sensor window, as shown below, will be displayed. Fill in the Name field and press the OK button.
Create virtual sensor

Create other virtual switches for the other X10 devices. Once that is done, click on the Switches tab. The newly created On/Off virtual switches will be visible, Click on the Edit button in the Floor lamp box to change its switch type to Dimmer.

A device editing page will be shown. As you can see there are a number of changes that can be made including changing the name of the device and its switch type.

Set Switch Type: to Dimmer and press the Save button. Return to the Switches tab sheet by pressing on the Back and change the switch type of any other virtual switch corresponding to a dimmable X10 device.

The result is something like this:

This is very nice looking, but clicking on an icon to toggle a lamp or sconce off or on or moving the slider to change the light intensity of either lamp will do nothing. These are virtual switches, scripts that invoke Heyu have to be written to do something. First I will describe the scripts for the sconces, dimmers are somewhat more complicated.

Scripting on/off switches

Rules about Lua scripts are quite strict. A Lua script for a virtual sensor

pi@rpi2b:~ $ cd domoticz/scripts/lua pi@rpi2b:~/domoticz/scripts/lua $ nano script_device_heyu_b2.lua
commandArray = {} NAME = 'North sconce' ADDR = 'B2' VERBOSE = 0 -- 0 = quiet, 1 = logs os.execute commands, 2 = debug if devicechanged[NAME] then cmd = string.lower(devicechanged[NAME]) if (VERBOSE > 1) then print('Device: ' .. NAME) print('Change: ' .. cmd) end if (cmd == 'on') or (cmd == 'off') then cmd = 'sudo -u pi /usr/local/bin/heyu '.. cmd .. ' ' .. ADDR os.execute(cmd) if (VERBOSE > 0) then print(cmd) end end end return commandArray
(Download script_device_heyu_b2.lua)

There are numerous lines that merely add information in the Domoticz log to help in debugging. They may hide how simple the script actually is. Here it is stripped of that information:

commandArray = {} NAME = 'North sconce' ADDR = 'B2' if devicechanged[NAME] then cmd = string.lower(devicechanged[NAME]) if (cmd == 'on') or (cmd == 'off') then os.execute('sudo -u pi /usr/local/bin/heyu '.. cmd .. ' ' .. ADDR) end end return commandArray

Of course, the status of the lights is updated when a light is turned off or on with the web interface. However, I have a couple of old Radio Shack (RS) programmable controllers and some wireless controllers that I use to toggle the lights at set times or to manually turn then on or off. If I do that, the corresponding status will not be updated in Domoticz, which is a problem.

Fortunately, Heyu has a scripting mechanism that can be used to notify Domoticz of a change in status of a module. This is what should happen in principle:

In practice care must be exercised to avoid creating a loop because in that last step Domoticz does not just update its database, it also performs the associated action. In other words it executes the script script_device_heyu_b2.lua which can restart the sequence. This problem is discussed at length in the forums and solutions have been suggested. The solution I have adopted is specific to Heyu using its own script trigger conditions.

The index number (idx) of the virtual switches must be used in the Heyu scripts. Those can be found on the Devices list found in the Setup sheet.
Devices idx numbers

The next step is to edit the Heyu configuration file to

Open a terminal and launch an ssh session with the Raspberry Pi. Then edit the x10config and restart Heyu.

pi@rpi2b:~ $ nano .heyu/x10config
# Start the Heyu Engine daemon automatically (needed to execute scripts) START_ENGINE AUTO # Note that the log file will continue to grow. Manually delete # or trim it from time to time, or configure a Unix utility like # 'logrotate' to manage this task automatically. #LOG_DIR NONE LOG_DIR /home/pi # Scripts to update Domoticz devices on receiving power line message # SCRIPT B2 on rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=13&nvalue=1" SCRIPT B2 off rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=13&nvalue=0" SCRIPT B3 on rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=14&nvalue=1" SCRIPT B3 off rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=14&nvalue=0"
pi@rpi2b:~ $ heyu stop pi@rpi2b:~ $ heyu start starting heyu_relay starting heyu_engine pi@rpi2b:~ $
(Download x10config)

Test everything, turn the sconce on and off with Domoticz and with a manual controller. It may take a while, but when doing the latter, the bulb icon should reflect the state of the sconce.

If you open the Heyu monitor on the Raspeberry Pi, you will see what happens.

pi@rpi2b:~ $ heyu monitor 12/03 13:37:39 Monitor started North sconce turned on with Domoticz: 12/03 13:37:50 sndc addr unit 2 : hu B2 (_no_alias_) 12/03 13:37:51 sndc func On : hc B North sconce turned on with a manual controller: pi@rpi2b:~ $ heyu monitor 12/03 13:46:59 Monitor started 12/03 13:47:28 rcvi addr unit 2 : hu B2 (_no_alias_) 12/03 13:47:29 rcvi func On : hc B Script B2 on rcvi... executed 12/03 13:47:29 sndc addr unit 2 : hu B2 (_no_alias_) 12/03 13:47:30 sndc func On : hc B

The Heyu log file, if enabled, will confirm that in the second case the script was launched in between the two power line commands issued to the CM11A. And that explains the second power line command as Heyu responded to Domestics action. Note how Heyu distinguishes between power line transmissions received by the CM11A rcvi and commands it sends on to the CM11A for transmission over the power lines sndc. Including rcvi in the script trigger ("B2 on rcvi" ::) a loop is avoided.

Scripting dimmer switches

Scripts for dimmer switches are a little bit more complex. For one thing, it is necessary to keep track of the bulb intensity in a user variable. This is because the slider control manipulated by the user in the web interface sets the brightness level wanted, but the CM11A and Heyu dim and bright commands are decrements or increments in the level. So the first step is to create two user variables. Click on the Setup tab, chose More Options and then User variables.
Add variables

The variable Bright_Level_B1 is already defined. Another variable for the other dimmer switch is about to be added. Its name Bright_Level_B4, is entered in the Variable name: field, Integer is selected for the Variable type, there remains to click on the Add button.

If you want to use different variable names, you will have to adjust the Lua scripts. It would be wise to end the name with the X10 address of the switches as this simplifies the script.

The Lua script for the floor lamp, script_device_heyu_b1.lua dimmer switches is

commandArray = {} NAME = 'Floor lamp' ADDR = 'B1' VERBOSE = 0 -- 0 = quiet, 1 = logs os.execute commands, 2 = debugs if devicechanged[NAME] then cmd = string.lower(string.sub(devicechanged[NAME], 1, 9)) if (VERBOSE > 1) then print('NAME: ' .. NAME) print('devicechanged[NAME]: '..devicechanged[NAME]) print('otherdevices_svalue[NAME]: ' .. otherdevices_svalues[NAME]) print('cmd: ' .. cmd) end if (cmd == 'on') or (cmd == 'off') then cmd = 'sudo -u pi /usr/local/bin/heyu ' .. cmd .. " " .. ADDR os.execute(cmd) if (VERBOSE > 0) then print(cmd) end elseif (cmd == 'set level') then name_level_var = 'Bright_Level_'..ADDR wanted_level = math.ceil(tonumber(otherdevices_svalues[NAME]) * 0.22) current_level = math.ceil(uservariables[name_level_var] * 0.22) commandArray['Variable:'..name_level_var] = otherdevices_svalues[NAME] delta = wanted_level - current_level if (VERBOSE > 1) then print('otherdevices_svalues[NAME]: '..otherdevices_svalues[NAME]) print('name of internal variable (name_level_var): ' .. name_level_var) print('wanted heyu dim level (0..22): ' .. wanted_level) print('current heyu dim level (0..22): ' .. current_level) print('delta (-22..22): ' .. delta) end if delta > 0 then action = 'bright ' elseif delta < 0 then action = 'dim ' delta = -delta end if delta ~= 0 then cmd = 'sudo -u pi /usr/local/bin/heyu '..action.." "..ADDR.." ".. delta os.execute(cmd) if (VERBOSE > 0) then print(cmd) end end end end return commandArray
(Download script_device_heyu_b1.lua)

Again, all the debugging code obfuscates the simple logic. Here is a version without that extra baggage:

commandArray = {} NAME = 'Lampe sur pied' ADDR = 'B1' if devicechanged[NAME] then cmd = string.lower(string.sub(devicechanged[NAME], 1, 9)) if (cmd == 'on') or (cmd == 'off') then os.execute('sudo -u pi /usr/local/bin/heyu ' .. cmd .. " " .. ADDR) elseif (cmd == 'set level') then name_level_var = 'Bright_Level_'..ADDR wanted_level = math.ceil(tonumber(otherdevices_svalues[NAME]) * 0.22) current_level = math.ceil(uservariables[name_level_var] * 0.22) commandArray['Variable:'..name_level_var] = otherdevices_svalues[NAME] delta = wanted_level - current_level if delta > 0 then action = 'bright ' elseif delta < 0 then action = 'dim ' delta = -delta end if delta ~= 0 then os.execute('sudo -u pi /usr/local/bin/heyu '..action.." "..ADDR.." ".. delta) end end end return commandArray

The string parameter otherdevices_svalues[NAME] is the light level chosen with the slider control. The string is converted to an integer, in the range 0 to 100, with the function tonumber. I am not too sure about the details but reading the Heyu documentation there seems to be 22 steps between full off and full on for X10 lamp modules. So the percentage level is converted to an integer between 0 and 22 and stored in wanted_level. The same interpolation is done to convert the saved percentage saved in Bright_Level_xx to a 0 to 22 level. That value is assigned to variable current_level. The current value of the user variable is no longer needed so it is updated immediately to the value of the slider control. That's the meaning of the arcane looking line commandArray['Variable:'..name_level_var] = otherdevices_svalues[NAME]. The difference between the wanted and current levels is stored in delta. If it is 0 nothing will happen. If delta is positive, a bright command will be sent to Heyu. If delta is negative, a dim command will be sent with the absolute value of delta.

As before, copy the file script_device_heyu_b1.lua to script_device_heyu_b4.lua and adjust the NAME and ADDR assigns for the table lamp.

We now need to write two bash scripts that Heyu will call to handle the dim and bright events.

pi@rpi2b:~ $ nano .heyu/dim_B1.sh
#!/bin/bash # Script to update X10 dimmer module status in Domoticz when the level has been changed manually # # Adjust the value assigned to IDX to reflect the module idx number in Domoticz # Adjust the value assigned to ADDR to reflect the module house and device code # IDX=8 ADDR=B1 # get the module's current dim level LEVEL=`sudo -u pi heyu dimlevel $ADDR` # debugging messages #echo 'LEVEL = ' $LEVEL >> debug_dim.log & #curl "http://192.168.0.22:8080/json.htm?type=command&param=addlogmessage&message=Updating%20B1%20to%20level%20$LEVEL" ## These will switch the lamp off and on & set the dim level at the same time probably not the way to go ##curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=8&nvalue=0&svalue=$LEVEL" ##curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=8&nvalue=1&svalue=$LEVEL" # Update the Domoticz user variable holding the dim level. This MUST be done before updating the device to avoid action curl "http://192.168.0.22:8080/json.htm?type=command&param=updateuservariable&vname=Bright_Level_$ADDR&vtype=0&vvalue=$LEVEL" # This will update the slider (svalue) of the Domoticz device, but when the lua script is called, there will be # no action, the calculated change in dim level, delta, will be 0. curl "http://192.168.0.22:8080/json.htm?type=command&param=switchlight&idx=$IDX&switchcmd=Set%20Level&level=$LEVEL"
(Download dims_B1.sh)

This is a three line script:

It is important to do these steps in order. That is the way to avoid an unwanted loop that will cause the slider (and lamp) will be more or less quickly go to full on or full off whenever it is changed.

Copy this script to create another named dim_B4.sh and edit it, setting the correct values for NAME and ADDR. Make sure that the two scripts are executable:

pi@rpi2b:~ $ sudo chmod +x .heyu/dim_B1.sh pi@rpi2b:~ $ sudo chmod +x .heyu/dim_B4.sh

The last step is to add scripts for the two modules in the Heyu configuration file:

pi@rpi2b:~ $ nano .heyu/x10config
... # B1 and B4 are dimmer switches: do not set the device's svalue # (which is the dim level) when updating on/off status # SCRIPT B1 on rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=12&nvalue=1" SCRIPT B1 off rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=12&nvalue=0" SCRIPT B1 dim rcvi :: /home/pi/.heyu/dim_B1.sh SCRIPT B1 bright rcvi :: /home/pi/.heyu/dim_B1.sh SCRIPT B4 on rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=15&nvalue=1" SCRIPT B4 off rcvi :: curl "http://192.168.0.22:8080/json.htm?type=command&param=udevice&idx=15&nvalue=0" SCRIPT B4 dim rcvi :: /home/pi/.heyu/dim_B4.sh SCRIPT B4 bright rcvi :: /home/pi/.heyu/dim_B4.sh
(Download x10config)

The Bash and Lua scripts as well as the Heyu configuration file are available for download.

Previous: Part 3 - Gateways Next: Part 5 – Extension