Workflow when Working with a Headless Raspberry Pi
June 3, 2017
<-Domoticz on a Raspberry Pi - Déjà Vu All Over Again MQTT with Domoticz->
X10 On/Off Switches with a CM11A Interface in Domoticz->

The Raspberry Pi does not provide the most comfortable working environment. Friendly programs with graphical user interfaces are not available in the light version of Raspbiam installed on the single board computer. Even if the full version of Raspbian had been installed, the Raspberry Pi would be rather slow compared to a i7 quad core desktop computer. I have developed some work habits that make the process tolerable. Some may find them useful or may find inspiration for developing their own approach.

In other words, as I began writing scripts on the Raspberry Pi, I quickly tired of using the nano editor and pined for my usual text editor. Obviously, I wanted to work mostly on the desktop computer and quickly transfer file back and forth between my main machine and the home automation server.

Table of Contents

  1. Transfer Files by SFTP
  2. Transfer Files with FileZilla
  3. Synchronizing Directories with FileZilla
  4. Edit files in FileZilla
  5. Raspberry Pi SD Card Backup
  6. Domoticz Database Backup
  7. Better Acces to the Domoticz Web Site

  1. Transfer Files by SFTP
  2. Since the Raspberry Pi is running an SSH server, files can be transferred back and forth using the SSH file transfer protocol (SFTP). And there is no change to be made to the Raspberry Pi. Its all on the desktop computer.

    You can do SFTP transfers from a terminal window. The following will download all the Domoticz Lua scripts to the Documents directory in my account

    michel@hp:~$ sftp pi@ pi@'s password: remember nothing is shown not even '*'s Connected to sftp> get domoticz/scripts/lua/*lua Documents Fetching /home/pi/domoticz/scripts/lua/JSON.lua to Documents/JSON.lua /home/pi/domoticz/scripts/lua/JSON.lua 100% 49KB 49.4KB/s 00:00 ... sftp>quit
    Its all very old school FTP and not particularly friendly.

  3. Transfer Files with FileZilla
  4. I prefer to use FileZilla which can be installed on Linux, Mac OS X and Windows. It was a snap to install on my Linux desktop using Ubuntu Software Center

    Once it is installed, start FileZilla and open the Site Manager in the File menu. Click on the New Site button and then give the site (the Raspberry Pi) a meaningful name. Go on to fill Host, User and Password fields the values used to log into an SSH session on the Raspberry Pi. Set the Protocol to SFTP. Finally click on the Ok button to save the settings.

    From then on, to start a session, go back to the Site Manager, make sure to select the correct entry from the list on the left, and then click on the Connect button.

    On my desktop computer, there is an exact copy of the pi user directory on the Raspberry Pi. When I need to add a script or modify one, I do it on the desktop using my preferred text editor, and then I copy the new or modified files back to the Raspberry Pi.

    To upload a file from the desktop to the Raspberry Pi, you just have to double click on its name in the left hand side listing in FileZilla. The file will be copied to the directory shown on the right side.

    The procedure is the same to download a file from Raspberry Pi to the desktop: double click on its name in the right hand side listing in FileZilla. The file will be copied to the directory shown on the left side.

    Luckily, FileZilla makes it easy to synchronize the files on the desktop (local computer) and the Raspberry Pi (remote computer). Go to the Site Manager, click on the Advanced tab and fill in the default directories.

    If you check Use synchronized browsing, it will be easier to navigate both directory trees:
    If your FielZilla freezes, uncheck that option and look into updating to the latest version.

  5. Synchronizing Directories with FileZilla
  6. If you add or edit numerous files then it is easier to see the differences in the content of the local and remote directories by enabling Directory comparison in the View menu.

    The file listings will be colour coded. A yellow background identifies a file missing from the other directory, while a dusty rose background identifies a file that is of a different size from the similarly named file in the other directory. A green background identifies a file that has a different modification date compared to a file with the same name in the other directory. The clear or white background is reserved for files deemed the same in both directories. Those files can be hidden by checking the Hide identical files option in the menu. Files with the same name found in both the local and remote directory are deemed different if their size is different or if their modification time is different. Only one of the two criteria can be applied at any one time.

    Some care must be used if using file size as the criterion. If the only change in a file is in a IP address "" instead of "" then the files will not show up as different.

    Using file date criterion is not foolproof either. There may be false positives as it were, where an unchanged file was saved on top of itself which changed the notification date. Of course that is not too dramatic; it may cause you to copy unnecessarily a file but since they are identical there would be no bad consequence.

    More consequential is the fact that, by default Filezilla considers the files equal if the difference in their notification date is less than one minute. This is a problem because you can easily make a minor change in a file and save it within a minute. Also you may get a lot of false positives if the desktop computer has a different file system. Time stamps could be rounded to different number of seconds or fractions of seconds. Other file managers use a much smaller threshold: 2 seconds in one case at least. While the threshold can be increased, (Edit/Preferences/Interface/File lsits/Directory comparison), it cannot be set at a fraction of a minute in Filezilla.

    I guess the moral of the story is to be careful when using the directory comparison function to synchronize directories in Filezilla.

  7. Edit Files in FileZilla
  8. Sometimes, I need to make only small changes to a script. In that case I can edit the file on the Raspberry Pi by clicking with the right mouse button on the file name in the right hand listing. Then I select View/Edit in the pop up menu. FileZilla will then transparently download the file to my desktop, open my default editor with that file. When editing is done, the file need to be saved without changing its name! FileZilla will check and if it finds that the saved file is different from the original, it will offer to copy the revised version on top of the original file on the Raspberry Pi.

    If you do that, and if you keep a copy of the files on your desktop computer then at some point in the future you should download the modified file back to the desktop. Otherwise, the copied and original directories will get out of sync.

  9. Raspberry Pi SD Card Backup
  10. Given how long it has taken to get here, especially with the older Raspberry Pi, it may be a good idea to backup the SD card. That means copying the content of the SD card to a file image on your desktop computer. In fact, it amounts to performing the initial Debian image installation process in the opposite direction. Unfortunately, ETCHER does not work backwards, so you will have to resort to the traditional applications for copying to and from an SD card.

    The specific instructions for installing the Debian image to an SD card with a Linux desktop also explain how to do the reverse. As I remember it was easy to use the Win32DiskImager utility to do the same sort of thing in Windows. It looks like the instructions for Mac OSX machine are quite similar to those for Linux. Here are the series of commands used on Linux:

    michel@hp:~$ df -h ... identify the physical devices containing the boot partition (fat) ... and the operating system (ext) partition, in my case, sde1 and sde2. michel@hp:~$ sudo umount /dev/sde1 michel@hp:~$ sudo umount /dev/sde2 michel@hp:~$ cd Downloads/raspberrypi michel@hp:~/Downloads/raspberrypi$ sudo dd bs=4M if=/dev/sde of=backup.img

  11. Domoticz Database Backup
  12. Saving the full SD card is time consuming and even involves sneaker net. I do try to minimize my movements out of a deep concern for the environment: excessive oxygen consumption, release of gaz, noise polution, wear and tear to the stairs and flooring, that type of thing. So a doing a backup of the Domoticz database after significant changes to devices, timers, scenes, etc. is good practice and it can be done from the comfort of the desktop. Here are the (virtual) steps involved:

    1. Log on to the Domoticz web interface.
    2. Click on the Setup tab.
    3. Choose Settings in the drop down menu.
    4. Click on the Backup/Restore tab.
    5. Click on the Backup Database button.
    6. Select Save File or something equivalent depending on your web browser and system settings.

  13. Better Acces to the Domoticz Web Site
  14. Adding new devices, timers, scenes and so on or editing the same is done in the Web site run by the Domoticz server on the Raspberry Pi. I have found that if I just open a tab to log onto that Web site, I eventually lose it among all the open tabs in the numerous browser windows I have or I accidently close it. It is useful to create a launch bar icon to open the browser at the correct page. When I have lost the window, I then just click on the icon and it comes to the forefront.

    The instructions for doing this with Firefox on Ubuntu that are available here: A Domoticz application in Linux. I imagine something similar can be done Firefox on Windows. Something similar can be done with Chrome on Android: Domoticz with Android.

<-Domoticz on a Raspberry Pi - Déjà Vu All Over Again MQTT with Domoticz->
X10 On/Off Switches with a CM11A Interface in Domoticz->