2021-08-10
md
E-mail Notification of a Changed or Lost Public IP Address

Every day, I get e-mails that look like this most of the time:

Subject: Information from domohost.local Message: Public IP Addresse (no change) : 171.45.8.239 IP Address of DDNS Host (modomo.twilightparadox.com) : 171.45.8.239

That message says that my local area network (where machine domohost runs) can be reached from outside using IP address 171.45.8.239 or the domain name modomo.twilightparadox.com. The home automation sever runs the script once a day at around four in the morning so that I will find the result in my mailbox when I start the day. A remote system used for offsite backups sends a similar message also once a day at about the same time. Finally, my desktop machine, also on my local area network, executes the same script as domohost during the middle of the day, but it sends e-mail notifications only when there has been a change.

When the IP addresses do not match, the message constitutes a warning that the dynamic domain name service has been disrupted and it may be necessary to rectify the situation. But even if the IP addresses are the same, I find such messages comforting when away from home because they implicitly indicate that host system for the home automation system is functioning.

This post presents the Python script that sends these e-mail notifications. However before getting to the script, there is an overview of the dynamic domain name system. That section is followed by a discussion of a couple of failures encountered in the past which prompted me to create the script. I often provide context in that way, but understandably, many will want to skip ahead directly to the script.

All IP addresses and dynamic host names shown in this post have fictitious values which I plucked out of the air. I am very sorry if any such value happens to correspond to your public IP address or dynamic host name. Please accept my apologies if someone starts poking at your site in the hope of accessing my top secret system.

Table of Contents

  1. Private Local Area Networks
  2. Accessing the Wide Area Network (Internet)
  3. Accessing a Local Area Network from Outside
  4. Dynamic Domain Name Service
  5. DDNS Failures
  6. E-mail Notification
  7. Final Thoughts and References

Private Local Area Networks toc

To start, let's look at a simplified representation of my LAN and its connection to the Wide Area Network (WAN) or Internet (which is basically a collection of networks and many backbones, bridges, gateways, routers, switches, name servers and so on to stitch everything together). Click on the image to see it full sized.

Every device connected to my home network has a unique IP address used to route data (divided into chunks called packets or datagrams) to and from the device to other devices on the LAN. Along with hundreds of thousands if not millions of others, I use the 192.168.1.0/24 subnet of the private 192.168.0.0/16 network which means that all devices on the LAN have an IP address in the range 192.168.1.1 to 192.168.1.254 (addresses 192.168.1.0 and 192.168.1.255 are reserved for special purposes). The fact that my neighbour also has a desktop with an IP address of 192.168.1.101 causes no problems since our two LANs are not physically connected.

Only a small portion of the data traversing my LAN is strictly for internal purposes such as when a Web browser on the 192.168.1.135 desktop is used to look at the Web interface of the home automation system at 192.168.1.22. Most data transfers is with outside machines when exchanging e-mails, participating in video conferences, viewing video streams or listening to audio streams and so on.

Accessing the Wide Area Network (Internet) toc

Things can get a bit more complicated when accessing machines on the wide area network. Just imagine if my neighbour and I both ask the same search engine for some information using distinct desktops that happen to have the same IP address, say 192.168.1.101. How can the reply from the search engine get to the correct machine then? An important part of the answer is that the search engine did not get two requests from 192.168.1.101. Instead it got a request from 171.45.8.239, which is the public IP address assigned by my ISP to the whole of my home network and another request from 95.23.9.8, which is my neighbours public IP address assigned to his home network by his ISP. Those two addresses, 171.45.8.239 and 95.23.9.8 are unique and not found anywhere else in the whole of the Internet, so the reply will get to the correct LAN.

Of course we have run into a problem: how does the reply from the search engine get to the correct computer once it has reached the LAN? The IP addressing system is actually more complicated than just an IP address. Data is actually transfered between "end points" which are really processes running in computers. End points are identified by the IP address of the computer and a "port" number. For example, the Web interface of my home automation system is at 192.168.1.22:8080, while the default Web server on the same Raspberry Pi is at 192.168.1.22:80. So what the router/gateway does is translate private IP addresses and port numbers to public IP addresses and port numbers on outgoing data and the opposite with data coming in from the outside. We are on the brink of falling into the complicated network address translation (NAT) hole and that is outside the purview of this post. Let's stick to IP translation and trust that the rest gets done without problems.

Accessing a Local Area Networks from Outside toc

As long as I am initiating data exchanges from within my LAN to the WAN, things are very simple from my point of view, because NAT takes care of the details in a completely transparent way. But what happens when I want to reach my home automation system on my LAN which has the private IP address 192.168.1.22 from a hotel room with Wi-Fi? See the image on the right and click on it to see it full sized. Remember, private IP address cannot be routed through the public Internet (source). In other words, the 192.168.1.22:8080 URL will go nowhere if used in a web browser on the portable computer. However, if I know the public IP address of the LAN, then I can reach the home automation sever. In other words, the URL 171.45.8.239:8080 works on the portable computer.

As simple minded as this sounds, I have used this technique for testing purposes. It turns out that my ISP grants rather long DHCP lease periods. The lease is the length of time that an IP address assigned to a client is reserved for that client even when the connection is broken. More than once, it was possible to test some aspect of remote access to the LAN by getting the assigned public IP address from the router/gateway and then travelling about 15 km to the nearest coffee shop to log into my home network from the WAN using that address. The long DHCP lease time is a double-edged sword as will be seen later.

Dyanmic Domain Name Service toc

The next level of sophistication is to send e-mails at regular intervals with the public IP address. This is one function of the script shown below. I have never relied on that technique, but it has been implemented as a backup for about three years now. It is just a DIY version of the technique used by many applications such as Zerotier, Teamviewer or even MyDomoticz that provide remote access or control of devices (this is not a recommendation of any of these applications). Both devices being connected by such applications call back to the "mothership" when launched, which means that it knows their public IP addresses and can pass them back so that the devices can then exchange data directly with each other. Basically, the Python script deposits the IP address in an easily accessible place on the Internet, an email account, so that I know where to find that address if it is needed to access the home network.

There is a better approach to keeping track of a LAN's public IP address that is transparent, works most of the time, and is widely used. Indeed it is popular enough that the "free" router/gateway provided by my ISP supports it... more or less. I am talking about the dynamic domain name service (DDNS). We are all users of the Internet domain name service (DNS). It's what makes possible using host names such as www.google.com or www.duckduckgo.com instead of their IP address 142.251.32.100 52.149.246.39 to reach the respective search engines in a web browser. DDNS is basically the same as DNS except that the IP address associated with a host name is not necessarily fixed. How does this work from the user point of view? I simply use the modomo.twilightparadox.com:8080 URL on my portable computer in that lonely hotel room and automagically I get to see the Web page of my home automation system.

Here is what happens behind the scenes, again simplifying as much as possible. The browser must resolve the host name modomo.twilightparadox.com to an IP address. That is done by querying the domain name server system 1. Exactly which DNS server is first queried is not that important. If it knows the IP address it returns the address, if not, it passes the query on to another server. These recursive queries should eventually succeed and the found IP address is handed back up the chain until the browser gets the public IP address of my network 2. Now IP data can be exchanged between the two devices at 10.10.5.205:80 and 171.45.8.239:8080 3 without further involvement with the domain name system. Of course, NAT will be performed in both the home lan gateway and the hotel LAN gateway.

The preceding is outrageously simplified, especially when it comes to part 3. As shown above, IP datagrams with a destination end point of 171.45.8.239:8080 arrive at the home LAN gateway, they will simply be dropped. For access to the home automation server to work as described, it would be necessary to "port forward" TCP datagrams for inbound port 8080 to end point 192.168.1.22:8080. One could certainly do that for a short time to test things, but before punching permanent holes in the LAN firewall, security must be carefully considered. One should never allow insecure HTTP traffic into the LAN. At a minimum, the HTTPS protocol with user name and password protection should be used. Using SSL/TLS certificates would be better. In practice, I entrust security to a virtual private network (VPN). This is getting us away from the main subject of this post. Look at Google Home Mini with Domoticz using IFTTT. While I do not use Google Home and IFTTT anymore, I believe that the discussion and practical information on HTTPS, TLS/SSL encryption, port forwarding and dynamic domain names remain pertinent. As for the VPN, I use the ever more popular Wireguard (see Installing and Configuring WireGuard on Raspberry Pi OS).

Ultimately, who knows the IP address of modomo.twilightparadox.com? It's my dynamic domain name service provider as shown with the nslookup utility that is part of the dnsutils package.

michel@bomv:~$ nslookup -type=ns modomo.twilightparadox.com Server: 8.8.8.8 Address: 8.8.8.8#53 Non-authoritative answer: *** Can't find modomo.twilightparadox.com : No answer Authoritative answers can be found from: twilightparadox.com origin = ns3.afraid.org mail addr = dnsadmin.afraid.org serial = 2108070065 refresh = 86400 retry = 7200 expire = 2419200 minimum = 3600

How does FreeDNS (i.e. afraid.org) know the public IP of my home address? I told it what it was when I signed up for the dynamic domain name service and I tell it the new IP address whenever my ISP changes the public address of my home network. Technically, I don't manually do that, my router/gateway does it. As for the domain name itself, it was made available by FreeDNS which owns the twilightparadox.com domain among others. All I had to do was to come up with the subdomain modomo to ensure that modomo.twilightparadox.com was unique. That is one of the best deals on the Web because it is all free. Apparently, individuals or organizations that need premium DNS services are subsidizing free riders such as myself and I thank them very much for their help.

What can go wrong with this system? I have run into two problems with a common cause.

DDNS Failures toc

The ISP can change the public IP address of my LAN whenever it wants. That IP address is assigned with the usual DHCP protocol which allows a server to order its clients to reconnect whenever it wants to. When they do that, the server can assign a different IP address. Whenever the public IP address is changed and FreeDNS is informed of the change, it can take up to an hour before DNS caches are flushed and the new IP address is propagated. So during that one hour (maximum), my home LAN cannot be reached with the modomo.twilightparadox.com domain name which may be resolved to the old IP address.

Of course, that maximum one-hour delay before the domain name system is updated assumes that my router/gateway notifies FreeDNS of the change in its public IP address immediately. To be honest, I am not sure that this is done correctly. There is no real documentation about setting up the DDNS update service on the router. All there is really is a few fields to fill in that have rather cryptic names. It is also very difficult to test because of the lengthy DHCP lease period set by my ISP. I have disconnected my router/gateway for over two hours hoping to be assigned a new IP address when reconnecting (and hence test DDNS updating by the router/gateway), but unfortunately the same old IP address was assigned. Since I am not the only person using the Internet in the household, it is a difficult to suspend all access to the outside for longer periods.

One way around this problem is to forego the DDNS update support built into the router. I could use any of the scripts found at FreeDNS to update the public IP address at regular intervals. Indeed, I do that with another account running on a different LAN with an old router that does not support DDNS, and it seems to work well. Of course, my stubborn streak means that I will not do this because the built-in support "should work".

The other problem I have run into seems to be related to FreeDNS. Apparently, if there is "no activity", the DDNS service provider suspends the account. I am not too sure just was constitutes activity. Is a request for the IP address of a dynamic domain name enough? Is it sufficient to perform an update of the public IP address associated with a dynamic domain name (even if the IP has not changed) to restart the timer? Or is it necessary to log into the account at freedns.afraid.org every six months or whatever? During the exceptional circumstances, we have experienced during the Covid19 pandemic, I did not access the LAN from outside during more than a year. At the same time, the ISP did not change the public IP address of the LAN during that period as far as I know. It looked like the router/gateway did not update the public IP address at all during this period (because if IP address did not change or have yet to enable the update service on the router correctly) and I know I did not log into my account at FreeDNS. The end result was that at one point I could not access the LAN when trying to test a new Wireguard server installed on it. Thankfully, reactivating the service only required logging into the account.

The Python script described in the next section should provide notification of either type of problem.

E-mail Notification toc

The Python script starts by getting three IP addresses:

If the script was able to get the publicIP address then it immediately saves it to a disk file for retrieval the next time the script is run. The script then creates the body of the e-mail message reporting the current public IP address and the IP address mapped to the LAN dynamic domain name. The next step is to create the message subject appending an obvious "*** ERROR ***" string if an IP address could not be found or if they do not match. The last step is to send the e-mail.

#!/usr/bin/python3 # coding: utf-8 ## Python script to send an e-mail with the public IP address of the ## host on which the script is executed and the IP address of a dynamic ## domain name ## See E-mail Notification of a Changed or Lost Public IP Address ## https://sigmdel.ca/michel/program/python/public_ipv4_en.html # Author: Michel Deslierres # Version: 2021-08-06 # Licence: BSD Zero Clause # SPDX-License-Identifier: 0BSD ###################################### ### Adjust the following constants ### # File holding the previous public IP address IPFILE = '/home/XXXXXXXXX/.local/bin/oldIP' # Dynamic host name DDNSNAME = 'modomo.twilightparadox.com' # E-mail stuff SMPT = 'smpt.amail.com' # smpt sever of sender PORT = 465 # smpt port SRC = 'me@amail.com' # smpt user PWD = 'xxxxxxxxxxxxxxxxxx' # smpt password TGT = 'me@omail.com' # destination of e-mail # E-mail strings - translate these as needed HOST = 'myhost.local' # host name of device running the script # e-mail title OBJ = 'Information from {}' # title will be OBJ + HOST ERR = ' (*** ERROR ***)' # + ERR if there is an error WARN = ' (Warning)' # + WARN if no error but a get IP site failed # public IP string MSG = 'Public IP Address' # will be MSG + (NEWIP|OLDIP) + publicIP NEWIP = ' (new): ' # if found or MSG + NOIP if not found OLDIP = ' (unchanged): ' NOIP = ' not found' BADIPSITE = '\nFailed to return the public IP address:' # DDNS IP string DDNSIP = "IP Address of DDNS Host ({}) : " # resolved DDNS host name will be NODDNS = 'not resolved' # DDNSIP + DDSNAME + hostIP if ok # or DDNSIP + DDSNAME + NODDNS if not resolved # Send e-mail flag # If True, # the IP addresses are sent in an e-mail each time the script is run # If False, # the IP addresses are sent only if there is a change or an error SEND_ALWAYS = False # List of web sites that return the IP address of a client request # In other words they return the LAN's public IP address getIpSites = ['checkip.dyndns.org', 'v4.ident.me', 'api.ipify.org', 'ipinfo.io/json'] ### No changes needed below ### ############################### # Import URL request from requests import get # Import host name resolver from socket import gethostbyname # Import smtplib for the actual sending function import smtplib, ssl # Import the email modules we'll need from email.mime.text import MIMEText # Import regular expression engine import re # Import function to select get Ip site randomly from the list from random import shuffle def sendMessage(theMsg, theSub): msg = MIMEText(theMsg) msg['Subject'] = theSub msg['From'] = SRC msg['To'] = TGT context = ssl.create_default_context() with smtplib.SMTP_SSL(SMPT, PORT, context=context) as server: server.login(SRC, PWD) server.sendmail(SRC, TGT, msg.as_string()) # Get publicIP IPv4Pattern = re.compile('\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}') headers = {'Pragma': 'no-cache', 'Cache-Control' : 'no-cache'} def getIp(url): try: hf = get('http://'+url, headers, timeout=15) ip = re.findall(IPv4Pattern, hf.text) if ip: return ip[0] else: return '' except: return '' ipSiteComments = '' shuffle(getIpSites) for site in getIpSites: publicIP = getIp(site) if publicIP: break else: ipSiteComments += '\n {}'.format(site) # Get hostIP try: hostIP = gethostbyname(DDNSNAME) except Exception: hostIP = '' # Get oldIP try: with open(IPFILE, "r+") as ipfile: oldIP = ipfile.read().rstrip() except Exception: oldIP = "" # Save current publicIP to oldIP file try: with open(IPFILE, "w") as ipfile: ipfile.write(publicIP) except Exception: pass # Do we send a message? if publicIP and publicIP == hostIP and publicIP == oldIP and not ipSiteComments and not SEND_ALWAYS: exit() # Compose message text isOK = True msg = MSG if publicIP: if publicIP == oldIP: msg += OLDIP else: msg += NEWIP msg += publicIP else: msg += NOIP isOK = False msg += "\n" + DDNSIP.format(DDNSNAME) if hostIP: msg += hostIP if hostIP != publicIP: isOK = False else: msg += NODDNS isOK = False if ipSiteComments: msg += BADIPSITE msg += ipSiteComments # Compose message subject subject = OBJ.format(HOST) if not isOK: subject += ERR elif ipSiteComments: subject += WARN # Send e-mail sendMessage(msg, subject)

Download the script using this link: checkip4.py.

The most complicated part of the script is getting the public IP address of the LAN which is also called the outward facing or wide area network IP address of the router/gateway. One would think that getting the information from the router/gateway would be the simplest thing to do. Indeed the router provided by my ISP router does show this information in its Web interface on the System Information Status page as shown on the right. Unfortunately, I cannot find the correct way to pass on the login information to retrieve that page directly. Instead I have to rely on web sites that display the IP address of HTTP clients that connect to them. I learned the hard way that such sites can disappear so the current version of the script contains a list (getIpSites) of four sites that still work (at least as of August 9, 2021). Here are the sites and the raw response obtained from each.

Obviously, the first two responses are much easier to parse, but the other two are simple enough that a regular expression is able to find all IP (version 4) addresses without problems and, thankfully, the wanted public IP address is always the first found.

Mindful that the service is provided at no cost to me, only three requests are made each day as described in the introduction. To try to balance the number of requests made, the getIPSites list is randomly scrambled before the script tries each site in turn until it finds one that does return a valid IP address. When adding multiple sites that could be queried for the public IP address, it seemed best to report when a site no longer responded in the ipSiteComments string which will be added to the e-mail message if not empty.

Hopefully the rest of the script is clear enough.

Final Thoughts and References toc

Be careful when receiving an email that says the public IP address is not the same as the IP address associated with a dynamic host name. Remember that it can take up to an hour before DNS records are updated if the Internet service provider (ISP) changed the public IP address it assigned to the local area network. Actually, it can take longer if the modem/gateway or the update script is slow to inform the dynamic name service provider that the public IP has changed. If I am desperate to reach my LAN from outside and can't wait an hour or two, I can always modify the Wireguard configuration file (in directory /etc/wireguard on a Linux machine) and change the last line.

[Interface] Address = 192.168.99.2/24 PrivateKey = gH5xInhP2NZw0t8hVgJPhTRDUh3Bir7FEynRcW8IHlg= [Peer] PublicKey = /y4PnCDdei8dIWCnCD84wOtUewzzWgLH/XY8o/p3Pxc= AllowedIPs = 192.168.99.1/32, 192.168.1.0/24 Endpoint = modomo.twilightparadox.com:53133

to

... Endpoint = 168.102.82.120:53133

where the 168.102.82.120 IP address is the public IP address found in the message. To be honest, I would rather wait an hour or so.

It was pretty cheeky on my part to present the checkip4.py script given that I can only write Python scripts with the help of a good search engine. Please forward any improvements or suggestions by clicking on the e-mail link at the bottom of this post. If you do decide to try it, do not forget to adjust all the constants near the start of the script to your particular circumstances.

I should point out that should it be impossible to obtain a valid public IP address, there's every chance that it will not be possible to send an e-mail. Some sort of error logging should be added to the script. If I recall correctly, I have done this elsewhere and it was not that difficult, but given my usual lackadaisical attitude, I will not look into this until I run into the problem. As stated, some form of this script has been running for years without ever running into a problem except for the initial reliance on a single web site to obtain the IP address.

There is no mention of IPv6 at all here for a simple reason. I avoid IPv6 as much as possible. I am just too lazy to work out what that addressing scheme changes and I'll wait until I am forced to do it.

It seems fitting to give references for bits of Python code used in the script.

Finally, the network diagrams were partly created with diagrams.net (formerly draw.io)