Measuring home energy consumption with the PZEM004T and ESP8266

First of all a very BIG WARNING: This project works with AC mains current, which, where I live, is 220V AC, meaning that extra precautions must be taken, since risk of serious injuries and/or death is possible.

The PZEM004T
The Peacefair PZEM004T device (available at the usual far east shopping web sites) is a device that can measure energy consumption by monitoring a live AC mains wire using an inductor as the measuring sensor. Take notice there are currently at least (3/2020) two versions: V2.0 which uses the standard serial protocol, and V3.0 that uses Modbus. This project uses V2.0.

One of the wires that carries the current (normally the AC power phase) goes through the inductor so that the current that flows through it can be measured and hence the other measurements, including power consumption, can be also measured.

The PZEM004T can be bought with two types of inductor, one that opens up and can clip on the wire of interest, and the other type that requires to disconnect and connect the wire of interest, so that it passes through the inductor core. I’ve chosen the former, since in this case I do not need to do any disconnection/connections on the electric mains board, and so it is way safer and easier to add and remove the measurement device.

PZEM 004T

The PZEMM04T outputs the collected data through an opto-coupled isolated serial port that allows to retrieve values for voltage, current/intensity, current power consumption and energy accumulated consumption.

The device that connects to the PZEM serial port must provide power to it (5V), and so the serial port data lines are 5V level, which means that we should use a 5v to 3.3V level converter to connect to the ESP8266. While there are several hacks to make the PZEM004T serial port to use 3.3V on the serial port, and hence have 3.3V data lines, I just used a simple level converter to connect the serial port to the ESP8266, and avoid in this way any modifications to the PZEM-004T. The serial port connector is a 4 Pin JST-XH connector.

So the basic schematics for using the PZEM004T is as simple as the following highly professionally drawn schematic shows:

PZEM004T And Wemos D1 connection schematic

Two things of notice:

  • The Ground connection – The serial port uses the same ground as the Wemos D1.
  • The Power supply – Wemos D1 is powered through the 5V pin, NOT through the 3.3v pin, since we need 5V to power up the PZEM serial port.

The level converter is just a simple, cheap I2C level converter, used in this case to level convert the serial data lines.
Also the above schematic shows that the TX and RX pins connect to the Wemos D6 and D5 pins, since I’ll be using software serial, but the depicted connections are just an example, since the pins to be used can be software defined.
In my code I use the connections the other way around ( D5-TX, D6-RX) so beware to how the pins are connected and how they are defined at the software level.

Powering up the ESP8266 Wemos D1
I’ll be using the Wemos D1 ESP8266 based boards, as we can see on the above schematics (associated to a prototype shield to solder the connections and the level converter), we need to power it up using 5V. The ESP8266 uses 3.3v, but the Wemos board has a 5V input and a 5V to 3.3V converter, so no issues there. The PZEM004T on the other hand uses 220V, and since the ESP8266 will be near the PZEM004T, it makes sense to get the 5V CC from the 220V AC to power up the Wemos D1 board.

The 220V AC to 5V CC can be achieved in several ways, and since I’ll be installing all this in a DIN case on my home electricity mains board, the easiest solution is just to buy a 5V output 220V based DIN power supply for around 10/15€. This is the easiest and safest solution.

There are other solutions, including the one that I’m using that is based on the 5V HLK-PM01 based modules. This requires some assembly and also be aware that there are fake HLK modules around.

Do not connect the HLK-PM01 without the associated protection components, namely fuses, VDR, and the most important component the thermal fuse of 72ºC (Celsius!) that will cut off the power to anything after it (including the VDR) if the temperature of the HLK module or it’s surroundings rises above the 72ºC temperature. I’ve not soldered the thermal fuse, since the heat from the soldering iron can destroy it, just used a two terminal with screws to connect it.

The schematic used is the following one:

5V Power supply

The PZEM-004T, the HLK based power supply and the Wemos D1 ESP8266 module are inside a double length project DIN case so that all components can be safely installed on the mains electricity board.

Since all is self contained on the DIN case, all is needed is to clip the inductor on the main phase wire entering the mains board (and it is easy since the inductor is an open clip on type), and connect the components to the 220V AC power. I’ve derived the power from one of the circuit breakers that already protects a house circuit, which adds an additional layer of protection.

The software
On the next post I’ll discuss the software for driving the ESP8266 to gather data from the PZEM004T and how it works.
The firmware for driving this is already available at: https://github.com/fcgdam/PowerMeter

MQTT-SN Paho gateway and UDP6

Sometimes simple things can take a lot of time, such as this: How to make the Paho MQTT-SN gateway to listen to UDP6 instead of the standard UDP4 ports…

Introduction:
A lot can be written about MQTT-SN and it’s counterpart MQTT. To keep it simple and short, MQTT-SN (SN for Sensor Networks) uses UDP instead of TCP/IP and is streamlined to be more efficient in transmitting information, and so more adequate to be used on constrained devices than MQTT on WSN (Wireless Sensor Networks).
One important key element in using MQTT-SN is that MQTT-SN clients use UDP broadcasts to find gateways, and so they have no need to hardcode a broker address on the firmware code, since gateways can be dynamically discover at run time. Still MQTT-SN depends on MQTT broker to connect to the outside world (and that’s why MQTT-SN is a gateway, not a broker), but that is configured at the gateway level, not at the node level.

Compiling and configuring:
If we just clone the MQTT-SN Github repository, we can for UDP version 4, just do the following:

git clone https://github.com/eclipse/paho.mqtt-sn.embedded-c
cd MQTTSNGateway/
make 

And that’s it. On the Build/ directory there are the executables, among them the MQTT-SNGateway, and on the current directory there is the configuration file gateway.conf that is used by the gateway to startup. Out of the box the only thing that we need to change is the upstream MQTT broker address that by default points to the iot.eclipse.org MQTT broker, the gateway name, and maybe the ports:

BrokerName=iot.eclipse.org
BrokerPortNo=1883
BrokerSecurePortNo=8883

GatewayID=1
GatewayName=PahoGateway-01
KeepAlive=900

# UDP
GatewayPortNo=10000
MulticastIP=225.1.1.1
MulticastPortNo=1883

So if we start with this config, the output is:

20190612 142837.189 PahoGateway-01 has been started.

 ConfigFile: ./gateway.conf
 PreDefFile: ./predefinedTopic.conf
 SensorN/W:  UDP Multicast 225.1.1.1:1883 Gateway Port 10000
 Broker:     172.17.0.4 : 1883, 8883
 RootCApath: (null)
 RootCAfile: (null)
 CertKey:    (null)
 PrivateKey: (null)

We can see that in this case my upstream broker is at 172.17.0.4 (a docker image of the Mosquitto broker).

Checking which ports the gateway is listening:

root@b74f5ad3fd8e:/app/mqttsn# lsof -c MQTT-SNGateway -a -i4
COMMAND   PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
MQTT-SNGa 923 root    3u  IPv4 5017719      0t0  UDP *:10000 
MQTT-SNGa 923 root    4u  IPv4 5017720      0t0  UDP *:1883 
root@b74f5ad3fd8e:/app/mqttsn# lsof -c MQTT-SNGateway -a -i6
root@b74f5ad3fd8e:/app/mqttsn# 

We can see that the gateway is listening on UDP4 and not on UDP6.

Supporting UDP6:
Sensor networks such as Openthread networks work only with IPv6… and so the MQTT-SN gateway must listen to UDP6 instead of UDP4. It can still communicate with the MQTT broker over IPv4, and hence bridging the IPv4 network with the IPv6 sensor network.

To add support to UDP6 for the MQTT-SN gateway we need to change the Makefile (and hence the time to find information for this little bit… ) to change the protocol used by the sensor network and recompile.

So again on the source folder for the MQTT-SN gateway, edit the Makefile and change the following line from

SENSORNET := udp

to

SENSORNET := udp6

Or just change the parameter at the make call: make SENSORNET=udp6

And then clean and build:

make clean
make SENSORNET=udp6

We have now a MQTT-SN gateway version for UDP6. We need now to add the configuration entries to the gateway configuration file so that the UDP6 configuration is set:

#UDP v6
GatewayUDP6Broadcast = ff03::1
GatewayUDP6Port = 47193
GatewayUDP6If = wpan0

We can now start the UDP6 MQTT-SN enabled version:

20190612 144124.280 PahoGateway-01 has been started.

 ConfigFile: ./gateway.conf
 PreDefFile: ./predefinedTopic.conf
 SensorN/W:   Gateway Port: 47193 Broadcast Address: ff03::1 Interface: wpan0
 Broker:     172.17.0.4 : 1883, 8883
 RootCApath: (null)
 RootCAfile: (null)
 CertKey:    (null)
 PrivateKey: (null)

Specifically in this case the GatewayUDP6If interface that is defined is the NCP OpenThread interface, but it should be the interface where we want the gateway to listen. The broadcast address is the address defined on the Nordic SDK for Thread, namely on the file mqttsn_packet_internal.h.

Checking the ports:

root@b74f5ad3fd8e:/app/mqttsn# lsof -c MQTT-SNGateway -a -i4
root@b74f5ad3fd8e:/app/mqttsn# lsof -c MQTT-SNGateway -a -i6
COMMAND   PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
MQTT-SNGa 947 root    4u  IPv6 5050130      0t0  UDP *:47193 
root@b74f5ad3fd8e:/app/mqttsn# 

And that’s it.

Conclusion:
A simple lack of documentation regarding MQTT-SN and UDP6 took a while to solve since it was so “simple”.

Still one still needs two compiled versions of MQTT-SN for supporting UDP4 and UDP6 based networks, but since MQTT-SN is light there is no big issue with this.

Self hosted mobile phone push notifications

Google cloud messaging (GCM), Pushbullet, among others, allows to notify users though their mobile phone or portable device for events that where trigger either, programmatically or by any other mean.

A nice self hosted solution that does not depend in form of third party services, and as far as I can see is being actively developed, is Gotify that offers a self hosted server, and mobile application for Android.

Gotify offers a docker images (and unfortunately in the last few days they stopped producing the Docker image for Raspberry Pi), that allows to have a running Gotify server in less than 2 minutes.

Anyway, I’m still running the latest docker image for the RPI. Since I have already running something at the HTTP port 80, I’ve started the Gotify container with an alternate port:

docker run -p 1080:80 -v /var/gotify/data:/app/data gotify/server-arm7

After docker pulls the image and starts it up, we can access the Notify Web Server interface at http://rpi:1080/ and access it with the default user/password admin/admin.

The first thing that we need to do is to change the default admin password and add an application. Both steps are straight forward.

The key aspect for the application after it is created is to take notice of the application token. This token will be necessary for inject messages on Gotify and associate the injected message to the application. Also there is the possibility to associate an icon to the application, that also be shown on the mobile application, by pressing the pen icon and adding an image.

After this, to inject a message from anywhere we can use CURL:

curl -X POST "http://rpi:1080/message?token=AhV-43qgmkKSj07" -F "title=RPI" -F "message=Status Alert"
{"id":16,"appid":2,"message":"Status Alert","title":"RPI","priority":0,"date":"2019-02-25T12:13:35.777531495Z"}

Anyway, to make things short, Gotify works more or less well. I still have some issues with the web interface and with the mobile application, but those issues might be associated to being using an older version of the server.

Synology reverse proxy and websockets
Gotify uses websockets as the transport medium for the push notifications.

As a final note, since I’m using the Synology reverse proxy to access the Gotify server from the internet, some special configuration must be done so that the Synology reverse proxy let’s websockets to “pass through” to the Gotify server.

Basically we need on to go to Control Panel -> Application Portal -> Reverse Proxy and create our rule that maps out the Gotify server to the external URL. I’ve create something like http://notify.myserver.myds.me to be pointing to the internal Gotify server http://rpi:1080:

The important bit is that after creating the reverse proxy rule, we must edit and add the support for websockets by going to Custom Header and press the Create -> WebSocket button.

And that’s it. We have now a self hosted notification system. We can now, for examble, by using Node-Red or bash scripts, to start sending notifications to our clients when important events happen.

Lorawan/TTN with the TTGO ESP32 board

So I’m testing my TTGO Lora32 board with the RIOT-OS operating system to connect to the The Things Network, and while at first I had some success, then out of the blue things didn’t worked anymore.

The code that I’m using is based on the standard RIOT-Os Loramac example but with supporting code for ABP instead of the OTA activation.

The code is available at this Github repository.

Since I saw no activity on any channel for the Lora transmission by using the RTLSDR dongle, I suspected that something was up.
The Arduino based code from for connecting to TTN using Lorawan worked fine, and so it excluded an issue with the hardware.

By going to RIOT-OS installation directory, under drivers and on the sx127x directory, I’ve enabled on th sx127x.c file the debug to 1, since enabling debug on the semtech_loramac.c file held no useful information:

#define ENABLE_DEBUG (1)

With debug enabled, lo and behold, at startup:

2019-01-22 14:11:02,993 - INFO # [sx127x] error: failed to initialize SPI_0 device (code -1)
2019-01-22 14:11:02,993 - INFO # [sx127x] error: failed to initialize SPI

This is rather strange since at the initial tests the sx127x was detected and worked.
The only thing that changed is that I periodically update the RIOT local repository, and checking my board definition with other ESP32 boards that have a sx127x transceiver, the way that the pins where declared were quite different from what I had initially.

So I’ve changed the way that the pins for the SPI and the sx127x peripheral where defined, and the result:

2019-01-22 14:28:00,847 - INFO #  -> Node activation by: ABP
2019-01-22 14:28:00,847 - INFO # [sx127x] SPI_0 initialized with success
2019-01-22 14:28:00,848 - INFO # Set ABP information.

And the data arrives at TTN without any issue:

So either I had initially had the pins for SPI and the sx127x defined wrong or the latest RIOT build changed the way pins where defined.

A more complete example, working with multiple threads, is also available at this Github repository.

The ESP32 TTGO board using RIOT-OS

In the last few months, RIOT-OS operating system gained support for the ESP32 and ESP8266 based boards. RIOT-OS also supports several communication protocols, namely Lorawan through the usage of the Semtech implementation.

My post connecting the ESP32 TTGO board to TTN (The Things Network) describes (more or less) how to connect this board to the The Things Network using the Arduino and LMIC stack for the ESP32.

So why don’t give it a go with RIOT-OS?

Board definition:
RIOT-OS works with board definitions that describes several key features of the targeting board. In the case of the TTGO ESP32 Version 1 that I have several things needed to be changed from the base ESP32 board definitions that are available now on the RIOT-OS code base.

The most important issue was the clock speed definition for this ESP32 board, since it seems it uses a 26MHz clock instead of the standard(?) 40Mhz clock. Without the change for the clock speed all the serial output on the USB port was garbled and unreadable.

With the clock speed correctly set, important information regarding the pins for the I2C, SPI buses and the sx1276 chip needed to be defined. That was also correctly defined, and preliminary tests show that the Oled and sx1276 is detected using the Semtech Loramac stack.

I haven’t yet change the other pin definitions, so for the basic I2C, SPI, Button, and sx1276 are correctly defined.

The board definition can be found in my repository located at RIOT TTGO ESP32 repository under the directory boards.

To use this board definition, we just need to copy the board definition directory to the boards directory of the cloned RIOT-OS repository.

Testing:
For testing we can just compile the examples under the example directory for the RIOT TTGO ESP32 repository.

There are two examples: One for testing the OLED screen and other to test the sx1276 and the Semtech Loramac stack.
This last example is based on the provided RIOT-OS example for Lorawan, but I’ve added support for ABP and debug prints to see what is going on.
On the Make file, we should select the type of activation and fill out the required data from the TTN device console. All data just needs to be copied and pasted directly and it should work.

To build the examples I’m using Docker according to this instructions.

Additionally, for the ESP32 flashing to work, the ESP-IDF repository must be cloned and its location set for RIOT build by setting the ESP32_SDK_DIR variable.

Results:
The following things work fine:

– The serial console works fine.
– The blue led, on and off
– The button, detecting it release and pressed.

The OLED works fine through I2C, but the I2C speed must be set exactly as defined on the board definition, otherwise it won’t work. It took me several hours to debug this issue.

The sx1276 works, it is detected and the Semtech stack seems to be able to use it. Adding some debugging to Loramac stack shows that the DIO# interrupts are detected but there are issues. For example I have two boards and with the same code I have different behaviours which seems rather strange… I was able to send data to the TTN gateway, but when waiting for the Semtech stack it hangs waiting for a stack status message. The other board is always complaining regarding MAC timeout.

So, all in all, as a first experience with RIOT-OS on the TTGO board, excepting the Lorawan stack, everything seems to work as expected. Regarding the Lorawan stack, probably going to try the LMIC Arduino implementation to see if it is possible to port it to RIOT-OS.

A simple start with the RiotOS operating system for IoT devices

The RiotOS operating system is an Open Source operating system that targets different embedded platforms while allowing to use the same code base for some aspects of IoT development: Connectivity, protocols and security. It also supports other key aspects such as threads, Inter process communication, synchronization support on the kernel side. It also supports several communications protocols, suche BLE, 6LoWPAN, Lorawan, and so on.

At the target level for the RiotOS operating system, RiotOS supports a wide scope of different platforms, including a special native platform. What does this means? It means that within certain limitations, it is entirely possible to develop an IoT application that runs where the code is developed, this includes our PC and any SBC such as the RaspberryPI. This support opens a new window of opportunities where it is possible to integrate different classes of devices, such as Arduino, STM32 and RPI while leveraging the knowledge on the same supporting code base.

How to start:

This is a very quick start instructions for building a native demo application with network support. As usual all the instructions apply to a Linux Operating system, in my case Arch Linux.

Clone the RiotOS repository:

Just run:

cd /opt
git clone https://github.com/RIOT-OS/RIOT.git

I’ve chosen the /opt directory for the sake of example.

Compile one the example application:

All example applications are under the example applications.
In each directory, there is a Makefile file with a specific content.

For example on the examples/default there is Makefile that targets the native environment.

Of key interest is the line in this file that defines the target board:

BOARD ?= native

If the BOARD variable is not defined on the running environment it will use the native board.

We just need now to run the make command:

[pcortex@pcortex:default|master]$ make
Building application "default" for "native" with MCU "native".

"make" -C /opt/RIOT/boards/native
"make" -C /opt/RIOT/boards/native/drivers
"make" -C /opt/RIOT/core
"make" -C /opt/RIOT/cpu/native
...
...
...
"make" -C /opt/RIOT/sys/shell
"make" -C /opt/RIOT/sys/shell/commands
   text    data     bss     dec     hex filename
  88189    1104   72088  161381   27665 /opt/RIOT/examples/default/bin/native/default.elf

We can see that the compilation output was placed at /opt/RIOT/examples/default/bin/native/default.elf.

But if we ran the default.elf executable, we get:

[pcortex@pcortex:default|master]$ bin/native/default.elf 
usage: bin/native/default.elf  [-i ] [-d] [-e|-E] [-o] [-c ]
 help: bin/native/default.elf -h

Options:
    -h, --help
        print this help message
    -i , --id=
        specify instance id (set by config module)
    -s , --seed=
        specify srandom(3) seed (/dev/random is used instead of random(3) if
        the option is omitted)
    -d, --daemonize
        daemonize native instance
    -e, --stderr-pipe
        redirect stderr to file
    -E, --stderr-noredirect
        do not redirect stderr (i.e. leave sterr unchanged despite
        daemon/socket io)
    -o, --stdout-pipe
        redirect stdout to file (/tmp/riot.stdout.PID) when not attached
        to socket
    -c , --uart-tty=
        specify TTY device for UART. This argument can be used multiple
        times (up to UART_NUMOF)

A little side note, under bin there is a native directory but if we target the compilation to a different platform, a new directory under bin will be created to that platform.

Returning to result to the above command, we notice that the command requires a TAP (terminal interface point) interface. The fact is that the RiotOS code to access the network resources of the hosting operating system doesn’t access the network interface directly, but does it through the TAP interface. We can create only one TAP interface, or several TAP interfaces, which in such case, means that we can have several RiotOS applications using an TAP based network to communicate between themselves and also with the external network. Neat!

RiotOS offers a script to the TAP intefaces, but before running the script, just make sure if any Linux operating system Kernel was done recently, a reboot migh be needed to the interfaces be successfully created.

So:
[pcortex@pcortex:RIOT|master]$ ip a

1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp6s0:  mtu 4000 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:24:8c:02:dd:7e brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.68/24 brd 192.168.1.255 scope global noprefixroute enp6s0
       valid_lft forever preferred_lft forever
    inet6 fe80::224:8cff:fe02:dd7e/64 scope link 
       valid_lft forever preferred_lft forever

Now we run the tapsetup script that creates the TAP interfaces. This script when called without any parameters, it creates two TAP interfaces. If we pass the -c argument with a number, for example -c 4, it will create 4 TAP interfaces. The -d parameter deletes all interface that where created.

[pcortex@pcortex:tapsetup|master]$ pwd
/opt/RIOT/dist/tools/tapsetup                
[pcortex@pcortex:tapsetup|master]$ sudo ./tapsetup 
creating tapbr0
creating tap0
creating tap1
[pcortex@pcortex:tapsetup|master]$ ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp6s0:  mtu 4000 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:24:8c:02:dd:7e brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.68/24 brd 192.168.1.255 scope global noprefixroute enp6s0
       valid_lft forever preferred_lft forever
    inet6 fe80::224:8cff:fe02:dd7e/64 scope link 
       valid_lft forever preferred_lft forever
8: tapbr0:  mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 2e:bd:d8:88:64:55 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::2cbd:d8ff:fe88:6455/64 scope link 
       valid_lft forever preferred_lft forever
9: tap0:  mtu 1500 qdisc fq_codel master tapbr0 state DOWN group default qlen 1000
    link/ether 9a:a0:bc:fa:c7:61 brd ff:ff:ff:ff:ff:ff
10: tap1:  mtu 1500 qdisc fq_codel master tapbr0 state DOWN group default qlen 1000
    link/ether 2e:bd:d8:88:64:55 brd ff:ff:ff:ff:ff:ff

We can see that both tap0 and tap1 where created and are down.

Within two different windows we can run the default.elf with each interface now:

Window one: > sudo ./default.elf tap0
Window two: > sudo ./default.elf tap1

The two instances will start communicate and the tap interface have addresses and are up now.

11: tapbr0:  mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 0e:db:a6:77:3b:e5 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::cdb:a6ff:fe77:3be5/64 scope link 
       valid_lft forever preferred_lft forever
12: tap0:  mtu 1500 qdisc fq_codel master tapbr0 state UP group default qlen 1000
    link/ether 0e:db:a6:77:3b:e5 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::cdb:a6ff:fe77:3be5/64 scope link 
       valid_lft forever preferred_lft forever
13: tap1:  mtu 1500 qdisc fq_codel master tapbr0 state UP group default qlen 1000
    link/ether be:65:4f:4d:a0:ed brd ff:ff:ff:ff:ff:ff
    inet6 fe80::bc65:4fff:fe4d:a0ed/64 scope link 
       valid_lft forever preferred_lft forever

Conclusion:
The above is a simple example, but RiotOS offers several examples including COAP, MQTT-SN and OpenThread examples.

The next step is to test RiotOS on ESP8266 and ESP32.

Node-Red: Checking network service port status + UI status indicator

This post is about how to do two simple things using Node-Red:

  1. Check if network service on the machine running Node-Red is available by checking the corresponding listening port.
  2. The Node-Red UI doesn’t have a status indicator available, so I’ve built one

The only limitation on the following solution is that it only tests for ports for services that are running on the same server, where Node-Red is also running.

Preparation:

We need to install the Is Port Available NPM Module and make it available into our Node-Red instance.

For doing so, in Linux we must do the following:

root@server:~# cd .node-red/
root@server:~/.node-red# npm i --save is-port-available

We need now to make this node module available to Node-Red by editing the settings.js file:

root@server:~/.node-red# vi settings.js

Add the module to the global context on the function named functionGlobalContext:

    functionGlobalContext: {
        // os:require('os'),
        // octalbonescript:require('octalbonescript'),
        // jfive:require("johnny-five"),
        // j5board:require("johnny-five").Board({repl:false})

        portavail:require('is-port-available')
    },

You might have other modules configured, so we need to add the above portavail:require(‘is-port-available’) line to that list preceded by a comma.

We need to restart Node-Red to make the module available to the flows.

The testing flow
In our Function nodes, we can now use the global context object portavail to access the is-port-available module.

For example for testing the InfluxDB server port (1086/TCP) we can write the following function:

    // Instantiate locally on the flow the is-port-available module
    const isPortAvailable = context.global.portavail;

    msg.payload = {};   // Zero out the message. Not really necessary
     
    var port = 1086; // Replace this with your service port number. In this case 1086 is the Influx DB port
    
    isPortAvailable(port).then( status => {
        if(status) {
            //console.log('Port ' + port + ' IS available!');
            msg.payload = {'InfluxDB':false,"title":"InfluxDB","color":"red"};   // The port is available, hence the server is NOT running
            node.send(msg);
        } else {
            //console.log('Port ' + port + ' IS NOT available!');
            //console.log('Reason : ' + isPortAvailable.lastError);
            msg.payload = {'InfluxDB':true,"title":"InfluxDB","color":"green"};    // The port is not available, so the server MIGHT be running
            node.send(msg);
           
        }
    });

    // Note that we DO NOT return a message here since the above code is asynchronous and it will emit the message in the future. 

Since the test is using promises, Node-Red will continue executing without waiting for the test response (the isPortAvailable(port) code ). So we do not send any message further on the normal Node-Red execution flow (hence there is no return msg; object) and the message is only emitted when the promise fulfils. When that happens we just send the message with the node.send(msg) statement.

The message payload can be anything, being the only important properties the title and color that are used for creating the UI status indicator.

The status indicator is a simple Angularjs template that displays the title and a status circle with the chosen colour.

Since pasting CSS and HTML code in WordPress is recipe to disaster, the template code can be accessed on this gist or on the complete test flow below:

[{"id":"1f506795.4be25","type":"inject","z":"53f8b852.885c6","name":"Check todos os 60s","topic":"","payload":"","payloadType":"date","repeat":"60","crontab":"","once":true,"x":260,"y":96,"wires":[["5d180fc7.9ad06","27e67f9b.4f9158"]]},{"id":"5d180fc7.9ad06","type":"function","z":"53f8b852.885c6","name":"Test Influx DB","func":"    const isPortAvailable = context.global.portavail;\n    msg.payload = {};\n     \n    var port = 8086;\n    \n    isPortAvailable(port).then( status =>{\n        if(status) {\n            //console.log('Port ' + port + ' IS available!');\n            msg.payload = {'InfluxDB':false,\"title\":\"InfluxDB\",\"color\":\"red\"};   // The port is available, hence the server is NOT running\n            node.send(msg);\n        } else {\n            //console.log('Port ' + port + ' IS NOT available!');\n            //console.log('Reason : ' + isPortAvailable.lastError);\n            msg.payload = {'InfluxDB':true,\"title\":\"InfluxDB\",\"color\":\"green\"};    // The port is not available, so the server MIGHT be running\n            node.send(msg);\n           \n        }\n    });\n    ","outputs":1,"noerr":0,"x":533.5,"y":97,"wires":[["3f3f8226.c9bfb6"]]},{"id":"3f3f8226.c9bfb6","type":"ui_template","z":"53f8b852.885c6","group":"44e5d7ea.043b2","name":"Status Icon","order":0,"width":0,"height":0,"format":"\n.dot {\n    height: 25px;\n    width: 25px;\n    background-color: #bbb;\n    border-radius: 50%;\n    display: inline-block;\n    float: right;\n}\n\n\n
{{msg.payload.title}}\n \n
","storeOutMessages":true,"fwdInMessages":true,"x":780,"y":96,"wires":[[]]},{"id":"27e67f9b.4f9158","type":"function","z":"53f8b852.885c6","name":"Test MongoDB","func":" const isPortAvailable = context.global.portavail;\n msg.payload = {};\n \n var port = 27017;\n \n isPortAvailable(port).then( status =>{\n if(status) {\n //console.log('Port ' + port + ' IS available!');\n msg.payload = {'MongoDB':false,\"title\":\"MongoDB\",\"color\":\"red\"}; // The port is available, hence the server is NOT running\n node.send(msg);\n } else {\n //console.log('Port ' + port + ' IS NOT available!');\n //console.log('Reason : ' + isPortAvailable.lastError);\n msg.payload = {'MongoDB':true,\"title\":\"MongoDB\",\"color\":\"green\"}; // The port is not available, so the server MIGHT be running\n node.send(msg);\n \n }\n });\n ","outputs":1,"noerr":0,"x":533,"y":158,"wires":[["2e85d9d.cc25126"]]},{"id":"2e85d9d.cc25126","type":"ui_template","z":"53f8b852.885c6","group":"44e5d7ea.043b2","name":"Status Icon","order":0,"width":0,"height":0,"format":"\n.dot {\n height: 25px;\n width: 25px;\n background-color: #bbb;\n border-radius: 50%;\n display: inline-block;\n float: right;\n}\n\n\n
{{msg.payload.title}}\n \n
","storeOutMessages":true,"fwdInMessages":true,"x":781,"y":161,"wires":[[]]},{"id":"44e5d7ea.043b2","type":"ui_group","z":"","name":"System Status","tab":"7011ff77.15cb18","disp":true,"width":"6"},{"id":"7011ff77.15cb18","type":"ui_tab","z":"","name":"Home","icon":"dashboard"}]

The result:

The above flow and Node UI status indicator template should produce the following result:

NR UI Status Indicator
Node-Red UI Status Indicator

Simple BLE bridge to TTN Lora using the TTGO ESP32 LoRa32 board

The TTGO LoRa32 is an ESP32 based board that features Wifi and BlueTooth low energy but also includes an external Lora chip, in my case the SX1276 868Mhz version.

The following code/hack is just to test the feasibility of bridging BLE devices over the ESP32 and then to Lorawan, more specifically sending BLE data to the LoraWan TTN network.

I’m using Neil Koban ESP32 BLE library, that under platformIO is library number 1841 and the base ABP code for connecting to TTN.

In simple terms this code just makes the ESP32 to emulate a BLE UART device for sending and receiving data. It does that by using the Nordic UART known UUID for specifying the BLE UART service and using also the Nordic mobile applications, that supports such device, for sending/receiving data.

Using the Nordic mobile Android phone applications, data can be sent to the Lora32 board either by using the excellent Nordic Connect application or by also using the simpler and direct Nordic UART application.

The tests program just receives data through BLE and buffers it onto an internal message buffer that, periodically, is sent through Lora to the TTN network. I’ve decided arbitrary that the buffer is 32 bytes maximum. We should keep our message size to the necessary minimum, and also just send few messages to keep the lorawan duty factor usage within the required limits.

So, using the following code we can use our phone to scan from the ESP32 BLE device named TTGOLORAESP32 connect to it and send data to the device.

After a while, when the transmission event fires up, data is transmitted, and the BLE device just receives a simple notification with the EV_TXCOMPLETE message.

That’s it.

The ESP32 Oled Lora TTGO LoRa32 board and connecting it to TTN

The TTGO LoRa32 board is an ESP32 based board that has both an Oled and a Lora transceiver, in my case, the SX1276 transceiver for the 868Mhz band. So it is very similar to some of the ESP32 Oled boards available on the Internet. The board looks like this:

And the interesting part of this board is the new Wifi antenna located in the back that is made of bend metal:

The board also has a LiPo connector, and probably a charger circuit, since I haven’t tried it yet, a user controlled blue led, and a very dim red power led. The led is so dim that at first I thought the board was broken/short circuited, but it is normal.
The Lora Antenna is connected by U.FL/IPEX connector. Both a U.FL to SMA adapter cable is provided and also a cable to connect to the LiPo connector.

An important information to use this board for the LMIC LoraWan based communication is the location of the Lora transceiver DI01 and DIO2 pins. Fortunately they are exposed and connected internally to the ESP32 processor GPIO33 and GPIO32 pins respectively. I’ve updated the pin out for this board:

TTGOESP32Lora_wrongPins

EDIT: Thanks to Andreas on the comment section to point out that this image, while is correct for my board version (with the “3D” metal antenna under the board), the pin labels ARE WRONG. So much for copy it from the seller page.

The (so far yet…) pins mapping are on the bellow image. I’ve checked with my physical board and it seems right now.

I hope this corrects definitely the issue.

So back to basics, the LMIC definition pins for using this board are:

const lmic_pinmap lmic_pins = {
    .nss = 18,
    .rxtx = LMIC_UNUSED_PIN,
    .rst = 14,
    .dio = {26, 33, 32}  // Pins for the Heltec ESP32 Lora board/ TTGO Lora32 with 3D metal antenna
};

The Blue Led Pin is at Pin 2, and according to the sample code the Oled Display is at I2C address 0x3C. The I2C bus where the OLed is at SDA pin 4 and SCLK pin 15.

Also it seems there are at least two revisions for the ESP32 silicon, Revision 0 (Zero) for the initial one, and the latest, at the current date, Revision one.

By executing the Andreas Spiess revision check code it seems that my board is using the latest revision:

REG_READ(EFUSE_BLK0_RDATA3_REG) 1000000000000000
EFUSE_RD_CHIP_VER_RESERVE_S 1100
EFUSE_RD_CHIP_VER_RESERVE_V 111

Chip Revision (official version): 1
Chip Revision from shift Operation 1

Programming the board:
The board can be programmed easily with Platformio IDE by selecting as the target board the Heltec Wifi Lora board. Probably both boards are identical.

The platformio.ini file is as follows:

[env:heltec_wifi_lora_32]
platform = espressif32
board = heltec_wifi_lora_32
framework = arduino

For supporting the OLed and the Lora transceiver we also need to install the ESP8266_SSD1306 lib (ID: 562) and the IBM LMIC library (ID: 852) by either manually installing them on the project root or by adding the following line to the platformio.ini file:

[env:heltec_wifi_lora_32]
platform = espressif32
board = heltec_wifi_lora_32
framework = arduino
lib_deps= 852, 562

With this, the sample TTN INO sketchs for connecting either through ABP or OTAA work flawlessly without any issue by using the above LMIC pins configuration.

The sample sketch for the board: Connecting to TTN and display the packet RSSI:
Since we have the OLed, we can use the RX window to display the received RSSI of our messages on the gateway. This only works if the downlink messages from the gateway can reach back our node, so it might not work always. In my case, I’m about 3Km from the gateway in dense urban area, and not always I can display the packet RSSI.

How this works? Simple, just send our packet, and on the backend we send back the received RSSI as downlink message by using Node-Red, the TTN nodes, and some code:

Since our packet can be received by several gateways, we iterate over the TTN message and calculate the better RSSI and SNR:

// Build an object that allows us to track
// node data better than just having the payload

//For the debug inject node. Comment out when in real use
//var inmsg = msg.payload;
var inmsg = msg;  // from the TTN node

var newmsg = {};
var devicedata = {};
var betterRSSI = -1000;  // Start with a low impossible value
var betterSNR = -1000;

// WARNING only works with String data
// Use TTN decode functions is a better idea
var nodercvdata = inmsg.payload.toString("utf-8");

devicedata.device = inmsg.dev_id;
devicedata.deviceserial = inmsg.hardware_serial;
devicedata.rcvtime = inmsg.metadata.time;
devicedata.nodedata = nodercvdata;

// Iterate over the gateway data to get the best RSSI and SNR data
var gws = inmsg.metadata.gateways;

for ( var i = 0 ; i  betterRSSI )
        betterRSSI = gw.rssi;
        
    if ( gw.snr > betterSNR )
        betterSNR = gw.snr;
}

devicedata.rssi = betterRSSI;
devicedata.snr = betterSNR;

newmsg.payload = devicedata;

return newmsg;

We build then the response object and send it back to the TTN servers that send it to our node. The received data is then displayed on the Oled.

The Node-Red code is as follows:

[{"id":"d4536a72.6e6d7","type":"ttn message","z":"66b897a.7ab5c68","name":"TTN APP Uplink","app":"b59d5696.cde318","dev_id":"","field":"","x":140,"y":220,"wires":[["facbde95.14894"]]},{"id":"facbde95.14894","type":"function","z":"66b897a.7ab5c68","name":"Calculate better RSSI","func":"// Build an object that allows us to track\n// node data better than just having the payload\n\n//For the debug inject node. Comment out when in real use\n//var inmsg = msg.payload;\nvar inmsg = msg;  // from the TTN node\n\nvar newmsg = {};\nvar devicedata = {};\nvar betterRSSI = -1000;  // Start with a low impossible value\nvar betterSNR = -1000;\n\n// WARNING only works with String data\n// Use TTN decode functions is a better idea\nvar nodercvdata = inmsg.payload.toString(\"utf-8\");\n\ndevicedata.device = inmsg.dev_id;\ndevicedata.deviceserial = inmsg.hardware_serial;\ndevicedata.rcvtime = inmsg.metadata.time;\ndevicedata.nodedata = nodercvdata;\n\n// Iterate over the gateway data to get the best RSSI and SNR data\nvar gws = inmsg.metadata.gateways;\n\nfor ( var i = 0 ; i  betterRSSI )\n        betterRSSI = gw.rssi;\n        \n    if ( gw.snr > betterSNR )\n        betterSNR = gw.snr;\n}\n\ndevicedata.rssi = betterRSSI;\ndevicedata.snr = betterSNR;\n\nnewmsg.payload = devicedata;\n\nreturn newmsg;","outputs":1,"noerr":0,"x":400,"y":220,"wires":[["1ac970ec.4cfabf","94515e56.904228"]]},{"id":"1ac970ec.4cfabf","type":"debug","z":"66b897a.7ab5c68","name":"","active":false,"console":"false","complete":"payload","x":670,"y":260,"wires":[]},{"id":"2bea15d8.18f88a","type":"ttn send","z":"66b897a.7ab5c68","name":"TTN APP Downlink","app":"b59d5696.cde318","dev_id":"","port":"","x":970,"y":100,"wires":[]},{"id":"94515e56.904228","type":"function","z":"66b897a.7ab5c68","name":"set Payload","func":"msg.dev_id  = msg.payload.device;\nmsg.payload = Buffer.from(\"RSSI: \" + msg.payload.rssi);\n\nreturn msg;","outputs":1,"noerr":0,"x":670,"y":100,"wires":[["2bea15d8.18f88a","cd04abb9.ccd278"]]},{"id":"cd04abb9.ccd278","type":"debug","z":"66b897a.7ab5c68","name":"","active":true,"console":"false","complete":"true","x":930,"y":200,"wires":[]},{"id":"b59d5696.cde318","type":"ttn app","z":"","appId":"TTNAPPLICATIONID","region":"eu","accessKey":"ttn-account-v2.CHANGEMECHANGEME"}]

Just make sure that we have the TTN nodes installed, and change the credentials for your TTN Application.

On the TTGO ESP32 Lora32 board we just modify the event handling code to display the downlink message:

void onEvent (ev_t ev) {
    if (ev == EV_TXCOMPLETE) {
        display.clear();
        display.drawString (0, 0, "EV_TXCOMPLETE event!");


        Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
        if (LMIC.txrxFlags & TXRX_ACK) {
          Serial.println(F("Received ack"));
          display.drawString (0, 20, "Received ACK.");
        }

        if (LMIC.dataLen) {
          int i = 0;
          // data received in rx slot after tx
          Serial.print(F("Data Received: "));
          Serial.write(LMIC.frame+LMIC.dataBeg, LMIC.dataLen);
          Serial.println();

          display.drawString (0, 20, "Received DATA.");
          for ( i = 0 ; i < LMIC.dataLen ; i++ )
            TTN_response[i] = LMIC.frame[LMIC.dataBeg+i];
          TTN_response[i] = 0;
          display.drawString (0, 32, String(TTN_response));
        }

        // Schedule next transmission
        os_setTimedCallback(&sendjob, os_getTime()+sec2osticks(TX_INTERVAL), do_send);
        digitalWrite(LEDPIN, LOW);
        display.drawString (0, 50, String (counter));
        display.display ();
    }
}

For example we can now see on the serial port monitor:

EV_TXCOMPLETE (includes waiting for RX windows)
Sending uplink packet...
EV_TXCOMPLETE (includes waiting for RX windows)
Sending uplink packet...
EV_TXCOMPLETE (includes waiting for RX windows)
Sending uplink packet...
EV_TXCOMPLETE (includes waiting for RX windows)
Data Received: RSSI: -118
Sending uplink packet...
EV_TXCOMPLETE (includes waiting for RX windows)
Data Received: RSSI: -114
Sending uplink packet...
EV_TXCOMPLETE (includes waiting for RX windows)
Data Received: RSSI: -105

Thats it!

Some final notes:
Probably not related to the board, but when connecting it to an USB3 port, the Linux Operating system was unable to configure a device for the board. Connecting it to an USB2 port worked flawlessly:

usb 2-1: new full-speed USB device number 2 using xhci_hcd
usb 2-1: string descriptor 0 read error: -71
usb 2-1: can't set config #1, error -71      

As additional information the serial chip on this board is an umarked CP210x chip:

usb 4-1.3: new full-speed USB device number 6 using ehci-pci
cp210x 4-1.3:1.0: cp210x converter detected
usb 4-1.3: cp210x converter now attached to ttyUSB0

lsusb:

Bus 004 Device 006: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP2102/CP2109 UART Bridge Controller [CP210x family]

I haven’t yet tried the WiFi and checked if the metal antenna is any good, but with my preliminary tests, it seems it’s not very good.

Sample code:

Sample code for the board is on this github link: https://github.com/fcgdam/TTGO_LoRa32

Using the BSFrance Lora32U4 board to connect to the Things Network Lorawan

The BSFrance Lora32u4 II (Lora32U4II for helping Google out) board is an Atmega32U4 processor with a HDP13 Lora transceiver on the same board. As far as I’m aware, the HDP13 is similar to the RFM95W (including pinout), and in my case it seems it has an original Semtech SX1276 (868Mhz radio transceiver) chip installed on the HDP13 module.
This board is similar to the Adafruit 32U4 Lora feather, if not equal… (possible schematics for the Lora32u4 board)

The board hardware includes beside the Lora HDP13 module a LiPo connector with an 2 pin JST PH 2.0mm pin spacing connector and the power supporting electronics.
There are two leds: one orange LED for LiPo and charger status, that blinks very fast when no LiPo is connected, and a very bright white led that fades in and out when the bootloader is in the programming mode or programming is ongoing. After the bootloader exits and starts the main program, the led shuts off.
This led, as usual in Arduino boards, is connected to I/O pin 13, so it is software controllable.

Also the only way to power up the board is either trough the USB port, LiPo battery or 5V to an input pin. No other voltages, like RAW voltages above 5V are supported.

As a final note, the board that I’ve bought also came with an uFL adapter cable for SMA, an antenna and a link for accessing documentation, so, excluding the LiPo battery, the complete kit.

Starting up using the board:

I’m testing the board to send data to the Things Network and doing so by using PlatformioIO as the developing IDE. Platformio IDE is much better than the Arduino IDE, since each project has it’s own depending libraries directory .piolibdeps which we can modify and edit the library code without breaking other projects.

The platformio.ini board definition for the Lora32u4II board is just a clone of Adafruit feather 32u4:

[env:feather32u4]
platform = atmelavr
board = feather32u4
framework = arduino

As the code to send data to the TTN network, I’ve just used ABP lorawan device connection that I’ve used on my previous hand build node.

I’m testing the node with both the IBM LMIC Library (ID: 852) and the Arduino LMIC Library (ID: 1729).

After setting the correct keys and device ID, all we need is to change the LMIC pins configuration for this board: LoRa32u4II pinout diagram

According to documentation the pins are:

  1. nss (SS – Chip Select): Pin 8
  2. rst (Reset): Pin 4
  3. DIO (Lora TX/RX indicator): Pin 7

So the LMIC Pins configuration is:

const lmic_pinmap lmic_pins = {
    .nss = 8,
    .rxtx = LMIC_UNUSED_PIN,
    .rst = 4,
    .dio = {7, 6 , LMIC_UNUSED_PIN}
};

Regarding Pin 6, is the chosen pin to connect to the DIO1 pin. This pin signals receive timeouts generated by the radio module.

The connection of this pin is required for LMIC and for the onEvent() function signaling of EV_TXCOMPLETE to be triggered/fired, otherwise the onEvent() funciton is never called.

Since this is a LoraWan Class A node, after the transmission, two receive windows are opened for any downlink data that might be sent to the node.

The DIO1 pin signals the receive timeout, and at the end of the receive windows, triggers the EV_TXCOMPLETE event.

I’ve tried to use other pins, for example, pin 3, but then the EV_TXCOMPLETE event was never fired… Strange.

Anyway, with the above configuration and with DIO1 connected through a wire bridge to pin 6 works fine.

If we do not connect DIO1 by removing the DIO1 pin configuration:

 .dio = {7, LMIC_UNUSED_PIN , LMIC_UNUSED_PIN}

with the platformio IBM Lmic library (Id: 852), or with the Arduino LMIC Library the LMIC fails. An example:

pio device monitor --port /dev/ttyACM0 --baud 115200
[cortex@brightlight:TTN32u4ABP]$ pio device monitor --port /dev/ttyACM0 --baud 115200
--- Miniterm on /dev/ttyACM0  115200,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
Starting...
FAILURE
.piolibdeps/IBM LMIC framework_ID852/src/hal/hal.cpp:24

The line hal.cpp:24 point to an ASSERT that doesn’t allow a LMIC_UNUSED_PIN for DIO1.

Putting pin 6 and making sure that it is connected to DIO1 is required. Otherwise if the pin is defined but not connected we have the following behaviour:

--- Miniterm on /dev/ttyACM0  115200,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
Starting...
Sending uplink packet...

As we can see the EV_TXCOMPLETE event is never fired, and the associated reschedule of another transmission never happens, since the code that triggers the next transmission is inside the code for the EV_TXCOMPLETE event. The only way, in this case, to exit this situation is to reset the board so another transmission can happen.

So if using the above LMIC pins configuration and connecting DIO1 to pin 6, sending data to the The Things Network works just fine:

Data received at the TTN side

Some final notes, tips and tricks:

The ATMega 32U4 USB Serial port:
The ATMega 32U4 USB serial port is a bit fiddly when using it from the Arduino framework. At reset or connection first the USB port is used by the bootloader (white led fading in and out). After a while the board starts to execute the flash program (white led off), but it resets the USB port. The host computer might have an issue with this and fails to assign an USB address.

The solution is just to add at the start of the setup function a delay:

void setup() {
  delay(2500);   // Give time to the ATMega32u4 port to wake up and be recognized by the OS.
  
  Serial.begin(115200);
...
...

Using minicom instead of PlatformIO serial monitor:
This one is quite simple to explain, since minicom survives to the USB port resets since they appear and disappear through the board reset.
Against it, is that we need to explicitly exit minicom to be able to program the board.

# minicom -D /dev/ttyACM0 -b 115200

The PlatformIO Arduino LMIC library is outdated:
This is solved now. Lib 852 is now updated.
The Arduino LMIC version (1729) on the PlatformIO is outdated, since, for example doesn’t have neither the LMIC_UNUSED_PIN definition and the LMIC_setClockError function needed for a successful OTAA TTN network join.

The solution is just clone the Arduino LMIC library and copy the src folder to .piolibdeps/IBM LMIC framework_ID852/ removing the original src folder version.

Comparing Library sizes:

Using the IBM LMIC Library (ID:852) with PINGS and BEACONS disabled on the config.h file, otherwise it doesn’t fit on the 32u4 32K flash space, our sketch uses the following space:

AVR Memory Usage
----------------
Device: atmega32u4

Program:   26040 bytes (79.5% Full)
(.text + .data + .bootloader)

Data:       1014 bytes (39.6% Full)
(.data + .bss + .noinit)

Using the Arduino LMIC library (ID: 1729) with PINGS and BEACONS enabled, but a more efficient AES implementation, we get:

AVR Memory Usage
----------------
Device: atmega32u4

Program:   22776 bytes (69.5% Full)
(.text + .data + .bootloader)

Data:        954 bytes (37.3% Full)
(.data + .bss + .noinit)

With PINGS and BEACONS disabled we get:

AVR Memory Usage
----------------
Device: atmega32u4

Program:   19032 bytes (58.1% Full)
(.text + .data + .bootloader)

Data:        903 bytes (35.3% Full)
(.data + .bss + .noinit)

So we get, with this last change, and while keeping support for OTTA, at least 8K/9K for program space not related to the Lorawan/TTN code support.