A simple Grafana dashboard for the Power Meter

Regarding the data that is collected by the PZEM-004T ESP8266 based power meter, data is published into a MQTT topic (the telemetry topic). A Node-Red flow listens to that topic and saves the data into an InfluxDB database and also builds a Node-Red UI dashboard:

Power Meter Node-Red UI

So far so good, but the Node-Red Dashboard only shows the latest collected data. If we want to see more information across time, then Grafana can really helps on this.

So I’ve updated the Github repository with a simple Grafana Dashboard, that can be imported and tweaked to fulfil any requirements.

The simple dashboard looks like this:

Power Meter Grafana Dashboard

Requirements are that Grafana is installed and can access the InfluxDB used by the Node-Red flow to save the Power Meter data.

Both Node-Red (and of course) Grafana use the same database named PowerMeter, so we need to make sure it exists, or adjust Node-Red and Grafana Data Sources accordingly.

PZEM-004T ESP8266 software

Following up the home energy meter post based on an ESP8266 and PZEM-004T hardware, this post describes succinctly the software for using the energy meter.

There are at least two components to the solution:

  1. ESP8266 software for driving the power meter and make the measurements.
  2. The backend software for receiving and processing data.

The ESP8266 software:
The power meter software for the ESP8266 available on this GitHub repository, uses an available PZEM-004T library for accessing the power meter, and sends the collected data through MQTT to any subscribers of the power meter topic.
I’m using the convention that is also used on Thingsboard, namely an MQTT attributes topic to publish the device status, and a telemetry topic to post the data in JSON format.
Around lines 80 on main.cpp of PowerMeter sources, the topics are defined as:

  1. Attributes: “iot/device/” + String(MQTT_ClientID) + “/attributes”
  2. Telemetry: “iot/device/” + String(MQTT_ClientID) + “/telemetry”

MQTT_ClientID is defined on the secrets.h file, where we also define a list of available WIFI connections for our ESP8266. The attributes topic periodically sends the current device status (RSSI, HEAP, wifi SSID), while the data on the telemetry topic is fed into a timeseries database such as InfluxDB where then a Grafana Dashboard shows and allows to see the captured data across time.

As also my previous post regarding framework and libraries versions, I needed to block the ESP8266 framework version and the SoftwareSerial library because the combination of these with the PZE-004T library was (is ?) broken of more recent versions. As is currently defined on the platformio.ini file, the current set of versions, work fine.

A lot of people had problems working with the use of SoftwareSerial library for the PZEM library to communicate with the hardware. The issue, that I accidentally found out, are related with timing issues to communicate with the PZEM hardware. There are periods of time that the PZEM is not responsive, probably because is making some measurement.

The solution to this issue is at start up to try the connection during some time, at 3 seconds interval until it succeeds. After the connection is successful, we need to keep an interval around one minute between reads to encounter no issues/failures . If this interval is kept, the connection to the PZEM hardware works flawlessly, at least with the hardware that I have.

So the connection phase is checked and tried several times to synchronize the ESP8266 with the PZEM, and them every single minute there is a data read. If the interval is shorter, lets say, 30s, it will fail, until the elapsed time to one minute is completed.

The firmware solves the above issue, and after reading the data, it posts it to a MQTT broker. The firmware also makes available a web page with the current status and measurements:

Power Meter Web Page

Then there are other bits, namely since the meter will be on the electric mains board, an UDP logging facility that allows on the computer to run an UDP server and see what is going on.

The back-end software:
I’ve not done much on this area, since most of it is just standard stuff. An MQTT broker and Node-Red flow. The flow just receives the data, saves it into an InfluxDB database and creates a Node-Red UI dashboard.

Power Meter Node-Red UI

This screenshot shows some of the information that was collected on the last minute and it is updated in real time as soon the PowerMeter information arrives to the MQTT broker.

Future work:
Basically what is missing is two things:

  1. Grafana Dashboard based on the InfluxDB data (Already done, to be described in a future post).
  2. Some kind of exporter to CSV or Spreadsheat to allow further data analysis such as the daily power consumption totals.

Measuring home energy consumption with the PZEM004T and ESP8266

First of all a very BIG WARNING: This project works with AC mains current, which, where I live, is 220V AC, meaning that extra precautions must be taken, since risk of serious injuries and/or death is possible.

The PZEM004T
The Peacefair PZEM004T device (available at the usual far east shopping web sites) is a device that can measure energy consumption by monitoring a live AC mains wire using an inductor as the measuring sensor. One of the wires that carries the current (normally the AC power phase) goes through the inductor so that the current that flows through it can be measured and hence the other measurements, including power consumption, can be also measured.

The PZEM004T can be bought with two types of inductor, one that opens up and can clip on the wire of interest, and the other type that requires to disconnect and connect the wire of interest, so that it passes through the inductor core. I’ve chosen the former, since in this case I do not need to do any disconnection/connections on the electric mains board, and so it is way safer and easier to add and remove the measurement device.


The PZEMM04T outputs the collected data through an opto-coupled isolated serial port that allows to retrieve values for voltage, current/intensity, current power consumption and energy accumulated consumption.

The device that connects to the PZEM serial port must provide power to it (5V), and so the serial port data lines are 5V level, which means that we should use a 5v to 3.3V level converter to connect to the ESP8266. While there are several hacks to make the PZEM004T serial port to use 3.3V on the serial port, and hence have 3.3V data lines, I just used a simple level converter to connect the serial port to the ESP8266, and avoid in this way any modifications to the PZEM-004T. The serial port connector is a 4 Pin JST-XH connector.

So the basic schematics for using the PZEM004T is as simple as the following highly professionally drawn schematic shows:

PZEM004T And Wemos D1 connection schematic

Two things of notice:

  • The Ground connection – The serial port uses the same ground as the Wemos D1.
  • The Power supply – Wemos D1 is powered through the 5V pin, NOT through the 3.3v pin, since we need 5V to power up the PZEM serial port.

The level converter is just a simple, cheap I2C level converter, used in this case to level convert the serial data lines.
Also the above schematic shows that the TX and RX pins connect to the Wemos D6 and D5 pins, since I’ll be using software serial, but the depicted connections are just an example, since the pins to be used can be software defined.
In my code I use the connections the other way around ( D5-TX, D6-RX) so beware to how the pins are connected and how they are defined at the software level.

Powering up the ESP8266 Wemos D1
I’ll be using the Wemos D1 ESP8266 based boards, as we can see on the above schematics (associated to a prototype shield to solder the connections and the level converter), we need to power it up using 5V. The ESP8266 uses 3.3v, but the Wemos board has a 5V input and a 5V to 3.3V converter, so no issues there. The PZEM004T on the other hand uses 220V, and since the ESP8266 will be near the PZEM004T, it makes sense to get the 5V CC from the 220V AC to power up the Wemos D1 board.

The 220V AC to 5V CC can be achieved in several ways, and since I’ll be installing all this in a DIN case on my home electricity mains board, the easiest solution is just to buy a 5V output 220V based DIN power supply for around 10/15€. This is the easiest and safest solution.

There are other solutions, including the one that I’m using that is based on the 5V HLK-PM01 based modules. This requires some assembly and also be aware that there are fake HLK modules around.

Do not connect the HLK-PM01 without the associated protection components, namely fuses, VDR, and the most important component the thermal fuse of 72ºC (Celsius!) that will cut off the power to anything after it (including the VDR) if the temperature of the HLK module or it’s surroundings rises above the 72ºC temperature. I’ve not soldered the thermal fuse, since the heat from the soldering iron can destroy it, just used a two terminal with screws to connect it.

The schematic used is the following one:

5V Power supply

The PZEM-004T, the HLK based power supply and the Wemos D1 ESP8266 module are inside a double length project DIN case so that all components can be safely installed on the mains electricity board.

Since all is self contained on the DIN case, all is needed is to clip the inductor on the main phase wire entering the mains board (and it is easy since the inductor is an open clip on type), and connect the components to the 220V AC power. I’ve derived the power from one of the circuit breakers that already protects a house circuit, which adds an additional layer of protection.

The software
On the next post I’ll discuss the software for driving the ESP8266 to gather data from the PZEM004T and how it works.
The firmware for driving this is already available at: https://github.com/fcgdam/PowerMeter

MQTT-SN Paho gateway and UDP6

Sometimes simple things can take a lot of time, such as this: How to make the Paho MQTT-SN gateway to listen to UDP6 instead of the standard UDP4 ports…

A lot can be written about MQTT-SN and it’s counterpart MQTT. To keep it simple and short, MQTT-SN (SN for Sensor Networks) uses UDP instead of TCP/IP and is streamlined to be more efficient in transmitting information, and so more adequate to be used on constrained devices than MQTT on WSN (Wireless Sensor Networks).
One important key element in using MQTT-SN is that MQTT-SN clients use UDP broadcasts to find gateways, and so they have no need to hardcode a broker address on the firmware code, since gateways can be dynamically discover at run time. Still MQTT-SN depends on MQTT broker to connect to the outside world (and that’s why MQTT-SN is a gateway, not a broker), but that is configured at the gateway level, not at the node level.

Compiling and configuring:
If we just clone the MQTT-SN Github repository, we can for UDP version 4, just do the following:

git clone https://github.com/eclipse/paho.mqtt-sn.embedded-c
cd MQTTSNGateway/

And that’s it. On the Build/ directory there are the executables, among them the MQTT-SNGateway, and on the current directory there is the configuration file gateway.conf that is used by the gateway to startup. Out of the box the only thing that we need to change is the upstream MQTT broker address that by default points to the iot.eclipse.org MQTT broker, the gateway name, and maybe the ports:




So if we start with this config, the output is:

20190612 142837.189 PahoGateway-01 has been started.

 ConfigFile: ./gateway.conf
 PreDefFile: ./predefinedTopic.conf
 SensorN/W:  UDP Multicast Gateway Port 10000
 Broker: : 1883, 8883
 RootCApath: (null)
 RootCAfile: (null)
 CertKey:    (null)
 PrivateKey: (null)

We can see that in this case my upstream broker is at (a docker image of the Mosquitto broker).

Checking which ports the gateway is listening:

root@b74f5ad3fd8e:/app/mqttsn# lsof -c MQTT-SNGateway -a -i4
MQTT-SNGa 923 root    3u  IPv4 5017719      0t0  UDP *:10000 
MQTT-SNGa 923 root    4u  IPv4 5017720      0t0  UDP *:1883 
root@b74f5ad3fd8e:/app/mqttsn# lsof -c MQTT-SNGateway -a -i6

We can see that the gateway is listening on UDP4 and not on UDP6.

Supporting UDP6:
Sensor networks such as Openthread networks work only with IPv6… and so the MQTT-SN gateway must listen to UDP6 instead of UDP4. It can still communicate with the MQTT broker over IPv4, and hence bridging the IPv4 network with the IPv6 sensor network.

To add support to UDP6 for the MQTT-SN gateway we need to change the Makefile (and hence the time to find information for this little bit… ) to change the protocol used by the sensor network and recompile.

So again on the source folder for the MQTT-SN gateway, edit the Makefile and change the following line from




Or just change the parameter at the make call: make SENSORNET=udp6

And then clean and build:

make clean
make SENSORNET=udp6

We have now a MQTT-SN gateway version for UDP6. We need now to add the configuration entries to the gateway configuration file so that the UDP6 configuration is set:

#UDP v6
GatewayUDP6Broadcast = ff03::1
GatewayUDP6Port = 47193
GatewayUDP6If = wpan0

We can now start the UDP6 MQTT-SN enabled version:

20190612 144124.280 PahoGateway-01 has been started.

 ConfigFile: ./gateway.conf
 PreDefFile: ./predefinedTopic.conf
 SensorN/W:   Gateway Port: 47193 Broadcast Address: ff03::1 Interface: wpan0
 Broker: : 1883, 8883
 RootCApath: (null)
 RootCAfile: (null)
 CertKey:    (null)
 PrivateKey: (null)

Specifically in this case the GatewayUDP6If interface that is defined is the NCP OpenThread interface, but it should be the interface where we want the gateway to listen. The broadcast address is the address defined on the Nordic SDK for Thread, namely on the file mqttsn_packet_internal.h.

Checking the ports:

root@b74f5ad3fd8e:/app/mqttsn# lsof -c MQTT-SNGateway -a -i4
root@b74f5ad3fd8e:/app/mqttsn# lsof -c MQTT-SNGateway -a -i6
MQTT-SNGa 947 root    4u  IPv6 5050130      0t0  UDP *:47193 

And that’s it.

A simple lack of documentation regarding MQTT-SN and UDP6 took a while to solve since it was so “simple”.

Still one still needs two compiled versions of MQTT-SN for supporting UDP4 and UDP6 based networks, but since MQTT-SN is light there is no big issue with this.

Self hosted mobile phone push notifications

Google cloud messaging (GCM), Pushbullet, among others, allows to notify users though their mobile phone or portable device for events that where trigger either, programmatically or by any other mean.

A nice self hosted solution that does not depend in form of third party services, and as far as I can see is being actively developed, is Gotify that offers a self hosted server, and mobile application for Android.

Gotify offers a docker images (and unfortunately in the last few days they stopped producing the Docker image for Raspberry Pi), that allows to have a running Gotify server in less than 2 minutes.

Anyway, I’m still running the latest docker image for the RPI. Since I have already running something at the HTTP port 80, I’ve started the Gotify container with an alternate port:

docker run -p 1080:80 -v /var/gotify/data:/app/data gotify/server-arm7

After docker pulls the image and starts it up, we can access the Notify Web Server interface at http://rpi:1080/ and access it with the default user/password admin/admin.

The first thing that we need to do is to change the default admin password and add an application. Both steps are straight forward.

The key aspect for the application after it is created is to take notice of the application token. This token will be necessary for inject messages on Gotify and associate the injected message to the application. Also there is the possibility to associate an icon to the application, that also be shown on the mobile application, by pressing the pen icon and adding an image.

After this, to inject a message from anywhere we can use CURL:

curl -X POST "http://rpi:1080/message?token=AhV-43qgmkKSj07" -F "title=RPI" -F "message=Status Alert"
{"id":16,"appid":2,"message":"Status Alert","title":"RPI","priority":0,"date":"2019-02-25T12:13:35.777531495Z"}

Anyway, to make things short, Gotify works more or less well. I still have some issues with the web interface and with the mobile application, but those issues might be associated to being using an older version of the server.

Synology reverse proxy and websockets
Gotify uses websockets as the transport medium for the push notifications.

As a final note, since I’m using the Synology reverse proxy to access the Gotify server from the internet, some special configuration must be done so that the Synology reverse proxy let’s websockets to “pass through” to the Gotify server.

Basically we need on to go to Control Panel -> Application Portal -> Reverse Proxy and create our rule that maps out the Gotify server to the external URL. I’ve create something like http://notify.myserver.myds.me to be pointing to the internal Gotify server http://rpi:1080:

The important bit is that after creating the reverse proxy rule, we must edit and add the support for websockets by going to Custom Header and press the Create -> WebSocket button.

And that’s it. We have now a self hosted notification system. We can now, for examble, by using Node-Red or bash scripts, to start sending notifications to our clients when important events happen.

Lorawan/TTN with the TTGO ESP32 board

So I’m testing my TTGO Lora32 board with the RIOT-OS operating system to connect to the The Things Network, and while at first I had some success, then out of the blue things didn’t worked anymore.

The code that I’m using is based on the standard RIOT-Os Loramac example but with supporting code for ABP instead of the OTA activation.

The code is available at this Github repository.

Since I saw no activity on any channel for the Lora transmission by using the RTLSDR dongle, I suspected that something was up.
The Arduino based code from for connecting to TTN using Lorawan worked fine, and so it excluded an issue with the hardware.

By going to RIOT-OS installation directory, under drivers and on the sx127x directory, I’ve enabled on th sx127x.c file the debug to 1, since enabling debug on the semtech_loramac.c file held no useful information:

#define ENABLE_DEBUG (1)

With debug enabled, lo and behold, at startup:

2019-01-22 14:11:02,993 - INFO # [sx127x] error: failed to initialize SPI_0 device (code -1)
2019-01-22 14:11:02,993 - INFO # [sx127x] error: failed to initialize SPI

This is rather strange since at the initial tests the sx127x was detected and worked.
The only thing that changed is that I periodically update the RIOT local repository, and checking my board definition with other ESP32 boards that have a sx127x transceiver, the way that the pins where declared were quite different from what I had initially.

So I’ve changed the way that the pins for the SPI and the sx127x peripheral where defined, and the result:

2019-01-22 14:28:00,847 - INFO #  -> Node activation by: ABP
2019-01-22 14:28:00,847 - INFO # [sx127x] SPI_0 initialized with success
2019-01-22 14:28:00,848 - INFO # Set ABP information.

And the data arrives at TTN without any issue:

So either I had initially had the pins for SPI and the sx127x defined wrong or the latest RIOT build changed the way pins where defined.

A more complete example, working with multiple threads, is also available at this Github repository.

The ESP32 TTGO board using RIOT-OS

In the last few months, RIOT-OS operating system gained support for the ESP32 and ESP8266 based boards. RIOT-OS also supports several communication protocols, namely Lorawan through the usage of the Semtech implementation.

My post connecting the ESP32 TTGO board to TTN (The Things Network) describes (more or less) how to connect this board to the The Things Network using the Arduino and LMIC stack for the ESP32.

So why don’t give it a go with RIOT-OS?

Board definition:
RIOT-OS works with board definitions that describes several key features of the targeting board. In the case of the TTGO ESP32 Version 1 that I have several things needed to be changed from the base ESP32 board definitions that are available now on the RIOT-OS code base.

The most important issue was the clock speed definition for this ESP32 board, since it seems it uses a 26MHz clock instead of the standard(?) 40Mhz clock. Without the change for the clock speed all the serial output on the USB port was garbled and unreadable.

With the clock speed correctly set, important information regarding the pins for the I2C, SPI buses and the sx1276 chip needed to be defined. That was also correctly defined, and preliminary tests show that the Oled and sx1276 is detected using the Semtech Loramac stack.

I haven’t yet change the other pin definitions, so for the basic I2C, SPI, Button, and sx1276 are correctly defined.

The board definition can be found in my repository located at RIOT TTGO ESP32 repository under the directory boards.

To use this board definition, we just need to copy the board definition directory to the boards directory of the cloned RIOT-OS repository.

For testing we can just compile the examples under the example directory for the RIOT TTGO ESP32 repository.

There are two examples: One for testing the OLED screen and other to test the sx1276 and the Semtech Loramac stack.
This last example is based on the provided RIOT-OS example for Lorawan, but I’ve added support for ABP and debug prints to see what is going on.
On the Make file, we should select the type of activation and fill out the required data from the TTN device console. All data just needs to be copied and pasted directly and it should work.

To build the examples I’m using Docker according to this instructions.

Additionally, for the ESP32 flashing to work, the ESP-IDF repository must be cloned and its location set for RIOT build by setting the ESP32_SDK_DIR variable.

The following things work fine:

– The serial console works fine.
– The blue led, on and off
– The button, detecting it release and pressed.

The OLED works fine through I2C, but the I2C speed must be set exactly as defined on the board definition, otherwise it won’t work. It took me several hours to debug this issue.

The sx1276 works, it is detected and the Semtech stack seems to be able to use it. Adding some debugging to Loramac stack shows that the DIO# interrupts are detected but there are issues. For example I have two boards and with the same code I have different behaviours which seems rather strange… I was able to send data to the TTN gateway, but when waiting for the Semtech stack it hangs waiting for a stack status message. The other board is always complaining regarding MAC timeout.

So, all in all, as a first experience with RIOT-OS on the TTGO board, excepting the Lorawan stack, everything seems to work as expected. Regarding the Lorawan stack, probably going to try the LMIC Arduino implementation to see if it is possible to port it to RIOT-OS.