MQTT Mosquitto broker – Client Authentication and Client Certificates

After seeing how to set up transport layer security for the Mosquitto MQTT broker by using the Transport layer security on this post, we need to see how to setup client authentication (only authorized clients can connect to the broker) either by using the common user/password based authentication method or using client certificates.

Authentication by user and password:
First let’s enable authentication to the broker by setting up user and password authentication. For enabling this kind of authentication we need to modify the broker configuration (/etc/mosquitto/mosquitto.conf in my case) file and change the following entries:

allow_anonymous false
password_file /etc/mosquitto/passwd_mqtt

Before restarting the MQTT broker we need to add some users to the passwd_mqtt file with the command mosquitto_passwd:

mosquitto_passwd -c passwd_mqtt user1

The -c parameter is for creating the initial password file, if doesn’t exist, otherwise it will overwrite it!. For adding new users or update the passwords just run the command without the -c parameter.

After restarting the broker will should not be able to logon anonymously:

# mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883 -d
Client mosqpub/25688-pcortex sending CONNECT
Client mosqpub/25688-pcortex received CONNACK
Connection Refused: not authorised.
Error: The connection was refused.

But providing the user name and password:

# mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883 -d -u user1 -P secret
Client mosqpub/25709-pcortex sending CONNECT
Client mosqpub/25709-pcortex received CONNACK
Client mosqpub/25709-pcortex sending PUBLISH (d0, q0, r0, m1, 'test', ... (7 bytes))
Client mosqpub/25709-pcortex sending DISCONNECT 

For subscription, the same has to be done, again by providing an user and password:

# mosquitto_sub -t \$SYS/broker/bytes/\# -v --cafile /etc/mosquitto/certs/ca.crt -p 8883 -u user1 -P secret
$SYS/broker/bytes/received 217
$SYS/broker/bytes/sent 20

Authentication by using client certificates
Using client certificates, signed by a certificate authority, assures the client identity. The certificate authority used must be the same used by the server certificates and is only supported over TLS/SSL.
For using client certificates for authentication, we need to change the listener configuration for TLS/SSL by adding the following directives:

# MQTT over TLS/SSL
listener 8883
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/certs/hostname.crt
keyfile /etc/mosquitto/certs/hostname.key
require_certificate true
use_identity_as_username true

The require_certificate directive with the value true means that clients must now provide a client certificate to connect.
The use_identity_as_username means that the user name of the connecting user is taken from the CN (Common Name) property of the certificate, otherwise we still need to provide an user and password.

With the above new configuration, we now can’t access the broker with user/password token:

mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883 -d -u user1 -P secret
Client mosqpub/26549-pcortex sending CONNECT
Error: A TLS error occurred.

the result on the log file is:

1478537500: New connection from 127.0.0.1 on port 8883.
1478537500: OpenSSL Error: error:140890C7:SSL routines:ssl3_get_client_certificate:peer did not return a certificate
1478537500: Socket error on client , disconnecting.

For accessing now the broker we must provide a client certificate and a private key. The client certificate must be generated from the same CA (Certificate authority) that created the server certificate, otherwise, the client certificate can’t be validated, and the connection fails:

1478537629: New connection from 127.0.0.1 on port 8883.
1478537629: OpenSSL Error: error:14089086:SSL routines:ssl3_get_client_certificate:certificate verify failed
1478537629: Socket error on client , disconnecting.

So on the same directory where the ca.crt and ca.key are residing execute the script to create the client certificate. For example for creating a client certificate for user1, we need to execute the following command:

# ./gen_CA.sh client user1

And we should have three files, two of them the user1.crt, the user certificate, and the user1.key, the user1 private key.

We can now logon to the broker:

# mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883 -d  --cert user1.crt --key user1.key 
Client mosqpub/30264-pcortex sending CONNECT
Client mosqpub/30264-pcortex received CONNACK
Client mosqpub/30264-pcortex sending PUBLISH (d0, q0, r0, m1, 'test', ... (7 bytes))
Client mosqpub/30264-pcortex sending DISCONNECT

If we check the log, we have the following:

1478539830: New client connected from 127.0.0.1 as mosqpub/27159-pcortex (c1, k60, u'user1').
1478539830: Client mosqpub/27159-pcortex disconnected.
1478601507: New connection from 127.0.0.1 on port 8883.

We can see that the broker extracted the username from the certificate property CN. If we don’t use the use_identity_as_username as true we need to provide the username and password and the client certificate. In this case, the certificate is validated, and the user used to logon might not be the same as the one defined on the CN certificate property. So without this directive we either need to allow anonymous logon again or define user and passwords.

Additional thoughts:
With user and password authentication we can revoke access to an user, by deleting it from the password file or changing the password.
But what about the authentication based on client certificates? As long the certificate is valid, the user can logon at will, since the broker will always accept it until the end of the certificate validation date.
The solution for this is to revocate the client certificate so when it is used, the broker rejects it. For this functionality, most Certificate Authorities provide the revocation list by providing either by CRL (Certificate Revocation list) file, or by OSCP (Online Status Certificate Protocol), and the server checks the client certificate on this list before allowing access. Mosquitto broker only works with CRL files.

We need to modify the listener configuration to verify the client certificates if it was revocated or not:

# MQTT over TLS/SSL
listener 8883
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/certs/halcyon.crt
keyfile /etc/mosquitto/certs/halcyon.key
require_certificate true
use_identity_as_username true
crlfile /etc/mosquitto/certs/ca.crl

The issue now is that the script that we are using for generating certificates doesn’t generate CRLs, and so we can’t revocate certificates by using this script.

We can do it by hand, but that wouldn’t make our life much easier, since revocating certificates means that the CA server must now track all certificates and their status.
One possible solution is to use scripts that allow to create (a not very secure) CA: easy-ca. These scripts will replace the genCA.sh script and allow the revocation of certificates.

Getting the easy-CA scripts:

git clone https://github.com/fcgdam/easy-ca.git

I’m omitting a lot of output that these scripts do, namely asking for the Organization name, CA passwords and so on.

Somewhere where we find suitable we create our CA:

Generating the CA:

# cd ~ 
# ~/easy-ca/create-root-ca -d CA
# cd CA

Then at the CA directory we created, we can now create our broker certificate and user certificate. Make sure that the names make sense.

Generating the broker certificate:

# bin/create-ssl -s brokername
# bin/create-client -c user1

At the end we have the following files:

ca/ca.crt  <- CA certificate
crl/ca.crl  <- CA CRL list

certs/brokername.server.crt  <- Server certificate
certs/user1.server.crt <- Client certificate

private/brokername.server.key
private/user1.server.key

We now have all the needed files for setting up our broker with support for CRL.

# sudo -s
# cp ~/CA/ca/ca.crt /etc/mosquitto/certs
# cp ~/CA/crl/ca.crl /etc/mosquitto/certs
# cp ~/CA/certs/brokername.server.crt /etc/mosquitto/certs/brokername.crt
# cp ~/CA/private/brokername.server.key /etc/mosquitto/certs/brokername.key

And restarting the broker we have the new certificates in place.
We can test now by using our CA user certificate:

# cd ~/CA
# mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883 -d  --cert certs/user1.client.crt --key private/user1.client.key 
Client mosqpub/7424-pcortex sending CONNECT
Client mosqpub/7424-pcortex received CONNACK
Client mosqpub/7424-pcortex sending PUBLISH (d0, q0, r0, m1, 'test', ... (7 bytes))
Client mosqpub/7424-pcortex sending DISCONNECT

It works. Now let’s revoke the user certificate:

# bin/revoke-cert -c certs/user1.client.crt 

Revoking certificate 'certs/user1.client.crt'

Reason for revocation: 

1. unspecified
2. keyCompromise
3. CACompromise
4. affiliationChanged
5. superseded
6. cessationOfOperation
7. certificateHold

Enter 1-7 [1]: 2
You are about to revoke this certificate with reason 'keyCompromise'.
Are you SURE you wish to continue? [y/N]: y
Using configuration from conf/ca.conf
Revoking Certificate 02.
Data Base Updated
Using configuration from conf/ca.conf

Server certificate revoked.

We need to copy the new CRL to the directory where mosquitto expects the CRL:

# sudo cp crl/ca.crl /etc/mosquitto/certs

And after restarting the broker we have:

mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883 -d  --cert certs/user1.client.crt --key private/user1.client.key  
Client mosqpub/7640-pcortex sending CONNECT
Error: A TLS error occurred.

And on the logs:

1478625909: OpenSSL Error: error:14089086:SSL routines:ssl3_get_client_certificate:certificate verify failed
1478625909: Socket error on client , disconnecting.
1478625910: New connection from 127.0.0.1 on port 8883.

MQTT Mosquitto broker with SSL/TLS transport security

Just a quick note in setting up transport layer security on the MQTT Mosquitto broker for both supported protocols: MQTT and WebSockets.

There are several posts on the web regarding this, namely:

SSL Client certs to secure mqtt and Mosquitto websocket support

Those posts explain more or less what is needed to be done to have TLS/SSL transport security. These are just my notes:

Generating the server certificates:
This can be quite easily accomplished by using the following script: https://github.com/owntracks/tools/blob/master/TLS/generate-CA.sh.
This script will generate a self signed certificate to be used by Mosquito for providing TLS for the MQTT and WebSocket protocol. All that is needed to run the script is to have openssl installed on your Linux machine.

If the script is called without parameters, it will generate a self signed certificate for the hostname where the script is running. Otherwise a we can pass a hostname as the first parameter to the script.

After running the script, the following files are generated:

  1. ca.crt – The CA (Certificate Authority, who published the host certificate) public certificate.
  2. hostname.crt – The hostname, that will run the mosquitto broker, public certificate.
  3. hostname.key – The hostname private key.

After having these files, we need to configure the Mosquitto Broker to use them.

Mosquitto configuration:
To configure the Mosquito broker we need first to copy the certificates and key files to a known directory. We will create a certs directory under /etc/mosquitto:

sudo -s
mkdir -p /etc/mosquitto/certs
cp ca.crt /etc/mosquitto/certs
cp hostname.* /etc/mosquitto/certs

After this we can modify the mosquitto configuration file. One important thing to keep in mind is that lines must be following each other without blank lines after the listener directive.

So:

# Plain MQTT protocol
listener 1883

# End of plain MQTT configuration

# MQTT over TLS/SSL
listener 8883
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/certs/hostname.crt
keyfile /etc/mosquitto/certs/hostname.key

# End of MQTT over TLS/SLL configuration

# Plain WebSockets configuration
listener 9001
protocol websockets

# End of plain Websockets configuration

# WebSockets over TLS/SSL
listener 9883
protocol websockets
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/certs/hostname.crt
keyfile /etc/mosquitto/certs/hostname.key

We will make one more change, but restart mosquitto broker now and do some testing.

Testing MQTT TLS/SSL configuration
We can use Mqtt-Spy to subscribe to our defined test topic: test. We can use plain MQTT or use MQTT over TLS/SSL:

MQTT Spy simple TLS configuration

MQTT Spy simple TLS configuration

We can use then the MQTT spy tool to publish or subscribe MQTT topics.

By command line, the mosquitto_sub and mosquitto_pub only worked if the port number for MQTTS is provided, otherwise it gives a TLS error:

mosquitto_pub --cafile /etc/mosquitto/certs/ca.crt -h localhost -t "test" -m "message" -p 8883

mosquitto_sub -t \$SYS/broker/bytes/\# -v --cafile /etc/mosquitto/certs/ca.crt -p 8883

This should work without any issues.

Testing MQTT websockets over TLS/SSL configuration
The issue with this testing is that we are using a self signed certificate, so only useful for local, restricted, testing.
Before we can use the MQTT websockets with TLS/SSL enabled we need to use the browser and visit the following URL:

  https://MQTT_BROKER_IP_OR_HOSTNAME:9883/mqtt

Note that we are using HTTPS. When connecting to the above URL, the browser should complain about the insecure connection, due to the self signed certificate, and we need to add an exception and always accept that certificate. After that the error should be something like connection reset or failed to load page. This is normal, since the browser won’t upgrade the connection to a web socket.
We can now use the Hive MQTT Websockets Client to test our connection, and it should work fine (Note the connected green icon and SSL is selected):
Hive MQTT WebSocket client

Forcing TLSv1.2
All this work of enabling TLS/SSL on the Mosquitto Broker is needed, since most IoT clouds that have MQTT interface need that the connection is over TLS/SSL. More specifically AWS IoT cloud needs the connection to be protected by TLS/SSL, but that connection must be only on version 1.2 of the TLS protocol. AWS IoT cloud also requires client authentication through client certificates, but we are not dealing with this part on this post.

So we are now configuring our Mosquitto broker to only accept TLSv1.2 connections. To do that we modify the mosquitto.conf file and add the following line:

# MQTT over TLS/SSL
listener 8883
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/certs/hostname.crt
keyfile /etc/mosquitto/certs/hostname.key
tls_version tlsv1.2

# WebSockets over TLS/SSL
listener 9883
protocol websockets
cafile /etc/mosquitto/certs/ca.crt
certfile /etc/mosquitto/certs/hostname.crt
keyfile /etc/mosquitto/certs/hostname.key
tls_version tlsv1.2

and restart the broker.

Testing TLS V1.2
We can specify the TLS version on the mosquitto command line utils:

[pcortex@pcortex:~]$ mosquitto_pub --cafile ./ca.crt --tls-version tlsv1.2 -h localhost -t "test" -m "mes" -p 8883 -d
Client mosqpub/26994-pcortex sending CONNECT
Client mosqpub/26994-pcortex received CONNACK
Client mosqpub/26994-pcortex sending PUBLISH (d0, q0, r0, m1, 'test', ... (3 bytes))
Client mosqpub/26994-pcortex sending DISCONNECT
[pcortex@pcortex:~]$ mosquitto_pub --cafile ./ca.crt --tls-version tlsv1.1 -h localhost -t "test" -m "m3224" -p 8883 
Error: A TLS error occurred.

As we can see the lower versions of the TLS protocol are now not accepted.
The Websockets client should work without any issues.

Conclusion:
This configuration only solves the transport security, not the authentication security. The later can be accomplished by using the username/password process or using client certificates, which is the process that Amazon AWS IoT cloud uses. But those are topics for other posts. Edit: Follow-up at: Client authentication

MQTT Dashboards

The freeboard.io dashboard is one of the most popular dashboards to build an interface for displaying data. Still, for supporting the MQTT protocol some configuration is needed that is not exactly straight forward. Also if don’t want to use MQTT but web sockets, a similar not straight forward configuration is also needed. After that configuration (MQTT support for freeboard.io and WebSocket support for freeboard.io) we can add MQTT and Web Sockets datasources to our dashboards, and we get a nice customable dashboard framework.

Anyway there are some nice alternatives available that are worth some attention for someone that wants a simpler way of doing things:

CroutonCrouton MQTT Dashboard
Node Red UINode-Red-ui

Crouton
This dashboard is really nice since it has something that we might call as dashboard auto-configuration. We only need to configure the MQTT broker (with web sockets enabled, off course) and add device names to the configuration list. The device name is very important because it will be used in MQTT topic subscriptions (see below).

The Crouton dashboard configuration is stored on the browser local storage, so it’s not central. This means, that if you change browser or machine the configuration needs to be done again for that specific instance of browser and/or machine. Also, based on my experience, this is rather fidly, sometime it saves, other times it won’t, and need to reconfigure everything again after closing and opening the browser again.

The Crouton MQTT configuration is simple:

Crouton MQTT Configuration

MQTT Configuration

After the dashboard is connected to the MQTT broker, it searchs for specific topics based on the names configured on the Devices list. For each device it publishes to topic /inbox/devicename and subscribes to /outbox/devicename. At the initialization phase the Dashboard sends a get message to /inbox/devicename/deviceInfo. We can see that the device name is used to “find” a MQTT subscriber that answers to this topic. The subscriber that “owns” the topic /inbox/devicename answers to the get message with a JSON description of the device parameters to be shown at the dashboard on the /outbox/devicename/deviceInfo topic.

For example, on the following configuration:

Device Configuration

Device Configuration

The Crouton Dashboard sends to /inbox/Odroid/deviceInfo (Case Sensitive!!) and to /inbox/DiskStation/deviceInfo the message get and waits for a specific JSON response on /outbox/Odroid/deviceInfo and /outbox/DiskStation/deviceInfo to know how to automatically create the dashboard. With another words, the JSON response defines the dashboard. This means that to add a new metric, for example, we just need to add on the device the new metric and report it, no need to mess again with the dashboard configuration.

Auto Generated MQTT Dashboard

We can now use the mouse to move around the tiles, but the it seems also that this configuration isn’t stored.

This is the neat part of Crouton Dashboard, this auto-configuration.

The configuration is defined on the JSON response to /outbox/deviceName/deviceInfo MQTT topic. The JSON object defines the number of metrics and display format that device publishes and automatically associated to the metric name there is the MQTT topic /outbox/deviceName/Metric that the dashboard listens to.

On the Crouton GitHub there are two examples of this, one in Python and other in Lua for the ESP8266 NodeMCU based firmware.

I have a simpler one in Python, derived from the original Python example, for retrieving and send metrics from my Odroid SBC. The code is available here.

On the example note two things:

At the all.py file line 9, the clientName variable is set to Odroid. This is the device name to set on the Crouton dashboard.

Then for example, I publish my root disk space with the following JSON object segment:

        ....
        ....
        "endPoints": {
            "RootDisk": {
                "values": {
                    "labels": [],
                    "series": [0]
                },
                "total": 100,
                "centerSum": true,
                "units": "%",
                "card-type": "crouton-chart-donut",
                "title": "Root Disk"
            },....
            .....

The above configuration means that the end point /outbox/Odroid/RootDisk will have some value published that should be shown as a doughnut chart with % units.

A bit below on the code, there is an infinite loop, that each 30 seconds publishes to the topic the available disk space in percentage:

   client.publish("/outbox/"+clientName+"/RootDisk" , '{"series":['+str(disk_usage(rootPath))+']}')

As a conclusion, this Dashboard is rather interesting and worth a look, even with it’s, for now, issues with saving the configuration.

Node-Red-UI
While the Crouton Dashboard is an standalone Node.JS based application, the Node-Red-UI is an extension the Node-Red itself. It adds a new range of UI nodes to the Node-Red available nodes that allows to also build a Dashboard, but this time, not limited to MQTT. In fact any node that can inject data into a node-red-ui works.

The UI is available on the same address as Node-Red, but at the path /ui: Example:http://localhost:1880/ui/

For example, the following flow:

Node-Red-UI Flow

Node-Red-UI Flow

Generates the following UI:

Node-Red-UI

Node-Red-UI

The UI is based on the same MQTT topics and data published by my Crouton Python Client. The only thing needed is to extract the data from the MQTT message:

var space = msg.payload.series[0];
var msg ={"value":space , "payload":space};
return msg;

And the data feed from the “agents” work on both dashboards.

Anyway, regarding Node-Red-UI, it is an all other world, and I recommend to join and/or read the discussion group for this project.

ODroid – Mosquitto MQTT Broker install

Just for a quick reference, the following instructions detail how to install the latest Mosquitto MQTT broker with Websockets enabled on the ODroid C1+ running (L)Ubuntu release. The instructions are probably also valid for other platforms, but not tested.

1. Install the prerequisites
As the root user, install the following libraries:

apt-get update
apt-get install uuid-dev libwebsockets-dev

Probably the SSL libraries, and a few others are also needed, but in my case they where already installed.

2. Mosquitto install
Download and compile the Mosquitto broker:

mkdir ~/mosq
cd ~/mosq
wget http://mosquitto.org/files/source/mosquitto-1.4.5.tar.gz
tar xvzf mosquitto-1.4.5.tar.gz
cd mosquitto-1.4.5/

Edit the config.mk file to enable the websockets support:

# Build with websockets support on the broker.
WITH_WEBSOCKETS:=yes

and compile and install:

make
make install

3. Configuration
Copy the file mosquitto.conf to /usr/local/etc, and edit the file:

cp mosquitto.conf /usr/local/etc
cd /usr/local/etc

Add at least the following lines to mosquitto.conf file to enable websockets support.

# Port to use for the default listener.
#port 1883
listener 1883
listener 9001
protocol websockets

Add an operating system runtime user for the Mosquitto daemon:

useradd -lm mosquitto
cd /usr/local/etc
chown mosquitto mosquitto.conf

If needed, or wanted, change also on the configuration file the logging level and destination.
For example:

# Note that if the broker is running as a Windows service it will default to
# "log_dest none" and neither stdout nor stderr logging is available.
# Use "log_dest none" if you wish to disable logging.
log_dest file /var/log/mosquitto.log

# If using syslog logging (not on Windows), messages will be logged to the
# "daemon" facility by default. Use the log_facility option to choose which of
# local0 to local7 to log to instead. The option value should be an integer
# value, e.g. "log_facility 5" to use local5.
#log_facility

# Types of messages to log. Use multiple log_type lines for logging
# multiple types of messages.
# Possible types are: debug, error, warning, notice, information, 
# none, subscribe, unsubscribe, websockets, all.
# Note that debug type messages are for decoding the incoming/outgoing
# network packets. They are not logged in "topics".
log_type error
log_type warning
log_type notice
log_type information

# Change the websockets logging level. This is a global option, it is not
# possible to set per listener. This is an integer that is interpreted by
# libwebsockets as a bit mask for its lws_log_levels enum. See the
# libwebsockets documentation for more details. "log_type websockets" must also
# be enabled.
websockets_log_level 0

# If set to true, client connection and disconnection messages will be included
# in the log.
connection_messages true

# If set to true, add a timestamp value to each log message.
log_timestamp true

4. Automatic start
The easiest way is to add the following file to the init.d directory and link it to the current runlevel:

Create the file named mosquitto under /etc/init.d

#!/bin/bash

case "$1" in
    start)
        echo "Starting Mosquitto MQTT Broker"
        touch /var/log/mosquitto.log
        chown mosquitto /var/log/mosquitto.log
        su - mosquitto -c "/usr/local/sbin/mosquitto -c /usr/local/etc/mosquitto.conf" & > /dev/null 2>/dev/null
        ;;
    stop)
        echo "Stopping Mosquitto MQTT Broker"
        killall mosquitto
        ;;
    *)
        echo "Usage: /etc/init.d/mosquitto start|stop"
        exit 1
        ;;
esac

exit 0

Find out the current run level. On Odroid it seems that is level 2:

root@odroid:/etc/init.d# runlevel
N 2
root@odroid:/etc/init.d# 

And link the automatic start for Mosquitto broker at run level 2:

cd /etc/rc2.d
ln -s  /etc/init.d/mosquitto S97mosquitto

And that’s it.

We can start manually the broker with the command /etc/init.d/mosquitto start, and stop it with the mosquitto stop command.

Sming – MQTT (part 2)

While doing some test with MQTT using the Sming Framework for the ESP8266 I had some issues that are worthy to be documented.
The Sming Framework uses the libemqtt library for MQTT support, and it seems it works fine. Connection, subscribing and publishing work out of the box, and one important thing, the keep alive mechanism also works. We can see how MQTT can be used with the Sming framework in here.

But what is this keep alive mechanism on the MQTT protocol? The MQTT protocol specification defines this:

It is the responsibility of the Client to ensure that the interval between Control Packets being sent does not exceed the Keep Alive value. In the absence of sending any other Control Packets, the Client MUST send a PINGREQ Packet.

This means that before a keep alive time is reached the client must do a request to the MQTT broker, and in case it doesn’t, it should send a PINGREQ.

For example, on a Mosquitto broker with logs enabled I can have this:

1450189246: New client connected from 192.168.1.200 as ESP8266 (c1, k10)

It shows that a client connected from 192.168.1.200 with the client identification ESP8266 and it defined as the keep alive interval 10s.

This means that at least before a ten second period is reached, the esp8266 MQTT client should send a PINGREQ or do something, like publishing to a topic.

The above requirements are probably the root of my problems and the fact that the ESP8266 chip/firmware can’t do too many things at once….

The test that I was doing was that I didn’t published nothing from the ESP8266 to the MQTT broker. All I was doing was to subscribe some MQTT topic and print the messages on the I2C 16×02 LCD that I have connected to the ESP8266.

Without any activity the esp8266 stayed connected to broker without any problems.

If I published something to the topic subscribed by the esp8266, it worked fine.

But if I published something massively to the topic subscribed, then the broker would drop the connection:

1450190071: Socket error on client ESP8266, disconnecting.

And that was a bit odd…

So at a first approach, I added a check timer to see if the MQTT connection was valid, and to reconnect if it wasn’t:

// Connect to the MQTT broker.
// If BRUSER is not empty, connect with user and password.
void startMqttClient()
{
    Serial.println("Mqtt server connect & subscribe..");   
    String mqttId = "esp8266-" + String(system_get_chip_id());

    if ( strlen( BRUSER ) == 0 )   
        mqtt.connect( mqttId );
    else
        mqtt.connect( mqttId, BRUSER , BRPWD );
    
    // Let's subscribe topics    
    mqtt.subscribe("/data");
    
}

void checkMQTT() {
        if (mqtt.getConnectionState() != eTCS_Connected) {
            Serial.println("MQTT Server: Lost connection... ");
            Serial.println("MQTT Server: Reconnect");
            startMqttClient(); // Auto reconnect
        }
}

And the timer that is defined to check the MQTT connection:

Timer checkTimer;

void startMain() {
  //...
  //...
  
  startMqttClient();  // Connect to the MQTT broker.
  checkTimer.initializeMs( 60 * 1000 , checkMQTT).start();
}

So every minute the connection to the broker is checked, and the reconnection, if needed, happens automatically.

But this solution doesn’t explain why the connection is lost when the burst of messages arrive at the topic.

Anyway to keep the long story short, the burst duration of messages where superior to the keep alive interval. Since that during that period I didn’t publish nothing, AND, the Mqtt library was busy to receive the incoming data burst, the broker assumed that the esp8266 was dead, and drop the connection.

Since the default keep alive was 10s, that value was too short to keep the connection stable under load.

The real solution is to increase the MQTT keep alive interval, to something that is bigger that any duration of probable data bursts. So what is needed is to specify a higher keep alive interval:

void startMqttClient()
{
    Serial.println("Mqtt server connect & subscribe..");   
    String mqttId = "esp8266-" + String(system_get_chip_id());

    mqtt.setKeepAlive(180);  // Keep alive of 180 seconds...

    if ( strlen( BRUSER ) == 0 )   
        mqtt.connect( mqttId );
    else
        mqtt.connect( mqttId, BRUSER , BRPWD );
    
    // Let's subscribe topics    
    mqtt.subscribe("/data");
    
} 

And with this change we got at the broker:

1450190936: New client connected from 192.168.1.200 as ESP8266 (c1, k180)

I can now burst messages to the esp8266 MQTT client without the connection being dropped.

So as long that incoming burst of messages are under 180 seconds ( which is easy) or some publishing from the ESP8266 happens, the connection stays up. Anyway, if it fails, the check timer will reconnect again to the broker.

SparkFun Phant server data stream graphing

This application Phant Graphs is a simple AngularJs and ChartJs single page web application that allows to configure and create a dashboard of graphs that display stream data available on Sparkfun Phant servers, either localy, or on the internet (data.sparkfun.com for example).

The datasources ( servers and streams) can be configured rapidly, the graphs types can be chosen and also the fields that make up the graphs can also be chosen, so we can have some nice graphs without coding a single line. The application also pools periodically the data streams for the latest data and updates the graphs if needed.
For example, on the following image, for the same data stream there are several graphs, where each one show a different set of fields on different graph type. This is useful, and needed, for example for graphing temperature and pressure. Due to the scale differences between unit values a single graph won’t show properly the temperature for a graph that also has the pressure field.

PhantGraphs Dashboard

PhantGraphs Dashboard

How to deploy
Deploy the supporting rest server: https://github.com/fcgdam/PG-RestServer.git, and start the server. The provided database already has some configuration done.
Deploy PhantGraphs on a web server, and change the REST server endpoint on the restapi.js file to point to the correct location of the Rest Server.

Configuring
To configure, goto Config->Phant Servers where we can add, edit and delete servers. We should provide a name and the base URL for the Phant Server.
Please notice, that at least at this stage, the REST server doesn’t provide referential integrity on the data, so it’s a bad idea to delete a server that has streams and graphs associated…

Phant Servers

Phant Servers

After a server is configured, we can associate data streams to it. Just goto Config->Phant Streams and create a stream. The stream public key is needed, so we have to get it on the Phant server web interface. The PhantGraphs application will try to fetch from the stream the available fields.

Phant Streams

Phant Streams

If it fails, an error will appear at the top of the page.

After all the streams are configured, we can now create the graphs for them, one or more, with different graph types and for different stream fields.
Some things are not quite ready here, namely the graph display order, that must be defined, but it’s not used for now.

Phant Graphs configuration

Phant Graphs configuration

We can play around with the Graph type, like Line, Bar, Pie, and so on. The option Single Bars only applies to Bar Graphs, that for these type of graphs instead of using a time series for the X axis, it uses only bars for the fields defined.

After this, we just go to Dashboard and enjoy our graphs.

On Config->Dashboard we can change the refresh period for getting the latest data. This value shouldn’t be to low because it will overload the browser and the servers. We can also define the number of datapoints shown on the graphs, the number of errors that the application will endure before stopping the automatic refresh. To resume, just refresh the page or click on any graph. The label interval is used in the Line graph charts so that the X axis doesn’t become too overcrowded. Anyway, just play with the values.

MQTT
The MQTT is just a simple MQTT over web sockets monitor, that can be configured in Config->MQTT Server. The MQTT broker must support MQTT over web sockets, for example Mosquitto 1.4 or Mosca.

The code is based on the flm-ui application available on Github.

IoT Device Provisioning
The application also allows to list IoT devices, namely the ESP8266, that registered on the RestServer, check their status, and make a simple JSON based properties configuration that is requested and loaded by the devices:

IoT Device Provisioning

IoT Device Provisioning

Configuring properties for the device:

Selection_240

Check out for more information: IoT device provisioning and tracking

And that’s it. Enjoy your new graphs.

Mosquitto Broker with Websockets enabled on the Synology NAS

Synology NAS are great devices, mainly due to the bundled software (CloudStation, Photo Station, to name a few), but also because it can run other software that is provided on non official package sources.
One of such providers is the SynoCommunity that provides a Mosquitto MQTT broker packaged for the Synology NAS. While I’ve successfully was able to cross compile the Mosquitto broker for my DS212+ NAS with websockets enabled (I do have a draft post with the steps that I’ve never published), the SynoCommunity package allows a simpler way to install and use a Mosquitto broker that has the WebSockets protocol support compiled in.

So, what is needed to be done?

Setting up:

  • 1st: Add to your Synology server the SynoCommunity package source. See the detailed instructions on the community web page.
  • 2nd: Install the Mosquitto package from the Synology package manager.
  • 3rd: Edit the file located at /usr/local/mosquitto/var/mosquitto.conf. You need to connect through ssh as root to do this.

At the end of this file add the following lines:

listener 1883
listener 9001
protocol websockets

Note that the 1883 is the standard MQTT broker port, and 9001 is the port that I’ve choose have the websockets server listening/answering.

After the change, stop and start the Mosquitto broker on the package manager action dropdown box for the Mosquitto package.

Still using ssh, and logged in as root, check now that the port 9001 is active:

root@DiskStation:/volume1/homes/root# netstat -na | grep 9001 
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN       
tcp        0      0 192.168.1.250:9001      192.168.1.32:55331      ESTABLISHED

In the above case we can see that there is a websocket client connected to the broker from my workstation.

Testing:

For testing we can use the HiveMQ websocket client: http://www.hivemq.com/demos/websocket-client/ that allows to communicate using the websockets transport.

Despite the fact that the page is available on an external, internet based server, the websocket client will be running on YOUR web browser, on your machine in your own (internal) network, so at the connection settings for the broker, for Host I use my internal IP address: 192.168.1.250, and for the Port: I use the 9001 value.

I can now subscribe and publish to topics using the browser, and also using MQTT-SPY we can see and check the MQTT Websockets communication.

And that’s it!