With the low cost of the esp8266 chip and boards, it is now easy and affordable to add connectivity to almost anything that we can think of. Still if we build an Internet of Things device by enabling it with the esp8266 chip to connect to the internet, or other networks, what we can do with the data that we gather? Basically we do what we do with any data that we get, store it, process it and show it, and that’s what we are going to see on this post.
The software (running on Linux platform) that I’ll use is the following:
MQTT Broker Mosca: (http://www.mosca.io/) There are several MQTT brokers available, but this one runs on Node-Js and so is platform agnostic, and runs where Node-Js runs. Also it has websocket support out of the box, which allows to use the Freboard.io dashboard in a straight foward way.
Freeboard.io Dashboard: (http://freeboard.io/) Is a great dashboard and freely available at the project GitHub page (https://github.com/Freeboard/freeboard). It allows to design and store several dashboards with some cool widgets.
Step 1: Setting up NodeRed:
First of all make sure that you have node and npm (the node package manager) installed on your distribution. You can see that with the commands node -v and npm -v for checking out their versions. In my case, and for now, it’s version v0.10.33 for nodejs and 1.4.28 for npm.
First I’ve created a directory named IoTServer where I’ll keep all software and configurations.
Download the zip file from nodered GitHub page https://github.com/node-red/node-red or just clone the repository. I clone the repository, because it’s simpler in the future to update it with just git pull. First just make sure that you have the icu/libicu libraries installed.
cd ~ mkdir IoTServer cd IoTServer git clone https://github.com/node-red/node-red cd node-red npm install
We can now run the nodered server with the command node red.js -v. I use -v at the end to see if any error crops up and also to see if any node modules are missing, and need to be installed. For example:
pcortex@cloudsrv:~/IoTServer/node-red$ node red.js -v Welcome to Node-RED =================== 24 Feb 11:35:16 - [info] Node-RED version: v0.10.3.git 24 Feb 11:35:16 - [info] Node.js version: v0.10.33 24 Feb 11:35:16 - [info] Loading palette nodes 24 Feb 11:35:59 - [warn] ------------------------------------------ 24 Feb 11:35:59 - [warn] [arduino] Error: Cannot find module 'arduino-firmata' 24 Feb 11:35:59 - [warn] [rpi-gpio] Info : Ignoring Raspberry Pi specific node. 24 Feb 11:35:59 - [warn] [redisout] Error: Cannot find module 'redis' 24 Feb 11:35:59 - [warn] [mongodb] Error: Cannot find module 'mongodb' 24 Feb 11:35:59 - [warn] ------------------------------------------ 24 Feb 11:35:59 - [info] Server now running at http://127.0.0.1:1880/ 24 Feb 11:35:59 - [info] Flows file not found : flows_clouds.json 24 Feb 11:35:59 - [info] Starting flows
As we can see the node modules arduino, rpi-gpio, redisout and mongodb are missing. If we are going to use them, we need to install them with the npm tool. Also the following message: Flows file not found : flows_clouds.json informs us that no saved nodered workflows where found. We can now access the nodered at adress http://localhost:1880/. If we want to change the bind address or add authentication, we should change the settings.js file. Take a look at http://nodered.org/docs/configuration.html for more information, and start nodered with: node red.js -v -s settings.js
We can now start designing our nodered workflows that receive data from the esp8266 via MQTT protocol or even by simple REST based HTTP protocol, process it and store the data payload, if any.
For storing data into a database, we can use MongoDB, or mysql, for example, installing the needed modules.
For mysql we can do:
cd ~/IoTServer/node-red npm install bignumber.js require-all readable-stream npm install mysql npm install node-red-node-mysql
And we need to restart nodered. The mysql node should be available now in storage pallet.
Step 2: Installing Mosca MQTT broker (Alternative 1 for the MQTT Broker)
We can use several brokers, but since we already are running nodered on node-js, we will use Mosca. Both nodered and Mosca can provide a websocked based interface. This is of interest because we can use browser based websocket applications, like freeboard.io, to consume data from our IoT devices.
cd ~ sudo -s sudo npm install mosca bunyan -g
And to run it just do: mosca -v | bunyan
For running it with websockets enabled, on TCP port 3000, just do:
mosca -v --http-port 3000 --http-bundle --http-static ./ | bunyan
Check out the documentation for more information at: https://github.com/mcollina/mosca/wiki/MQTT-over-Websockets
Step 2: Mosquitto MQTT broker with Websockets support (Alternative 2 for the MQTT Broker)
If having trouble installing or using Mosca, we can use the latest Mosquitto version 1.4 that has websockets support.
For that we need to make sure that cmake and uuid/uuid-dev are installed (For example: sudo apt-get install cmake uuid uuid-dev).
Also we need to download the libwesockets library, compile it and install it:
cd ~/IoTServer git clone git://git.libwebsockets.org/libwebsockets cd libwesockets mkdir build cd build cmake .. make sudo make install
cd ~/IoTServer wget http://mosquitto.org/files/source/mosquitto-1.4.tar.gz tar xvzf mosquitto-1.4.tar.gz cd mosquitto-1.4
make sudo make install
listener 1883 listener 9001 127.0.0.1 protocol websockets
Then we run it with:
pcortex@cloudsrv:~/IoTServer$ mosquitto -c /etc/mosquitto/mosquitto.conf
1424790588: mosquitto version 1.4 (build date 2015-02-24 14:38:47+0000) starting 1424790588: Config loaded from /etc/mosquitto/mosquitto.conf. 1424790588: Opening ipv4 listen socket on port 1883. 1424790588: Opening ipv6 listen socket on port 1883. 1424790588: Opening websockets listen socket on port 9001.
Step 3: Installing freeboard.io dashboard
cd ~/IoTServer git clone https://github.com/Freeboard/freeboard.git
For now I’ll just use apache to serve the freeboard dashboards that is installed in my server. Because freeboard is not installed on the root web directory we need to make the freeboard available to Apache. The easiest way to do this, as long as Apache has the FollowSymLinks option enabled for the document root, is to create a link to the freeboard directory on the web document root:
sudo -s cd /var/www ln -s /home/pcortex/IoTServer/freeboard iot
And now freeboard is available at the url http://myserveraddres/iot.
We need now, and finally to add MQTT over Websockets support to freeboard… We are almost there. This post http://jpmens.net/2014/11/12/freeboard-a-versatile-dashboard/ shows how it’s done but I’ll repeat it here again:
We will now download the freeboard MQTT plugin:
cd ~/IotServer git clone https://github.com/alsm/freeboard-mqtt cd freeboard/plugins/mqtt cp ~/IoTServer/freeboard-mqtt/paho.mqtt.plugin.js .
We need to edit the paho.mqtt.plugin.js file and change the line:
"external_scripts" : [ "
and finally we need to change the index.html file from:
That’s it. We are finally ready. All infrastructure is done. We need now just to configure it.
Final step: Testing it out with mosquitto websockets!:
Let’s start Mosquitto: nohup mosquitto -c /etc/mosquitto/mosquitto.conf &
And check if the WS protocol is active:
netstat -nap | grep 9001 tcp 0 0 0.0.0.0:9001 0.0.0.0:* LISTEN 18973/mosquitto
Setting up Freeboard.io: It’s very important to notice that the Dashboard will run on our browser, making in fact our browser the client. This means that when we configure freeboard, we must take notice of this when filling up the MQTT broker address. It will be localhost if the browser AND the broker runs on the same machine, but for the dashboard to work anywhere we should use the public ip of the machine running the broker:
Access the freeboard.io dashboard, and select Datasources -> Add. From the dropdown box, we select the Paho MQTT provider.
We need to configure the data source as follows:
Change the MQTT server address to something that makes sense in your config.
For Mosquitto with websockets enabled, the port is 9001, and for Mosca, the port is 3000, but of course this can change, depending of the configuration used.
We can now add a Panel and a gauge to see data from the topic /data that we defined on the above datasource.
So, Add Pane, and on the new Pane select the + (plus) to add a widget. In our case we will select Gauge. We fill out the required info and on the Value field we press Datasource, and select the newly previous created datasource named Data and because we are not using JSON for data, we just append .msg at the end. So something like datasource[“DS_data”].msg should be writen.
We press Save, and that’s it.
We can now publish into the topic, and the dashboard should update:
For example, in this case: mosquitto_pub -t /data -m 23
That’s it. It works fine with Mosquitto 1.4 with websockets enabled and with the Mosca MQTT broker.
We should make our dashboard permanent and for doing so we need to save the dashboard locally as a JSON file, and move it to the server running the dashboard to the root directory of freeboard.
After that we call directly the saved dashboard with the following URL:
Edit: For Websocket support on the Freeboard.io Dashboard, please checkout Freeboard and WebSockets
Warning:The current freeboard.io project at Github (as of 12/2015) has problems saving dashboards when using Firefox. For saving dashboards use Chrome/Chromium.