When updating breaks projects…

A quick post regarding updating platforms and libraries for projects, specifically projects for the ESP8266 platforms:

The PZEM004T is device available on eBay and other sites, that allows to measure energy consumption. The PZEM004T has a serial output port and when connecting it to, for example, an ESP8266, we can access the collected data trough WIFI and process it for finding out how much electricity we are using, and so on.

Anyway, when using the ESP8266 WEMOS D1 mini, to be able to still use the USB serial port, we need to use Software serial emulation. In fact the Arduino PZEM004T library available on Github and on the Platformio library registry allows the use of the Software Serial to communicate with the PZEM004T (and it works just fine).

So what is the issue?

I’m using Platformio to develop the ESP8266 application, and normally when running, it checks and offers any updates that might be available. So, I’ve updated to the latest Espressif ESP8266 platform and ESPSoftwareSerial, and then everything just break down:

  • ESPSoftwareSerial last version just completely breaks the previous existing API which made the PZEM004T library also broken.
  • The new ESP8266 platform removes Esp8266 a SDK attachInterruptArg function which renders the ESPSoftwareSerial library unbuildable

The solution?

The solution with Platformio is quite easy: use semantic versioning.

In fact something like this on the platformio.ini file:

[env:d1_mini]
platform = espressif8266
...
...

can be locked to a working previous version:

[env:d1_mini]
platform = espressif8266@2.0.3
...
...

The same can be done with the project libraries. While the ESPSoftwareSerial (the PZEM004T dependency loads it) does not need to be defined, specifying it allows to use a specific version:

[env:d1_mini]
platform = espressif8266@2.0.3
board = d1_mini_lite
framework = arduino
upload_speed = 921600
monitor_speed = 115200

lib_deps =  ESPSoftwareSerial@5.0.3
            PZEM004T
            MQTT
            LiquidCrystal_I2C
            SimpleTimer
            ESPAsyncTCP
            ESP Async WebServer
            Time

And with this, the ESP8266 platform and ESPSoftwareSerial versions locked, the issues with the newer versions are avoided, and the code compiles and works as it should.

So, updating is fine, but when it breaks it can be an issue. Fortunately Platformio allows the usage of specific version for building our projects, and even allows to deploy our specific library version under the project lib directory.

Have fun!

Advertisements

ESP32 TTGO board and Nordic Thingy:52 sensor device

The Nordic Thingy:52 is a 40€/50€ device based on the nRF52832 Nordic microcontroller that has, in a ready to use package, several environmental sensors, that can be accessed by low power Bluetooth (BLE). Nordic provides a complete solution that comprises the Thingy:52 firmware already flashed on the device (Source at GitHub) and an very nice Android Nordic Thingy:52 application, with also sources available at GitHub.

Anyway I have some of these devices for some months now, for other uses, but I decided to test the ESP32 based boards, since the ESP32 has Bluetooth and theoretically can connect and gather gather data from the Thingy. So this post is about the use of the TTGO ESP32 Lora based boards with an OLED to gather data, show it on the OLED, and send it to The Things Network. Seems simple, right?

So when a application connects to the Thingy:52 it can be notified when a sensor value changes throught the standard BLE notification mechanisms. The way the Thingy firmware works, this notification happens at a fixed intervals instead of a value change, and that interval, 5 seconds, 10 seconds, be defined by the Android App or programmatically by our application.

The application is developed by using the PlatformIO and for using the ESP32 Bluetooth interface, I’ve used the NKolban ESP32 BLE Library that happens to be library 1841 at the Platformio repository.

To cut a long story short, as still of today, the ESP32 BLE library doesn’t work correctly with the Thingy:52 notifications. This means that the application subscribes to have notifications, but those never happen. Fortunately someone already hit this problem and solved the issue, but the correction still hasn’t hit the library.

So basically to have my code example to work the following steps are needed:

  1. 1. Clone the TTGO ESP32 repository from . The repository uses the PlatformIO to build the application.
  2. 2. At the root of the repository run the command pio run so that the libraries are downloaded and installed.

At this point we need to correct the Arduino ESP32 library to add the patch to the notification issue.
Just execute the command:

[pcortex@pcortex:ESP32_NordicThingy|master *]$ cd .piolibsdeps/ESP32\ BLE\ Arduino_ID1841/src

At this directory (.piolibsdeps/ESP32\ BLE\ Arduino_ID1841/src edit the file BLERemoteDescriptor.cpp and at around line 151 (the exact line number will probably change in the future) we must change the ::esp_ble_gattc_write_char_descr function parameters:

/**
 * @brief Write data to the BLE Remote Descriptor.
 * @param [in] data The data to send to the remote descriptor.
 * @param [in] length The length of the data to send.
 * @param [in] response True if we expect a response.
 */
void BLERemoteDescriptor::writeValue(
        uint8_t* data,
        size_t   length,
        bool     response) {
    ESP_LOGD(LOG_TAG, ">> writeValue: %s", toString().c_str());
    // Check to see that we are connected.
    if (!getRemoteCharacteristic()->getRemoteService()->getClient()->isConnected()) {
        ESP_LOGE(LOG_TAG, "Disconnected");
        throw BLEDisconnectedException();
    }

    esp_err_t errRc = ::esp_ble_gattc_write_char_descr(
        m_pRemoteCharacteristic->getRemoteService()->getClient()->getGattcIf(),
        m_pRemoteCharacteristic->getRemoteService()->getClient()->getConnId(),
        getHandle(),
        length,                           // Data length
        data,                             // Data
        ESP_GATT_WRITE_TYPE_NO_RSP,
        ESP_GATT_AUTH_REQ_NONE
    );
    if (errRc != ESP_OK) {
        ESP_LOGE(LOG_TAG, "esp_ble_gattc_write_char_descr: %d", errRc);
    }
    ESP_LOGD(LOG_TAG, "<< writeValue");
} // writeValue

We need to change the highlighted line to:

    esp_err_t errRc = ::esp_ble_gattc_write_char_descr(
        m_pRemoteCharacteristic->getRemoteService()->getClient()->getGattcIf(),
        m_pRemoteCharacteristic->getRemoteService()->getClient()->getConnId(),
        getHandle(),
        length,                           // Data length
        data,                             // Data
        response ? ESP_GATT_WRITE_TYPE_RSP : ESP_GATT_WRITE_TYPE_NO_RSP,
        ESP_GATT_AUTH_REQ_NONE
    );

With this change, the code at my github repositories has a working example:

– The ESP32 connects to the Nordic Thingy:52 device.
– It programs the Nordic Device to notify sensor values each 5 seconds (in real use cases it should be much larger period)
– Current sensor data is shown on the serial terminal.

What needs to be done:
– When notified by the Thingy:52, the ESP32 shows the new data on the OLED screen (WIP – Work in progress).
– To keep the application obeying the ISM bands duty cycle, it collects the data, calculates the medium, and sends the data to the Things network each 10 minutes (Also work in progress).

Docker container web interface – Portainer and Riot-OS Development

This post is a follow up of starting up with RIOT-OS. To be able to develop with RIOT-OS an easy (and easier) way to do so is just to install docker and web UI docker interface Portainer to control docker.

So we will install Docker, Portainer, and finally the RIOT-OS building environment.

Installing Docker and Portainer, is an initial stepping stone for using the dockerized development environment for RIOT-OS, since I don’t want to install all the development environments in my machine.

Installing Docker:
On Arch-Linux is as simple as installing the Docker package using pacman, enabling the services and rebooting.
Basically we need to run, as root the following commands:

pacman -S docker
systemctl enable docker.service
reboot

After rebooting the following command should return some information

docker info

A sample output is:

Containers: 2
 Running: 0
 Paused: 0
 Stopped: 2
Images: 9
Server Version: 18.09.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
...
...
...

Installing Portainer
Installing the Docker Portainer Web UI is as simple as:

docker pull portainer/portainer

To run Portainer a set of complete instructions on this page, but basically on the simplest way is:

$ docker volume create portainer_data
$ docker run -d -p 9000:9000 --name portainer --restart always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

We can now check if the docker image is up:

$ docker ps
CONTAINER ID        IMAGE                 COMMAND             CREATED             STATUS              PORTS                    NAMES
7a38ae7fc922        portainer/portainer   "/portainer"        4 seconds ago       Up 3 seconds        0.0.0.0:9000->9000/tcp   portainer

Since I have already ran the Portainer container, the initial setting up steps when accessing the URL HTTP://localhost:9000 do not appear, but we need to choose:

  1. A set of credentials to use as de administrator for portainer
  2. The local machine registry to connect to the local docker containers.

1- At initial access we define an user and password:

Portainer Credentials

2- Then we connect to our local docker instance:
Portainer Local Docker

Press Connect and then we can now access our Docker instance from Portainer:
Portainer Main Screen

Pressing the Local Docker Connection we can now manage our docker resources.

Installing the build environment for RIOT-OS
We can do it by two ways:

From the command line:

docker pull riot/riotbuild

or use Portainer:

This container is very big, so we need to wait some time for the container image download. The command line shows in greater detail the download process.

After the image is downloaded, we can follow these instructions for building our apps using the docker container as the build environment.

After the image is installed:

To use is is as simple as going the the examples directory and do:

make BUILD_IN_DOCKER=1

From this we are now able to build based RIOT-OS applications for several targets, including the ESP8266/ESP32.

As we can see we even don’t need to have a running container, just the image.

Simple BLE bridge to TTN Lora using the TTGO ESP32 LoRa32 board

The TTGO LoRa32 is an ESP32 based board that features Wifi and BlueTooth low energy but also includes an external Lora chip, in my case the SX1276 868Mhz version.

The following code/hack is just to test the feasibility of bridging BLE devices over the ESP32 and then to Lorawan, more specifically sending BLE data to the LoraWan TTN network.

I’m using Neil Koban ESP32 BLE library, that under platformIO is library number 1841 and the base ABP code for connecting to TTN.

In simple terms this code just makes the ESP32 to emulate a BLE UART device for sending and receiving data. It does that by using the Nordic UART known UUID for specifying the BLE UART service and using also the Nordic mobile applications, that supports such device, for sending/receiving data.

Using the Nordic mobile Android phone applications, data can be sent to the Lora32 board either by using the excellent Nordic Connect application or by also using the simpler and direct Nordic UART application.

The tests program just receives data through BLE and buffers it onto an internal message buffer that, periodically, is sent through Lora to the TTN network. I’ve decided arbitrary that the buffer is 32 bytes maximum. We should keep our message size to the necessary minimum, and also just send few messages to keep the lorawan duty factor usage within the required limits.

So, using the following code we can use our phone to scan from the ESP32 BLE device named TTGOLORAESP32 connect to it and send data to the device.

After a while, when the transmission event fires up, data is transmitted, and the BLE device just receives a simple notification with the EV_TXCOMPLETE message.

That’s it.

Cloud based deployment for IOT devices

Following up on my previous post Cloud based CI with Platformio, after we have the build output from the Continuous Integration process, we are able now to deploy to our devices.

This last deploy phase of the cycle Develop, CI, Deliver using Cloud infrastructure, only makes sense to devices that are powerful enough to have permanent or periodic network connectivity and have no problems or limitations with power usage, bandwidth, are in range and are able to remotely be updated.

In reality this means that most low power devices, devices that use LPWAN technologies like LoraWan or SigFox, devices that are sleeping most of the time and are battery powered are not able to be easily updated. For these cases the only solution is really out of band management by upgrading locally the device.

So the scope of this post is just to simply build a cloud based process to allow ESP8266 devices to get update firmware from the CI output. On it’s simplest form all we need is to create a web server, make the firmware available at the server and provide the URL for OTA updates to the ESP8266 that use the HTTP updater.

One can already use from the squix blog the PHP file to be deployed on PHP enabled web server that delivers the latest builds for devices requesting over the air updates.

Openshift PaaS Cloud Platform

The simplest way of making the Squix PHP page available on the cloud is to use the great Platform as a Service Openshift by RedHat. The free tier allows to have three applications (gears) available and the sign up is free. At sign up time we need to name our own domain suffix so that, for example I choose primal I’ll have URL’s such as application-primal.rhcloud.com.

Openshift offers a series of pre-configured applications ready to be deployed such NodeJs, Java, Python and PHP.

Openshift preconfigured platforms

So after sign up, all we need is to create a new application based on the PHP 5.4 template, give it an URL (it can be the default PHP), and that’s it: we have our PHP enabled web server.

Deploying code to Openshift

To deploy code to Openshift we use the Git tool for manipulating our application repository on the PaaS cloud platform.

So we must first clone our repository locally, modify it and then upload the changes.

For obtaining the repository URL and connection details, we must first setup our local machine with the rhc command line tool and upload our public SSH key to the Openshift servers:

 [pcortex@pcortex:~]$ gem install rhc

If the gem tool is not available, first install Ruby (sudo pacman -S ruby).

We then setup the rhc tool with the command rhc setup. Complete details here.

The command rhc apps should list now our Openshift applications:

[pcortex@pcortex:~]$ rhc apps
nodejs @ http://nodejs-primal.rhcloud.com/ (uuid: 9a72d50252d09a72d5)
-----------------------------------------------------------------------------
  Domain:     primal
  Created:    Aug 26  3:43 PM
  Gears:      1 (defaults to small)
  Git URL:    ssh://9a72d50252d09a72d5@nodejs-primal.rhcloud.com/~/git/nodejs.git/
  SSH:        9a72d50252d09a72d5@nodejs-primal.rhcloud.com
  Deployment: auto (on git push)

  nodejs-0.10 (Node.js 0.10) 
----------------------------             
    Gears: 1 small 
                    
php @ http://php-primal.rhcloud.com/ (uuid: c0c157c41271b559e66) 
-----------------------------------------------------------------------                    
  Domain:     primal          
  Created:    12:16 PM  
  Gears:      1 (defaults to small) 
  Git URL:    ssh://c0c157c41271b559e66@php-primal.rhcloud.com/~/git/php.git/                
  SSH:        c0c157c41271b559e66@php-primal.rhcloud.com 
  Deployment: auto (on git push) 

  php-5.4 (PHP 5.4)
  -----------------
    Gears: 1 small

You have access to 2 applications.

We pull now the remote repository to our machine:

[pcortex@pcortex:~]$ mkdir Openshift
[pcortex@pcortex:~]$ cd Openshift
[pcortex@pcortex:Openshift]$ git clone ssh://c0c157c41271b559e66@php-primal.rhcloud.com/~/git/php.git/
[pcortex@pcortex:Openshift]$ cd php
[pcortex@pcortex:php]$ wget https://raw.githubusercontent.com/squix78/esp8266-ci-ota/master/server/firmware.php 

We should now change the PHP file so it uses our repository to bring up our firmware:

 <?php
    $githubApiUrl = "https://api.github.com/repos/squix78/esp8266-ci-ota/releases/latest";
    $ch = curl_init();

And then it’s just to commit the change to Openshift:

[pcortex@pcortex:php]$ git add firmware.php
[pcortex@pcortex:php]$ git commit -m "Added firmware.php file"
[pcortex@pcortex:php]$ git push
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 924 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: Stopping PHP 5.4 cartridge (Apache+mod_php)
remote: Waiting for stop to finish
remote: Waiting for stop to finish
remote: Building git ref 'master', commit a72403a
remote: Checking .openshift/pear.txt for PEAR dependency...
remote: Preparing build for deployment
remote: Deployment id is 8fdecb3f
remote: Activating deployment
remote: Starting PHP 5.4 cartridge (Apache+mod_php)
remote: Application directory "/" selected as DocumentRoot
remote: -------------------------
remote: Git Post-Receive Result: success
remote: Activation status: success
remote: Deployment completed with status: success
To ssh://php-primal.rhcloud.com/~/git/php.git/
   321e48b..a72403a  master -> master

And that’s it: the link for HTTP OTA is available at http://php-primal.rhcloud.com/firmware.php

Final notes:

With the above firmware.php file we can deliver a single firmware file to any device that calls the page.

But a better solution is needed if we want to:

– Deliver multiple firmware files to different devices
– Deliver different versions of firmware files, for example be able to lock a specific version to some devices
– Know which devices have updated
– Know which version of firmware the devices are running

and of course, add some security.

ESP8266 – Logging data in a backend – AES and Crypto-JS

After building, on the previous posts, the Node-Red based backend to support E2EE (End to End Encryption) so we can log data into a central server/database, from our devices securely, without using HTTPS, we need now to build the firmware for the ESP8266 that allows it to call our E2EE backend.

The firmware for the ESP8266 must gather the data that it wants to send, get or generate the current sequence number for the data (to avoid replay attacks), encrypt the data and send it to the backend.
On the backend we are using the Java script library for cryptographic functions Crypto-js, and specifically we are encrypting data with the encryption algorithm AES. So all we need is to encrypt our data with AES on the ESP8266, send it to the Node-Red Crypto-js backend, decrypt it and store it, easy right?

Not quite, let’s see why:

Crypto-js and AES:
We can see that on my Node-Red function code and testing programs I’m using something similar to the following code example:

var CryptoJS = require("crypto-js");
var message  = "Message to encrypt";
var AESKey   = '2B7E151628AED2A6ABF7158809CF4F3C';

// Encrypt
var ciphertext = CryptoJS.AES.encrypt(message, AESKey );

console.log("Cypher text in Base64: " ,  ciphertext.toString(CryptoJS.enc.base64) );

// Decrypt
var bytes  = CryptoJS.AES.decrypt(ciphertext.toString(), AESKey );
var plaintext = bytes.toString(CryptoJS.enc.Utf8);

console.log("Decrypted message UTF8 decoded: ", plaintext);

Several points regarding the above code need clarification:

The code variable AESKey the way it is used on the above example encrypt and decrypt functions isn’t really a key but a passphrase from where the real key and an initialization vector or salt value is generated (I’m using the names interchangeably, but they are not exactly the same thing except they are public viewable data that must change over time/calls).
The use for the generated key is self explanatory, but the initialization vector (IV) or salt value is used to allow that the encrypted data output to be not the same for the same initial message. While the key is kept constant and secret to both parties, the IV/salt changes between calls, which means that the above code, when run multiple times, will never produce the same output for the same initial message.

Still referring to the above code, the algorithm that generates the key from the passphrase is the PBKDF2 algorithm. More info at Crypto-js documentation. At the end of the encryption the output is a OpenSSL salted format that means that the output begins by the signature id: Salted_, followed by an eight byte salt value, and after the salt, the encrypted data.

So if we want use the API has above on the node-js/crypto-js side, we need to implement on the ESP8266 side both the AES and PBKDF2 algorithms.

I decided not to do that, first because finding a C/C++ implementation of the PBKDF2 algorithm that could be portable and worked on the ESP822 proved difficult, and second the work for porting it to the ESP8266 won’t be needed if I use a KEY/IV directly, and so I decided to use the more standard way of providing an AES key and an initialization vector for encrypting and decrypting data.

In the case of Node-JS and Crypto-JS when using an explicit key and IV the code looks like:

var CryptoJS = require("crypto-js");
var request = require('request');

// The AES encryption/decription key to be used.
var AESKey = '2B7E151628AED2A6ABF7158809CF4F3C';

// The JSON object that we want to encrypt and transmit
var msgObjs = {"data":{"value":300}, "SEQN":145 };

// Convert the JSON object to string
var message = JSON.stringify(msgObjs);

var iv = CryptoJS.enc.Hex.parse('0000000000000000');
var key= CryptoJS.enc.Hex.parse(AESKey);

// Encrypt
var ciphertext = CryptoJS.AES.encrypt(message, key , { iv: iv } );

//console.log("Cypher: ", ciphertext );
console.log("Cypher text: " ,  ciphertext.toString(CryptoJS.enc.base64) );
console.log(" ");

console.log("=============================================================================");
console.log(" ");
console.log("Let's do a sanity check: Let's decrypt: ");

// Decrypt
var bytes  = CryptoJS.AES.decrypt(ciphertext.toString(), key , { iv: iv} );
var plaintext = bytes.toString(CryptoJS.enc.Utf8);

console.log("Decrypted message UTF8 decoded: ", plaintext);

Now, with above code, where the IV is always initialized to the same value, in this case ‘0000000000000000’, we can see when running the above code several times that the output is always the same since the IV is kept constant. Also the encrypted output is now just the raw encrypted data and not the Openssl format.

So to make the above code secure we must randomize the IV value for producing an output that is always different, even from several program runs when encrypting the same source data.

As a final note, if we count the number of HEX characters on the Key string, we find out that they are 16 bytes, which gives a total of 128 key bits. So the above example is using AES128 encryption, and with default Crypto-js block mode and padding algorithms which are CBC (Chain block mode) and pkcs7.

Interfacing Crypto-js and the ESP8266:
Since we are using AES for encrypting data and decrypting data, we need first to have an AES library for the ESP8266. The AES library that I’m using is this one Spaniakos AES library for Arduino and RPi. This library uses AES128, CBC and pkcs7 padding, so it ticks all boxes for compatibility with Crypto-js…

I just added the code from the above library to my Sming project and also added this Base64 library so that I can encode to and from Base64.

The only remaining issue was to securely generate a truly random initialization vector. And while at first I’ve used some available libraries to generate pseudo-random numbers to real random numbers, I’ve found out that the ESP8266 seems to have already a random number generator that is undocumented: Random number generator

So to generate a random IV value is as easy as:

uint8_t getrnd() {
    uint8_t really_random = *(volatile uint8_t *)0x3FF20E44;
    return really_random;
}

// Generate a random initialization vector
void gen_iv(byte  *iv) {
    for (int i = 0 ; i < N_BLOCK ; i++ ) {
        iv[i]= (byte) getrnd();
    }

So our ESP8266 code is as follow:

Global variables declarations:
The N_Block defines the encryption block size, that for AES128 is 16 bytes.

#include "AES.h"
#include "base64.h"

// The AES library object.
AES aes;

// Our AES key. Note that is the same that is used on the Node-Js side but as hex bytes.
byte key[] = { 0x2B, 0x7E, 0x15, 0x16, 0x28, 0xAE, 0xD2, 0xA6, 0xAB, 0xF7, 0x15, 0x88, 0x09, 0xCF, 0x4F, 0x3C };

// The unitialized Initialization vector
byte my_iv[N_BLOCK] = { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};

// Our message to encrypt. Static for this example.
String msg = "{\"data\":{\"value\":300}, \"SEQN\":700 , \"msg\":\"IT WORKS!!\" }";


The example function ESP8266 Sming code is:

void testAES128()  {

    char b64data[200];
    byte cipher[1000];
    byte iv [N_BLOCK] ;
    
    Serial.println("Let's encrypt:");
    
    aes.set_key( key , sizeof(key));  // Get the globally defined key
    gen_iv( my_iv );                  // Generate a random IV
    
    // Print the IV
    base64_encode( b64data, (char *)my_iv, N_BLOCK);
    Serial.println(" IV b64: " + String(b64data));
       
    Serial.println(" Mensagem: " + msg );
 
    int b64len = base64_encode(b64data, (char *)msg.c_str(),msg.length());
    Serial.println (" Message in B64: " + String(b64data) );
    Serial.println (" The lenght is:  " + String(b64len) );
    
    // For sanity check purpose
    //base64_decode( decoded , b64data , b64len );
    //Serial.println("Decoded: " + String(decoded));
    
    // Encrypt! With AES128, our key and IV, CBC and pkcs7 padding    
    aes.do_aes_encrypt((byte *)b64data, b64len , cipher, key, 128, my_iv);
    
    Serial.println("Encryption done!");
    
    Serial.println("Cipher size: " + String(aes.get_size()));
    
    base64_encode(b64data, (char *)cipher, aes.get_size() );
    Serial.println ("Encrypted data in base64: " + String(b64data) );
      
    Serial.println("Done...");
}

When the above code/function is executed on the ESP8266 it outputs the following:

Let's encrypt:
 IV b64: cAFviaDMHejlteGn9/4eQQ==
 Mensagem: {"data":{"value":300}, "SEQN":700 , "msg":"IT WORKS!" }
 Message in B64: eyJkYXRhIjp7InZhbHVlIjozMDB9LCAiU0VRTiI6NzAwICwgIm1zZyI6IklUIFdPUktTISIgfQ==
 The lenght is:  76
Encryption done!
Cipher size: 80
Encrypted data in base64: /1aZRwVaw3jv+ct8HS4pCV5lThvTG70M90ARiyAsIDYMkfJE3w8F3bgxaOKVA0rX4m1Mq50VVN0u9gRw9F2gKE4r2OcY8oECv8bKT80F9pY=
Done...

And now we can feed the above Base64 IV and encrypted data to our decoding program in Node-Js using Crypto-JS:

var CryptoJS = require("crypto-js");
var request = require('request');

var esp8266_msg = '/1aZRwVaw3jv+ct8HS4pCV5lThvTG70M90ARiyAsIDYMkfJE3w8F3bgxaOKVA0rX4m1Mq50VVN0u9gRw9F2gKE4r2OcY8oECv8bKT80F9pY=';
var esp8266_iv  = 'cAFviaDMHejlteGn9/4eQQ==';

// The AES encryption/decryption key to be used.
var AESKey = '2B7E151628AED2A6ABF7158809CF4F3C';

var plain_iv =  new Buffer( esp8266_iv , 'base64').toString('hex');
var iv = CryptoJS.enc.Hex.parse( plain_iv );
var key= CryptoJS.enc.Hex.parse( AESKey );

console.log("Let's decrypt: ");

// Decrypt
var bytes  = CryptoJS.AES.decrypt( esp8266_msg, key , { iv: iv} );
var plaintext = bytes.toString(CryptoJS.enc.Base64);
var decoded_b64msg =  new Buffer(plaintext , 'base64').toString('ascii');
var decoded_msg =     new Buffer( decoded_b64msg , 'base64').toString('ascii');

console.log("Decrypted message: ", decoded_msg);

and the output is:

Decrypted message:  {"data":{"value":300}, "SEQN":700 , "msg":"IT WORKS!" }

So, as the message shows, it WORKS!. AES encryption on the ESP8266 to a Node-JS Crypto-JS based code where decryption occurs.

Final notes:
So all is needed now is to build on the ESP8266 side the message with the encrypted data and the IV, and send through plain HTTP a JSON object to the Node-Red back end.

{
  "msg":"/1aZRwVaw3jv+ct8HS4pCV5lThvTG70M90ARiyAsIDYMkfJE3w8F3bgxaOKVA0rX4m1Mq50VVN0u9gRw9F2gKE4r2OcY8oECv8bKT80F9pY=",
  "iv":"cAFviaDMHejlteGn9/4eQQ=="
}

On the Node-Red back end the decryption can now be done easily as was shown on the above testing code.

ESP8266 and the Micropython firmware

One of the alternative firmwares available for the ESP8266 is MicroPython Python interpreter. I’ve found by chance a great tutorial at Adafruit for building the Micropython firmware and I thought to give it a try.

Building the firmware:
The Adafruit tutorial uses a Vagrant based virtual machine to build the firmware, but since I’m already running Linux (Arch Linux to be more specific) and already have the Falcon open ESP8266 sdk installed (see here) and the esptool.py also available since I’m using the Sming firmware, I’ve just downloaded only the latest Micropython source code from the Github repository to a local directory.

cd ~
git clone https://github.com/micropython/micropython
cd ~/micropython
git submodule update --init
cd ~/micropython/esp8266
export PATH=/opt/esp-open-sdk/xtensa-lx106-elf/bin:$PATH
make axtls
make

So far nothing different from the Adafruit tutorial except that I’m not using the vagrant VM. Also make sure that you first execute the command make axtls otherwise the main make command will compiling about not finding a version.h file. Also make sure that the export command that adds the path to the Xtensa compiler points to the right location.

After compiling, which was fast, I’ve just flashed the firmware on my Wemos mini D1 board. Again I had trouble flashing this board with other speeds than the default 115200 bps.

So:

cd ~/micropython/esp8266/build
esptool.py -p /dev/ttyUSB0 --baud 115200 write_flash 0 firmware-combined.bin

And after flashing, we can connect through the serial terminal, and pressing CTRL-D we should be greeted with the following message:

PYB: soft reboot
could not open file 'main.py' for reading
MicroPython v1.8-157-g480159c on 2016-05-29; ESP module with ESP8266
Type "help()" for more information.

Some basic tests:
When doing some tests I’ve found out that most information is outdated regarding the version of Micropython that I’ve flashed. For example:

>>> import pyb
Traceback (most recent call last):
  File "", line 1, in 
ImportError: no module named 'pyb'
>>> 

The common refered module pyb doesn’t exist, because it’s now machine:

>>> import machine
>>> dir(machine)
['__name__', 'mem8', 'mem16', 'mem32', 'freq', 'reset', 'reset_cause', 'unique_id', 'deepsleep', 'disable_irq', 'enable_irq', 'RTC', 'Timer', 'Pin', 'PWM', 'ADC', 'UART', 'I2C', 'SPI', 'DEEPSLEEP', 'PWR_ON_RESET', 'HARD_RESET', 'DEEPSLEEP_RESET']
>>>

So the following code:

>>> import pyb
>>> pin = pyb.Pin(14, pyb.Pin.OUT)  # Set pin 14 as an output.
>>> for i in range(10):
...    pin.value(0)     # Set pin low (or use pin.low())
...    pyb.delay(1000)  # Delay for 1000ms (1 second)
...    pin.value(1)     # Set pin high (or use pin.high())
...    pyb.delay(1000)

should be now:

>>> import machine
>>> pin = machine.Pin(14, machine.Pin.OUT)  # Set pin 14 as an output.
>>> for i in range(10):
...    pin.value(0)     # Set pin low (or use pin.low())
...    pyb.delay(1000)  # Delay for 1000ms (1 second)
...    pin.value(1)     # Set pin high (or use pin.high())
...    pyb.delay(1000)

Other interesting stuff is for example, the esp module:

>>> import esp
>>> dir(esp)
['__name__', 'osdebug', 'sleep_type', 'deepsleep', 'flash_id', 'flash_read', 'flash_write', 'flash_erase', 'flash_size', 'neopixel_write', 'apa102_write', 'dht_readinto', 'freemem', 'meminfo', 'info', 'malloc', 'free', 'esf_free_bufs', 'SLEEP_NONE', 'SLEEP_LIGHT', 'SLEEP_MODEM', 'STA_MODE', 'AP_MODE', 'STA_AP_MODE']
>>> esp.freemem()
17672
>>> esp.meminfo()
data  : 0x3ffe8000 ~ 0x3ffe8410, len: 1040
rodata: 0x3ffe8410 ~ 0x3ffe9038, len: 3112
bss   : 0x3ffe9038 ~ 0x3fff6bd0, len: 56216
heap  : 0x3fff6bd0 ~ 0x3fffc000, len: 21552

There are examples on GitHub to use the deepsleep functions:

Regarding the Wifi connectivity, by default when starting up the ESP8266 sets up a Wifi access point with the name Micropython-XXXXX where XXXX are some digits from the MAC address. Following the documentation the AP is protected with the password micropythoN, and sure enough the connection works. Still I haven’t tested it enough, for example, accessing the Python interpreter over Wifi, instead of through the serial port.

Anyway, one final test is to use Python to connect to make the ESP8266 to connect to my network. The instructions are simple, just write help(), and the micropython shows how to do it:

import network
>>> help()
Welcome to MicroPython!

For online docs please visit http://docs.micropython.org/en/latest/esp8266/ .
For diagnostic information to include in bug reports execute 'import port_diag'.

Basic WiFi configuration:

import network
sta_if = network.WLAN(network.STA_IF)
sta_if.active(True)
sta_if.scan()                             # Scan for available access points
sta_if.connect("", "") # Connect to an AP

and we can see if it connected successfully:

>>> sta_if.isconnected() 
True

and what IP configuration was set:

>>> sta_if.ifconfig()
('10.42.0.173', '255.255.255.0', '10.42.0.1', '10.42.0.1')

Also I was unable to access the Python interpreter through the access point connection. Supposedly there should be a listener running on port 8266 that allows access over WIFI, but I my tests found that the port 8266 was closed. Probably I need to initialize something first…
Anyway, there is a tool webrepl that allows to use the browser through websockets to connect to the ESP8266 and access the Python prompt and also to copy files to the ESP8266, namely the main.py startup file.

To finish. during my tests I had no crashes or surprise reboots. Using Python also has the advantage, in my opinion, that is more mainstream than Lua, since we leverage desktop programming with device programming. Also the useful tool ESPLorer already supports Micropython, it means that probably it is a better alternative for quick hacks using the ESP8266 instead of Nodemcu running LUA.