TTN LoraWan Atmega32U4 based node – ABP version

TTN is the The Things Network that provides the required backend services and infra-structure for supporting IoT (Internet of Things) connectivity that uses the LORAWAN protocol.

Anybody can participate on the Things Network by either providing the radio gateways that feed the received data to the TTN backend that, then, delivers it to the user applications, and so increasing the coverage of the TTN network, or just use the network by building TTN Lorawan nodes.

This post is regarding the later case, the build of a simple node based on an Arduino board: the Arduino Micro Pro. So why the Micro PRO, these are quite more expensive than the normal Arduinos, but come in two versions: 5V and 3.3V.
Since I’m using the SX1276 Lora radio that works with 3.3V, I’ve chosen the 3.3V Arduino Pro version so that I do not need to use level shifters if using a 5V based board. Also the Arduino Micro PRO chip, the Atmega32u4 has embedded USB connectivity/port, so no need for serial adapters and/or supporting chips which, at the end, might lead to lower power consumption.

Right now, on sites like eBay and Aliexpress, boards like the Lora32u4 come at least in two versions: with the Atmega328p and with the Atmega32u4. Both suffer the same problem, the Atmel micro processor used only has 32K of RAM available which might be too short to be used for some applications.
This is because the LMIC, the Lorawan stack, takes a huge amount of space if using the original IBM version. A much more memory efficient version for Arduino, originally ported from IBM code, but using a different AES encryption algorithm also exists and saves a lot of memory space. We will see about that. The great advantage of these boards is they also have connection and charger for a LiPo battery, so in reality all we need is to add sensors, battery and our code. An example of such board is the BSFrance Lora32u4 board.

The node build:
While I’m waiting for my Atmega32U4 based Lora32u4 board, I’m using an Hoperf RFM95 radio soldered on board/shield designed for the Wemos ESP8266: Wemos RFM95 Lora shield. this way I can use the RFM radio either on the ESP8266 Wemos based set of boards, or, as in this case, with the Arduino 32u4.

The Hallard shield as one interesting feature that is that merges all the Lora transceiver status pins by using diodes and hence only use one Arduino pin for inquiring Lora SX1276 radio status. This is needed due to the lack of I/O pins on the Wemos ESP8266 board. For this to work on Arduino we need to add a pull-down resistor to the Arduino pin that connects to the merged output. In my case I used a 10K resistor.
The RFM95 radio is controlled using SPI, so we need to use also the SPI Arduino Pins, and also need to connect the Chip Select pin.
The schematics is as follows:

Arduino Pro Micro and RFM95 Wemos Shield

The node software:
After the node hardware build is done, from the software perspective the node needs now at least another two things: the LMIC stack for implementing the Lorawan protocol support over the Lora radio and, at the TTN site, the device configuration.

Since I’m using Platformio to develop, the LMIC library is the library 852: pio lib show 852. We need to install it and add the reference to it on the file platformio.ini. Also since there is no ATMega 32U4 board on the Platformio IDE available boards, we can use the Adafruit Feather 32u4 board, which is the same thing:

platform = atmelavr
board = feather32u4
framework = arduino
lib_install= 852

The device registration can be done so that the node device access the TTN network in two different ways:

  1. ABP – Activation by personalisation – This means that all set of keys required to join the Lorawan network are embedded into the software.
  2. OTAA – Over the Air Activation – The network session keys needed to join the Lorawan network are generated when the device tries to join the network.

On this post we will ABP first, since I have no nearby TTN gateway capable o OTTA (I’m using a single channel gateway without downlink support.).

Anyway, the node code is really nothing special, except the necessary configuration for the LMIC to communicate with our RFM95 board.

On the ABP device registration TTN page we need to register our device, so that, on main.cpp code file we can fill the required keys and device ID.

As a quick introduction, after registering onto the TTN site, we go to the console and choose Applications. We can there create or reuse an existing application and register the device, making sure we choose ABP as the method to join the network.

On the Device EUI field, either we fill it or press the crossing arrows to generate an ID. We let the system generate an ID, and then we can finally press the Register button.

The newly register device is configured as an OTAA device:

So we go to Settings and change the OTAA to ABP. After this step we have the required data to put on our code.

Since our node doesn’t have any memory to track frame counting that survives reboots or power cycles, we disable the frame counter checks.

Don’t forget to press save. Again on the main device screen we can now copy the keys to the code:

We can now copy the keys:

static u1_t NWKSKEY[16] = { 0xEE, ... ... ... ... }; // <- Put here the NETWORK KEY
static u1_t APPSKEY[16] = { 0x4E, 0x12, ... ... ... ... };  // <- Put here the APPLICATION KEY
static u4_t DEVADDR = 0x26304050;   // Put here the device id in hexadecimal form.


Compiling the code with the pio run command, we have the following output when using the original IBM LMIC library:

Calculating size .pioenvs/feather32u4/firmware.elf
AVR Memory Usage
Device: atmega32u4

Program:   28542 bytes (87.1% Full)
(.text + .data + .bootloader)

Data:        957 bytes (37.4% Full)
(.data + .bss + .noinit)

And we can flash the firmware with the command: pio run -t upload.

The result is data on the TTN console referring to our device:

The problem… :
So, everything runs OK, and we can send data to the TTN Network, everything looks good, right?

As soon we start to add functionality to our code, for example reading some I2C sensors, our some serial debug messages, we hit this problem:

Linking .pioenvs/feather32u4/firmware.elf
Checking program size
text       data     bss     dec     hex filename
Error: The program size (28756 bytes) is greater than maximum allowed (28672 bytes)
28548       208     749   29505    7341 .pioenvs/feather32u4/firmware.elf
*** [.pioenvs/feather32u4/firmware.elf] Explicit exit, status 1

So in reality we can’t add much functionality to our code if using a full LMIC stack, since it occupies a lot of the available flash memory.

Trimming down the IBM LMIC stack:
Since our node is ABP only we can strip out some LMIC functionality for OTAA an other Lorawan features. For this we need to edit the config.h file from the LMIC library. Since we are using platformio, this file is located at project_root/.piolibdeps/IBM LMIC framework_ID852/src/lmic

We only leave support for ABP by enabling the disable lines for other LMIC functionality:

// Any runtime assertion failures are printed to this serial port (or
// any other Print object). If this is unset, any failures just silently
// halt execution.
#define LMIC_FAILURE_TO Serial

// Uncomment this to disable all code related to joining
// Uncomment this to disable all code related to ping
// Uncomment this to disable all code related to beacon tracking.
// Requires ping to be disabled too

// Uncomment these to disable the corresponding MAC commands.
// Class A
//#define DISABLE_MCMD_DCAP_REQ // duty cycle cap
//#define DISABLE_MCMD_DN2P_SET // 2nd DN window param
//#define DISABLE_MCMD_SNCH_REQ // set new channel
// Class B
#define DISABLE_MCMD_PING_SET // set ping freq, automatically disabled by DISABLE_PING
#define DISABLE_MCMD_BCNI_ANS // next beacon start, automatical disabled by DISABLE_BEACON

By uncommenting the above lines, our code now takes (we can and should ignore the LMIC compile warnings):

AVR Memory Usage
Device: atmega32u4

Program:   23324 bytes (71.2% Full)
(.text + .data + .bootloader)

Data:        796 bytes (31.1% Full)
(.data + .bss + .noinit)

So around 5KB less without the OTAA and Class B support.

So we have a bit more memory to do something useful.

Enabling OTAA by commenting the line //#define DISABLE_JOIN:

AVR Memory Usage
Device: atmega32u4

Program:   25048 bytes (76.4% Full)
(.text + .data + .bootloader)

Data:        912 bytes (35.6% Full)
(.data + .bss + .noinit)

We still have around 3K free. Tight but might be enough.

If using the Arduino ported LMIC library (852) we have:

AVR Memory Usage
Device: atmega32u4

Program:   18944 bytes (57.8% Full)
(.text + .data + .bootloader)

Data:        813 bytes (31.8% Full)
(.data + .bss + .noinit)

Much better!

The availability of boards with the AtMega32u4 processor, Lora Radio and LiPo charge and battery connectivity, is a great step to start using the TTN (or other) Lorawan networks. But with only with 32K or flash memory, for some applications, these boards might not be the best solution.

Also the price for such boards are still a bit on the expensive side, since a discrete 32u4 + RFM95 + Lipo charger is a bit cheaper than the single board solution.

Anyway, the STM32F103 blue pill boards cost half of the 32U4 price and have double the flash size and 9x the clock, are also 3.3v compatible and so it would be great that such single Lora boards used the STM32F103 instead of the 328p or 32u4…

So my conclusion is, without power considerations taken into account, a STM32F103 + RFM95 and LiPo charger, is a better alternative than the one that I’ve used here.


LPWAN – Starting up with LoraWAN and The Things Network

LPWAN Networks – A simple introduction
Low Power Wide Area Communications (LPWAN) classifies a group of communication protocols featuring low power usage and a high communication range. For Internet of Things communications, where battery powered devices and constrained devices (weak CPU, low RAM/ROM) are the norm, LPWAN use as the communication protocol for IoT makes sense, since it makes possible to have standalone devices with batteries that last years, instead of hours or days, and might be hundreds of meters to Kms away from a base station/gateway.

But LPWAN protocols, in contrast have low communication bandwidth, since from power, range and bandwidth, we can only have two of those, and while this might be a problem for certain applications, IoT devices don’t require high bandwidth and since most have sparse communication requirements, and when they do communicate they don’t need to transmit a lot of data.

Starting up with LoraWan and The Things Network
One of the possible ways of starting to use LPWAN networks in our IoT devices, is to use a LPWAN protocol named LoraWan, which is based on the Lora physical protocol, and if available at our area, the crowd sourced LPWAN network The Things Network as the backend network for receiving data.

Lora uses the 868Mhz ISM radio band and it uses a form of signal transmission called CSS. The ISM radio band can be used freely by everybody, but under certain rules that depends of the country where the devices are located. So to use that band, rules like maximum emission power and duty cycle (usage of the available spectrum) are defined, and in Europe, the maximum power is 20mW and 1% duty cycle per day. Additionally the back end operator can add other restrictions.

Over Lora, the Lorawan protocol is an open standard and source code for implementing it is available and already ported to several devices, including the Arduino, Raspberry PI and the ESP8266, among others.

The software for implementing the LoraWan protocol at the end devices (nodes) is IBM’s LMIC library, available at Github and on Platformio libs:

[ ID  ] Name             Compatibility         "Authors": Description
[ 852 ] IBM LMIC framework arduino, atmelavr, atmelsam, espressif8266, intel_arc32, microchippic32, nordicnrf51, teensy, timsp430 "IBM, Matthijs Kooijman": Arduino port of the LMIC (LoraWAN-in-C, formerly LoraMAC-in-C) framework provided by IBM. | @1.5.0+arduino-1

Based on this library we can build code and nodes that are able to transmit data through LoraWan to a network backend. Specifically for the Arduino, the Atmega328 is the bare minimum to begin with, since the library, due to use of AES encryption functions occupies a lot of memory.

The backend can be provided by really anyone, including Telco operators, or private and crowd source operators like The Things Network (TTN). TTN provides the backend and management services, but depends on crowd sourced gateways, and might not be available at our area. Still it is possible, for testing, to build our own gateways, our buy them, and connect them to the Things Network. TTN doesn’t require any access fees (yet).

So with LoraWan an the Things Network, we can build our own nodes and gateways since code and hardware can be easily obtained, connect them and use it to build applications.

Regarding LoraWan we can also read this great introduction from Design Spark.

Lora hardware:

Anyway the easiest way for starting up using Lora and Lorawan, is to buy an Dragino Lora Shield and connect it to an Arduino Uno or Mega.

Dragino Lora Shield

Dragino Lora Shield

This is a great shield to startup since doesn’t need any soldering or complex configuration, just plug it on an Arduino shield and use the LMIC library and some code samples. Also the Dragino Shield can be used to build a simple gateway by connecting it to a Raspberry PI or ESP8266 to build what is called a single channel gateway, that allows to test gateway functionality, but it isn’t quite compatible with the LoraWan specifications. Anyway it gets the job done if there is no gateway nearby. Just make sure that you by version 1.3 or 1.4 of the shield. Mine also came with an SMA antenna.

Other alternatives to start using Lorawan are available at eBay/Aliexpress and other renowned name shops, namely Lora radio transceivers at around 8/16€, for example the HopeRF RFM95. Those we can also use them with Arduino or ESP8266 to build our nodes or single channel gateways.

Just make sure that the frequency for these modules and shields must match the allowed radio transmission frequency in your area. In my case, in Europe is 868Mhz, but for example at USA is 900Mhz.

Dragino Lora Shield Jumpers
The Shield has some jumpers and only looking at the schematic and cross referenced them with the RFM95 module (used by the shield as the Lora transceiver) I could see what they where for:

– There are two set of jumpers:

  • One defines the pins for SPI communication: SV# Jumpers;
  • The other set defines which data pins of the RFM95 module are exposed to the shield allowing them to be connected to the Arduino: JP# Jumpers.

The JP# jumpers connect the following if closed:

  • JP1 – RFM95 DI1 – Arduino pin D6
  • JP2 – RFM95 DI2 – Arduino pin D7
  • JP3 – RFM95 DI5 – Arduino pin D8

The RFM95 pin DI0 is permanently connected to Arduino pin D2.
The RFM95 pin RST (Reset) is permanently connected to Arduino pin D9.

Their function for Lora might be the following (The function depends how the RFM95 module is used)

  • RFM95 DI0 – Indicates if data was received and is ready to be read through the SPI bus. Indicates end of transmission for data that was previously sent: RXReady/TXReady
  • RFM95 DI1 – Receive timeout for Lora mode.
  • RFM95 DI2 – Receive timeout for FSK mode.
  • RFM95 DI5 – Used by Semtech Library. LMIC library doesn’t use it.

The SV# jumpers connect:

  • SV2 – SPI Clock line. Default on pin D13, otherwise on Arduino SPI CLK on the ICSP header.
  • SV3 – SPI Data line In (MOSI). Default on pin D11, otherwise on Arduino MOSI pin on the ICSP header.
  • SV4 – SPI Data line Out (MISO). Default on pin D12, otherwise on Arduino MISO pin on the ICSP header.

The SPI Chip Select line is always at pin D10.

So now we know that D10, D9 and D2 are used permanently connected and used by the shield, and the others can be connected or disconnected if needed or not.

LMIC software with Dragino Lora Shield:

To start using the Dragino Lora Shield so it connects to a LPWAN network, we can start using the following example: TTN node by ABP. ABP means Activation by Personalization, which means that all data for joining the network will be explicitly defined on the code. Other alternative would be OTAA: Over the air activation, where the gateway sends the configuration data to the node. Since we don’t know if we have a gateway in range, let’s start first with ABP mode.

The above code uses the LMIC library for implementing the LoraWan stack.
Since LMIC library doesn’t use DI05, we can remove the JP3 jumper, and free this IO line for other things, like another shield.

To use the LMIC library we must define first the pins that are used according to the shield configuration:

// Pin mapping
const lmic_pinmap lmic_pins = {
    .nss = 10,                   //Chip Select pin. In our case it is always D10.
    .rxtx = LMIC_UNUSED_PIN,     //Antenna selection pin: not used in our case.
    .rst = 9,                    //Reset pin used to reset the RFM95: D9
    .dio = {2, 6, 7}             //DIO pin mapping for DIO0, DIO1 and DIO2  

But DIO2 pin is only used for FSK modulation, and so if only using LoraWan we can also open up the JP2 jumper, and our LMIC pin configuration can be as follows:

// Pin mapping
const lmic_pinmap lmic_pins = {
    .nss = 10,                   //Chip Select pin. In our case it is always D10.
    .rxtx = LMIC_UNUSED_PIN,     //Antenna selection pin: not used in our case.
    .rst = 9,                    //Reset pin used to reset the RFM95: D9
    .dio = {2, 6, LMIC_UNUSED_PIN} //DIO pin mapping for DIO0, DIO1. DIO2 is not used  

Connecting to TTN
Connecting to The Things Network (TTN) depends of course of an available TTN gateway at the nodes range. Still we need to configure some parameters to allow the node to connect.

On this example the connection is done through Activation by Personalization. This means that we should put on our code the Network Session key and Application Session key on the code. To obtain these values we need to register on the TTN site, TTN Dashboard and add an ABP device..

From this site, then we can get the three configuration parameters that we need:

  • Device ID
  • Network Session key
  • Application Session key

Note by pressing the icon we can get the values in a form ready to paste it on the code:

 * TODO: Change the following keys
 * NwkSKey: network session key, AppSKey: application session key, and DevAddr: end-device address
static u1_t NWKSKEY[16] = { 0xAA, 0x0F, 0x29, 0xD3, 0x9D, 0x7A, 0xAE, 0x3B, 0x54, 0xCF, 0xDF, 0x2F, 0x2A, 0x23, 0x55, 0xB5 };
static u1_t APPSKEY[16] = { 0x2F, 0x85, 0x43, 0x5B, 0x34, 0x9C, 0x80, 0xC6, 0xA8, 0xFA, 0x27, 0x49, 0x5A, 0x36, 0x82, 0x21 };
static u4_t DEVADDR = 0x95337738;

And that’s it, when running the sketch, and if in range of a gateway, the sent message should appear at the Dashboard:

Messages received for the device

Messages received for the device

Final thoughts:
In my area there are at least, supposedly, two active TTN gateways and I’m around 2Km’s away from them in a dense urban area.
But when running the sketch the first times, I had no results what so ever.

One of the configurations options for LoraWan is what is called a Spread Factor that, in a simplest way, exchanges range with on-air time for transmission, being SF7 the fastest speed and shorter range and SF12 the slowest speed and higher range. The sketch default value for the sprea factor was SF7, and changing it to SF12:

    // TTN uses SF9 for its RX2 window.
    LMIC.dn2Dr = DR_SF9;

    // Set data rate and transmit power for uplink (note: txpow seems to be ignored by the library)
    LMIC_setDrTxpow( DR_SF12 ,14);

    // Start job

With SF12 the message started be received and the sequence numbers of the messages received where continuous, so no lost messages.

Dropping to SF11, also worked fine, and the message sequence received and shown on the TTN Dashboard where still continuous.

At SF10, some of the messages where lost, almost 75% of them. In this case, arranging the antenna position (to exactly vertical) and placement did alter the reception successes.

Processing received data
After the data is at the TTN backend there are several ways of getting it. For reference on the TTN site there are instructions in how to access the data.

Crash course on Lora
For further information the next one hour video is a great starting point:

The slides for the presentation are available here: