CubicSDR and SoapyRemote on Orange PI PC

So I’m using remotely my RTL-SDR dongle connected to an Orange PI PC using GQrx on my desktop computer while on the Orange Pi PC I’m running the rtl_tcp server.

This combination works fine, but still some times I have some lag, not common, but it happens.

Anyway Gqrx is great, but I wanted to try another SDR programs, and one of those programs is Cubic SDR. Cubic SDR uses an abstration layer for accessing the SDR hardware either locally connected or over the network. So I wanted to see how the behaviour of CubicSdr, comparing to Gqrx when accessing the RTLSDR over the network.

Installing CubicSDR on the Desktop
I’ve not used the available binaries, but I’ve used the code from the Git repository: https://github.com/cjcliffe/CubicSDR. So far the git repository works fine.

I’m not posting here the instructions for building the CubicSDR because the full instructions are at the Cubic SDR wiki.

Just make sure, when compiling, to give the correct path to wxWidgets when building Cubic SDR.

So at the desktop, we need to obtain, build and install the following components:

  • SoapySDR – Abstraction layer
  • Liquid-dsp – The digital signal processing libs
  • wxWidgets – The display widgets
  • CubicSDR – The Sdr program itself
  • SoapyRTLSDR – The Soapy abstraction layer driver for the RTLSDR USB type dongles. To use the dongles locally

Installing the above software we can use the attached locally RTLSDR dongle.

Installing SoapySDR Remote on the remote server
The remote server where my RTLSDR dongle is connected is an Orange Pi PC running Armbian. To let my desktop running CubicSDR program to access remotely the dongle, we need to install SoapySDR Remote that allows remote access to the SDR.

So at the remote server we need to obtain, build and install the following components:

  • SoapySDR – The abstraction layer
  • SoapyRTLSDR – The driver for our RTL USB dongle
  • SoapySDR Remote – The server for remotely access the RTL dongle

So basically the instructions are something like this:

  mkdir ~/SDR 
  cd ~/SDR
  git clone https://github.com/pothosware/SoapySDR.git
  cd SoapySDR/
  mkdir build
  cd build
  cmake ../ -DCMAKE_BUILD_TYPE=Release
  make -j4
  sudo make install
  sudo ldconfig
  SoapySDRUtil --info

Building the SoapySDR remote:

 cd ~/SDR
 git clone https://github.com/pothosware/SoapyRemote.git
 cd SoapyRemote/
 cd build
 cmake ..
 make
 sudo make install

and build the RTLSDR driver:

  cd ~/SDR
  sudo apt-get install librtlsdr-dev
  git clone https://github.com/pothosware/SoapyRTLSDR.git
  cd SoapyRTLSDR/
  mkdir build
  cd build
  cmake .. -DCMAKE_BUILD_TYPE=Release
  make
  sudo make install
  sudo ldconfig

So at the end we should have the following outputs:

opi@opi:~# SoapySDRUtil --probe
######################################################
## Soapy SDR -- the SDR abstraction library
######################################################

Probe device 
Found Rafael Micro R820T tuner
Found Rafael Micro R820T tuner

----------------------------------------------------
-- Device identification
----------------------------------------------------
  driver=RTLSDR
  hardware=RTLSDR
  origin=https://github.com/pothosware/SoapyRTLSDR
  rtl=0

----------------------------------------------------
-- Peripheral summary
----------------------------------------------------
  Channels: 1 Rx, 0 Tx
  Timestamps: NO
  Other Settings:
     * Direct Sampling - RTL-SDR Direct Sampling Mode
       [key=direct_samp, default=0, type=string, options=(0, 1, 2)]
     * Offset Tune - RTL-SDR Offset Tuning Mode
       [key=offset_tune, default=false, type=bool]
     * I/Q Swap - RTL-SDR I/Q Swap Mode
       [key=iq_swap, default=false, type=bool]

----------------------------------------------------
-- RX Channel 0
----------------------------------------------------
  Full-duplex: YES
  Supports AGC: YES
  Stream formats: CS8, CS16, CF32
  Native format: CS8 [full-scale=128]
  Stream args:
     * Buffer Size - Number of bytes per buffer, multiples of 512 only.
       [key=bufflen, units=bytes, default=16384, type=int]
     * Buffer Count - Number of buffers per read.
       [key=buffers, units=buffers, default=15, type=int]
  Antennas: RX
  Full gain range: [0, 49.6] dB
    TUNER gain range: [0, 49.6] dB
  Full freq range: [23.999, 1764] MHz
    RF freq range: [24, 1764] MHz
    CORR freq range: [-0.001, 0.001] MHz
  Sample rates: [0.25, 3.2] MHz

And SoapySDR should have the following configuration:

opi@opi:~# SoapySDRUtil --info
######################################################
## Soapy SDR -- the SDR abstraction library
######################################################

API Version: v0.5.0-gfec33c63
ABI Version: v0.5-2
Install root: /usr/local
Module found: /usr/local/lib/SoapySDR/modules/libremoteSupport.so
Module found: /usr/local/lib/SoapySDR/modules/librtlsdrSupport.so
Loading modules... done
Available factories...null, remote, rtlsdr, 

So all we need is to start our server:

opi@opi:~# SoapySDRServer --bind
######################################################
## Soapy Server -- Use any Soapy SDR remotely
######################################################

ba73bc08-3d0f-1458-8337-93d5a4502801
Launching the server... tcp://[::]:55132
Server bound to [::]:55132
Launching discovery server... 
Press Ctrl+C to stop the server

Using CubicSDR and SoapySDRRemote
So all we need is now on the startup SDR device screen selection add (by pressing the Add button) the remote IP of the Orange Pi PC server to access remotely the RTLSDR dongle.

My Orange PI PC IP address is 192.168.1.19:

CubicSDR device selection

CubicSDR device selection

And here it is the CubicSDR in action.

CubicSDR in action

CubicSDR in action

Conclusion and final notes
The CPU usage and temperature on the Orange PI PC is not a problem when using the server. CPU usage floats around 40%, and no meaningful or worrying changes on the CPU temperature. So the Orange PI PC is up to the task without any issues when serving data with SoapySDRRemote.

Also with CubicSDR and SoapySDRRemote, I’ve experienced no lag when changing frequencies, namely when dragging the frequency selector.. It seems that all changes are instantaneous and note that my desktop connects to the remote server through a 200Mbps PLC and only then it is cable network to the Orange Pi. According to my desktop PC network widget, when receiving data, I have around 6.5Mbps of data comming in when using the maximum sample rate of 3.2MHz.

Also it took me a while to get used to the CubicSDR user interface, but overall for things like fine tuning, since it has a dedicaded codec screen, is much better than Gqrx.

CubicSDR  fine tuning

CubicSDR fine tuning

Also one great feature is if we keep dragging the spectrogram window, the central frequency changes so it keeps up with the SDR bandwidth, shile in Gqrx we need to dial in.

Still I’m using Gqrx and rtl_tcp since CubicSDR has no data output, other then piping audio. Gqrx can pipe to UDP, that allows the decoding of digital modes locally or on other servers without messing around with PulseAudio and Jackd.
Also bookmarking isn’t as direct/easy as with Gqrx. Not sure if I can give labels/names to bookmarks and search them, like I can in Gqrx, but then the problem might be between the chair and computer…

Anyway CubicSDR is a great SDR application and the future looks bright.

I do recommend to give it a test drive.

Dropbox doesn’t start (Linux)

So a quick note regarding Dropbox running on ArchLinux with Nvidia drivers. Since I use KDE Plasma 5 that doesn’t show system tray icons of some applications that have not migrated to the new system tray protocol, I didn’t notice that after upgrading Dropbox it stopped working.

A quick investigation, I’ve found out that starting Dropbox from the command line did nothing. It started and stopped. Something is going on…

So I’ve used the strace command to see if I could catch something meaningful:

[pcortex@pcortex:~]$ cd .dropbox-dist/dropbox-lnx.x86_64-3.10.8/
[pcortex@pcortex:dropbox-lnx.x86_64-3.10.8]$ strace ./dropbox 2> stracedump.txt

And sure enough in the stracedump.txt I had the following error:

open("/usr/lib/dri/tls/swrast_dri.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/dri/swrast_dri.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0} ---
rt_sigaction(SIGSEGV, {0x7f09e1dea010, [SEGV], SA_RESTORER|SA_RESTART, 0x7f09e1a88d60}, NULL, 8) = 0
write(4, "Fatal Python error: ", 20)    = 20
write(4, "Segmentation fault", 18)      = 18
write(4, "\n\n", 2)                     = 2
write(4, "Traceback (most recent call firs"..., 36) = 36
write(4, "  File ", 7)                  = 7
write(4, "\"", 1)                       = 1
write(4, "d", 1)                        = 1

So the error is related to the Software Raster library, that I don’t have because I’m running Nvidia proprietary drivers.

The confirmation came with the following command:

[pcortex@pcortex:dropbox-lnx.x86_64-3.10.8]$ LIBGL_DEBUG=verbose ./dropbox
libGL: OpenDriver: trying /usr/lib/dri/tls/swrast_dri.so
libGL: OpenDriver: trying /usr/lib/dri/swrast_dri.so
libGL error: dlopen /usr/lib/dri/swrast_dri.so failed (/usr/lib/dri/swrast_dri.so: cannot open shared object file: No such file or directory)
libGL error: unable to load driver: swrast_dri.so
libGL error: reverting to indirect rendering

The solution?
Quite simple, just remove (backup) the libGL.so.1 library that exists on the Dropbox directory.
After the removal, Dropbox starts just fine.

Odroid-C1+ – The low power super server

On the ARM based small computers front, running Linux or Android, there are several contenders, being the most famous one the Raspberry PI. Due to the huge community and documentation available the Raspberry Pi is the most chosen for a whole range of projects including low power servers, controllers, media players and so on.
Anyway, there are several alternatives, even cheaper, than the RPi. For example the 15$ Orange Pi looks promising on the price front, but not on the software side due to the Allwinner chip, it doesn’t look very promising without closed binary blobs. Other alternatives are the Banana Pi, Beaglebone board, Cubie board, and so on.

Anyway, since I needed something low power for running Node-Red, Sparkfun’s Phant, for logging data from the ESP8266, the RPi, was the way to go. But for the same price and with better specs, the Odroid-C1+ boosts also a very good community, supports Android, Ubuntu and even Arch Linux for ARM V7.

The advantages from the Odroid-C1+ over the RPi are:
– Same price (at least for me, 45€ for the RPi vs 44€ for the Odroid)
– Powerful 1.5 GHz quad core cpu
– Dedicated Ethernet chip not sharing the USB ports with 1GBps port.
– An emmc slot for storage allowing faster I/O than the SD cards. Sizes available between 8GB and 64Gb
– IR receiver
– Supports Ubuntu, Android, Arch Linux…

And so on.

As same as the RPi, the board by itself can’t do much, so I’ve also bought the power supply, acrylic box and a 32Gb emmc solid disk.
I was torn between the cheaper typical sd card and this emmc disk. Since the odroid supports it, why not?

As we can see, while it’s not faster than a typical SSD, it has the same performance than a spinning regular harddisk:

root@odroid:/etc/NetworkManager# hdparm -Tt /dev/mmcblk0p2

/dev/mmcblk0p2:
 Timing cached reads:   1440 MB in  2.00 seconds = 720.23 MB/sec
 Timing buffered disk reads: 240 MB in  3.02 seconds =  79.56 MB/sec
root@odroid:/etc/NetworkManager# 

Anyway, while the emmc is expensive, more than the price of the Odroid for the 32GB, it feels snappy and boots quite fast.

The emmc comes with a standard 4GB partition, and we need to expand it to the full emmc capacity using the provided odroid utility, or also, the gparted utility that is shipped on the ubuntu release.

So after powering up:

– Went to my router web interface and under the DHCP server tried to find which IP the Odroid got.
– Note, if connected to a HDMI monitor and if a keyboard and mouse is also connected, we can use the Odroid utility on the LXE window manager to expand the partition size from the default 4GB.
– The odroid utility can also be used from the command line: odroid-utility.sh
– With the IP I’ve got, I’ve ssh to the odroid and logon as root with odroid as password.
– Started up Gparted and increased the partition from 4GB to the 32GB emmc capacity.

root@odroid:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p2 29G 4.6G 23G 17% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 420M 4.0K 420M 1% /dev
tmpfs 425M 8.0K 425M 1% /tmp
tmpfs 85M 2.6M 83M 4% /run
none 5.0M 0 5.0M 0% /run/lock
none 425M 8.0K 425M 1% /run/shm
none 100M 24K 100M 1% /run/user
/dev/mmcblk0p1 129M 7.2M 122M 6% /media/boot
root@odroid:~#

– Updated the system: apt-get update and apt-get upgrade and it took a while to upgrade a lot of packages.

– After the upgrade I added a user: pcortex and made it to belong to the sudo group. This will be my working user, to avoid mishaps with the root user….

adduser –home /home/pcortex pcortex
passwd pcortex
adduser pcortex sudo

After the initial updating, and setting up, the remaining tasks where to install nodejs, node-red, phant and sqlite3.
Still as root:

apt-get install nodejs npm sqlite3
npm install -g node-red
npm install -g phant
npm install -g node-red-node-sqlite

For Android Notifications using the Google Cloud Messaging infrastructure:
npm install -g node-gcm (Check out: Node-Red GCM notifications

And finally making a link to let node be an alias of nodejs: ln -s /usr/bin/nodejs /usr/bin/node

Now with the user pcortex, we can start Node-Red and phant and starting doing interesting things with a low power super server 🙂

Edit: I’ve also changed the IP from a dynamic IP to a fixed IP. I’ve kept the hostname, odroid is cool enough 🙂

So I’ve edit the NetworkManager.conf file located at /etc/NetworManager, and changed the managed=false to managed=true under the section [ifupdown]. The final file is this one:

[main]
plugins=ifupdown,keyfile,ofono
dns=dnsmasq

[ifupdown]
managed=true

 
And then changed the network interface configuration to the fixed IP by editing the file interfaces located in /etc/network

# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
   address 192.168.1.45
   netmask 255.255.255.0
   broadcast 192.168.1.255
   gateway 192.168.1.254
   dns-nameservers 8.8.8.8 208.67.222.123

Note: I’ve also installed nginx instead of Apache just to keep resource usage low, but that’s another story…

Arch Linux e Cartão do Cidadão

Só um quick post para lembrar como se pode por o Midleware/software do cartão do cidadão a funcionar em Arch Linux.

– Primeiro determinar que o cartão é detectado pelo sistema:

lsusb
Bus 007 Device 002: ID 058f:9520 Alcor Micro Corp. EMV Certified Smart Card Reader

– Segundo instalar o software opensc:

[root@pcortex ~]# pacman -S opensc

Isto só vai servir para verificar que conseguimos aceder correctamente ao leitor do cartão.

– Testar o acesso ao leitor:

[root@pcortex ~]# opensc-tool -l
No smart card readers found.

Se for este o caso, arrancar com o daemon de suporte de acesso aos leitores:

[minime@pcortex ~]# sudo -s
[root@pcortex ~]# pcscd

Testar novamente:

[minime@pcortex ~]# opensc-tool -l
# Detected readers (pcsc)
Nr. Card Features Name
0 No Alcor Micro AU9520 00 00

– Podemos agora instalar o software do cartão do cidadao. No meu caso usei a versão disponível no repositório AUR:

yaourt -S cartao-cidadao

E no fim, executa-se:

[minime@pcortex ~]# pteigui

Pode-se ignorar as mensagens de erros, e o icon deverá aparecer no SysTray.

Para ter acesso aos certificados do cartão a partir do FireFox, por exemplo, bastará em princípio seguir estas instrucções: https://faqs.up.pt/faq/content/29/557/pt/como-configurar-o-mozilla-firefox-para-usar-o-cart%C3%A3o-de-cidad%C3%A3o.html

Setting up an IoT framework/dashboard with NodeRed, Mosca/Mosquitto and Freeboard.io dashboard

With the low cost of the esp8266 chip and boards, it is now easy and affordable to add connectivity  to almost anything that we can think of. Still if we build an Internet of Things device by enabling it with the esp8266 chip to connect to the internet, or other networks, what we can do with the data that we gather? Basically we do what we do with any data that we get, store it, process it and show it, and that’s what we are going to see on this post.

The software (running on Linux platform) that I’ll use is the following:

IBM NodeRed: (http://nodered.org/) This great software will allow us to get data, process it, store and publish it, with a workflow based designer. It runs under Node-Js Javascript based server engine.

MQTT Broker Mosca: (http://www.mosca.io/) There are several MQTT brokers available, but this one runs on Node-Js and so is platform agnostic, and runs where Node-Js runs. Also it has websocket support out of the box, which allows to use the Freboard.io dashboard in a straight foward way.

Freeboard.io Dashboard: (http://freeboard.io/) Is a great dashboard and freely available at the project GitHub page (https://github.com/Freeboard/freeboard). It allows to design and store several dashboards with some cool widgets.

Step 1: Setting up NodeRed:

First of all make sure that you have node and npm (the node package manager) installed on your distribution. You can see that with the commands node -v and npm -v for checking out their versions. In my case, and for now, it’s version v0.10.33 for nodejs and 1.4.28 for npm.

First I’ve created a directory named IoTServer where I’ll keep all software and configurations.

Download the zip file from nodered GitHub page https://github.com/node-red/node-red or just clone the repository. I clone the repository, because it’s simpler in the future to update it with just git pull. First just make sure that you have the icu/libicu libraries installed.

cd ~
mkdir IoTServer
cd IoTServer
git clone https://github.com/node-red/node-red
cd node-red
npm install

We can now run the nodered server with the command node red.js -v. I use -v at the end to see if any error crops up and also to see if any node modules are missing, and need to be installed. For example:

pcortex@cloudsrv:~/IoTServer/node-red$ node red.js -v  
   Welcome to Node-RED 
   =================== 
 
24 Feb 11:35:16 - [info] Node-RED version: v0.10.3.git 
24 Feb 11:35:16 - [info] Node.js  version: v0.10.33 
24 Feb 11:35:16 - [info] Loading palette nodes 
24 Feb 11:35:59 - [warn] ------------------------------------------ 
24 Feb 11:35:59 - [warn] [arduino] Error: Cannot find module 'arduino-firmata' 
24 Feb 11:35:59 - [warn] [rpi-gpio] Info : Ignoring Raspberry Pi specific node. 
24 Feb 11:35:59 - [warn] [redisout] Error: Cannot find module 'redis' 
24 Feb 11:35:59 - [warn] [mongodb] Error: Cannot find module 'mongodb' 
24 Feb 11:35:59 - [warn] ------------------------------------------ 
24 Feb 11:35:59 - [info] Server now running at http://127.0.0.1:1880/ 
24 Feb 11:35:59 - [info] Flows file not found : flows_clouds.json 
24 Feb 11:35:59 - [info] Starting flows

As we can see the node modules arduino, rpi-gpio, redisout and mongodb are missing. If we are going to use them, we need to install them with the npm tool. Also the following message: Flows file not found : flows_clouds.json informs us that no saved nodered workflows where found. We can now access the nodered at adress http://localhost:1880/. If we want to change the bind address or add authentication, we should change the settings.js file. Take a look at http://nodered.org/docs/configuration.html for more information, and start nodered with: node red.js -v -s settings.js

We can now start designing our nodered workflows that receive data from the esp8266 via MQTT protocol or even by simple REST based HTTP protocol, process it and store the data payload, if any.

For storing data into a database, we can use MongoDB, or mysql, for example, installing the needed modules.

For mysql we can do:

cd ~/IoTServer/node-red
npm install bignumber.js require-all readable-stream
npm install mysql
npm install node-red-node-mysql

And we need to restart nodered. The mysql node should be available now in storage pallet.

Step 2: Installing Mosca MQTT broker (Alternative 1 for the MQTT Broker)

We can use several brokers, but since we already are running nodered on node-js, we will use Mosca. Both nodered and Mosca can provide a websocked based interface. This is of interest because we can use browser based websocket applications, like freeboard.io, to consume data from our IoT devices.

cd ~
sudo -s
sudo npm install mosca bunyan -g

And to run it just do: mosca -v | bunyan

For running it with websockets enabled, on TCP port 3000, just do:

mosca -v --http-port 3000 --http-bundle --http-static ./ | bunyan

Check out the documentation for more information at: https://github.com/mcollina/mosca/wiki/MQTT-over-Websockets

Step 2: Mosquitto MQTT broker with Websockets support (Alternative 2 for the MQTT Broker)

If having trouble installing or using Mosca, we can use the latest Mosquitto version 1.4 that has websockets support.

For that we need to make sure that cmake and uuid/uuid-dev are installed (For example: sudo apt-get install cmake uuid uuid-dev).

Also we need to download the libwesockets library, compile it and install it:

cd ~/IoTServer
git clone git://git.libwebsockets.org/libwebsockets
cd libwesockets
mkdir build
cd build
cmake ..
make
sudo make install
We then need to download and compile Mosquitto 1.4:
cd ~/IoTServer
wget http://mosquitto.org/files/source/mosquitto-1.4.tar.gz
tar xvzf mosquitto-1.4.tar.gz
cd mosquitto-1.4
We need to edit the config.mk file and change the option WITH_WEBSOCKETS:=no  to WITH_WEBSOCKETS:=yes
And finally:
make
sudo make install
To activate websocket support we need to edit the mosquitto.conf file located at /etc/mosquitto or some other directory and add the following lines, for example at the end of the file:
listener 1883

listener 9001 127.0.0.1
protocol websockets

Then we run it with:

pcortex@cloudsrv:~/IoTServer$ mosquitto -c /etc/mosquitto/mosquitto.conf
1424790588: mosquitto version 1.4 (build date 2015-02-24 14:38:47+0000) starting 
1424790588: Config loaded from /etc/mosquitto/mosquitto.conf. 
1424790588: Opening ipv4 listen socket on port 1883. 
1424790588: Opening ipv6 listen socket on port 1883. 
1424790588: Opening websockets listen socket on port 9001.

Step 3: Installing freeboard.io dashboard

We are almost at the end of this long post. For our infrastructure we need now to install the dashboard that will allow us to see the data in the browser.
cd ~/IoTServer
git clone https://github.com/Freeboard/freeboard.git

For now I’ll just use apache to serve the freeboard dashboards that is installed in my server. Because freeboard is not installed on the root web directory we need to make the freeboard available to Apache. The easiest way to do this, as long as Apache has the FollowSymLinks option enabled for the document root, is to create a link to the freeboard directory on the web document root:

sudo -s
cd /var/www
ln -s /home/pcortex/IoTServer/freeboard iot

And now freeboard is available at the url http://myserveraddres/iot.

We need now, and finally to add MQTT over Websockets support to freeboard… We are almost there. This post http://jpmens.net/2014/11/12/freeboard-a-versatile-dashboard/   shows how it’s done but I’ll repeat it here again:

1st: Download the development version of mqtt.js here: (http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.javascript.git/plain/src/mqttws31.js?h=develop) and save it:

cd ~/IoTServer/freeboard/plugins
mkdir mqtt
cd mqtt
wget --output-document mqttws31.js http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.javascript.git/plain/src/mqttws31.js?h=develop

We will now download the freeboard MQTT plugin:

cd ~/IotServer
git clone https://github.com/alsm/freeboard-mqtt
cd freeboard/plugins/mqtt
cp ~/IoTServer/freeboard-mqtt/paho.mqtt.plugin.js .

We need to edit the paho.mqtt.plugin.js file and change the line:

                "external_scripts" : [
                        "<full address of the paho mqtt javascript client>" 
                ],

to

               "external_scripts" : [
                        "plugins/mqtt/mqttws31.js" 
                ],

and finally we need to change the index.html file from:

   <scripttype="text/javascript">
        head.js("js/freeboard+plugins.min.js", 
                // *** Load more plugins here *** 
                function(){

to

   <scripttype="text/javascript">
        head.js("js/freeboard+plugins.min.js", 
                "plugins/mqtt/paho.mqtt.plugin.js", 
                // *** Load more plugins here *** 
                function(){

That’s it. We are finally ready. All infrastructure is done. We need now just to configure it.

Final step: Testing it out with mosquitto websockets!:

Let’s start Mosquitto: nohup mosquitto -c /etc/mosquitto/mosquitto.conf &

And check if the WS protocol is active:

netstat -nap | grep 9001

tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN      18973/mosquitto

Setting up Freeboard.io:  It’s very important to notice that the Dashboard will run on our browser, making in fact our browser the client. This means that when we configure freeboard, we must take notice of this when filling up the MQTT broker address. It will be localhost if the browser AND the broker runs on the same machine, but for the dashboard to work anywhere we should use the public ip of the machine running the broker:

Access the freeboard.io dashboard, and select Datasources -> Add. From the dropdown box, we select the Paho MQTT provider.

Selection_055

We need to configure the data source as follows:

Selection_056

Change the MQTT server address to something that makes sense in your config.

For Mosquitto with websockets enabled, the port is 9001, and for Mosca, the port is 3000, but of course this can change, depending of the configuration used.

We can now add a Panel and a gauge to see data from the topic /data that we defined on the above datasource.

So, Add Pane, and on the new Pane select the + (plus) to add a widget. In our case we will select Gauge. We fill out the required info and on the Value field we press Datasource, and select the newly previous created datasource named Data and because we are not using JSON for data, we just append .msg at the end. So something like datasource[“DS_data”].msg should be writen.

Selection_057

We press Save, and that’s it.

We can now publish into the topic, and the dashboard should update:

Selection_058

For example, in this case: mosquitto_pub -t /data -m 23

That’s it. It works fine with Mosquitto 1.4 with websockets enabled and with the Mosca MQTT broker.

Final setup:

We should make our dashboard permanent and for doing so we need to save the dashboard locally as a JSON file, and move it to the server running the dashboard to the root directory of freeboard.

After that we call directly the saved dashboard with the following URL:

http://myserveraddress/iot/index.html?load=dashboard.json

Edit: For Websocket support on the Freeboard.io Dashboard, please checkout Freeboard and WebSockets

Warning:The current freeboard.io project at Github (as of 12/2015) has problems saving dashboards when using Firefox. For saving dashboards use Chrome/Chromium.

Upgrading to an SSD, moving from Kubuntu to Arch Linux and setting up KDE

This is a single post just to make notice of some points:

My home desktop PC, was running since 2009 the Kubuntu distribution. I’ve never reinstalled it and did always the upgrades from 8.10 to 9.04, 9.10 and so on until 14.04. Almost six (!) years with no virus, trojan horses, and a single initial installation. I’ve upgraded through the years the KDE desktop since the ill famed initial 4.0 release, to the latest 4.6 release.

But things where starting to go awry… At the beginning I had a ATI 4870 graphic board, but ATI dropped support for it on Linux (and Windows) and since I had so many troubles with the proprietary driver, and to add it up I had no support for the board, I sold it and bought an Nvidia GTX660.

But with both graphic boards, I had suffered from a long time problem: I could only do a sleep/resume a number of times before the system hanged. The symptoms starting from loosing network connectivity and then restoring right back, and some cycling of these symptoms to a complete lockup. At first I thought that was due to the network driver used by my motherboard, a Asus P6T with  a Realtek RTL8111/8168/8411 board, but with the correct driver for it, the issue continued.  Booting up was a painful two/three minute wait from BIOS to desktop.

But after a while I could pin point my issues not to the network board but to the Xorg and video driver combination. Some EQ buffer overflow, hanged X, and finally the machine.

Ubuntu and Kubuntu based distributions have PPA’s repositories where some alternative software and versions of mainstream software are available. I had trouble with some of these alternative PPA’s when upgrading, and had to remove them (with ppa-purge) to be able to upgrade the Kubuntu distribution. Anyway, there is an PPA, xorg-edgers,  where alternative and more recent versions of Xorg Software and Nvidia drivers are provided and that almost solved my sleep/resume problem. The problem was that after adding this PPA the Ubuntu/Kubuntu nvidia driver and the Xorg-edgers driver, after updates the X server was unable to find the nvidia driver, despite being installed, and had to reinstall the driver again to get my desktop working.

That was it. I had enough with Ubuntu/Kubuntu distributions. My bad and my issues, sure…

I’ve being testing Arch Linux on my work laptop for almost an year with great success. I’ve chosen Arch Linux because I didn’t had to loose an unknown number of hours per year upgrading to the next distribution version, and an rolling release version makes more sense. And Arch has the greatest Linux Wiki out there even solving problems that happen on the Ubuntu/Kubuntu line, and also mainly there are only one (well two) repositories. No PPA’s mess.

So I completely moved all my systems to Arch Linux, and my sleep/resume issue is 100% solved.

SSD on a SATA II based motherboard

My desktop computer has an Asus P6T motherboard. This motherboard only has SATA II (SATA 2) ports. Is it worth to use an SSD on such a board without upgrading to a SATA III board?

The answer is YES. If you have a SATA II based motherboard AND not buying an SSD because of that, do not wait any more. Buy it now. It is totally worth it.

Arch Linux is of course way lighter than Kubuntu, and with an SSD ( by the way it’s a Crucial MX100 256GB) my two/three minute boot to desktop time came down to 15s…

hdparm -Tt /dev/sdc
/dev/sdc:
Timing cached reads:   16710 MB in  2.00 seconds = 8361.40 MB/sec
Timing buffered disk reads: 728 MB in  3.00 seconds = 242.59 MB/sec

Note the Samsung EVO 120GB that I have on my work computer, that has the flawed firmware that slows down with time, on a SATA III based motherboard:

/dev/sda:
Timing cached reads: 23864 MB in 2.00 seconds = 11943.41 MB/sec
Timing buffered disk reads: 164 MB in 3.01 seconds = 54.47 MB/sec

Compare this with my desktop old boot disk, an WD 640GB blue:
/dev/sdb:
Timing cached reads: 16580 MB in 2.00 seconds = 8296.87 MB/sec
Timing buffered disk reads: 356 MB in 3.00 seconds = 118.59 MB/sec

I hadn’t yet had the time to upgrade the firmware on the EVO…

Making KDE looking good on Arch Linux

To end this rather long and tedious post, and also to my future reference, to make KDE looking good:

    • Make sure that the infinality modifications are installed: https://wiki.archlinux.org/index.php/Infinality. Make sure that you update.
    • Make sure that the DPI detected by your X server are correct for your monitor. Install xdpyinfo with pacman -S xorg-xdpyinfo. Figure out what might be your monitor dpi using this site: http://dpi.lv/ . Execute xdpyinfo | grep -B2 resolution and see if are similar. If not you need to set the correct dpi either in xorg.conf or in the Kde settings.
      In my case I have a dual monitor setup, and hence the weird dpi 95×96.screen

      #0: dimensions: 3200×1080 pixels (856×286 millimeters)

      resolution: 95×96 dots per inch

      So my font settings are:

Selection_002

  • While the above settings make my fonts look good, the taskbar fonts where awful and took me a while to figure out that was not the dpi settings but the Desktop Theme.
  • To solve this first install Krita with pacman -S calligra-krita . This will install the Krita color themes that are, in my opinion very good. And on KDE System Settings -> Workspace Appearance -> Desktop Themes select Get New Themes and get the AndroidBit theme.
  • That’s it:Selection_004
  • I’ve selected the Produkte Desktop Theme
  • And the colors:Selection_005

ESP8266: Setting up native SDK’s on Linux and exploring some applications

The information for installing and setting up the native SDK for the esp8266 from Expressif is readily available in the Internet:

With the native SDK we can develop our own firmware using the Espressif SDK and the C or C++ language. Then we can use directly the internal esp8266 processor, allowing to do interesting things in a more direct way, like implementing MQTT, driving hardware, and so on.

So why this post?  Basically is to put into writing some of my own annotations to the issues that I had installing the SDK and the cross compiler tools on my Linux machines.

Starting up:

The original instructions that I’ve followed where these: https://github.com/esp8266/esp8266-wiki/wiki/Toolchain
I installed the tool chains in two computers, both running Linux: an Arch Linux distribution and a Kubuntu 14.04 distribution (and time is up for this one because I’m moving it also to Arch Linux. All I’m missing is to install a SSD).

1st) Installed the required pre-requisites that are documented on the above link. On Arch Linux the package installation tool is pacman and probably the package names can be different. I didn’t need to install anything on Arch, but on Kubuntu I needed to install the 64-bit pre-requisites.

2nd) Create the /opt/Espressif directory. The owner should already be your user, except if you have no permissions on the /opt directory.

3rd) Install the Xtensa Crosstool-NG that has the cross compiler and tools for the esp8266 processor:

cd /opt/Espressif
git clone -b lx106-g++ https://github.com/jcmvbkbc/crosstool-NG.git
cd crosstool-NG
./bootstrap && ./configure --prefix=`pwd` && make && make install
./ct-ng xtensa-lx106-elf
./ct-ng build

Some notes regarding this:

  •  We are going to clone the git branch lx106-g++ that has the C compiler and the C++ compiler. There is another branch named lx106 that only has the C compiler.  There is some software out there like Ardunet that need the c++ compiler and so that is the reason why I’ve chosen this git branch.
  • I’ve also changed the url for the git repository from git:// to https:// because in I’m behind a web proxy that needs authentication. Just make sure that the http_proxy, https_proxy and ftp_proxy variables are set. The ftp_proxy variable is needed because at the ct-ng build step will connect to a ftp server to retrieve files. One example for setting these environment variables: export https_proxy=http://myproxyuser:myproxypass@proxyip:port. If you have an internet direct connection, like at home, no need to set these variables.
  • The ./ct-ng build step will take a long time, around 30 minutes in one of my computers and 15 in another.
  • The tool will connect to the ftp server on IP: http://208.118.235.20/ where are located the Gnu compiler sources.

At the end we need to add the path to the crosstools to our PATH variable.
export PATH=/opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf/bin/:$PATH
You may want to add this line at your .bashrc file. In my case (I’m using KDE desktop, I’ve created a Konsole profile for this type of work and add the new settings to the PATH variable,)

Installing the Expressif SDK

The instructions on the above link are for the 0.9.3 version of the SDK. I’ve installed the most recent version at this time that was 0.9.4:

cd /opt/Espressif
wget -O esp_iot_sdk_v0.9.4_14_12_19.zip https://github.com/esp8266/esp8266-wiki/raw/master/sdk/esp_iot_sdk_v0.9.4_14_12_19.zip
unzip esp_iot_sdk_v0.9.4_14_12_19.zip
mv esp_iot_sdk_v0.9.4 ESP8266_SDK
mv License ESP8266_SDK/
cd /opt/Espressif/ESP8266_SDK
wget -O lib/libc.a https://github.com/esp8266/esp8266-wiki/raw/master/libs/libc.a
wget -O lib/libhal.a https://github.com/esp8266/esp8266-wiki/raw/master/libs/libhal.a
wget -O include.tgz https://github.com/esp8266/esp8266-wiki/raw/master/include.tgz
tar -xvzf include.tgz

Except the change of the SDK version, the instructions are the same as the original reference post.

Installing the esptool

The esptool will pick up the compiler output and extract/create the files needed for flashing the esp8266. The instructions for this step are the same as in the original reference post. In my case, I’ve got the tool from the sources and not installed the deb file:

cd /opt/Espressif
wget https://github.com/esp8266/esp8266-wiki/raw/master/deb/src/esptool_0.0.2.orig.tar.gz
tar xvzf esptool_0.0.2.orig.tar.gz
cd esptool

Let’s put the tool on the search path. In my case I’ve choose the Xtensa bin directory. (Edit: it should be put at /usr/bin. See below at building the blinky example)

cd /opt/Expressif/esptool
chmod +w /opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf/bin
ln -s /opt/Espressif/esptool/esptool ../crosstool-NG/builds/xtensa-lx106-elf/bin
sudo cp /opt/Espressif/esptool/esptool /usr/bin

Installing the firmware uploader tool esptool-py

cd /opt/Espressif
git clone https://github.com/themadinventor/esptool esptool-py
chmod +w /opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf/bin
ln -s /opt/Espressif/esptool-py/esptool.py crosstool-NG/builds/xtensa-lx106-elf/bin/

With this tool we can upload the generated firmware into the esp8266.

Building

So now we are ready and we can follow the instructions here: https://github.com/esp8266/esp8266-wiki/wiki/Building to starting building things.

There are some code examples, one of them named blinky but the make file assumes that the esptool is located /usr/bin. This is because I’ve not installed the esptool through the deb package. So we can copy the esptool command to the /usr/bin directory: sudo cp  /opt/Espressif/esptool/esptool   /usr/bin.

We can now build the AT demo and the IoT demo. Just follow the instructions. For example for the IoT example just do the following (Make sure that the path is set: export PATH=/opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf/bin/:$PATH):

cd /opt/Espressif/ESP8266_SDK
sed -i -e 's/xt-ar/xtensa-lx106-elf-ar/' -e 's/xt-xcc/xtensa-lx106-elf-gcc/' -e 's/xt-objcopy/xtensa-lx106-elf-objcopy/' Makefile
mv examples/IoT_Demo .
cd /opt/Espressif/ESP8266_SDK/IoT_Demo 
make 
cd .output/eagle/debug/image
esptool -eo eagle.app.v6.out -bo eagle.app.v6.flash.bin -bs .text -bs .data -bs .rodata -bc -ec 
xtensa-lx106-elf-objcopy --only-section .irom0.text -O binary eagle.app.v6.out eagle.app.v6.irom0text.bin 

Uploading:

For uploading just follow the instructions here: https://github.com/esp8266/esp8266-wiki/wiki/Uploading but basically is using the esptool.py to upload the firmware to the esp8266. Just make sure tha GPIO0 is grounded at boot time.

The IoT RTOS based example:

There is a beta SDK for the esp8266 based on the freeRtos according to this: https://github.com/espressif.

To use this SDK (Make sure that the PATH environment variable is correctly defined):

cd /opt/Expressif
git clone https://github.com/espressif/esp_iot_rtos_sdk.git
git clone https://github.com/espressif/esp_iot_rtos_sdk_lib.git
cp esp_iot_rtos_sdk_lib/lib/*  esp_iot_rtos_sdk/lib
cd esp_iot_rtos_sdk

make

We should have now the IoT RTOS main image ready. To generate the firmware files:

cd app/.output/eagle/debug/image/
esptool -eo eagle.app.v6.out -bo eagle.app.v6.flash.bin -bs .text -bs .data -bs .rodata -bc -ec
xtensa-lx106-elf-objcopy --only-section .irom0.text -O binary eagle.app.v6.out eagle.app.v6.irom0text.bin

And we can flash it:

/opt/Espressif/esptool-py/esptool.py --port /dev/ttyUSB0 write_flash 0x00000 eagle.app.v6.flash.bin 0x40000 eagle.app.v6.irom0text.bin

The default baud rate for the IoT example is 74880…

 Exploring some application:

Now that we have all the tools for developing natively we can check out some applications:

  • esp-mqtt – A native MQTT client for the esp8266
  • Ardunet – A work in progress that mimics the Arduino way of making applications on the esp8266.

The download links are:

esp-mqtt:

For building it, first change your connection parameters at include/user_config.h

cd /opt/Espressif
git clone https://github.com/tuanpmt/esp_mqtt.git
cd esp_mqtt
cp Makefile.linux Makefile
make
make flash

Ardunet:

For building it:

cd /opt/Espressif
git clone https://github.com/Karang/Ardunet.git
cd Ardunet
cp -R /opt/Espressif/esp_iot_rtos_sdk/ld .
make
cd /opt/Espressif/Ardunet/src/.output/eagle/debug/image
esptool -eo eagle.app.v6.out -bo eagle.app.v6.flash.bin -bs .text -bs .data -bs .rodata -bc -ec
xtensa-lx106-elf-objcopy --only-section .irom0.text -O binary eagle.app.v6.out eagle.app.v6.irom0text.bin

And we can flash it.

Conclusion:

So after this long post I hope that things now are a bit more documented from the installation and setting up point of view.