Docker container web interface – Portainer and Riot-OS Development

This post is a follow up of starting up with RIOT-OS. To be able to develop with RIOT-OS an easy (and easier) way to do so is just to install docker and web UI docker interface Portainer to control docker.

So we will install Docker, Portainer, and finally the RIOT-OS building environment.

Installing Docker and Portainer, is an initial stepping stone for using the dockerized development environment for RIOT-OS, since I don’t want to install all the development environments in my machine.

Installing Docker:
On Arch-Linux is as simple as installing the Docker package using pacman, enabling the services and rebooting.
Basically we need to run, as root the following commands:

pacman -S docker
systemctl enable docker.service
reboot

After rebooting the following command should return some information

docker info

A sample output is:

Containers: 2
 Running: 0
 Paused: 0
 Stopped: 2
Images: 9
Server Version: 18.09.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
...
...
...

Installing Portainer
Installing the Docker Portainer Web UI is as simple as:

docker pull portainer/portainer

To run Portainer a set of complete instructions on this page, but basically on the simplest way is:

$ docker volume create portainer_data
$ docker run -d -p 9000:9000 --name portainer --restart always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer

We can now check if the docker image is up:

$ docker ps
CONTAINER ID        IMAGE                 COMMAND             CREATED             STATUS              PORTS                    NAMES
7a38ae7fc922        portainer/portainer   "/portainer"        4 seconds ago       Up 3 seconds        0.0.0.0:9000->9000/tcp   portainer

Since I have already ran the Portainer container, the initial setting up steps when accessing the URL HTTP://localhost:9000 do not appear, but we need to choose:

  1. A set of credentials to use as de administrator for portainer
  2. The local machine registry to connect to the local docker containers.

1- At initial access we define an user and password:

Portainer Credentials

2- Then we connect to our local docker instance:
Portainer Local Docker

Press Connect and then we can now access our Docker instance from Portainer:
Portainer Main Screen

Pressing the Local Docker Connection we can now manage our docker resources.

Installing the build environment for RIOT-OS
We can do it by two ways:

From the command line:

docker pull riot/riotbuild

or use Portainer:

This container is very big, so we need to wait some time for the container image download. The command line shows in greater detail the download process.

After the image is downloaded, we can follow these instructions for building our apps using the docker container as the build environment.

After the image is installed:

To use is is as simple as going the the examples directory and do:

make BUILD_IN_DOCKER=1

From this we are now able to build based RIOT-OS applications for several targets, including the ESP8266/ESP32.

As we can see we even don’t need to have a running container, just the image.

Advertisements

Arch Linux with full encrypted disk

So, I’ve bought a new light weight laptop, an HP Envy 13.3 1.2Kg I7, where I wiped out Windows 10 and installed Arch Linux.

Just for security reasons I’ve decided to do a full disk encryption install, including boot.

There are several instructions on the Web, including videos on youtube in how to do it, and so on this gist I have my instructions for the installation, based off course in other gists and instructions.

In this post is the configuration instructions that worked for me and also to allow better find from google/bing/ddg.

# WARNING: WORK IN PROGRESS, USE THESE STEPS WITH CAUTION. IT WILL CLEAR ALL DISK DATA!!
I REALLY recommend first to use a Virtual Box machine with EFI support enabled to test everything before doing it on a real machine.

# Arch installation on a HP ENVY 13 inch laptop (ah0006np part number: 16GB Ram, 512GB SSD)

OBJECTIVE:
Install Arch Linux with encrypted boot, root and swap filesystems and boot from UEFI, completly dumping Windows on the process.
No dual boot.
Windows, if necessary will be run on a Virtual Machine and re-use the Windows key that came with the laptop.

The configuration will be LVM on LUKS. Also a major difference from other tutorials is that the boot partition is also encrypted, and not a standard partition.

# Results do far:

– Disk encryption ok. GRUB boots slow (20s). Otherwise works fine.
– Wireless works ootb, but errors on dmesg output from time to time when high traffic.
– Sound and microfone works ok
– Webcam does work but needs configuration: See below at the end.
– Keyboard special keys work fine (brightness, Sound, Mute), including keyboard background lights, but F6 sound Mute Led does not work.
– Some screen corruption with the Intel Driver either SNA or UXA. Nouveau crashes, nvidia driver didn’t work. To be checked -> Issue with QT 5 and Konsole/Kate applications, not a Intel Driver issue.
– KDE SDDM doesn’t recover well if screen DPMS is activated. I’ve disable it so far to solve it.
– Suspend/resume works fine.
– Battery time so far, around 4/5 hours.


# Desired disk layout:

+---------------+----------------+----------------+----------------+
|ESP partition: |Boot partition: |Volume 1:       |Volume 2:       |
|               |                |                |                |
|/boot/efi       |/boot           |root            |swap            |
|               |                |                |                |
|               |                |/dev/vg0/root   |/dev/vg0/swap   |
|/dev/sda1      |/dev/sda2       +----------------+----------------+
|unencrypted    |LUKS encrypted  |/dev/sda3 encrypted LVM on LUKS  |
+---------------+----------------+---------------------------------+

The final result is to have an Arch Linux Installation with full disk encryption and with a basic set of applications such as the KDE Plasma Desktop.

These instructions have several sources, namely:
https://grez911.github.io/cryptoarch.html
and this WordPress post.

The installation process on this guide is for the Arch Linux installation onto an HP Envy 13, 16GB RAM with 512MB ssd laptop. This laptop comes with Windows 10 Home installed, and as far as my model goes, it comes with an Intel WiFi board and a WD Sandisk SN520 512GB NVME SSD.

The official Arch installation guide contains more details that you should refer to during this installation process.
That guide resides at: https://wiki.archlinux.org/index.php/Installation_Guide

## Boot from image

Download the archlinux-\*.iso image from https://www.archlinux.org/download/ and its GnuPG signature.
Use gpg –verify to ensure your archlinux-\*.iso is exactly what the Arch developers intended. For example
at the time of installation:

$ gpg --verify archlinux-2017.10.01-x86_64.iso.sig
gpg: Signature made Sun 01 Oct 2017 07:29:43 AM CEST using RSA key ID 9741E8AC
gpg: Good signature from "Pierre Schmitz "
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 4AA4 767B BC9C 4B1D 18AE  28B7 7F2D 434B 9741 E8AC

Currently the Arch ISO is archlinux-2018.11.01-x86_64.iso.

Burn the archlinux-\*.iso to a 1+ Gb USB stick. You can use the dd command, unetbootin or Etcher.

Connect the USB stick to the usb port and power on/cycle the machine to boot.
If your USB stick fails to boot, ensure that Secure Boot is disabled in your UEFI configuration.

Note: To access the BIOS on the Envy Laptop, turn on the laptop and press several times the ESC key or the F10 key to access the BIOS while the screen is black.
First I moved the boot order to have the USB boot at the top.
Then we need to disable the secure boot option and press F10 to save. Confirm saving it.

Attention now: There is a confirmation screen to really commit the secure boot option change. Enter the requested code and save.

After booting up:

Set your keymap only if not you are not using the default English keyboard.

$ loadkeys pt-latin1

We can now, if required backup the HP recovery partition, that I suppose is the Windows Install Media.

# Connect to the Internet.

Execute the wifi-menu command and select a Wifi network. On this HP Envy, the wireless card (Intel) was detected with no issues.

Check with the “ip a” command if there is network connection.

## Prepare your hard disk

In the next steps we will create necessary partitions and encrypt the main partition.

Find the correct block device

$ lsblk

In my case the correct block device (the NVME SSD of my laptop) is ‘nvme0n1’. (Depends on the machine)

Create and size partitions appropriate to your goals using gdisk.

$ gdisk /dev/nvme0n1

Press p to show the partitions.

In my case I have a 260Mb EFI partition, a 16MB Reserved Microsoft Partition, a 460GB partition and a 980MB and another 15GB partition.

From this point on, everything that is to be done, will destroy the disk data.

# Delete all partitions on disk

Use the d command to delete all partitions. Use d, then partition number. Repeat for all partitions

Press o to create the GPT.

Create three partitions: One for the EFI, one for boot and the other will be used to have the Arch Linux installation. To create a partition, press n:

1. Partition 1 = 512 MB EFI partition (Hex code EF00). Initial Sector: ; End: 512M; Type: EF00
2. Partition 2 = 1GB Boot partition (Hex code 8300)
3. Partition 3 = Size it to the last sector of your drive. (default) (Hex code 8E00 – Linux LVM Partition)

Review your partitions with the ‘p’ command.
Write your gdisk changes with ‘w’.

Check again the names with the blkid command to know the partitions name:

1. EFI: /dev/nvme0n1p1
2. BOOT: /dev/nvme0n1p2
3. Arch: /dev/nvme0n1p3

# Create filesystems
The EFI filesystem must be FAT32:

$ mkfs.vfat -F 32 /dev/nvme0n1p1

The other filesystems are to be encrypted.

(optional) Before creating the partitions we can use the command

cryptsetup benchmark 

to see how fast the different encryption algoritms are.

# Encrypted /boot partition

$ cryptsetup -c aes-xts-plain64 -h sha512 -s 512 --use-random luksFormat /dev/nvme0n1p2
$ cryptsetup open /dev/nvme0n1p2 cryptboot
$ mkfs.ext4 /dev/mapper/cryptboot

The first command will ask for the disk passphrase. Do not forget it!.

ATTENTION:
The first crypsetup command will set the LUKS with default iter-time parameters, which may or may not make grub to boot slow (around 20s). If this is not fine add the following parameter: –iter-time=5000 (This will affect security, so use a large key phrase)

The last command will create a /dev/mapper/cryptboot device.
We can check that it was created with the command ls /dev/mapper

# Create encrypted LUKS device for the LVM

cryptsetup -c aes-xts-plain64 -h sha512 -s 512 --use-random luksFormat /dev/nvme0n1p3
cryptsetup open /dev/nvme0n1p3 cryptlvm


## Create encrypted LVM partitions

These steps will create the required root partition and an optional partition for swap.
Modify this structure only if you need additional, separate partitions. The sizes used below are only suggestions.
The VG and LV labels ‘ArchVG, root and swap’ can be changed to anything memorable to you. Use your labels consistently, below!

$ pvcreate /dev/mapper/cryptlvm
$ vgcreate ArchVG /dev/mapper/cryptlvm
$ lvcreate -L +16G ArchVG -n swap
$ lvcreate -l +100%FREE ArchVG -n root

Again, we can see on /dev/mapper if the logical volumes where created.

## Create filesystems on your encrypted partitions

$ mkswap /dev/mapper/ArchVG-swap
$ mkfs.ext4 /dev/mapper/cryptboot
$ mkfs.ext4 /dev/mapper/ArchVG-root

Mount the new system

mount /dev/mapper/ArchVG-root /mnt
swapon /dev/mapper/ArchVG-swap
mkdir /mnt/boot
mount /dev/mapper/cryptboot /mnt/boot
mkdir /mnt/boot/efi
mount /dev/nvme0n1p1 /mnt/boot/efi

# Install the Arch system

This installation command provides a decent set of basic system programs which will also support WiFi when initally booting into your Arch system.

At this point we need to have a network connection. Since the HP only has Wifi connection, we need to setup the WiFi connection. Other alternative is to use an Ethernet USB dongle that is recognized by the Arch boot iso.
Also, if you are behind a proxy, you can set the http_proxy and https_proxy variables to access the internet.

(Optional) Use reflector to speedup download (credit goes to u/momasf)

Change COUNTRY to (surprise) your country name.

pacman -Sy reflector
reflector --country 'COUNTRY' --age 12 --protocol https --sort rate --save /etc/pacman.d/mirrorlist

I won’t install base-dev here to save time at the installation.

$ pacstrap /mnt base grub-efi-x86_64 efibootmgr dialog wpa_supplicant vim

# Create and review FSTAB
The -U option pulls in all the correct UUIDs for your mounted filesystems.

 $ genfstab -U /mnt >> /mnt/etc/fstab
 $ nano /mnt/etc/fstab  # Check your fstab carefully, and modify it, if required.
 

Enter the newly installed system

$ arch-chroot /mnt /bin/bash

Set the system clock, you can replace UTC with your actual timezone

$ ln -fs /usr/share/zoneinfo/Europe/Lisbon /etc/localtime
$ hwclock --systohc --utc

Assign your hostname

$ echo mylaptop > /etc/hostname

My requirements for the locale are:
– Metric system
– 24h time format
– dd/mm/yyyy date format
– Portuguese language
– A4 paper size
– But all help, error messages are in English

The *pt_PT.UTF-8* plus *en_US.UTF-8* locale meets those requirements. To set up this locale:

– In /etc/locale.gen

en_US.UTF-8 UTF-8
pt_PT.UTF-8 UTF-8

– In /etc/locale.conf, you should **only** have this line:

LANG=en_US.UTF-8

We will change other settings on Bash profile.

Now run:

$ locale-gen

Create a new file vconsole.conf so that the console keymap is correctly set at boot. Create the file and add the following line:

KEYMAP=pt-latin1

Set your root password

$ passwd

Create a User, assign appropriate Group membership, and set a User password.

$ useradd -m -G audio,games,log,lp,optical,power,scanner,storage,video,wheel -s /bin/bash memyselfandi
$ passwd memyselfandi

Configure mkinitcpio with the correct HOOKS required for your initrd image

$ nano /etc/mkinitcpio.conf

Use this HOOKS statement: (I’ve moved keyboard before keymap, encrypt and so on…)

HOOKS="base udev autodetect modconf block keyboard keymap encrypt lvm2 resume filesystems fsck"

Generate your initrd image

mkinitcpio -p linux

## Install and configure Grub-EFI
Since we have the boot partition INSIDE the encrypted disk, we need to add the following option to the Grub options:

Edit the file /etc/default/grub and uncomment the following line:

GRUB_ENABLE_CRYPTODISK=y

And then we can install Grub, which will create an EFI entry named ArchLinux

grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ArchLinux

Edit /etc/default/grub so it includes a statement like this:

GRUB_CMDLINE_LINUX="cryptdevice=/dev/nvme0n1p3:cryptlvm resume=/dev/mapper/ArchVG-swap i915.enable_guc=3"

I’ve also added the i915 configuration line.

Other way of doing it is to use UUID:

blkid /dev/nvme0n1p3 -s UUID -o value

And use the UUID outputed on this command line:

GRUB_CMDLINE_LINUX="cryptdevice=UUID=55994-XXXX-xXXXX-XXXXX:cryptlvm resume=/dev/mapper/ArchVG-swap"

Generate Your Final Grub Configuration:

$ grub-mkconfig -o /boot/grub/grub.cfg

At this point there are some errors regarding failing to connect to lvmetad, which are normal and can be ignored.

# Mounting /boot without password request
Grub will ask for passwords to access the encrypted volumes. We can do this automatically:

dd bs=512 count=8 if=/dev/urandom of=/etc/key
chmod 400 /etc/key
cryptsetup luksAddKey /dev/nvme0n1p2 /etc/key
echo "cryptboot /dev/nvme0n1p2 /etc/key luks" >> /etc/crypttab

# Mounting root LVM without password prompt

dd bs=512 count=8 if=/dev/urandom of=/crypto_keyfile.bin
chmod 000 /crypto_keyfile.bin
cryptsetup luksAddKey /dev/nvme0n1p3 /crypto_keyfile.bin
sed -i 's\^FILES=.*\FILES="/crypto_keyfile.bin"\g' /etc/mkinitcpio.conf
mkinitcpio -p linux
chmod 600 /boot/initramfs-linux*

The mkinitcpio.conf FILES line will look like:

FILES="/crypto_keyfile.bin"

# Enable Intel microcode CPU updates (if you use Intel processor, of course)

pacman -S intel-ucode
grub-mkconfig -o /boot/grub/grub.cfg

# Check EFI Boot Manager
Check that the EFI Boot manager has the ArchLinux entry:

$ efibootmgr

For example if ArchLinux entry is Boot0003, check if on the boot order, 0003 is on the head of the list.
If not change the order with:

$ efibootmg -o 0003,0002,0001,0000

Exit Your New Arch System

$ exit

Unmount all partitions

$ umount -R /mnt
$ swapoff -a

Reboot and Enjoy Your Encrypted Arch Linux System!

reboot

___

# Setup system

We need again to connect to the internet, so run again the *wifi-menu*.

Install bash completion for reduced typing effort and other packages if necessary:

$ pacman -S sudo bash-completion base-devel git

To be able to use sudo from your normal user add wheel to sudoers.

$ EDITOR=nano visudo

Uncomment the line

%wheel      ALL=(ALL) ALL

From this point on, it really depends of what need there is for the machine.

# Making the webcam to work.
The webcam id appears at the lsusb output:

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2a Intel Corp. 
Bus 001 Device 002: ID 04ca:7090 Lite-On Technology Corp. 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The webcam is the Bus 001:002 device: ID 04ca:7090.
Add the following rule at /etc/udev/rules.d

KERNEL=="video[0-9]*", SUBSYSTEM=="video4linux", SUBSYSTEMS=="usb", ATTRS{idVendor}=="04ca", ATTRS{idProduct}=="7090", SYMLINK+="video-cam"

Load the module to activate the webcam:

modprobe uvcvideo

The /dev/video0 and 1 devices should appear.

SdrPlay and ArchLinux

I’ve used the RTLSDR usb cheap dongles for a while for radio spectrum listening and scanning, but the fact is that in my case, they all are garbage. Yes they work fine for bands above 200/250Mhz (not too much noise there), but below that, my nearby powerful FM stations, overload the dongle and I can’t get much out of them (Even with a home made FM notch filter). Also in most cases, to hear HF bands (80m/40m/20) a hack (Q branch sampling) or an upconverter is needed, but still the FM stations blank out the other signals.

So for a bit of more money, I’ll give SDRPlay a go, since it is a full spectrum SDR (1.5Mz to 2GHz with a 10Mhz scope band) and also it can be used successfully as a pan adapter for Amateur band radios.

While waiting for my SDRPlay to arrive, and since the SDRUno program doesn’t run on Linux (I have Arch Linux), I started to get the software ready to use the SDR device with CubicSDR.

The software needed to access the SdrPlay devices in Linux is SoapySDR, the SDRPlay Driver for SoapySDR, and of course Cubic SDR. Also needed is the binary driver (yes I know… ) API/HW Driver – v2.13 to be installed.

While there are a lot of instructions to compile SoapySDR and Cubic SDR, including mine: Cubic SDR and SoapySDR, and a also very good video from SDRPlay I had a problem with the SoapySDRPlay component.

As we will se the problem is not with the SoapySDRPlay component but with the installation of the binary proprietary driver.

Basically during the binary driver installation I had the following output:

Press y and RETURN to accept the license agreement and continue with
the installation, or press n and RETURN to exit the installer [y/n] y
./install_lib.sh: line 17: arch: command not found
Architecture: 
API Version: 2.13
Remove old libraries...
Install /usr/local/lib/libmirsdrapi-rsp.so.2.13
cp: cannot stat '/libmirsdrapi-rsp.so.2.13': No such file or directory
chmod: cannot access '/usr/local/lib/libmirsdrapi-rsp.so.2.13': No such file or directory
Remove old header files...
Install /usr/local/include/mirsdrapi-rsp.h
Udev rules directory found, adding rules...
Libusb found, continuing...
Finished.

If we continue with this situation (the cp: cannot stat ‘/libmirsdrapi-rsp.so.2.13’: No such file or directory
chmod: cannot access ‘/usr/local/lib/libmirsdrapi-rsp.so.2.13’: No such file or directory
errors), the SoapySDRPlay compilation will fail:

[ 20%] Building CXX object CMakeFiles/sdrPlaySupport.dir/Registation.cpp.o
[ 40%] Building CXX object CMakeFiles/sdrPlaySupport.dir/Settings.cpp.o
[ 60%] Building CXX object CMakeFiles/sdrPlaySupport.dir/Streaming.cpp.o
[ 80%] Building CXX object CMakeFiles/sdrPlaySupport.dir/Version.cpp.o
make[2]: *** No rule to make target '/usr/local/lib/libmirsdrapi-rsp.so', needed by 'libsdrPlaySupport.so'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:68: CMakeFiles/sdrPlaySupport.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

To solve this we need to correcly install the driver, that in my case was the failure of copying the binary blob to the correct location.

So all we need is to go the directory where the binary installation is and create a working directory:

mkdir work
/SDRplay_RSP_API-Linux-2.13.1.run --target work

So the driver will install but in the working directory will be the missing driver to copy to the correct location:

sudo cp work/x86_64/libmirsdrapi-rsp.so.2.13 /usr/local/lib

Done.

I’ve also added to my home directory .bashrc file the following line to allow the new libraries to be found:

export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH

And then just do a source .bashrc so that changes take place.

We can now compile SoapySDRPlay with success:

Scanning dependencies of target sdrPlaySupport
[ 20%] Building CXX object CMakeFiles/sdrPlaySupport.dir/Registation.cpp.o
[ 40%] Building CXX object CMakeFiles/sdrPlaySupport.dir/Settings.cpp.o
[ 60%] Building CXX object CMakeFiles/sdrPlaySupport.dir/Streaming.cpp.o
[ 80%] Linking CXX shared module libsdrPlaySupport.so
[100%] Built target sdrPlaySupport

Execute the usual sudo make install and running the SoapySDRUtil command, the new driver should be available:

SoapySDRUtil --info
######################################################
##     Soapy SDR -- the SDR abstraction library     ##
######################################################

Lib Version: v0.7.0-g37429d89
API Version: v0.7.0
ABI Version: v0.7
Install root: /usr/local
Search path:  /usr/local/lib/SoapySDR/modules0.7
Module found: /usr/local/lib/SoapySDR/modules0.7/libremoteSupport.so  (0.5.0-6efb692)
Module found: /usr/local/lib/SoapySDR/modules0.7/librtlsdrSupport.so  (0.2.5-ad3d8e0)
Module found: /usr/local/lib/SoapySDR/modules0.7/libsdrPlaySupport.so (0.1.0-12c3db6)
Available factories... remote, rtlsdr, sdrplay
Available converters...
 -  CF32 -> [CF32, CS16, CS8, CU16, CU8]
 -  CS16 -> [CF32, CS16, CS8, CU16, CU8]
 -  CS32 -> [CS32]
 -   CS8 -> [CF32, CS16, CS8, CU16, CU8]
 -  CU16 -> [CF32, CS16, CS8]
 -   CU8 -> [CF32, CS16, CS8]
 -   F32 -> [F32, S16, S8, U16, U8]
 -   S16 -> [F32, S16, S8, U16, U8]
 -   S32 -> [S32]
 -    S8 -> [F32, S16, S8, U16, U8]
 -   U16 -> [F32, S16, S8]
 -    U8 -> [F32, S16, S8]

We can see now that the libsdrPlaySupport.so module is loaded.

So who ever encounters the issue that I had, hopefully this will be the solution since the sdrplay binary installation scripts might fail.

NodeJS BLE Applications using BLENO on Arch Linux

BLENO is a greate NodeJS based library for building applications that communicat with other devices (Smartphones, tables, sensor tags) using Bluetooth Low Energy (BLE).

This post is just to quickly document some requirements for successfully use the BLENO library, in my case, on Arch Linux running the latest Plasma (KDE) desktop.

The tools:

Most the information available on the internet for using and controlling the bluetooth adapter uses the now deprecated tools hcitool, hciconfig and so on. Check here the deprecated list of commands.

So we need to use the new tools from the latest Bluez (Bluetooth Linux implementation): btmgm, btinfo, …

Making Bleno examples work:

The simplest example to try out the BLENO library is the battery example located at: […]/bleno/examples/battery-service

First let’s check if our computer/laptop bluetooth adapter is available: Note that all commands must be ran as the root user:

root@pcortex:/opt/bleno/examples/battery-service# btinfo local
Bluetooth information utility ver 5.45
Failed to open HCI user channel
root@pcortex:/opt/bleno/examples/battery-service# 

This issue can be circumvented by stopping the higher level bluetooth stack:

root@pcortex:/opt/bleno/examples/battery-service# systemctl stop bluetooth
root@pcortex:/opt/bleno/examples/battery-service# btinfo local
Bluetooth information utility ver 5.45
HCI version: 6
HCI revision: 7869
LMP version: 6
LMP subversion: 64512
Manufacturer: 2
root@pcortex:/opt/bleno/examples/battery-service#

In case of previously disabling the Bluetooth through the graphical interface:

Disabling the Bluetooth here will have this behaviour (in this case the bluetooth service is still running):

root@pcortex:/opt/bleno/examples/battery-service# systemctl start bluetooth   (<- After this disable bluetooth on the graphical interface)
root@pcortex:/opt/bleno/examples/battery-service# btinfo local
Bluetooth information utility ver 5.45
Failed to open HCI user channel
root@pcortex:/opt/bleno/examples/battery-service# btmgmt power on
Set Powered for hci0 failed with status 0x12 (Blocked through rfkill)
root@pcortex:/opt/bleno/examples/battery-service# 

Even stopping the Bluetooth service keeps the BT adapter disabled:

root@pcortex:/opt/bleno/examples/battery-service# systemctl stop bluetooth
root@pcortex:/opt/bleno/examples/battery-service# btmgmt power on
Set Powered for hci0 failed with status 0x12 (Blocked through rfkill)
root@pcortex:/opt/bleno/examples/battery-service#

We can check this with the rfkill command:

root@pcortex:/opt/bleno/examples/battery-service# rfkill list
0: phy0: Wireless LAN
        Soft blocked: no
        Hard blocked: no
2: hci0: Bluetooth
        Soft blocked: yes
        Hard blocked: no
root@pcortex:/opt/bleno/examples/battery-service# 

We can unblock now the adapter:

root@pcortex:/opt/bleno/examples/battery-service# rfkill unblock 2
root@pcortex:/opt/bleno/examples/battery-service# rfkill list
0: phy0: Wireless LAN
        Soft blocked: no
        Hard blocked: no
2: hci0: Bluetooth
        Soft blocked: no
        Hard blocked: no
root@pcortex:/opt/bleno/examples/battery-service# btinfo local
Bluetooth information utility ver 5.45
HCI version: 6
HCI revision: 7869
LMP version: 6
LMP subversion: 64512
Manufacturer: 2
root@pcortex:/opt/bleno/examples/battery-service# btmgmt power on
hci0 class of device changed: 0x00010c
hci0 Set Powered complete, settings: powered bondable ssp br/edr le secure-conn 
root@pcortex:/opt/bleno/examples/battery-service# 

So why we are having all this work for making sure that the BT adapter is powered on AND the bluetooth stack is stopped (systemctl stop bluetooth).

The answer is quite simple. If we don’t do this the BLENO examples will seem to work (they start) but the BLE advertised services are the bluetooth Bluez services and not our code.

To explain, check the following behaviour where we start the BLENO Battery Service with the Bluetooth stack started:

root@halcyon:/opt/bleno/examples/battery-service# systemctl start bluetooth
root@halcyon:/opt/bleno/examples/battery-service# node main.js 
on -> stateChange: poweredOn
on -> advertisingStart: success
setServices: success

Using the Nordic nRF Connect Android App we can see the non working behaviour vs what we should expect from the Bleno Battery example:

BLE Scan Results

Pressing Connect we can see on Client that no service are provided. This is due to the fact that the desktop bluetooth is enabled):

Now let’s disable the bluetooth stack (which powers the BT adapter) and start again the Bleno Battery example:

root@pcortex:/opt/bleno/examples/battery-service# systemctl stop bluetooth
root@pcortex:/opt/bleno/examples/battery-service# node main.js 

Example hangs in here, because BT adapter is disabled/off

^Croot@pcortex:/opt/bleno/examples/battery-service# btmgmt power on
hci0 class of device changed: 0x00010c
hci0 Set Powered complete, settings: powered bondable ssp br/edr le secure-conn 
root@pcortex:/opt/bleno/examples/battery-service# node main.js 
on -> stateChange: poweredOn
on -> advertisingStart: success
setServices: success

And now if we scan again and connect to the Battery example with our mobile phone through the Nordic application we have:

It works now!

We can confirm that because on the file battery-service.js the service identifier is defined:

function BatteryService() {
  BatteryService.super_.call(this, {
      //uuid: '180F',
      uuid: 'ff51b30e-d7e2-4d93-8842-a7c4a57dfb07',
      characteristics: [
          new BatteryLevelCharacteristic()
      ]
  });
}

and it is the same detected by the Android application.

Dropbox doesn’t start (Linux)

So a quick note regarding Dropbox running on ArchLinux with Nvidia drivers. Since I use KDE Plasma 5 that doesn’t show system tray icons of some applications that have not migrated to the new system tray protocol, I didn’t notice that after upgrading Dropbox it stopped working.

A quick investigation, I’ve found out that starting Dropbox from the command line did nothing. It started and stopped. Something is going on…

So I’ve used the strace command to see if I could catch something meaningful:

[pcortex@pcortex:~]$ cd .dropbox-dist/dropbox-lnx.x86_64-3.10.8/
[pcortex@pcortex:dropbox-lnx.x86_64-3.10.8]$ strace ./dropbox 2> stracedump.txt

And sure enough in the stracedump.txt I had the following error:

open("/usr/lib/dri/tls/swrast_dri.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/dri/swrast_dri.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0} ---
rt_sigaction(SIGSEGV, {0x7f09e1dea010, [SEGV], SA_RESTORER|SA_RESTART, 0x7f09e1a88d60}, NULL, 8) = 0
write(4, "Fatal Python error: ", 20)    = 20
write(4, "Segmentation fault", 18)      = 18
write(4, "\n\n", 2)                     = 2
write(4, "Traceback (most recent call firs"..., 36) = 36
write(4, "  File ", 7)                  = 7
write(4, "\"", 1)                       = 1
write(4, "d", 1)                        = 1

So the error is related to the Software Raster library, that I don’t have because I’m running Nvidia proprietary drivers.

The confirmation came with the following command:

[pcortex@pcortex:dropbox-lnx.x86_64-3.10.8]$ LIBGL_DEBUG=verbose ./dropbox
libGL: OpenDriver: trying /usr/lib/dri/tls/swrast_dri.so
libGL: OpenDriver: trying /usr/lib/dri/swrast_dri.so
libGL error: dlopen /usr/lib/dri/swrast_dri.so failed (/usr/lib/dri/swrast_dri.so: cannot open shared object file: No such file or directory)
libGL error: unable to load driver: swrast_dri.so
libGL error: reverting to indirect rendering

The solution?
Quite simple, just remove (backup) the libGL.so.1 library that exists on the Dropbox directory.
After the removal, Dropbox starts just fine.

Linux: Slow or unbearable performance with high memory usage applications

Well this post title is a bit vague because the issue that have affected me can have several sources, not just the one that I’ll describe. But I suppose the solution is the same independently for all sources.

Anyway, I was suffering from a strange problem on both my computers, the desktop has 6GB of RAM, and the laptop, 16GB.

On the desktop machine, where I use KDE 4 on Arch Linux, the desktop can froze when using some resource intensive applications like Google Maps on Firefox or Chromium and Android Studio. When the frozen desktop situation happens, I just have to wait a few minutes, during which time, the keyboard is irresponsible and the mouse is “jumpy”, or doesn’t work at all. And so after a while, one random application, that could be anything, including the desktop, is killed, with some cryptic message regarding sacrifice on the system logs…

Anyway, on the Desktop, due to the fact that I’ve upgraded the main disk to an SSD, I didn’t enabled the swap partition. And that was one of the main reasons of this behaviour that I was having. Despite having 6GB of RAM, a swap file is good to have, and so I’ve used a swap partition on one of the spinning disks.

That apparently stopped the issues on the desktop and it never froze like it used to when performing the same tasks.

On the laptop it was another matter, mainly because of the fact that it only has a 120GB SSD disk, and no spinning disks, I had no swap file/partition created or enabled. I though that with 16GB of RAM, why bother… but due to professional related activities, I had to start to work with the HortonWorks HDP 2.3 big data/hadoop platform, and that is a heavy resource hog…

So, despite of having 16GB of RAM available, the Hadoop virtual machines based on Vmware could make the computer just froze for lenghty periods of time, just like my desktop computer.

But, now I know that was related to fact that I didn’t have a swap partition/file, and so I’ve created one.

The swap file solution for this specific case didn’t solve completely my issue, but I did managed to see a process named khugepaged consuming huge amounts of CPU.

And that was the reason that I had for my frozen desktop and virtual machine for some lengths of time.

So I’ve disable the Huge Page support on my host operating system, AND on my guest CentOS virtual machine that is running HDP 2.3:

[root@dune:cortex]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
[root@dune:cortex]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

[root@dune:cortex]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@dune:cortex]# cat /sys/kernel/mm/transparent_hugepage/enabled          
always madvise [never]

More info can be obtain here:

http://docs.mongodb.org/manual/tutorial/transparent-huge-pages/

So this might not be the magic bullet if we are having desktop freezes, and not using huge amounts of RAM, but it might help.

Upgrading to an SSD, moving from Kubuntu to Arch Linux and setting up KDE

This is a single post just to make notice of some points:

My home desktop PC, was running since 2009 the Kubuntu distribution. I’ve never reinstalled it and did always the upgrades from 8.10 to 9.04, 9.10 and so on until 14.04. Almost six (!) years with no virus, trojan horses, and a single initial installation. I’ve upgraded through the years the KDE desktop since the ill famed initial 4.0 release, to the latest 4.6 release.

But things where starting to go awry… At the beginning I had a ATI 4870 graphic board, but ATI dropped support for it on Linux (and Windows) and since I had so many troubles with the proprietary driver, and to add it up I had no support for the board, I sold it and bought an Nvidia GTX660.

But with both graphic boards, I had suffered from a long time problem: I could only do a sleep/resume a number of times before the system hanged. The symptoms starting from loosing network connectivity and then restoring right back, and some cycling of these symptoms to a complete lockup. At first I thought that was due to the network driver used by my motherboard, a Asus P6T with  a Realtek RTL8111/8168/8411 board, but with the correct driver for it, the issue continued.  Booting up was a painful two/three minute wait from BIOS to desktop.

But after a while I could pin point my issues not to the network board but to the Xorg and video driver combination. Some EQ buffer overflow, hanged X, and finally the machine.

Ubuntu and Kubuntu based distributions have PPA’s repositories where some alternative software and versions of mainstream software are available. I had trouble with some of these alternative PPA’s when upgrading, and had to remove them (with ppa-purge) to be able to upgrade the Kubuntu distribution. Anyway, there is an PPA, xorg-edgers,  where alternative and more recent versions of Xorg Software and Nvidia drivers are provided and that almost solved my sleep/resume problem. The problem was that after adding this PPA the Ubuntu/Kubuntu nvidia driver and the Xorg-edgers driver, after updates the X server was unable to find the nvidia driver, despite being installed, and had to reinstall the driver again to get my desktop working.

That was it. I had enough with Ubuntu/Kubuntu distributions. My bad and my issues, sure…

I’ve being testing Arch Linux on my work laptop for almost an year with great success. I’ve chosen Arch Linux because I didn’t had to loose an unknown number of hours per year upgrading to the next distribution version, and an rolling release version makes more sense. And Arch has the greatest Linux Wiki out there even solving problems that happen on the Ubuntu/Kubuntu line, and also mainly there are only one (well two) repositories. No PPA’s mess.

So I completely moved all my systems to Arch Linux, and my sleep/resume issue is 100% solved.

SSD on a SATA II based motherboard

My desktop computer has an Asus P6T motherboard. This motherboard only has SATA II (SATA 2) ports. Is it worth to use an SSD on such a board without upgrading to a SATA III board?

The answer is YES. If you have a SATA II based motherboard AND not buying an SSD because of that, do not wait any more. Buy it now. It is totally worth it.

Arch Linux is of course way lighter than Kubuntu, and with an SSD ( by the way it’s a Crucial MX100 256GB) my two/three minute boot to desktop time came down to 15s…

hdparm -Tt /dev/sdc
/dev/sdc:
Timing cached reads:   16710 MB in  2.00 seconds = 8361.40 MB/sec
Timing buffered disk reads: 728 MB in  3.00 seconds = 242.59 MB/sec

Note the Samsung EVO 120GB that I have on my work computer, that has the flawed firmware that slows down with time, on a SATA III based motherboard:

/dev/sda:
Timing cached reads: 23864 MB in 2.00 seconds = 11943.41 MB/sec
Timing buffered disk reads: 164 MB in 3.01 seconds = 54.47 MB/sec

Compare this with my desktop old boot disk, an WD 640GB blue:
/dev/sdb:
Timing cached reads: 16580 MB in 2.00 seconds = 8296.87 MB/sec
Timing buffered disk reads: 356 MB in 3.00 seconds = 118.59 MB/sec

I hadn’t yet had the time to upgrade the firmware on the EVO…

Making KDE looking good on Arch Linux

To end this rather long and tedious post, and also to my future reference, to make KDE looking good:

    • Make sure that the infinality modifications are installed: https://wiki.archlinux.org/index.php/Infinality. Make sure that you update.
    • Make sure that the DPI detected by your X server are correct for your monitor. Install xdpyinfo with pacman -S xorg-xdpyinfo. Figure out what might be your monitor dpi using this site: http://dpi.lv/ . Execute xdpyinfo | grep -B2 resolution and see if are similar. If not you need to set the correct dpi either in xorg.conf or in the Kde settings.
      In my case I have a dual monitor setup, and hence the weird dpi 95×96.screen

      #0: dimensions: 3200×1080 pixels (856×286 millimeters)

      resolution: 95×96 dots per inch

      So my font settings are:

Selection_002

  • While the above settings make my fonts look good, the taskbar fonts where awful and took me a while to figure out that was not the dpi settings but the Desktop Theme.
  • To solve this first install Krita with pacman -S calligra-krita . This will install the Krita color themes that are, in my opinion very good. And on KDE System Settings -> Workspace Appearance -> Desktop Themes select Get New Themes and get the AndroidBit theme.
  • That’s it:Selection_004
  • I’ve selected the Produkte Desktop Theme
  • And the colors:Selection_005