Mosquitto Broker with Websockets enabled on the Synology NAS

Synology NAS are great devices, mainly due to the bundled software (CloudStation, Photo Station, to name a few), but also because it can run other software that is provided on non official package sources.
One of such providers is the SynoCommunity that provides a Mosquitto MQTT broker packaged for the Synology NAS. While I was able to successfully cross compile the Mosquitto broker for my DS212+ NAS with websockets enabled (I do have a draft post with the steps that I’ve never published), the SynoCommunity package allows a simpler way to install and use a Mosquitto broker that has the WebSockets protocol support compiled in.

So, what is needed to be done?

Setting up:

  • 1st: Add to your Synology server the SynoCommunity package source. See the detailed instructions on the community web page.
  • 2nd: Install the Mosquitto package from the Synology package manager.
  • 3rd: Edit the file located at /usr/local/mosquitto/var/mosquitto.conf. You need to connect through ssh as root to do this.

At the end of this file add the following lines:

listener 1883
listener 9001
protocol websockets

Note that the 1883 is the standard MQTT broker port, and 9001 is the port that I’ve choose have the websockets server listening/answering.

After the change, stop and start the Mosquitto broker on the package manager action dropdown box for the Mosquitto package.

Still using ssh, and logged in as root, check now that the port 9001 is active:

root@DiskStation:/volume1/homes/root# netstat -na | grep 9001 
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN       
tcp        0      0 192.168.1.250:9001      192.168.1.32:55331      ESTABLISHED

In the above case we can see that there is a websocket client connected to the broker from my workstation.

Testing:

For testing we can use the HiveMQ websocket client: http://www.hivemq.com/demos/websocket-client/ that allows to communicate using the websockets transport.

Despite the fact that the page is available on an external, internet based server, the websocket client will be running on YOUR web browser, on your machine in your own (internal) network, so at the connection settings for the broker, for Host I use my internal IP address: 192.168.1.250, and for the Port: I use the 9001 value.

I can now subscribe and publish to topics using the browser, and also using MQTT-SPY we can see and check the MQTT Websockets communication.

And that’s it!

Advertisements

md5deep and hashdeep on Synology DS212+

My personal photos are located at my desktop computer, and I have backups of them on my Synology DS212+ and my old faithful Linksys NSLU2 with an external disk.

The issue that I have now is that to keep backup times short I only backup the current year, because, well, the other years are static data. Previous years are already backed up and that data is not to get modified. Right? So what if corruption happens? How do I detected it? How can I be warned of such event? Right now I have no method to check if my digital photos of 2003 are totally intact in all of my three copies. And if  I detect corruption on one bad file/photo, which file is the correct one?

So I’m investigating this issue, and one of the tools available to create checksums and then to verify if everything is ok and no corruption has appeared is the md5deep and hashdeep programs. These are available as sources at this link: http://md5deep.sourceforge.net/start-hashdeep.html

These are the instructions for cross compiling these tools for the ARM based Synology DS212+. As usual this is done on a Linux Desktop/server machine.

1st) Set up the cross-compiling environment: Check out this link on the topic: https://primalcortex.wordpress.com/2015/01/04/synology-ds-crosscompiling-eclipseibm-rsmb-mqtt-broker/

2nd) Setting up the cross compiling environment variables is now a bit different:

export INSTALLDIR=/usr/local/arm-marvell-linux-gnueabi
export PATH=$INSTALLDIR/bin:$PATH
export TARGETMACH=arm-marvell-linux-gnueabi
export BUILDMACH=i686-pc-linux-gnu
export CROSS=arm-marvell-linux-gnueabi
export CC=${CROSS}-g++
export LD=${CROSS}-ld
export AS=${CROSS}-as
export AR=${CROSS}-ar
export GCC=${CROSS}-g++
export CXX=${CROSS}-g++

We will use the C++ compiler.

3rd) Create a working directory and clone the repository to your local machine: git clone https://github.com/jessek/hashdeep.git
4th) Change to the hashdeep directory and execute the following commands:

[pcortex@pcortex:hashdeep]$ sh bootstrap.sh
[pcortex@pcortex:hashdeep]$ ./configure –host=arm-marvell-linux-gnueabi
[pcortex@pcortex:hashdeep]$ make

And that’s it. If everything went well then at hashdeep/src directory the commands are available:

[pcortex@pcortex:src]$ file hashdeep
hashdeep: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 2.6.16, not stripped

We can now copy the commands to the DiskStation and start using them.

Now all I need is to devise a method that creates and checks each year photo hash/signatures, and warns me if a difference is detected. I’m thing on using the audit mode of the hashdeep command, and do each day a check for one year, for example, on Mondays check 2003 and 2013, on Tuesday check 2004 and 2014 and so on.

Two Factor Authentication for Synology and others – Alternative to mobile Apps

One way to secure the access to our Synology Diskstation Web Management interface and File Manager tool is to enable the two factor authentication (2FA). This means that we need to have something we know (the username and password) and something that we have (a mobile phone) to access these interfaces.

Check out the following Synology page: https://www.synology.com/en-us/knowledgebase/tutorials/615

For that we need to install Google Authenticator that is a mobile application so that we can get the time depended code (TOTP)  needed on the two factor authentication process.

This works fine, but what if I loose my mobile phone? And what if I’m too lazy to get up and get my mobile phone or tablet to get the TOTP to login?

In the first case, if you have e-mail notification correctly configured on your Synology DiskStation you can get an emergency code to login again. But if you haven’t, only by accessing through ssh/telnet you can recover the 2FA key to get again a valid TOTP).  The keys and available emergency codes are located at /usr/syno/etc/preference/admin/google_authenticator

For the second situation there is a solution (well two, but I’ll use the simplest one) to achieve this. What we need is to install on our PC, a trusted one, at least, an application that mimics the mobile Google Authenticator application. This application is GAuth:  https://5apps.com/gbraad/gauth We can installed as an add-on on our browser or launch directly from a web site or in my case from a local directory. For this I download it and added a shortcut to the index.html file. The application is available at https://github.com/gbraad/gauth an we can get a copy with the command git clone https://github.com/gbraad/gauth

Accessing the application, it will store the Secret Key into the Browser Local Storage. It’s not stored anywhere else so it is safe. Now we only need to get the key from the above Synology directory and we are all set. We can check if the generated GAuth code is the same as the code generated by the mobile device, and if yes, we have a backup!

Synology DS: Cross compiling Eclipse/IBM RSMB MQTT broker

RSMB: Really Small Message Broker is an MQTT broker/server, simpler than the alternatives like Mosquitto, RabiitMQ, and so on. While it has its origins in IBM labs (as the MQTT protocol), it is now a project/sub project on Eclipse Paho MQTT related software. While downloading the RSBM does provide some binaries for some common platforms, it doesn’t offer any binaries for, in my case, the DS212+ Marvell 88F628x ARM processor.

So let’s see how to cross compile the RSMB for Synology DS and in this process also learn how to cross compile in your desktop computer native software for the Diskstation.

Requirements:

Setting up the cross compiling environment:

First, a read of the following document The 3rd party developer guide from Synology located here https://www.synology.com/en-global/support/developer#tool  is recommended. Based on this document (page 6 and 7)and on this page http://forum.synology.com/wiki/index.php/What_kind_of_CPU_does_my_NAS_have, we can know what version of the Synology tool chain we need to download from here:  http://sourceforge.net/projects/dsgpl/files/DSM%205.0%20Beta%20Tool%20Chains/

Download the required tool chain for your Synology version. In my case I have the Synology DS212+ that has the Marvel 88F628x ARM processor, so download this file: http://sourceforge.net/projects/dsgpl/files/DSM%205.0%20Beta%20Tool%20Chains/Marvell%2088F628x%20Linux%202.6.32/

Uncompress the file into the /usr/local directory. DO USE this directory. The tool chain is configured to get all files, libraries and so on from the /usr/local/… directory:

sudo tar xvzf gcc464_glibc215_88f6281-GPL.tgz -C /usr/local/

(Note: It’s a CAPITAL C. Check Synology documentation).

We can now get the RSMB sources.

Cross compiling RSMB:

Open an shell terminal (preferably bash but other shells might work) and create and change to a working directory. Clone the RSMB repository located in http://git.eclipse.org/gitroot/mosquitto/org.eclipse.mosquitto.rsmb.git with git tools:

mkdir work_dir
cd workdir
git clone http://git.eclipse.org/gitroot/mosquitto/org.eclipse.mosquitto.rsmb.git
cd org.eclipse.mosquitto.rsmb/rsmb/src/

While, for this case, not all the below settings are needed, for documentation purposes I document them here:

export INSTALLDIR=/usr/local/arm-marvell-linux-gnueabi
export PATH=$INSTALLDIR/bin:$PATH
export TARGETMACH=arm-marvell-linux-gnueabi
export BUILDMACH=i686-pc-linux-gnu
export CROSS=arm-marvell-linux-gnueabi
export CC=${CROSS}-gcc
export LD=${CROSS}-ld
export AS=${CROSS}-as
export AR=${CROSS}-ar

Just make sure that the INSTALLDIR variable and TARGETMACH and CROSS variables point to the correct settings.

Also, in this case, for compiling RSMB, we need also to add the following variable:

export GCC=${CROSS}-gcc

Otherwise we need to change the Makefile and change the line GCC=gcc to point to the correct compiler. We can compile now:

make

And we should have the broker executable among others.

Let’s make sure that it is ok:

pcortex@dune:~/01.Develop/org.eclipse.mosquitto.rsmb/rsmb/src$ file broker
broker: ELF 32-bit LSB  executable, ARM, EABI5 version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, stripped

If the output is this:

broker: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=815abb3a1aad7f430c6e825670601c4991b45bd5, stripped

The wrong compiler was called.

Synology installation:

Copy the following files to your synology station: broker and Messages.1.3.0.2. From your workstation:

scp broker root@diskstation:/usr/local/bin
scp Messages.1.3.0.2 root@diskstation:/usr/local/bin

Access through ssh the Synology terminal, and make sure that broker is executable and do a test run:

cd /usr/local/bin
chmod +x broker
./broker
20150104 190523.162 CWNAN9999I Really Small Message Broker
20150104 190523.162 CWNAN9998I Part of Project Mosquitto in Eclipse
(http://projects.eclipse.org/projects/technology.mosquitto)
20150104 190523.163 CWNAN0053I Version 1.3.0.2, Jan  2 2015 20:13:39
20150104 190523.163 CWNAN0054I Features included: bridge
20150104 190523.163 CWNAN9993I Authors: Ian Craggs (icraggs@uk.ibm.com), Nicholas O'Leary
20150104 190523.163 CWNAN0014I MQTT protocol starting, listening on port 1883

And success! We can now test with MQTT-Spy (https://code.google.com/p/mqtt-spy/), Android Client, or Eclipse Paho tools.

Configuration and start and stopping:

For configuring the RSMB, we really should really read the documentation… 🙂 that is provided…

A simple configuration file should be located at /usr/local/etc and named rsmb.conf with the following basic contents:

# sample configuration on port 1883 with retained persistence in /tmp
port 1883
max_inflight_messages 50
max_queued_messages 200
persistence_location /tmp/
retained_persistence true

And at the /usr/local/etc/rc.d create a file named S99rsmb.sh with the following content:

#!/bin/sh

case $1 in
start)
    nohup /usr/local/bin/broker /usr/local/etc/rsmb.conf >> /var/log/rsmb.log&
    /usr/bin/logger -p1 -t "rsmb: INFO  " " Service started."
;;
stop)
    /usr/bin/killall broker
    /usr/bin/logger -p1 -t "rsmb: INFO  " " Service stopped."
;;
esac

Save and chmod +x S99rsmb.sh

Now the broker should start and stop automatically.

Final notes:

To use the broker from the Internet the port 1883/TCP should be open/forward at your router/firewall, and you should add authentication to the MQTT broker.

Rising from the ashes: NSLU2

Despite having a Synology Diskstation DS212+ for storing my data (photos, videos and PC/laptop backups), I also backup that data to an external disk connected to my faithful Linksys NSLU2 bought in 2005 using rsync from the Diskstation.

The NSLU2 is flashed ith the openSlug 5.3Beta firmware since 2009 (when it came out), with the operating system installed in a crappy 2GB SD card.

But this weekend due to a power failure, the NSLU2 failed to boot up. It kept the amber led blinking signalling that it couldn’t forward from the initial stages of booting up.

Using my desktop computer, I’ve FSCK’ed the external disk filesystem, that had some inconsistencies, nothing serious (most of the time it is dormant), and FSCK’ed the SD card, and, well, most of the /etc and /var directory where gone.

Due to having a backup of the SD card (these things die…), I’ve recovered the /etc directory, but still the NSLU2 didn’t boot.

Booting up without SD card, the NSLU2 did finish booting up, but it wouldn’t ping, neither the original IP address (192.168.1.77) neither the configured IP address. All I had on my Linux machine was incomplete at the arp table…

nslu2 (191.168.1.32) at <incomplete> [ether] on enp4s0f2

Not good….

I’ve flashed it again with the openSlug firmare, but still I was unable to ssh to it so I could initialize. Because I was able to flash it again with the upslug2 tool, it mean that the ethernet port was ok, and probably everything was ok, except the NVRAM settings that define the ip address where pretty much corrupted… Let’s hope that.

So the solution was to boot into RedBoot and erase the NVRAM (http://www.nslu2-linux.org/wiki/HowTo/ResetSysConf) with the following command: fis erase -f 0x50040000 -l 0x20000  (Attention to this command!!!! Don’t get it wrong)

And then upgrade from the RedBoot interface. The original Linksys firmware was flashed and after rebooting this firmware initialized the NVRAM with default settings: IP address 192.168.1.77, and bingo, ping works, and I can access the original Linksys Web Interface. On the web interface I’ve configured the old IP address, DNS, host name, and so on, and rebooted.

After reseting the NVRAM from redboot you must install the original Linksys firmware, because the openSlug doesn’t initialize the NVRAM.

Everything was fine, and the NSLU2 was working on the new IP. From this point on I’ve just flashed again the openSlug firmware, and formatted the SD Card (turnup with the memstick otion), and configured everything again (crontab, ntpclient and rsync daemon).

In no time I had the Diskstation again backing up to the NSLU2 external disk.

So, welcome again NSLU2 🙂

Synology and MyDS Quickconnect

The issue: Quickconnect doesn’t work

After upgrading to the latest DSM version 5.0, it took a while to notice that my quickconnect id that I had chosen was not working…

On the DSM Control Panel, if I tried to change and apply the settings it gave a Unknow Error. On the logs, the only message related to the Quickconnect settings was network error: -23, and that was it…

On the myds site, my DS status was red, and clicking on the Quickconnect ID just gave a page that said that my DS was offline or with no network connection, but clicking on the host name just worked fine.

Using the Apps with the hostname and/or IP worked fine, just not with the Quickconnect ID.

The solution:

I don’t have a solution that might work for everyone, but the steps that I’ve taken solved the issue for me.

First on MyDS site I deleted the hostname, and on the DS Control Panel on DDNS settings I tried to register it again. This failed as said that the hostname doesn’t exist…

So, I also deleted the entry for the DDNS Synology provider and configured it again. I needed to enter again my login credentials to the MyDS site, and my hostname again.

This time, it worked, and on the MyDS page the hostname (after I deleted it from there) appeared again. Still clicking on the Quickconnect Id failed.

So, again on the DS Control Panel I went to the QuickConnect on Control Panel, and this time it said that I need to register a QuickConnect ID, so, I registered again my ID, providing the MyDS site credentials, and ID. And it worked.

Now my Quickconnect ID works and DDNS name also works.

The status of my DS on the MyDS site remained red for a large period of time, but at the end it turned green. Also clicking on the Quickconnect ID now works and gives me access to the Web frontend of DS.

This was quite a suprise for me as I didn’t expected to have the Web Administration console available to the internet.

I’ll have to see how to block this.

tl;dr:

Delete your DDNS configuration and register it again. Register again the QuickConnect ID.

Synology DSM 5 web station and virtual sites for FileStation

Edit: For DSM 6, follow this: Reverse Proxy with DSM 6.0

I’ve upgraded my Synology DS 212+ to the latest DSM version, version 5, a few weeks ago.

Many things have changed on this new DSM version, and one of the things that changed, was the way Web station and Apache/PHP works.

The configuration files are different from the previous version 4 and they use now upstart to start and stop the service:

To stop the Web Station: /sbin/initctl stop httpd-user

To start the Web Station: /sbin/initctl start httpd-user

There is also a status command, to show the state of the server: httpd-user stop/waiting (Stopped) or httpd-user start/running, process #### (running)

For FileStation and Audio station, these application/sites normally expect the browser to connect to application ports, namely in my case the port 7000 and 7002. This has potentially two problems: I need to open them up on my home router to allow access, and sometimes I’m behind a proxy that doesn’t allow anything else to be used as a port on the url (except for port 80-HTTP or 443-HTTPS).

So I have these applications behind a reverse proxy on the WebStation. With this configuration I have my “normal” http address: http://primalcortex.somedomain.com and I also have http://filestation.primalcortex.somedomain.com and http://audio.primalcortex.somedomanin.com.

What is needed to achieve this? Well first a a dns server that accepts wirldcard domains. The Myds.me is one of them and is provided by Synology and directly integrated into the DSM.

Second a file like this is needed to be created:

<IfModule !proxy_module>
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_http_module modules/mod_proxy_http.so
</IfModule>

ProxyRequests Off
 #ProxyPreserveHost On

NameVirtualHost *:80

#For File Station
 <VirtualHost *:80>
 ServerName filestation.mydomainname.myds.me
 <Location />
 RedirectPermanent / https://filestation.mydomainname.myds.me/
 </Location>
 </VirtualHost>

NameVirtualHost *:443

<VirtualHost *:443>
ServerName filestation.mydomainname.myds.me
SSLCipherSuite HIGH:MEDIUM
SSLProtocol all -SSLv2
SSLCertificateFile /usr/syno/etc/ssl/ssl.crt/server.crt
SSLCertificateKeyFile /usr/syno/etc/ssl/ssl.key/server.key
SSLEngine on
SSLProxyEngine on
ProxyRequests Off
ProxyVia Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / http://localhost:7000/
ProxyPassReverse / http://localhost:7000/

</VirtualHost>

(In case something is missing above due to WordPress formating issues see this ->
http://pastebin.com/C7WF8kTN)

Save this file as httpd-FILESTATION-vh.conf-user at /usr/syno/etc

Then add the following line at the end of this file /etc/httpd/conf/httpd.conf-user

Include /usr/syno/etc/httpd-FILESTATION-vh.conf-user

Repeat for other sites like Audio Station, changing the hostname and the localhost port.

You can now run the following command: /usr/syno/etc/rc.sysv/httpd-user-conf-writer.sh and check if the above include is added to the /etc/httpd/conf/httpd.conf file. This file is always regenerated at start of the Web Station.
Now we can restart the web station with /sbin/initctl stop httpd-user followed by the start command.

Check now if you can access the url: https://filestation.mydomain.myds.com

Edit: This works for version 5.0-4482

For more recent versions you need also to do the following:

Edit file /etc/httpd/conf/extra/httpd-ssl.conf-user and comment out the ServerName and ServerAlias like this

#ServerName *
#ServerAlias *

Save the files, write again the configuration (/usr/syno/etc/rc.sysv/httpd-user-conf-writer.sh), stop ( /sbin/initctl stop httpd-user) and start again ( /sbin/initctl start httpd-user), and it should work now.

Thanks for Markus (below at the comments for the solution) and Tensai for corrections.

EDIT: we can make all this setup to be done automatically, even between DSM upgrades. Thanks to Michi (see below) for his script: http://pastebin.com/raw.php?i=rjrU6X8K.

Just copy the content from the above link, and paste it into a file named, for example S99subdomains.sh in directory /usr/local/etc/rc.d

Make sure that the file is executable with the following command: chmod +x S99subdomains.sh and just run it: S99subdomains.sh start.

That’s it. Also just make sure that on DSM web interface the applications on Control Panel -> Application Portal have HTTP ports assigned that are coincident with those defined in the above file. Otherwise an Internal error 500 appear when trying to access the web page because Apache will try to forward the request to a non listening port.