NodeMcu and I2C: How to start and using the PCF8574A I2C port expander on ESP-01 board

Many of us bought a few ESP-01 boards when they came out, before other boards, with a lot more general purpose input/output pins (GPIO) exposed, became available, like the ESP-12.

The problem with the ESP-01 board is that it only has two GPIO pins available for doing something: GPIO0 and GPIO2. If we want more pins we need to buy another ESP board with more exposed pins, or use an I2C port expander. This is possible because the ESP8266 can create and use an I2C bus on those two pins, using either native programming or the NodeMcu firmware and the Lua language. On this post I’m using the NodeMcu and Lua language.

Setting up the hardware:

I’m using the PCF8574AP chips that I’ve bought on ebay (Original, fakes, who knows, they work..). The data sheet is here: PCF8574 datasheet. The important parts of this document are the I2C addresses that we need to used, and the way the quasi-bidirectional 8574 pins are used. The chip is quite simple to use because it only has one register to control the expander pins. We will get to that. Also these chips work fine with 3.3V and/or 5V, so we can connect them directly to the ESP8266 board pins.

On the ESP-01 board we need to connect the GPIO pins to the I2C bus pins of the PCF8574 chip. In my case I’ve defined the following:

  • SDA -> GPIO2 to PCF8574A pin 14
  • SCL -> GPIO0 to PCF8574A pin 13

We also need to add pull up resistors from these pins to VCC. I’m using 4K7 resistor. Regarding the address pins (PCF8574A pins 1,2,3), I’ve grounded them all, so the address is the base address for the chip. In my case the chip is the PCF8574A variant which means that the base address is 0x38. If I pull up A0 to VCC, then the address becomes 0x39 for example.

Regarding the address it’s important to note that NodeMcu uses 7 bit addresses and the I2C functions themselves set the LSB (least significant bit) for reading or writing data. So the I2 addresses to use on the I2C functions are the 7 MSB as the addresses as we can see on the PCF8574 datasheet.

The final schema is the following:

PCF8574 Schematics

PCF8574 Schematics

After powering up, all PCF8574 pins should be at level high (VCC).

Checking the hardware:

There is an I2C scanner available on the web that allows to check if there are I2C devices present on the I2C bus. For example on this thread there are some Lua script examples: http://www.esp8266.com/viewtopic.php?f=19&t=1049

I’ve used this variation of the script: i2_scan.lua

-- Based on work by sancho and zeroday among many other open source authors
-- This code is public domain, attribution to gareth@l0l.org.uk appreciated.

id=0  -- need this to identify (software) IC2 bus?
gpio_pin= {4,3,0,1,2} -- this array maps internal IO references to GPIO numbers

-- user defined function: see if device responds with ACK to i2c start
function find_dev(i2c_id, dev_addr)
     i2c.start(i2c_id)
     c=i2c.address(i2c_id, dev_addr ,i2c.TRANSMITTER)
     i2c.stop(i2c_id)
     return c
end

print("Scanning all pins for I2C Bus device")
for scl=1,7 do
     for sda=1,7 do
          tmr.wdclr() -- call this to pat the (watch)dog!
          if sda~=scl then -- if the pins are the same then skip this round
               i2c.setup(id,sda,scl,i2c.SLOW) -- initialize i2c with our id and current pins in slow mode :-)
               for i=0,127 do -- TODO - skip invalid addresses 
                    if find_dev(id, i)==true then
                    print("Device is wired: SDA to GPIO - IO index "..sda)
                    print("Device is wired: SCL to GPIO - IO index "..scl)
                    print("Device found at address 0x"..string.format("%02X",i))
                    end
               end
          end
     end
end

And the output is:

> dofile('i2c_scan.lua')
Scanning all pins for I2C Bus device
Device is wired: SDA to GPIO - IO index 4
Device is wired: SCL to GPIO - IO index 3
Device found at address 0x38

Nodemcu Lua I2C functions:

To use Nodemcu Lua I2C functions we need the following functions:

  • i2.setup ( bus_id , SDA_pin , SCL_pin , I2C_bus_speed) – This function should be called before any other I2C activity. The bus_id parameter is on this case always zero, and the SDA and SCL pins the Nodemcu GPIO pind definition. Check out the Github NodeMcu page for the mappings. In our case the GPIO2 is index 4 and GPIO0 is index 3. The last parameter is the I2C bus speed that always should be i2c.SLOW.

After initializing the i2c functions, we can now read or write to the device. The pattern for this is always the same:

  • i2c.start( bus_id )  -> Start the i2c transaction
  • i2c.address( bus_id , device_address, operation) -> For our case and for the PCF8574A the device_address is 0x38, and the operation is i2c.RECEIVER for reading data, or i2c.TRANSMITTER for writing data. Based on this parameter the LSB bit is added to the device address to define the operation, 1 for read, zero for writing and thus build the complete 8 bit address. This function also returns values: It returns true if the device acknowledges the operation.
  • i2c.read(bus_id, size) -> For the i2c.RECEIVER operation it reads size bytes into a string from the i2c bus. That is the value that is returned from the function.
  • i2c.write(bus_id, data) -> For the i2c.TRANSMITTER operation it writes the data on the i2c bus. The value returned is the number of bytes that where wrote.
  • i2c.stop(bus_id) -> Stops the i2c transaction.

For more complete information check the Nodemcu LUA api pages, for example: Nodemcu LUA api

Using the PCF8574

Now it is simple to use the I2C PCF8574 port expander from the ESP8266. We know the address and sine it only has one register it is easy to read and write into the chip. One interesting thing if something is written on the pins, the same value can be read if nothing is connected to it. Caution for not short circuiting… Please read the data sheet if wanting to use some pins as inputs.

Anyway, the simple code for blinking a led is the following: pcf8574_blinkled.lua

busid = 0  -- I2C Bus ID. Always zero
sda= 4     -- GPIO2 pin mapping is 4
scl= 3     -- GPIO0 pin mapping is 3

addr=0x38  -- the I2C address of our device
led = 0;  
-- Seting up the I2C bus.
i2c.setup(busid,sda,scl,i2c.SLOW)

-- Read from the pcf8574
function read_pcf8574(dev_addr)
     i2c.start(busid)
     i2c.address(busid, dev_addr , i2c.RECEIVER)
     bdata = i2c.read(busid,1)  -- Reads one byte
     i2c.stop(busid)
     return bdata
end

-- Writes to the pcf8574
function write_pcf8574(dev_addr, value)
     i2c.start(busid)
     i2c.address(busid, dev_addr, i2c.TRANSMITTER)
     i2c.write(busid,value)
     i2c.stop(busid)
end

-- timer controller blinking led on pin 0.
function blink_led()
    write_pcf8574( addr,  led )
    if led == 0 then
      led = 0x01
    else
      led = 0
    end
    --print("Blink Led...")
end

-- Main program.
i2c.setup(busid,sda,scl,i2c.SLOW)

for i=1, 254 do 
    write_pcf8574( addr,  i )
    tmr.delay(20000)
    result=read_pcf8574(addr)
    print(string.byte(result))
    tmr.wdclr()
end

tmr.alarm(2, 1000, 1, blink_led )

An that’s it. Eight new I/O pins expandable to much more on the ESP8266.

As we can see the read and write functions are quite simple since the PCF8574 chip only has one register. If we have a more complex device like an LCD or sensor, we might need first to write some command an the read the data. For example something like this:

function read_i2cdata(dev_addr, command, numbytes)
     i2c.start(busid)
     i2c.address(busid, dev_addr , i2c.TRANSMITTER)
     i2c.write(busid,command)  -- Sends the command
     i2c.stop(busid)

     i2c.start(busid)
     i2c.address(busid, dev_addr, i2c.RECEIVER)
     data = i2c.read(busid, numbytes)
     i2c.stop(busid)
     return data
end

Since there are so many I2C devices available, using I2C is really the only way of getting the most out of the ESP-01 boards.

ESP8266 NodeMCU firmware updating

My ESP-01 board has not being used for a few weeks due to other projects, and today, after updating my local nodemcu firmware repository I’ve found out that the latest build directory where the binary for the firmware was located was gone.
No problem.
We can find now the pre-compiled binaries on this link: NodeMcu Firmware Releases

There is also a cloud compiling option on this link: NodeMcu custom builds
If by any chance you are having second thoughts in providing your e-mail, use one from Fake Mails and wait for the firmware on the provided (temporary) e-mail.

Anyway, the procedure is always the same:

  • Close anything that using the serial port, for example the ESPlorer tool
  • Put the ESP8266 into firmware flashing mode by connecting the GPIO0 to ground and reset the board or power cycling.
  • Flash the firmware with ./esptool.py -p /dev/ttyUSB0 write_flash 0x000000 nodemcu_float_0.9.6-dev_20150704.bin
  • Remove the GPIO0 from GND and power cycle or reset
  • Connect to serial port and check the NodeMCU version

Update done!

Linux: Slow or unbearable performance with high memory consumption applications

Well this post title is a bit vague because the issue that have affected me can have several sources, not just the one that I’ll describe. But I suppose the solution is the same independently for all sources.

Anyway, I was suffering from a strange problem on both my computers, the desktop has 6GB of RAM, and the laptop, 16GB.

On the desktop where I’m using KDE 4 on Arch Linux, the desktop just froze when using some resource intensive applications like Google Maps on Firefox or Chromium and Android Studio. When the frozen desktop happened, I just had to wait a few minutes, during which time, keyboard was irresponsible and the mouse was “jumpy”, or did not worked at all. And so after a while, one random application, that could be anything, including the desktop, was killed, with some cryptic message regarding sacrifice on the system logs…

Anyway, on the Desktop, due to the fact that I’ve upgraded the main disk to an SSD, I didn’t enabled the swap partition. That was one of the reasons of the behaviour that I was having. Despite having 6GB of RAM, a swap file is good to have, and so I’ve used a swap partition on one of the spinning disks.

That apparently stopped the issues on the desktop and it never froze like it used to when using the same tasks.

On the laptop it was another matter, due to the fact that it only has a 120GB SSD disk, and no spinning disks, I had no swap file/partition created or enabled. I though that with 16GB of RAM, why bother… but due to professional related activities, I had to start to work with the HortonWorks HDP 2.3 big data/hadoop platform, and that is a heavy resource hog…

So, despite of having 16GB of RAM available and virtual machine based on Vmware, the computer just froze for lenghty periods of time, just like my desktop computer.

But, now I know that was related to fact that I didn’t have a swap partition/file, and so I’ve created one.

The swap file solution for this specific case didn’t solve completely my issue, but I did managed to see a process named khugepaged consuming huge amounts of CPU.

And that was the reason that I had for my frozen desktop and virtual machine for some lengths of time.

So I’ve disable the Huge Page support on my host operating system, AND on my guest CentOS virtual machine that is running HDP 2.3:

[root@dune:cortex]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
[root@dune:cortex]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

[root@dune:cortex]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@dune:cortex]# cat /sys/kernel/mm/transparent_hugepage/enabled          
always madvise [never]

More info can be obtain here:

http://docs.mongodb.org/manual/tutorial/transparent-huge-pages/

So this might not be the magic bullet if we are having desktop freezes, and not using huge amounts of RAM, but it might help.

Mosquitto Broker with Websockets enabled on the Synology NAS

Synology NAS are great devices, mainly due to the bundled software (CloudStation, Photo Station, to name a few), that also can run other software that is provided on non official package sources.
One of such providers is the SynoCommunity that provides a Mosquitto MQTT broker packaged for the Synology NAS. While I’ve successfully was able to cross compile the Mosquitto broker for my DS212+ NAS with websockets enabled (I do have a draft post with the steps that I’ve never published), the SynoCommunity package allows a simpler way to install and use a Mosquitto broker that has the WebSockets protocol support compiled in.

So, what is needed to be done?

Setting up:

  • 1st: Add to your Synology server the SynoCommunity package source. See the detailed instructions on the community web page.
  • 2nd: Install the Mosquitto package from the Synology package manager.
  • 3rd: Edit the file located at /usr/local/mosquitto/var/mosquitto.conf. You need to connect through ssh as root to do this.

At the end of this file add the following lines:

Listen 1883
Listen 9001
Protocol websockets

Note that the 1883 is the standard MQTT broker port, and 9001 is the port that I’ve choose have the websockets server listening/answering.

After the change, stop and start the Mosquitto broker on the package manager action dropdown box for the Mosquitto package.

Still using ssh, and logged in as root, check now that the port 9001 is active:

root@DiskStation:/volume1/homes/root# netstat -na | grep 9001 
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN       
tcp        0      0 192.168.1.250:9001      192.168.1.32:55331      ESTABLISHED

In the above case we can see that there is a websocket client connected to the broker from my workstation.

Testing:

For testing we can use the HiveMQ websocket client: http://www.hivemq.com/demos/websocket-client/ that allows to communicate using the websockets transport.

Despite the fact that the page is available on an external, internet based server, the websocket client will be running on YOUR web browser, on your machine in your own (internal) network, so at the connection settings for the broker, for Host I use my internal IP address: 192.168.1.250, and for the Port: I use the 9001 value.

I can now subscribe and publish to topics using the browser, and also using MQTT-SPY we can see and check the MQTT Websockets communication.

And that’s it!

ESP8266 – Updating the SDK to the latest version

So version 1.0 of the ESP8266 SDK as been released: https://espressif.com/new-sdk-release-2/.
This means that I need to update my cross compiling tools for the ESP8266 to the latest version.
I’ve installed and configured my environment according to this post: https://primalcortex.wordpress.com/2015/01/09/esp8266-setting-up-native-sdks-on-linux-and-exploring-some-applications/ and this post documents how I’ve updated my environment.

Updating the crosstools
This is quite easy, just go to the crosstool-NG directory and execute git pull. This will bring into your machine the latest release of the cross compiling tools.

Downloading and installing the Espressif ESP8266 SDK
At the date of this post I’ve downloaded this SDK version: http://bbs.espressif.com/viewtopic.php?f=5&t=321 This post refers to version 1.0.1_b1_15_04_02. At the bottom of this post there is a link for downloading the SDK. Just download the SDK: http://bbs.espressif.com/download/file.php?id=276

On my configuration, my SDK is on the directory ESP8266_SDK at /opt/Espressif. So I just make a backup of this SDK version and expand the new one:

cd /opt/Espressif
mv ESP8266_SDK ESP8266_SDK_old
unzip ~/Downloads/esp_iot_sdk_v1.0.1_b1_15_04_02.zip
mv esp_iot_sdk_v1.0.1_b1/ ESP8266_SDK
mv License ESP8266_SDK

We need finally to install some libraries:

cd /opt/Espressif/ESP8266_SDK
wget -O lib/libc.a https://github.com/esp8266/esp8266-wiki/raw/master/libs/libc.a
wget -O lib/libhal.a https://github.com/esp8266/esp8266-wiki/raw/master/libs/libhal.a
wget -O include.tgz https://github.com/esp8266/esp8266-wiki/raw/master/include.tgz
tar -xvzf include.tgz

And that’s it.

Compiling application and demos.
For compiling the IoT_Demo sample:

cd /opt/Espressif/ESP8266_SDK
mv example/IoT_Demo .
sed -i -e 's/xt-ar/xtensa-lx106-elf-ar/' -e 's/xt-xcc/xtensa-lx106-elf-gcc/' -e 's/xt-objcopy/xtensa-lx106-elf-objcopy/' -e 's/xt-objdump/xtensa-lx106-elf-objdump/' -e 's/xt-nm/xtensa-lx106-elf-nm/' Makefile
export COMPILE=GCC
export PATH=/opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf/bin/:$PATH

But before running make, check what version of python is the default:

python -V

If it is version 3, then the python scripts run only on python 2, and we need a workaround:


cd /opt/Espressif/ESP8266_SDK/bin
ln -s /usr/bin/python2 python
export PATH=/opt/Espressif/ESP8266_SDK/bin:/opt/Espressif/crosstool-NG/builds/xtensa-lx106-elf/bin/:$PATH

Running now the python -V command should report version 2.7.

We can now run make:

...
...
...
make[2]: Entering directory '/opt/Espressif/ESP8266_SDK/IoT_Demo/driver'
make[2]: Leaving directory '/opt/Espressif/ESP8266_SDK/IoT_Demo/driver'

!!!
No boot needed.
Generate eagle.flash.bin and eagle.irom0text.bin successfully in folder bin.
eagle.flash.bin-------->0x00000
eagle.irom0text.bin---->0x40000
!!!
make[1]: Leaving directory '/opt/Espressif/ESP8266_SDK/IoT_Demo'

Success. We can now flash the IoTDemo.

md5deep and hashdeep on Synology DS212+

My personal photos are located at my desktop computer, and I have backups of them on my Synology DS212+ and my old faithful Linksys NSLU2 with an external disk.

The issue that I have now is that to keep backup times short I only backup the current year, because, well, the other years are static data. Previous years are already backed up and that data is not to get modified. Right? So what if corruption happens? How do I detected it? How can I be warned of such event? Right now I have no method to check if my digital photos of 2003 are totally intact in all of my three copies. And if  I detect corruption on one bad file/photo, which file is the correct one?

So I’m investigating this issue, and one of the tools available to create checksums and then to verify if everything is ok and no corruption has appeared is the md5deep and hashdeep programs. These are available as sources at this link: http://md5deep.sourceforge.net/start-hashdeep.html

These are the instructions for cross compiling these tools for the ARM based Synology DS212+. As usual this is done on a Linux Desktop/server machine.

1st) Set up the cross-compiling environment: Check out this link on the topic: https://primalcortex.wordpress.com/2015/01/04/synology-ds-crosscompiling-eclipseibm-rsmb-mqtt-broker/

2nd) Setting up the cross compiling environment variables is now a bit different:

export INSTALLDIR=/usr/local/arm-marvell-linux-gnueabi
export PATH=$INSTALLDIR/bin:$PATH
export TARGETMACH=arm-marvell-linux-gnueabi
export BUILDMACH=i686-pc-linux-gnu
export CROSS=arm-marvell-linux-gnueabi
export CC=${CROSS}-g++
export LD=${CROSS}-ld
export AS=${CROSS}-as
export AR=${CROSS}-ar
export GCC=${CROSS}-g++
export CXX=${CROSS}-g++

We will use the C++ compiler.

3rd) Create a working directory and clone the repository to your local machine: git clone https://github.com/jessek/hashdeep.git
4th) Change to the hashdeep directory and execute the following commands:

[pcortex@pcortex:hashdeep]$ sh bootstrap.sh
[pcortex@pcortex:hashdeep]$ ./configure –host=arm-marvell-linux-gnueabi
[pcortex@pcortex:hashdeep]$ make

And that’s it. If everything went well then at hashdeep/src directory the commands are available:

[pcortex@pcortex:src]$ file hashdeep
hashdeep: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 2.6.16, not stripped

We can now copy the commands to the DiskStation and start using them.

Now all I need is to devise a method that creates and checks each year photo hash/signatures, and warns me if a difference is detected. I’m thing on using the audit mode of the hashdeep command, and do each day a check for one year, for example, on Mondays check 2003 and 2013, on Tuesday check 2004 and 2014 and so on.