MQTT Publishing on the ESP8266

I’ve done some tests with esp8266 and NodeMcu Lua based firmware, and I’ve come to the following conclusions:

  • We can only publish again in a MQTT topic if, and only if the previous topic call has finish. That’s why we need to use a semaphore to stop multiple publish request at the same time.
  • MQTT based code consumes a lot of heap. So much, that to make it work I have to strip empty lines and comments from the code. It seems that the latest NodeMcu firmware does have a node.compile() function that we can use.
  • Using unbound arrays for storing MQTT messages while waiting for their turn to be publish, will end up using all available heap space, and the esp8266 will reboot, because setting previous array entries to nil won’t free the used space at all.

So I’ve now a sample code that uses a circular buffer to publish messages. The code below has some flaws, namely:

  • Doesn’t check if the buffer is full, it will just overwrite any older unpublished messages.
  • Entries on the buffer have no QOS defined.
  • Has no “intelligence” for example to only override messages with QOS=0 that might get lost anyway, and keep QOS>0 messages. That would mean that eventually the buffer might get full, and all publishing from the user code can fail.

Here is the code:


-- Configuration to connect to the MQTT broker.
BROKER = ""   -- Ip/hostname of MQTT broker
BRPORT = 1883             -- MQTT broker port
BRUSER = "user"           -- If MQTT authenitcation is used then define the user
BRPWD  = "pwd"            -- The above user password
CLIENTID = "ESP8266-" ..  node.chipid() -- The MQTT ID. Change to something you like

-- MQTT topics to subscribe
topics = {"topic1","topic2","topic3","topic4"} 

-- Control variables.
pub_sem = 0         
current_topic  = 1  
topicsub_delay = 50

-- Publishing structures
pub_topic = {}
pub_message = {}
pub_head = 1
pub_tail = 1

print("heap: " .. node.heap() )
m = mqtt.Client( CLIENTID, 120, BRUSER, BRPWD)

print "Connecting to MQTT broker. Please wait..."
print("heap: " .. node.heap() )
m:connect( BROKER , BRPORT, 0, function(conn)
     print("Connected to MQTT:" .. BROKER .. ":" .. BRPORT .." as " .. CLIENTID )

function mqtt_sub()
     if table.getn(topics) < current_topic then
          m:subscribe(topics[current_topic] , 0, function(conn)          
          current_topic = current_topic + 1  
          tmr.alarm(5, topicsub_delay, 0, mqtt_sub )

function _publish_data()
   if pub_head ~= pub_tail then
     if pub_sem == 0 then
       pub_sem = 1
       m:publish( pub_topic[pub_head], pub_message[pub_head] ,0 ,0 , function(conn) 
          print("Sending to " .. pub_topic[pub_head] ..": " .. pub_message[pub_head])
          pub_sem = 0
          pub_topic[pub_head] = nil
          pub_message[pub_head] = nil
          pub_head = (pub_head + 1) % BFSIZE

function publish_data(topic , message )
  pub_topic[pub_tail] = topic
  pub_message[pub_tail] = message
  pub_tail = (pub_tail + 1) % BFSIZE

function publish_data1()
 publish_data("Out1","Mensagem para out1" )
 publish_data("Out2","Mensagem para out2" )

function run_main_prog()
     tmr.alarm(2, 5000, 1, publish_data1 )
     m:on("message", function(conn, topic, data)
        print(topic .. ":" )
        if (data ~= nil ) then
          print ( data )
      end )

Apparently this code can run for a long time.

Also at the beginning I print the heap space before running and connecting to a MQTT broker.

  • So after a reset: node.heap = 19600 ( with init.lua file)
  • Running the above file ( dofile(“mqtt_cbuf.lua”)): node.heap = 10560 after compiling and before connecting to the MQTT broker
  • After connecting: node.heap = 8136

If we use the new node.compile function with the function node.compile(“mqtt_cbuf.lua”) we should have now a new file named So:


  • So after a reset: node.heap = 19472 ( with init.lua file)
  • Running the above file ( dofile(“”)): node.heap = 12160 after compiling and before connecting to the MQTT broker
  • After connecting: node.heap = 9720

A bit better. Not much, but makes a difference.

Since a small code as the above can fail to run due to lack of heap space, solving some of the issues that I’ve referred on the beginning of this post might only be feasible if really needed. Let’s see if I can improve this.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.