ESP8266 NodeMcu and Lua language and some Arduino issues.

When using the ESP8266 chip with the original firmware, AT based command set, to perform operations, implies that something external, like an Arduino or RPi (Raspberry Pi), must be used to drive/command the chip through the serial port. This external device sends out AT commands and waits for responses.

While for some operations/applications, using the AT commands  might just work when using the Software Serial library from Arduino. In another words, using a software based serial port to communicate might work. The issue is that even at a slow rate of 9600 baud’s, Arduino has some trouble to follow the esp8266 output, due to the lack of flow control.

For example listing the available access points (AT+CWLAP) or even receiving simple web page from the internet, most of the time the Software Serial port only receives incomplete messages or garbled information. This can be worked around by increasing the software serial port buffer size, but is not a solid solution.

In fact without using the hardware port on the Arduino to connect to the ESP8266, I have doubts that some useful work can be done if we simultaneous also want to use the hardware port for debug ( connecting to the computer) and the software serial port to connect to the esp8266. We must use the hardware port for connecting to the esp8266 and use the software serial library for debugging purposes which brings other issues. This situation only arises on the Arduino Uno or Mini where only one hardware based serial port is available. The Arduino Mega has 4 hardware serial ports, so it doesn’t have this issue.

So why all about this ranting regarding Arduino, serial ports and the esp8266?

The fact is that the esp8266 is by itself a microprocessor and so it is able to do some processing on it’s own. So it’s is possible to only send or receive the already processed information needed from/for Arduino, without the Arduino being the processor that is controlling and defining the behaviour of the esp8266 chip like connecting to WIFI and so on. The AT command set is fine for some applications, but is so 80’s … 🙂

One of the solutions available is to replace the original based AT command set firmware for the ESP8266 with the NodeMcu firmware, that’s now open sourced, that allows to add some intelligence to the esp8266 chip, namely connecting, receiving, sending and waiting for data (server mode) using the Lua language running on the ESP8266 chip itself and not on an external device.

After flashing and restarting the esp8266 with the NodeMcu firmware, the esp8266 tries to load a file named init.lua.

Several words of warning regarding  when developing with this firmware:

– If init.lua file crashes, the NodeMcu restarts, and calls again init.lua that crashes again, that restarts NodeMcu and so on. You get the idea.  At initial stages do not develop using the init.lua file. The only way to recover from this crash loop is to flash the firmware again. Do your first developing in a file named testing.lua or init2.lua for example. When ready, just rename it to init.lua.

– NoceMcu is single threaded and the esp8266 has a WatchDog timer. Tight loops will trigger the watchdog timer and restart the firmware. If needed we can we use the tmr.wdclr() which resets the watchdog timer, but then other bad things happen due to the single threaded nature of the firmware, like losing wifi connection, timer triggers not activating, and so on.

A great way to develop is to use the ESPlorer IDE tool. It needs Java JDK 1.8 (Edit: It seems not anymore. JDK 7 might be fine), so make sure that version is installed in your Operation System. Then we can just do /opt/jre1.8/bin/java  -jar ESPlorer.jar to launch the IDE.

Let’s see how in Lua we can connect to WIFI.

The simplest code for init.lua is

-- This is a comment line
  print("Configuring WIFI....")   -- Print something to the serial port if connected
  wifi.setmode( wifi.STATION )    -- Client/Station mode. Other mode could be Access Point
  wifi.sta.config( SSID , APPWD)  -- Connect to SSID with password APPWD.
  -- Example: wifi.sta.config( "MYWNET" , "QuantumEncryptedPassword" )
  print( "Ip Address: " .. wifi.sta.getip() ) -- The .. is the string concatenate string operator. 
 dofile ( "RunMyCode.lua")

This code snippet will connect to WIFI, if successful print the IP, and then lauch the RunMyCode.lua file.

But what happens if the connection fails, or takes sometime? We may end lauching the RunMyCode.lua file without WIFI connectivity.

So I’ve lost a couple of hours, learning Lua and esp8266 quirks to develop my init.lua file, with a lot of help of the esp8266 forum where there are a lot of Lua code snippets and examples.

Due to the single threaded nature of the NodeMcu firmware is not possible to do this (a tight loop waiting for the wifi to connect):

isConnected = false
   -- Lets see if we are already connected by getting the IP
   ipAddr = wifi.sta.getip()
   if ( ( ipAddr ~= nil ) and  ( ipAddr ~= "" ) )then  -- Check if IP is valid
      isConnected = true

until ( isConnected == true )

This will hang the esp8266 and will kick the watch dog timer, restarting the firmware.  Also the only thread running is the the thread executing this repeat/until loop, and so this code snippet will never connect because the background functions that need to be run, so that the connection does happen, never have a chance to run and the isConnected will always be false. The fact is that after the firmware restart, on the lua prompt running the print ( wifi.sta.getip() ) shows that it connected… So the loop is in fact blocking everything else.

Another example:

isConnected = false
   -- Lets see if we are already connected by getting the IP
   ipAddr = wifi.sta.getip()
   if ( ( ipAddr ~= nil ) and  ( ipAddr ~= "" ) )then  -- Check if IP is valid
      isConnected = true
   tmr.wdclr()    -- Reset the watch dog timer.
until ( isConnected == true )

This is even worse because this code segment explicitly resets the watchdog, and so we might thing that the loop will succeed some time in the future, but it won’t. The only way to leave this is to reset by hardware. It seems that the only way that background functions can work, if we can say that, is to let the script end…

So how do we test the connection and we need to end the script? Well timers are the solution:

tmr.alarm ( Alarm_number , Number of mS , Repeat , function )

So this:  tmr.alarm ( 0 , 2500 , 1 , checkWifi )  will use timer #0 to call periodically each 2.5S the checkWifi function (NOTE the lack of () on the checkWifi parameter (checkWifi is a function)).  The Repeat parameter if zero is a one shot call, otherwise it will reset the alarm and trigger it again and again.

So we let the script end and periodically check if we have Wifi. If we have connectivity then we can lauch our code. And so this is my init.lua code:

-- Constants
APPWD   = "QuantumPassword"
CMDFILE = "ping.lua"   -- File that is executed after connection

-- Some control variables
wifiTrys     = 0      -- Counter of trys to connect to wifi
NUMWIFITRYS  = 200    -- Maximum number of WIFI Testings while waiting for connection

-- Change the code of this function that it calls your code.
function launch()
  print("Connected to WIFI!")
  print("IP Address: " .. wifi.sta.getip())
  -- Call our command file every minute.
  tmr.alarm(0, 60000, 1, function() dofile(CMDFILE) end )

function checkWIFI() 
  if ( wifiTrys > NUMWIFITRYS ) then
    print("Sorry. Not able to connect")
    ipAddr = wifi.sta.getip()
    if ( ( ipAddr ~= nil ) and  ( ipAddr ~= "" ) )then
      -- lauch()        -- Cannot call directly the function from here the timer... NodeMcu crashes...
      tmr.alarm( 1 , 500 , 0 , launch )
      -- Reset alarm again
      tmr.alarm( 0 , 2500 , 0 , checkWIFI)
      print("Checking WIFI..." .. wifiTrys)
      wifiTrys = wifiTrys + 1

print("-- Starting up! ")

-- Lets see if we are already connected by getting the IP
ipAddr = wifi.sta.getip()
if ( ( ipAddr == nil ) or  ( ipAddr == "" ) ) then
  -- We aren't connected, so let's connect
  print("Configuring WIFI....")
  wifi.setmode( wifi.STATION )
  wifi.sta.config( SSID , APPWD)
  print("Waiting for connection")
  tmr.alarm( 0 , 2500 , 0 , checkWIFI )  -- Call checkWIFI 2.5S in the future.
 -- We are connected, so just run the launch code.
-- Drop through here to let NodeMcu run

So we can see that to program the esp8266 in Lua using the NodeMcu firmware we need to change a bit our programming paradigms…

18 thoughts on “ESP8266 NodeMcu and Lua language and some Arduino issues.

    1. This is a safe mechanism used in embedded processors. If the processor enters a tight loop it might it thinks that it might have crashed and resets itself.
      Why is this important? because in harsh environments, things like static electricity might put the processor in a state where it hangs/loops without doing any more useful work, like eecuting instructions out of the program space. So the watch dog timer is an hardware device that checks the health status of the processor, and if it thinks that the processor has hanged it resets it. Now a tight loop makes the processor execute the same set of instructions over and over again, which might be considered by the WD to be a crashed state.
      Also the way that things are done on the esp8266, if background tasks aren’t executed, it does nothing useful. So we can’t really use tight loops (for too long), or actively wait for something to happen. We must always use callbacks or timers.
      Resetting the WD inside the loop doesn’t solve anything, only says that I’m not crashed, but because background tasks aren’t run at the end it does nothing.

      TL:DR: It’s a hardware safe mechanism to get the processor to a known state if it crashes.

      1. Thanks that’s a great explanation for me.

        So, as you say in your post, we must adopt a nerw programmingn paradigm for this environment, but are there any hard and fast rules?

        I have now modified my program to use a timer to re-trigger CheckWIFI() function at 10 second intervals. If WIFI is not connected, it tries again. This is just a copy of your code. But if WIFI IS connected, then I want it to fetch a webpage eveyr ten seconds instead. However, that part of CheckWIFI() never works. Nil is returns as the payload. I am using the Web Client code copied directly from the NodeMCU website. How can I adopt this new paradigm to that problem? ANd more generally -rather than messing around to see what works ands what doesn’t and nhow to struggle around misty obstacles – where is the definitive understanding of exactly what we can and cannot do with LUA?

        There is no reason why you should know this of course, but if you do I would really appreciate your help again.

        Thanks, Chris

  1. Hi. First the NodeMCU runs a subset of the Lua language, eLua: so it doesn’t have all the functionality of the Lua language. Better check out the eLua site regarding specific issues with eLua. To add to the complexity, eLua is implemented over the ExpressIF esp8266 SDK in the NodeMcu firmware, so it also has limitations that arise from that. Issues like tight loops exist in either NodeMcu and the native SDK… so we need to work around that.

    Regarding hard or fast rules, there is really nothing that I can say other than having some programming background either in normal “procedure oriented” languages, like C, Java and event oriented ones line Node.Js. The principles applied to those languages also applied here.

    Regard the copy of my code, if the esp8266 is already connected it should go to the lauch function and call the ping.lua file. See on my other posts the content of that file, that shows how to get a web page every single minute.

    If the checkWifi is not to be called directly, so it always returns null. That is your problem. The checkWifi function what it does is to check if it is connected, and if not it re-triggers itself in the future by setting up timer 0 (zero) to call itself again in 2,5s in the future. If it is connected it sets a timer to call launch() function that calls the main program. It does this also by setting a timer, and not calling it directly, because otherwise the environment crashes (who knows why…)

  2. Thanks again, Mr Cortex (or can I call you “Primal”??). I have experience in both procedural and object-oriented programming, but it’s always been made clear exactly what calling something will do. Behaviours are clearly defined in the documentation. A method blocks or it doesn’t, for example. But Lua seems to be “kind of object oriented” – we specify a callback function to be run once something is accomplished with a socket, but in some cases (we find them by trying and failing) thing don’t work as expected.

    I guess we are beggars to the folks who have put in large mounts of work for no reward in making LUA available at all on the ESP chip, but it does make for a frustrating time.

    Anyway, I have taken enough of your time. Thanks again.

    1. PS: The code which doesn’t work, in case you’re curious. I want it to flash a LED to show me WiFi is working and sotp flashing it when it isn’t. For range and reliability testing. WiFi connects, but pl is always Nil.

      SSID = “xxxxxx”
      APPWD = “xxxxxxx”

      ledPin = 8

      print(“Starting …”)

      function checkWIFI()
      ipAddr = wifi.sta.getip()
      if ( ( ipAddr ~= nil ) and ( ipAddr ~= “” ) ) then
      gpio.write(ledPin, gpio.LOW)
      conn=net.createConnection(net.TCP, false)
      conn:on(“receive”, function(conn, pl) print(pl) end)
      conn:send(“GET / HTTP/1.1\r\nHost:\r\n”)
      print(“Payload is “)
      gpio.write(ledPin, gpio.HIGH)
      print(“WIFI is off”)
      gpio.write(ledPin, gpio.LOW)
      tmr.alarm( 0 , 10000 , 0 , checkWIFI)


      1. I think you need to separate the testing and the output to the Led.
        Basically the checkWifi function if there is an IP, set’s a timer to call/do the GET, but first it makes the LED LOW/OFF. If the call succeeds then we turn on the led otherwise it is kept off.

    2. I can agree that we need to thank to the great folks that are making Lua available on the esp8266. But this is till in its infancy and so many things don’t work as expected and/or not possible to implement, and that makes things a bit frustrating. For the record the above simple code took me around 3/4 hours of work/debugging time, but I’m no Lua expert. That’s why I also checking out the native SDK for the esp8266 and moved my efforts to the native SDK.

  3. Thank you for saving me more than the two hours I already spent on this. Funny how my attempts look almost exact like yours…

    Hope you don’t mind if I borrow some snippets from this (with attribution, of course!)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.