ESP8266 NodeMcu Heap: Limitations and a small hack to graph heap usage

So I’m messing around with ESP8266 and the NodeMcu firmware that uses the Lua language.

The fact is that I’m hitting some limitations that are hindering development of something more complex than simple web servers or anything else for that matter. Limitations like it can’t be a network server and a client at the same time, no network sockets, like opening up a socket to a server and keep that connection open a use it across Lua files,  makes building things much harder when in fact this is a communication centred device… So at the end for more complex projects the native SDK and using the C language is the way to go.

Anyway, I’ve made a small hack that uses an Apache Web server with PHP installed and Dygraphs library to draw out the heap usage in a nice line graph:

Setting up the Apache Web Server: First make sure that you have Apache with PHP installed (for this hack I use PHP because it was already installed and configured). In my case, the Apache document root directory is /var/www .

Create a directory named data under /var/www and make sure that the user running the apache server process has rights to write on the directory. In my case the user is www-data:

ps -ef | grep apache
….
www-data 12118 697 0 Jan04 ? 00:00:00 /usr/sbin/apache2 -k start
….

So I needed to do chown www-data /var/www/data so that Apache can write data to that directory.
On that directory create a file named esp8266heap.csv where the first line has the following content:

Data , Heap

Just make sure that you add a new line at the end of this line.
Make sure that the rights are ok:

chmod o+w esp8266heap.csv
chown www-data:www-data esp8266heap.csv.

Setting up the code for receiving data:

Now the esp8266 will call a file named espdata.php passing through the query string the heap size: http://server_ip_name/espdata.php?heap=12345. This php file will get the heap query parameter and write it on the esp8266heap.csv file along with a time stamp. Here’s the code:

<?php
$now = new DateTime();
parse_str( html_entity_decode( $_SERVER['QUERY_STRING']) , $out);

if ( array_key_exists( 'heap' , $out ) ) {

echo "\n\r\n\rValor: " . $out['heap'];
$handle = fopen("/var/www/data/esp8266heap.csv", "a");

fwrite( $handle , $now->format('Y-m-d H:i:s') . " , " . $out['heap'] . "\n" );
fclose ( $handle ) ;

}
?>

Changing the esp8266 code:

My ping.lua file (check my older posts), that is called every minute looks like this now:

conn=net.createConnection(net.TCP, false)
conn:on("receive", function(conn, payload) print("Get done.", payload) end )
conn:connect(80,"xx.xx.xx.xx")
conn:send("GET /esp8266data.php?heap=" .. node.heap() .." HTTP/1.1\r\nHost: xx.xx.xx.xx\r\n" .. "Connection: keep-alive\r\nAccept: */*\r\n\r\n")

And that’s it. Every single minute an esp8266 timer triggers and calls the above ping.lua file that makes the espdata.php store the current heap size in the esp8266heap.csv file.

Graphing the data:

For graphing the data I’m using the dygraphs javascript library called remotely from my page that I’ve called esp8266graph.html:


<html>
<head>
<script src="//cdnjs.cloudflare.com/ajax/libs/dygraph/1.1.0/dygraph-combined.js"></script>
</head>
<body>
<div id="graphdiv2"
style="width:500px; height:300px;"></div>
<script type="text/javascript">
g2 = new Dygraph(
document.getElementById("graphdiv2"),
"data/esp8266heap.csv", // path to CSV file
{}          // options
);
</script>
</body>
</html>

And that’s it.

Seeing it all:

We can go now to http://server_ip_name/espgraph.html and see the esp8266 NodeMcu heap size variations in an interactive way. The final result show that with only the single file, heap keeps constant, then drops, and keeps constant again, then drops again, and so on, until the esp8266 reboots, and then the heap recovers and the process starts again…

Selection_042

We can see where it is heading… The sharp change at around 3AM was the NodeMcu reboot.

Anyway, this hack with dygraphs can be used for anything e can imagine…

Advertisements

19 thoughts on “ESP8266 NodeMcu Heap: Limitations and a small hack to graph heap usage

  1. Hello, I had question.

    I am searching for sending from the esp8266 directly to mysql database.

    I tought it will be possible with your code, but I am not sure.

    THe mysql server has a password.

    Could you maybe help me with a example code ?

    Thank you for your reply

    • Hi:

      Check out the following page: http://www.w3schools.com/php/php_mysql_insert.asp

      Basically the esp8266 calls the php page as above, but instead of writing into a file the query parameter value, it writes into a database.

      Something like this (Warning I didn’t test it. It might not work…):
      connect_error) {
      die(“Connection failed: ” . $conn->connect_error);
      }

      $sql = “INSERT INTO HeapTable ( datelog , heapvalue)
      VALUES ($now->format(‘Y-m-d H:i:s’), $out[‘heap’])”;

      if ($conn->query($sql) === TRUE) {
      echo “New record created successfully”;
      } else {
      echo “Error: ” . $sql . “
      ” . $conn->error;
      }
      }

      $conn->close();
      ?>

  2. Hello, I try it but if I do the file.php
    I get a error of ” unexpected symbol near ‘<' "

    I think it is the first symbole before <?php

    How could I fix this ?

    • I’ve made some changes on the code, and it works: Check out this: http://pastebin.com/Kd8k10mE. Please make sure that you copy the RAW section that has no formatting or other characters. Please note: The SQL insert table code is just an example!!! You must create your database, and tables with the schema that make sense for you.

      If you save the above pastbin as file, for example, dbheap.php, on your web server, then you must call it as following: http://yourserveraddress/dbheap.php?heap=334455

      If you want to put in the database other information, you must change the code

      • Hello primalcotex,thanks for your reply.
        Ah oke. I mean that the ESP8266 sent directly to the mysql server so the server doesn’t need the php webpage. is that possible? because now you send the information to server with the php file on it and that one sent it to the mysql server and save it, but I would like to sent it from the esp8266 directly to the mysql server. so the esp8266 connect to mysql server and write it on his own in the database.

  3. That would mean that the esp8266 needs to have a MySQL client on itself so it can communicate to the database server without any intermediary… As far as I’m aware, there isn’t such client available…

  4. It might be feasible to be done, but I foresee some problems. In my opinion, such tasks, as database access should be centralized.

  5. Hello, Really thanks for the tips it really works 🙂 only I have one problem the part I sent with the esp8266 doesn’t sent it.
    The part of thingspeak works but when he have to sent it to my mysql server he have some problems and the esp8266 reboots.

  6. Ah it looks that I can’t use them together. is it because the connection is still connected ? when I swith the thingsspeak as comment it works.

  7. I had a question. Do you also have example code for compelling directly with arduino esp8266 “generic esp8266 module”. So the esp8266 connect to the server and send values to the heap.You really help me last time.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s