2.0.0 Has A Memory Leak

Current Lua downloadable firmware will be posted here
 Reply

User avatar

2.0.0 Has A Memory Leak

By gschulz
- Sun Mar 19, 2017 12:17 am #63915 The latest built (2.0.0) has a memory leak (or at least a poorly functioning garbage collector). I upgraded from 0.9.6 to 2.0.0 and noticed an immediate speed increase. I also noticed that my server app was crashing after a couple dozen HTML page accesses. This is the exact same code and HTML sources that I was using on 0.9.6. I printed out the heap as I accessed pages and noticed the available memory was shrinking with every access. To validate this, I wrote a bare minimum server app that continually requests a page reload and displays the current heap, iteration count and LUA version number. Here is my test code:

Code: Select allwifi.setmode(wifi.STATIONAP)
srv=net.createServer(net.TCP)
x=1
srv:listen(80,function(conn)
    conn:on("receive", function(client,request)
        local a,b,c=node.info()
        client:send("HTTP/1.1 200 OK\r\nContent-type: text/html\r\nConnection: close\r\n\r\n<!DOCTYPE html><head></head><body><p>Iterations: "..x.."  Heap: "..node.heap().."</p><p>Version: "..a.."."..b.."."..c.."</p><script>setTimeout('location.reload();',300);</script></body></html>")
        x=x+1
    end)
    conn:on("sent", function() WiFiclose() end)
    function WiFiclose() conn:close() collectgarbage() print(node.heap()) end
end)


Run this on your ESP device and enter the IP address into your browser (it doesn't care what file you are accessing). You will see that 0.9.6 starts at around 27K and stabilizes at around 24K. In my case, 2.0.0 starts at around 37K and decreases at a rate of about 200 bytes per iteration until it crashes at around 3K. I noticed that if I increase the delay before reload to about a second, 2.0.0 will stabilize at around 4K and not crash. This implies that the garbage collection is not aggressive enough. I haven't tried this on any other versions, since I don't have a build environment. Perhaps this can be as simple as upping the garbage collection priority.
User avatar

Re: 2.0.0 Has A Memory Leak

By marcelstoer
- Sun Mar 19, 2017 6:20 am #63931 That's a nice piece of test code, it's not often that I see such a good MCVE. However, I can't reproduce this with either 1.5.4.1 or 2.0.

gschulz wrote:This is the exact same code and HTML sources that I was using on 0.9.6.


As a general remark, if the implied expectation is that anything that runs on 0.9.6. will run fine w/o any modifications on 2.0 than that'd be a fallacy.

I slightly modified your code for it to be a bit more legible and removed obsolete statements (conn:close(), GC).

Code: Select allsrv = net.createServer(net.TCP)
x = 1
srv:listen(80, function(conn)
  conn:on("receive", function(client, request)
    local a, b, c = node.info()
    local buf = "HTTP/1.1 200 OK\r\nContent-type: text/html\r\nConnection: close\r\n\r\n"
    buf = buf .. "<!DOCTYPE html><head></head><body>"
    buf = buf .. "<p>Iterations: " .. x .. "  Heap: " .. node.heap() .. "</p>"
    buf = buf .. "<p>Version: " .. a .. "." .. b .. "." .. c .. "</p>"
    buf = buf .. "<script>setTimeout('location.reload();',300);</script>"
    buf = buf .. "</body></html>"
    client:send(buf)
    x = x + 1
  end)
  conn:on("sent", function() print(node.heap()) end)
end)


I tested on a NodeMCU devkit with 4MB and NodeMCU 2.0 as follows:

NodeMCU custom build by frightanic.com
branch: dev
commit: 5feae3fee1debe6aef75b8836fa86f9967037f2f
SSL: true
modules: cron,dht,file,gpio,http,mqtt,net,node,rtctime,sntp,spi,tmr,uart,wifi,tls
build built on: 2017-02-26 20:37
powered by Lua 5.1.4 on SDK 2.0.0(656edbf)


I stopped the script after around 120 iterations. The heap was absolutely stable.

Then I tested the same Lua code on an ESP-01 with 512KB and NodeMCU 1.5.4.1 (i.e. before Espressif SDK was upgraded to 2.0).

NodeMCU custom build by frightanic.com
branch: 1.5.4.1-final
commit: 1885a30bd99aec338479aaed77c992dfd97fa8e2
SSL: false
modules: dht,file,gpio,mqtt,net,node,tmr,uart,wifi
build built on: 2017-03-18 22:53
powered by Lua 5.1.4 on SDK 1.5.4.1(39cb9a32)


Again, heap was absolutely stable, it bounced around +-20 bytes or so. I stopped after 140 iterations.
User avatar

Re: 2.0.0 Has A Memory Leak

By jankop
- Sun Mar 19, 2017 8:29 am #63938 Example from Marcel does not work for me. I had to modify it a little.

Code: Select allsrv = net.createServer(net.TCP)
x = 1
srv:listen(80, function(conn)
  conn:on("receive", function(client, request)
    local a, b, c = node.info()
    local buf = "HTTP/1.1 200 OK\r\nContent-type: text/html\r\nConnection: close\r\n\r\n"
    buf = buf .. "<!DOCTYPE html><head></head><body>"
    buf = buf .. "<p>Iterations: " .. x .. "  Heap: " .. node.heap() .. "</p>"
    buf = buf .. "<p>Version: " .. a .. "." .. b .. "." .. c .. "</p>"
    buf = buf .. "<script>setTimeout('location.reload();',300);</script>"
    buf = buf .. "</body></html>"
    client:send(buf)
    x = x + 1
   conn:on("sent", function() client:close() print(node.heap()) end)
  end)
  --conn:on("sent", function() print(node.heap()) end)
end)

But it takes about 90 iterations, and the program will collapse without memory. The same applies to gschulz version only need about 190 iterations. At start has Lua 40KB memory. Consequently is Lua unusable.
I have esp8266-12F with 4MB flash and integer firmware:
NodeMCU custom build by frightanic.com
branch: master
commit: b96e31477ca1e207aa1c0cdc334539b1f7d3a7f0
SSL: false
modules: adc,bit,file,gpio,net,node,pwm,rtcmem,tmr,uart,wifi
build built on: 2017-03-14 16:32
powered by Lua 5.1.4 on SDK 2.0.0(656edbf)
 Reply
Return to Lua Firmware