ESP8266 Community Forum

- Tue Mar 17, 2015 11:37 pm #12157 I can give a few pointers, but I don't have the free time to convert every one's module to flashMod for them... try it yourself and I'll help you with problems.

First, everything is global... you need to add local before most of your variables, or they will stick around in the global namespace forever, taking up space. The trick of memory optimization is usually NOT to avoid using memory, but to stop using it as soon as you are done with it.

Second, all those constants take up zero space in C, but they take up probably around 500 bytes in Lua. Why? INTEGRATIONTIME_700MS=0x00 is the same as _G["INTEGRATIONTIME_700MS"]=0.0 to Lua. That means the names of the variables themselves are key values in a table (which means they are represented as strings), and all numbers are 4 bytes (integers or floats, depending on your build). There's also some additional table overhead. The best approach from a memory standpoint is to use the numeric constants directly, and just add a comment of what it means. If the named constants are worth the memory usage, then go for it, but at least put them on your module's table rather than the global table.

Putting each of those functions on a module table and then running that table through flashMod() before returning it should make the heap after require much better. Again, try it and let me help you fix what doesn't work. Just remember to use colons instead of dots before functions, and use self instead of the module table to reference other functions or variables.

- Wed Mar 18, 2015 2:38 pm #12208 Hi, thanks for your reply.
Ive found other way to save a lot of heap, not using mod flash but, I learned usefull information about heap/flash thanks to your post's
Here are use results:

tcs_low.lc 2132 bytes
> node.heap()
> =node.heap()
22824
> require("tcs_low")
> =node.heap()
18592

As I see compared to other way that I used first I've saved about 5KB of heap space.
The secret is simple, only replaced static variables with respective value

This discussion post's lead me to learn that nodemcu lua uses a lot of heap memory for declaring variables, for example
> = node.heap()
18088
> t=1
> = node.heap()
18024

heap consumed per variable is about 64Bytes

but if i do

> = node.heap()
18024
> t=12345678901234567890
> = node.heap()
18024

This means that variable t allocates the same 64Bytes, suppose it can handle 64characters..
And about strings:

> = node.heap()
18024
> t="12345678901234567890"
> = node.heap()
18008
>
> t=12345678901234567890
> = node.heap()
18024

Ok, this means that the same string is eating more 16Bytes of heap for the same content.
So when programming on microcontrolers with limited resources using high level language lua you should declare variables only if really necessary, otherwise they are wasting a lot of space in heap.
If varible is dynamic so make sure u clean it after use, variable equal to nil.

So, if my thoughts are correct mod flash will put this variables into flash memory, releasing that heap space.
So need to have one precaution when using this mod flash, if there are some variables that are updated often this means writting to flash many times, and as we know flash memory have self life of some few thousands write operations.For exaple if u update this variable once a second means 86 400 writes/a day. Maybe enough to kill flash..
So this mod flash technik should be used only for static variables or the called #define

Other solution could be creating a file with static variables and read line index defined in a table to correspondent "variable" and insert that value directly into code dynamicaly.

The code for tcs34725 from Craig Scott is great and functional but has 5Kbytes allocated just for variables declaration. I just simplified the code with almost no declarations:


function initialise()
i2c.setup(0,3,4,i2c.SLOW)
if (string.byte(read_reg(0x29,0x12))==0x44) then
print("Found TCS34725")
end
end

function read_reg(dev_addr, reg_addr)
i2c.start(0)
i2c.address(0, dev_addr ,i2c.TRANSMITTER)
i2c.write(0,bit.bor(0x80,reg_addr))
i2c.stop(0)
i2c.start(0)
i2c.address(0, dev_addr,i2c.RECEIVER)
c=i2c.read(0,1)
i2c.stop(0)
return c
end

function write_reg(dev_addr, reg_addr, reg_val)
i2c.start(0)
i2c.address(0, dev_addr, i2c.TRANSMITTER)
i2c.write(0, bit.bor(0x80,reg_addr))
i2c.write(0, reg_val)
i2c.stop(0)
end

function getRawData ()
clear=bit.bor(bit.lshift(string.byte(read_reg(0x29,0x15)),8),string.byte(read_reg(0x29,0x14)))
red=bit.bor(bit.lshift(string.byte(read_reg(0x29,0x17)),8),string.byte(read_reg(0x29,0x16)))
green=bit.bor(bit.lshift(string.byte(read_reg(0x29,0x19)),8),string.byte(read_reg(0x29,0x18)))
blue=bit.bor(bit.lshift(string.byte(read_reg(0x29,0x1B)),8),string.byte(read_reg(0x29,0x1A)))
tmr.delay(100)
print(red,green,blue,clear)
red,green,blue,clear = nil,nil,nil,nil
end

function enable()
write_reg(0x29,0x00,0x01)
tmr.delay(30)
write_reg(0x29, 0x00, bit.bor(0x01,0x02))
end

function disable()
tmr.delay(3)
write_reg(0x29,0x00, bit.band(read_reg(0x29,0x00), bit.bnot(bit.bor(0x01,0x02))))
end

--Interaction with module after require("tcs34725") is:
--initialise(0x29)
--write_reg(0x29,0x01, 0xEB)
--write_reg(0x29,0x0F, 0x00)
--enable()
--getRawData()


The code is now fully functional but less 5KBytes used by heap
This is a lesson for all, remember that u are developing in a low resouce machine, all code must be optimized to do less operations and save RAM/FLASH memory .

Regards,Vito

- Wed Mar 18, 2015 6:13 pm #12220 Great work Vito!

One point: inside functions like getrawdata, you can declare the variables as local red = ... and they will be destroyed at the end of the function without setting to nil.

flashMod does not put variables in flash, only functions... This is why you should separate declaration [flashMod(table)] from use [flashMod(string)]. I haven't made that point clear, but require()ing the declaration every time will kill your flash fast.

- Wed Mar 18, 2015 7:43 pm #12224 Hi, thanks for local variable advice, it saved me more heap memory : )
I did not tryied yet your mod flash technik because it will handle the flash memory and i'm not well understanding the code, I was thinking that it saved functions and variables to flash, but it propouse is for functions only. My worry is that flash will die with many writes, so I must understand well the code before interact with flash.

My advice is not to mess too much with flash, only if well know what ur doing.
But your technik sounds very good.

Also found another way that I think will do similiar to mod flash, but mod flash can be personalized thats advantage
The other way is to compile xx.lua file (firmware>=13022015) Once u compile you get a xx.lc file
Simply load the lc file with dofile("xx.lc") or require("xx") and module is loaded
Here some output:

tcs_low.lua 1449 bytes
tcs_low.lc 2104 bytes
> = node.heap()
22824
>
dofile(tcs_low.lua) quinta-feira, 19 de Março de 2015 00:17:05

dofile("tcs_low.lua")
> = node.heap()
17072
>
Soft Restart quinta-feira, 19 de Março de 2015 00:17:17

node.restart()
> c_ÇÏRSöfJSúfJSúfîá

NodeMCU 0.9.5 build 20150311 powered by Lua 5.1.4
lua: cannot open init.lua
> = node.heap()
23072
>
dofile(tcs_low.lc) quinta-feira, 19 de Março de 2015 00:17:39

dofile("tcs_low.lc")
> = node.heap()
18944


As we can see using dofile("tcs_low.lua") it will consume 5725Bytes of heap, but much better is if use dofile("tcs_low.lc") , now the heap has only 4128Bytes allocated, its 1624Bytes less
This is the optimized way to run code.
To call module just use require("tcs_low")

Dont know if this can be used same time with mod flash




Regards,Vito