As the title says... Chat on...

User avatar
By dnc40085
#16324 Thank you for the quick reply.

cal wrote: As you may have noticed I am not a native speaker. So please correct me when I get non idiomatic.
No problem, aside from you saying "Moin"(hello in german?) I hadn't really noticed that you're not a native speaker. In future posts, to avoid confusion I will try to avoid using region specific vocabulary.

cal wrote:1. a stack has a stackpointer SP that point to the top of the stack
2. mostly you "push" values on the top of the stack and "pop" from the stack
3. If you address values on the stack you do it relative to the top. SP[0] is the top element SP[-1] the one below the top
and so on.

S:
push(100)
S: 100
^ SP
push(200)
S: 100 200
^SP
push(300)
S: 100 200 300
^SP
SP[0]: 300
SP[-1]: 200
SP[-2]: 100

pop()
S: 100 200
^SP
SP[0]: 200
SP[-1]: 100

Until your mind gets a "stack mode" just print current stack while following program flow on paper.
Thank you very much for this info, the visualization greatly clarifies how the Lua stack and the push and pop functions work and it also inspired me to do a little bit of experimenting on my own, so I found a function to dump the stack and wrote my own function to get some hands on expierence with the stack and figure out how exactly the stack and stack pointers work:
Code: Select allstatic void stackdump(lua_State* l)
{
    int i;
    int top = lua_gettop(l);

    c_printf("total in stack %d\n",top);

    for (i = 1; i <= top; i++)
    {  /* repeat for each level */
        int t = lua_type(l, i);
        switch (t) {
            case LUA_TSTRING:  /* strings */
               c_printf("string: '%s'\n", lua_tostring(l, i));
                break;
            case LUA_TBOOLEAN:  /* booleans */
               c_printf("boolean %s\n",lua_toboolean(l, i) ? "true" : "false");
                break;
            case LUA_TNUMBER:  /* numbers */
                c_printf("number: %g\n", lua_tonumber(l, i));
                break;
            default:  /* other values */
                c_printf("%s\n", lua_typename(l, t));
                break;
        }
        c_printf("  ");  /* put a separator */
    }
    c_printf("\n");  /* end the listing */
}

// Lua: mac = wifi.stack()
static int stack( lua_State* L)
{
  stackdump(L);
  lua_pushnumber(L, 100);
  lua_pushnumber(L, 200);
  lua_pushnumber(L, 300);
  lua_pushnumber(L, 400);
  lua_pushnumber(L, 500);
  lua_pushnumber(L, 600);
  lua_pushnumber(L, 700);
  lua_pushnumber(L, 800);
  lua_pushnumber(L, 900);
  stackdump(L);
  c_printf("index  0: %g\n", lua_tonumber(L, 1));
  c_printf("index  1: %g\n", lua_tonumber(L, 1));
  c_printf("index  2: %g\n", lua_tonumber(L, 2));
  c_printf("index  3: %g\n", lua_tonumber(L, 3));
  c_printf("index -1: %g\n", lua_tonumber(L, -1));
  c_printf("index -2: %g\n", lua_tonumber(L, -2));
  c_printf("index -3: %g\n", lua_tonumber(L, -3));
return 1;
}
I fooled around with it for a while putting arguments in on the Lua command prompt (e.g. wifi.stack(42, 88, 9001) and wifi.stackdump("hello world", true, function() return "over 9000" end )) to see the resulting stack and how positive and negative indexing worked and it helped me to better understand the stack and how it worked.

if anybody(who doesn't already know) is interested in the result of executing the stack() function. here ya go:
Code: Select allwifi.stack()
total in stack 0

total in stack 9
number: 100
  number: 200
  number: 300
  number: 400
  number: 500
  number: 600
  number: 700
  number: 800
  number: 900
 
index  0: 100
index  1: 100
index  2: 200
index  3: 300
index -1: 900
index -2: 800
index -3: 700
>


Code: Select all  if (lua_type(L, 1) == LUA_TFUNCTION || lua_type(L, 1) == LUA_TLIGHTFUNCTION){ // if index 1 of "L" is a function then...
What may index 1 be? Where may it come from?
previously I deduced that the index number corresponds to the position in the function as entered from the Lua prompt (lua_function("index 1", "index 2", "index 3", etc.). Now I realize that the index number corresponds to the position starting from the bottom of the lua stack.

Code: Select all if(wifi_scan_succeed != LUA_NOREF)// if variable(wifi_scan_succeed) is NOT equal to constant (LUA_NOREF(-2)) then...
      luaL_unref(L, LUA_REGISTRYINDEX, wifi_scan_succeed); //release previous reference and corresponding value???
Yep. (Maybe time to get suspicious?)
The questions marks were to indicate that I don't quite understand how luaL_unref() and luaL_ref() work.
Question: Is creating a reference with luaL_ref() in the LUA_REGISTRYINDEX similar to declaring a global C variable containing the entry from the top of the Lua stack, then returning a pointer to that global var(i.e. "lua_pushnumber(L, 100); global_var=lua_tonumber(L, -1); return &global_var;" ) and is luaL_unref() equivalent to setting the global_var to NULL(i.e. "global_var=NULL;"). would these scenarios correct?

Code: Select allwifi_scan_succeed = luaL_ref(L, LUA_REGISTRYINDEX); //create reference(lua pointer) to lua value and store the ID for the reference in the variable "wifi_scan_succeed"
What is this "value"? Where is it? Where does it come from?
I now know that this takes the function pushed on to the top of the stack by the command "lua_pushvalue(L, 1)" and places it into the LUA_REGISTRYINDEX then pops it from the stack.

code]wifi_station_scan(NULL,wifi_scan_done); //call station scan function from SDK
[/code]Correct. Notice what is passed and what it means.
The first argument passed is the scan config, but since NULL is passed instead of the struct for scan_config it scans for all access points, the second argument is the C callback function that handles converting the resulting access point info from the returned struct to a Lua table as well as executing the user's Lua callback.

Code: Select allstatic void wifi_scan_done(void *arg, STATUS status)
{
  uint8 ssid[33];  // Initialize variable "ssid"
Not really. It "defines" or declares the variable. The variable exists there after.
It is NOT initialialized which means "get a value". Its uninitialized. Typically contains garbage.
Note the size of the array.
thank you for clarifying this.
Side note: memory for this local variable comes from the c stack. I will vanish
This the reason why the concept of having to add local in front of variables to prevent them from being declared as global variables was so confusing at first.

Code: Select allif(wifi_scan_succeed == LUA_NOREF) //if variable "wifi_scan_succeed" contains no reference(lua pointer) then...
    return; // since there is no reference ID, return
Yep. Does that have any meaning on lua application code level?
I'm not sure... maybe it verifies that there exists a reference to the user defined callback function and if that fails it returns because there is nothing to process the args passed onto the C callback function by wifi_station_scan.

Code: Select allif(arg == NULL) //if variable "arg" is NULL then...
    return; //there is nothing to process so return
What is this arg? Where does it come from?
I think the variable "arg" containes the access point info returned by the function wifi_station_scan.

Code: Select alllua_rawgeti(gL, LUA_REGISTRYINDEX, wifi_scan_succeed); //push value corresponding to reference stored in "wifi_scan_succeed" onto the stack
Yep.
What type of object is it?
Just say: push <the-type-xxx> wifi_scan_succeed on the stack
the object should be the function stored in the LUA_REGISTRYINDEX previously by luaL_ref().

Code: Select allif (c_strlen(bss_link->ssid) <= 32) //if length of the string contained within "bss_link->ssid" is LESS THAN or EQUAL to 32 characters then...
      {
        c_memcpy(ssid, bss_link->ssid, c_strlen(bss_link->ssid)); //copy contents of "bss_link->ssid " to "ssid"
      }
      else //not sure why this here, maybe to copy only 32 characters of the string if it's longer than 32 chars
Look at definition of target "ssid".
My mistake, I forgot that c_strlen() also counts the trailing "\0", making c_strlen() return a size of 33 for an ssid comprising of 32 characters.

Code: Select all      {
        c_memcpy(ssid, bss_link->ssid, 32); //copy contents of "bss_link->ssid " to "ssid"

Be more precise about the semantics. My apologies, right around this point I started rushing to get the post finished since I had been trying to decipher the source all day and was getting frustrated with it so I started leaving out details. anyway the revised version is "//copy 32 characters from "bss_link->ssid " to "ssid" starting from index 0"
Remember how c strings are defined (especially what defines the length of a c string). String length is determined by the trailing"\0", correct?
Look at the definition of ssid.
Why does ssid contain a valid c string after this? It's a valid string because it is only copying 32 chars or less from bss_link->ssid leaving the \0 in index 32 (set by c_memset(ssid, 0, 33)) undisturbed?


Code: Select all      }
      c_sprintf(temp,"%d,%d,"MACSTR",%d", bss_link->authmode, bss_link->rssi,
                 MAC2STR(bss_link->bssid),bss_link->channel); // copy "authmode, rssi, bssid, channel" values from variable "bss_link" to variable "temp"

"copy" is too unprecise for me.
Here may be dragons.
Based on the format string and parameters given to the function a new string is generated and written to the buffer
defined by temp.
I typically say "print xxx to buffer temp".
Side note: Can the buffer temp overflow? What would happen if it overflows?
Can it overflow: I Believe not, in fact I think the size of the array could be decreased to 32. since if one were to add all the chars from the resulting string it should equal 32 chars, I could be wrong though.
What would happen if it overflows: As far as I've read an overflow would cause undefined behavior to occur, which could be: nothing at all, change a different variable, cause a segmentation fault,crash the chip, take your car for a joy ride, etc.

Code: Select alllua_pushstring(gL, temp); //set value of current key in lua table to contents of variable "temp"

Technically it just pushes a string on the stack.
What is "current key"?
Does it have a meaning?
I was referring to the Key in relation to the "key, value" returned by the lua function pairs()

Code: Select all    }
  }
  else //if "STATUS" NOT EQUAL "OK(0)" then...
  {
    lua_pushnil(gL); //push nil value onto stack for pointer "gL"? not sure what exactly this does... maybe this makes sure that nothing is returned to user in case info is old or non-existent.

Think about what "gL" is used for. It's just the home for the SP.
Why push nil on the stack if we don't have anything? Why don't just leave the stack unchanged?
I'm drawing a blank on this one... maybe push nil so there is no interference with the function put on the stack by "lua_rawgeti()" when the "wifi_scan_done" function is called again, if indeed it is called again.
Code: Select all  }
  lua_call(gL, 1, 0); //calls user's callback function???
Consider the state of the lua stack, the definition (http://pgl.yoyo.org/luai/i/lua_call), explain the parameters to lua_call.
If you are confident what function it is:
What type of parameter does it expect?
Can that help you understand the lua_setfield function?
http://pgl.yoyo.org/luai/i/lua_setfield
What do they mean with "t[k] = v" and how is that translated to the state of the lua stack and the parameters of that function when it is called?
first of all, lua_call(Lua_State, # of args passed to function, # of args returned by function). I believe the parameter it expects is a table since the user's function requires a table. as for the lua_setfield function, after reading the links you gave and I found and read THIS, I believe "lua_setfield(gL, -2, ssid)" takes the string pushed onto the stack by "lua_pushstring(gL, temp)" and creates an entry in the table created by "lua_pushstring(gL, temp)", which now resides at index -2 of the Lua stack, with the string contained within the variable "ssid", then pops the value from the top of the stack.

Next, I plan to add stackdump() throughout the functions, so I can figure out what exactly is going on in the stack.
Last edited by dnc40085 on Sun May 03, 2015 8:30 am, edited 1 time in total.
User avatar
By dnc40085
#16346 PROGRESS!!!!!1!!!!111!!!!11!1!!!1!!
I added stackdump() before and after each command that affects the Lua stack and learned how the stack is manipulated and where each function begins and ends. I figured out that the end of "wifi_scan_done" is the end of the line for the C function and I observed that when the function is done it does not remove the function stored in LUA_REGISTRYINDEX or set wifi_scan_succeed to "LUA_NOREF" so I added it. After this addition, the line "wifi.sta.getap(function() return end)" was no longer necessary and the memory use had dropped to 128 bytes.

This is what I added at the very end of wifi_scan_done:
Code: Select all  if(wifi_scan_succeed != LUA_NOREF)
  {
    luaL_unref(gL, LUA_REGISTRYINDEX, wifi_scan_succeed);
    wifi_scan_succeed = LUA_NOREF;
  }


This is the result:
Code: Select all=node.heap()
22072
> wifi.sta.getap(function(T) for k,v in pairs(T) do print(k..":"..v) end end)
>
> myssid:      1a:fe:34:xx:xx:xx,-1
xfinitywifi:      96:1a:ca:xx:xx:xx,-91
Linksys_dd-wrt:      00:1d:7e:xx:xx:xx,-64
=node.heap()
21944
>


Now I just have to figure out how to get the last 128 bytes back...
User avatar
By TerryE
#16506 Another trick which I use to clear down dead tmp.alarm() references is to set the alarm to a lightweight function which accepts a null argument (e.g. print or string.char) or in this case a table without barfing. I tried
Code: Select allwifi.sta.getap(function(T) for k,v in pairs(T) do print(k..":"..v) end end)

followed by:
Code: Select allwifi.sta.getap(type)
and this had the same effect: the RAM chunk tied up by the function(T) for k,v in pairs(T) do print(k..":"..v) end end got dereferenced and freed.

Messy, but a good workaround until the core ndoeMCU code is fixed. :)
User avatar
By dnc40085
#16511
TerryE wrote:Another trick which I use to clear down dead tmp.alarm() references is to set the alarm to a lightweight function which accepts a null argument (e.g. print or string.char) or in this case a table without barfing. I tried
Code: Select allwifi.sta.getap(function(T) for k,v in pairs(T) do print(k..":"..v) end end)

followed by:
Code: Select allwifi.sta.getap(type)
and this had the same effect: the RAM chunk tied up by the function(T) for k,v in pairs(T) do print(k..":"..v) end end got dereferenced and freed.

Messy, but a good workaround until the core ndoeMCU code is fixed. :)


my goal here is to fix the core NodeMCU code. so far i've managed to fix the issue for the most part, so neither wifi.sta.getap(function() return end) or wifi.sta.getap(type) is required to free up the ram anymore. since adding the lines to the end of wifi_scan_done takes care of the dereferencing and freeing he ram used by the users function(e.g. function(T) for k,v in pairs(T) do print(k..":"..v) end end).

Attached to this post there is the NodeMCU firmware release 20150406 with my partial fix(releases function memory but still uses 128-256 bytes RAM).
Attachments
(280.79 KiB) Downloaded 226 times
(280.78 KiB) Downloaded 213 times