ESP8266 Community Forum

- Sun Jul 05, 2015 3:16 pm #22510 I am experimenting with receiving data from a weather API but the amount of data received exceeds buffer several times over. Of course this data could be written to a file and parsed but the flash storage would not live long. I just need to be pointed in the right direction to handle this overflow.

Sample receive function:

Code: Select allconn:on("receive", function(conn,payload)
print(string.len(payload))
end)


The output of this code is:
1460
1460
114

Thanks in advance for any help.

- Thu Jul 09, 2015 11:05 am #22850 The http protocol includes size information, so the easiest method is to use the on("receive") callback to append the incoming records to an input array using

Code: Select allbuf[#buf+1] = payload
len = len + #payload

then process the payload buf array once the full content has been received

Please see my Developer FAQ for Q&A on nodeMCU Lua before asking a question here that's already been answered. This and the other official documentation should be your first reference for developing Lua Apps

- Thu Jul 09, 2015 7:22 pm #22886 I thought about that too, but the payload is JSON. Decode JSON from an array?

- Thu Jul 09, 2015 7:35 pm #22887 I think it would be useful to describe to us what you want to happen. If the server sends "X" bytes of data, your receive callback will be called multiple times in units of 1460 bytes ... however it will get to see all the data.

Let us assume that you were able to receive the data in one unit ... what would you do with it? You say that it is JSON data ... well that is a String representation of JavaScript objects ... but it is still string data. So in principle, as the other poster said, you could allocate enough buffer space for your whole message and then append the "parts" to that buffer on each chunk of 1460 bytes until you have received the whole message. How will you know that the end of the message has been reached? Does the sender disconnect at that point?

Free ESP8266 book available for download here.