- Tue Nov 15, 2016 8:28 am
#58178
Using freshly built Lua 5.1.4 on SDK 1.5.4.1(39cb9a32) net.socket:on("receive") callbacks are not being broken down into 1460 byte chunks but come in one callback with payloads up to the free RAM limit - failing if the payload is larger.
Background: I had been successfully using the HTTP module to place requests over my LAN to a LED Lighting Hub (that's had its cloud service discontinued!) but after adding a few more bulbs my code started failing when placing a discovery request. I determined that this was due to the response payload exceeding around 5Kbytes and hence running out of RAM.
Taking the advice in the docs:
Here's the function that performs the request (two example php?cmd are shown in skt:send, the first simple one that's commented out is to turn on a lamp. The second one performs the device discovery that fails to return the payload)
Note that I've tried using Connection: keep-alive and Transfer-Encoding: chunked in the requests.
First, the output from the succesful "Lamp on" request which triggers two "on receive" callbacks with payload lengths of 111 and 43:
Then the output from the "Discovery" request which only triggers a callback for the 113 byte header, not the 5697 byte payload:
If I remove a couple of bulbs it works but the whole ~5K payload comes back in one callback:
I thought there was a buffer size of 1460 and the callbacks would return up to this amount each time. This Post http://www.esp8266.com/viewtopic.php?f=24&t=3901 also seems to suggest that I should see that.
Background: I had been successfully using the HTTP module to place requests over my LAN to a LED Lighting Hub (that's had its cloud service discontinued!) but after adding a few more bulbs my code started failing when placing a discovery request. I determined that this was due to the response payload exceeding around 5Kbytes and hence running out of RAM.
Taking the advice in the docs:
If larger page/body sizes are necessary, consider using net.createConnection() and stream in the dataSo I re-wrote the request accordingly but It hasn't helped. While all requests that result in short responses succeed as before, the discovery request just returns the reply header (showing Content-Length >~5K) but then disconnects imediately. Sucessful requests return the reply header and the payload in successive callbacks - then disconnect after a few seconds timeout from the Lighting Hub.
Here's the function that performs the request (two example php?cmd are shown in skt:send, the first simple one that's commented out is to turn on a lamp. The second one performs the device discovery that fails to return the payload)
Code: Select all
function sendRequest()
skt = net.createConnection(net.TCP, 1) --SSL=1
skt:on("connection", function(skt)
print("connected to hub")
skt:send("GET /"
--"gwr/gop.php?cmd=DeviceSendCommand&data=<gip><version>1</version><token>5mcopjq4x2j5rv429pbbkp7tbvq0ubtpov4j5zs8</token><did>216771547807313123</did><value>0</value></gip>&fmt=xml"
.."gwr/gop.php?cmd=GWRBatch&data=<gwrcmds><gwrcmd><gcmd>RoomGetCarousel</gcmd><gdata><gip><version>1</version><token>5mcopjq4x2j5rv429pbbkp7tbvq0ubtpov4j5zs8</token><fields>name</fields></gip></gdata></gwrcmd></gwrcmds>&fmt=xml"
.." HTTP/1.1\r\nHost: 192.168.1.66\r\nConnection: keep-alive\r\nTransfer-Encoding: chunked\r\nAccept: */*\r\n\r\n"
)
end)
skt:on("reconnection", function(skt)
print("> reconnection")
end)
skt:on("disconnection", function(skt)
print("> disconnection")
skt:close()
end)
skt:on("sent", function(skt)
print("> sent")
end)
skt:on("receive", function(skt, payload)
print("> rx payload len:"..string.len(payload))
print(payload)
end)
skt:connect(443,"192.168.1.66")
endNote that I've tried using Connection: keep-alive and Transfer-Encoding: chunked in the requests.
First, the output from the succesful "Lamp on" request which triggers two "on receive" callbacks with payload lengths of 111 and 43:
Code: Select all
> connected to hub
> sent
> rx payload len:111
HTTP/1.1 200 OK
Date: Tue, 15 Nov 2016 12:42:08 GMT
Server:
Content-Type: text/html
Content-Length: 43
> rx payload len:43
<gip><version>1</version><rc>200</rc></gip>
> disconnection
Then the output from the "Discovery" request which only triggers a callback for the 113 byte header, not the 5697 byte payload:
Code: Select all
> connected to hub
> sent
> rx payload len:113
HTTP/1.1 200 OK
Date: Tue, 15 Nov 2016 12:42:57 GMT
Server:
Content-Type: text/html
Content-Length: 5697
> disconnection
If I remove a couple of bulbs it works but the whole ~5K payload comes back in one callback:
Code: Select all
> connected to hub
> sent
> rx payload len:113
HTTP/1.1 200 OK
Date: Tue, 15 Nov 2016 13:29:10 GMT
Server:
Content-Type: text/html
Content-Length: 5033
> rx payload len:5033
<gwrcmds><gwrcmd><gcmd>RoomGetCarousel</gcmd> ... ~5K of discovery details here!
> disconnection
I thought there was a buffer size of 1460 and the callbacks would return up to this amount each time. This Post http://www.esp8266.com/viewtopic.php?f=24&t=3901 also seems to suggest that I should see that.
What appeared to be arbitrary was the number of multiples of 1460 byte payload in each receive callback, being either one or four - which isn't necessarily an issue, but in the particular case I'm struggling with none of the expected callbacks are received other than for the initial header, until the payload drops below a certain byte-size (somewhere between 5033 - 5697). Can you help shed any light on why that might be?
