ESP8266 Community Forum

- Fri Nov 28, 2014 8:22 am #3457 Strange things happen when sending data longer than 2 physical packets...

Code: Select allsrv=net.createServer(net.TCP)
srv:listen(666,function(conn)
    conn:on("sent",function(conn)         
        print("packet sent")
    end)

    conn:on("receive",function(conn,payload)
        for i=1,500,1
        do
            conn:send(tostring(i) .. " 12345678901234567890\n")
        end
        -- conn:close()
    end)
end)

making a request gives this result:

Code: Select all1 12345678901234567890
2 12345678901234567890
[ .... many more lines ...]
118 12345678901234567890
119 12345678901234567890
120 12345678901234567890
121 12345678901234567890
122500 12345678901234567890

.. and fires the on:("sent") event 2 times. Looking at the network traffic (dumpfile attached) - there are 3 packets actually sent, the second ending with "122" (see the last line of the output above), the third one holds the last line actually sent: "500 12345678901234567890".
The count of lines sent doesn't matter as long as it's more data than fits in exactly two packets...

Additionally - enabling the "conn:close()" statement results in firing no "sent"-event at all but indeed sends the first packet (in the example above, the last lines received are:

Code: Select all121 12345678901234567890
122

Thomas

Attachments

tcpdump.txt

(16.46 KiB) Downloaded 700 times

- Fri Nov 28, 2014 9:10 pm #3494 This is likely the tcp buffer inside sdk overflowed.
try to send packets one by one after the previous one is sent.

Code: Select allsrv=net.createServer(net.TCP)
srv:listen(666,function(conn)
    local i = 0
    conn:on("sent",function(conn)         
        print("packet sent")
        i=i+1
        conn:send(tostring(i) .. " 12345678901234567890\n")
        if i==500 then conn:close() end
    end)

    conn:on("receive",function(conn,payload)
        conn:send(tostring(i) .. " 12345678901234567890\n")
        -- conn:close()
    end)
end)


the code above is not tested.

- Tue Dec 09, 2014 11:56 am #4396 oops, sorry, I thought I already replied... maybe just missed 'submit' after preview...

Thanks a lot, your code example works (adjusted it to send a file). Do you think it would be possible to get more control over the sending process - like a socket.flush() or a retryable error condition?

Thomas

- Wed Dec 10, 2014 4:05 am #4465 Thomas, You are quite correct.

Based on what you mentioned in the other thread (viewtopic.php?f=19&t=702&start=30), it seems that any transmissions over about 2-3 packets long is messing with the TCP stack - probably buffer overflow as you mentioned or who know what else. I suppose when port forwarding, there are additional TCP headers generated which reduce the data throughput even further and bring on the issue when the html data is even smaller; hence the discrepancy between local client and remote client.

I did a lot of testing on this today and I think by breaking up the data into packets manually, similar to the example zeroday presented here, I have a decent workaround. Not fully implemented yet but my plan is to count the length of each string or line that is being sent and when the counter exceeds a limit of something like 800 bytes I'll stop transmitting and wait for the next conn:send call to post the next chunk of data until all data is sent at which point I'll close the connection.

I still would like to know what's happening behind the scenes in terms of when data via conn:send is actually being sent. Perhaps a delay everynow and then to allow the buffers to be flushed could suffice. I don't know...

Anyway, thanks for shedding some light on this. I was getting to the point of dumping Lua yesterday based on this issue and some other memory related problems and also random reboots.