ESP8266 Community Forum

- Sat Feb 13, 2016 9:14 am #41032 I've been experimenting with nodemcu during the last two weeks. I've created a captive portal to get a feel about the development environment and the API nodemcu offers. I did develop a very simple HTTP server for the captive portal. For that, I did use the net API. And I'm a bit confused about the part to receive data over TCPIP sockets. The basic code for the HTTP server looks like this:

Code: Select allfunction handleHttpRequest(client, request)
   ...
end

function setupHttpServer()
  srv = net.createServer(net.TCP);
  srv:listen(80, function(conn)
    conn:on("receive", handleHttpRequest);
  end);
end 


I do understand asynchronous API's, but TCPIP is in essence a streaming API. So, I'm confused about the data that the callback e.g. handleHttpRequest gives me. How much of the data that is on the socket is being offered in the call back ? If the data does not fit in the buffer(s), then what is happening with the rest of the data on the socket ?

E.g., the 1460 byte buffer did get filled completely by Windows 10 trying to connect to my AP. Windows 10 does send a GET request with about 4000 bytes of data in cookies. I noticed that the data given to me in the callback function was just a part of this. The callback was called about 3 times in a row, each time with a part of the data.

I'm not sure how - as a developer - I need to deal with that. How do I know which data belongs together ? And - even if I would would path the buffer length - TCPIP is a streaming protocol that delivers data in chunks. So, I would expect the data to be delivered in pieces from time to time.

Using a more classical API, I did solve this by blocking receive calls on the socket, or - in case a Lua - to yield from a coroutine until more data is available. Is something similar possible in nodemcu ? If not, then how am I to deal with receiving data over a socket in the 'correct' way ?

- Sat Feb 13, 2016 6:11 pm #41069 Hi wiffel, I'm no expert on this subject but seem to have got a web page file server working from the examples floating around here. I've never been aware of incoming http requests of any significant size like you describe.
If it's any help at all my typical receive callback starts like this:

Code: Select allsrv=net.createServer(net.TCP)
srv:listen(80,function(conn)
    conn:on("receive", function(conn,payload)
        local _, _, method, path, vars = string.find(payload, "([A-Z]+) (.+)?(.+) HTTP")
        if(method == nil)then
         _, _, method, path = string.find(payload, "([A-Z]+) (.+) HTTP")
        end
        print(method, path, vars)   --just while debugging

--snip


I've connected both using the ESP8266 in soft ap and station modes from a variety of smartphones and PCs including windows 10 and have never been aware of cookies or any spurious data other than the standard GET headers. I'm not sure if I'm understanding your issue properly.

Stick with it though, of all the frameworks I've tried nodemcu/lua has proved to be the most productive from me. I'm rather incremental in my approach and like to test almost one line at a time so the ability to upload .lua files and run them independently is a massive time-saver. One thing that I would recommend is using the dev branch of http://nodemcu-build.com/ to get an up-to-date firmware. This has many bug fixes - but beware early lua code examples you might scrape from the net - for example, people were coding-in multiple sequential sends to concatenate outgoing server data and it's never been valid and reliably falls over now.

- Sun Feb 14, 2016 6:20 pm #41126 Try the new http server module in the dev branch.

But my general recommendation is to the first ESP chip as an IoT device and not as an http server. I use an RPi2 running Apache as my user interface and the interface app talks to the ESPs using a simply UDP protocol.

K.I.S.S.

Please see my Developer FAQ for Q&A on nodeMCU Lua before asking a question here that's already been answered. This and the other official documentation should be your first reference for developing Lua Apps

- Mon Feb 15, 2016 6:35 am #41156 AdrianM, thanks for your reply.

Hi wiffel, I'm no expert on this subject but ...

I've connected both using the ESP8266 in soft ap and station modes from a variety of smartphones and PCs including windows 10 and have never been aware of cookies or any spurious data other than the standard GET headers. I'm not sure if I'm understanding your issue properly.

I've got about the same code structure as yours. That indeed works fine most of the time, since GET requests are typically very short. But not always. The example I gave was Windows 10 trying to connect to a captive portal (so not to a normal web server). In that request Windows 10 is sending a lot of information in the request headers. (By the way, I'm a bit worried about that. I'm not sure why Microsoft wants that amount of information when I'm connecting to an access point. But that is a different story all together).

More in general, I'm not sure how to deal with that. In this case, the data is delivered to my call back function in chunks without any other information. E.g. the chunks could come from different requests, from different computers, ... Your code (which is almost the same as mine) will crash in such an event, since only part of the GET request is delivered. The next call will contain the rest of the data and does not even have a GET header anymore.

Since - in TCPIP - this can also happen with shorter data streams, I'm wondering if I was just lucky up to now. Lucky that the requests did come in one go because they are short. But that does not give me a happy feeling, because then I'm just awaiting random crashes in the future.

Stick with it though, .....

I do like lua and nodeMcu. It's a lot easier to work with then the alternatives I've used in the past. That's why I'm looking for a good - and safe - way to deal with reading data from sockets. Without that, I've got a hard time using it for something that needs to be 'live' for a long period of time. And that would be a pity, since I'd rather use lua and nodeMcu instead of the alternatives.