ESP8266WebServer to send larger amounts of data
Posted: Fri Apr 10, 2015 12:40 pmI have a project that involves transmitting larger amounts of HTML or images from the ESP8266 Arduino Web Server.
I noticed that both server.send() and client.write() were truncating my data, so I did some experimentation to figure out the best way to do this.
I used the example sketch as my base, and added some extra code to reliably send large responses to a client.
I initially tried to store my data in "PROGMEM" before realizing that the ESP8266 does not make a distinction between different types of memory, like the smaller platforms that Arduino has targetted, so you'll see some of the code reference PROGMEM but it isn't really necessary. All the data is actually just sitting in regular RAM. I was concerned about that, but a minimal sketch tells me I have 38k of heap available, so having 10-15k of HTML/image in memory isn't that concerning after all.
I also ran some performance statistics. It seems like every call to client.write() costs about 125 milliseconds, regardless of the amout of data transmitted. So if you want to send a bunch of data, it's better to buffer it and send it in big chunks. The largest amount of data that client.write() will transmit appears to be 2920 bytes.
This version of BBCode doesn't support tables, sorry. All the following stats are for transmitting the same 8344 byte PNG file with various block sizes.
2920 bytes/block; 3 blocks; 0.4198 seconds total transmission time
2048 bytes/block; 5 blocks; 0.6378 seconds total transmission time
1024 bytes/block; 9 blocks; 1.1282 seconds total transmission time
512 bytes/block; 17 blocks; 2.1300 seconds total transmission time
256 bytes/block; 33 blocks; 4.1289 seconds total transmission time
As you can see, sending smaller the data in smaller chunks SIGNIFICANTLY increases the transmission time. less than half a second to over 4 seconds for the same total amount of data!
Another trick I'm using is how to store long string literals (including line breaks, special characters, etc, in a variable. I haven't seen this mentioned elsewhere on the forum, so it may be worth a look if you want to store/send large blocks of static HTML. In another project, I'm using it to serve about 6-10k of static HTML with a lot of embedded javascript with no problems. Here is an example (same as the attached zip, but since you can't see attachments if you're not logged in, I will re-state it here...):
Basically everything between the two delimiters 'R"=====(' and ')====="' will go into your variable without being parsed for special characters or anything.
For the image file, I just wrote a tiny python script that read the file in byte by byte and spit out to the console each byte in decimal, separated by a comma. Then I copy/pasted that into a variable for the Arduino sketch. Sorry, I didn't think to save a copy of the python script...
I noticed that both server.send() and client.write() were truncating my data, so I did some experimentation to figure out the best way to do this.
I used the example sketch as my base, and added some extra code to reliably send large responses to a client.
I initially tried to store my data in "PROGMEM" before realizing that the ESP8266 does not make a distinction between different types of memory, like the smaller platforms that Arduino has targetted, so you'll see some of the code reference PROGMEM but it isn't really necessary. All the data is actually just sitting in regular RAM. I was concerned about that, but a minimal sketch tells me I have 38k of heap available, so having 10-15k of HTML/image in memory isn't that concerning after all.
I also ran some performance statistics. It seems like every call to client.write() costs about 125 milliseconds, regardless of the amout of data transmitted. So if you want to send a bunch of data, it's better to buffer it and send it in big chunks. The largest amount of data that client.write() will transmit appears to be 2920 bytes.
This version of BBCode doesn't support tables, sorry. All the following stats are for transmitting the same 8344 byte PNG file with various block sizes.
2920 bytes/block; 3 blocks; 0.4198 seconds total transmission time
2048 bytes/block; 5 blocks; 0.6378 seconds total transmission time
1024 bytes/block; 9 blocks; 1.1282 seconds total transmission time
512 bytes/block; 17 blocks; 2.1300 seconds total transmission time
256 bytes/block; 33 blocks; 4.1289 seconds total transmission time
As you can see, sending smaller the data in smaller chunks SIGNIFICANTLY increases the transmission time. less than half a second to over 4 seconds for the same total amount of data!
Another trick I'm using is how to store long string literals (including line breaks, special characters, etc, in a variable. I haven't seen this mentioned elsewhere on the forum, so it may be worth a look if you want to store/send large blocks of static HTML. In another project, I'm using it to serve about 6-10k of static HTML with a lot of embedded javascript with no problems. Here is an example (same as the attached zip, but since you can't see attachments if you're not logged in, I will re-state it here...):
Code: Select all
const char index_html[] PROGMEM = R"=====(
<!DOCTYPE HTML>
<html>
<head><title>ESP8266 Arduino Demo Page</title></head>
<body>ESP8266 power!<p><img src="logo.png"></body>
</html>
)=====";
Basically everything between the two delimiters 'R"=====(' and ')====="' will go into your variable without being parsed for special characters or anything.
For the image file, I just wrote a tiny python script that read the file in byte by byte and spit out to the console each byte in decimal, separated by a comma. Then I copy/pasted that into a variable for the Arduino sketch. Sorry, I didn't think to save a copy of the python script...