Reliable Wifi network of 100+ Wemos D1 Mini Pro
Posted: Tue Aug 18, 2020 10:46 pmI am working on a project that involves 100+ Wemos D1 Mini Pro controllers, spread across a warehouse. Each Wemos is acting as a monitor/logger, logging some environmental data to its onboard SPIFFS every 5 seconds, which is working fine. I need to be able to download the log files from each Wemos, both individually and as a group, which means they need to be on the same WiFinetwork.
So to test this on a small scale, I setup 3 Wemos boards which were happily logging away. I have a mini PC running a simple PHP webserver which I plug into my wifi router. When each Wemos is turned on, it connects to the WiFi and then sends a request to the webserver (it has a fixed IP) to register itself with the server. The webserver stores its IP address. I can then connect to the webserver over WiFi from any browser (on the LAN, not over the internet) and the webserver displays a page showing all Wemos controllers and their IPs. I can click on a "get log" button next to any controller to send a request to a /getlog endpoint on the controller. I can also tick checkboxes next to multiple controllers on the page to send the "get log" request to all of them at once. The sketch returns the log file like this:
So first question: The communication from webserver to the Wemos controllers over WiFi is currently using curl commands... the html page is using JavaScript to post to a PHP script which builds curl commands for each wemos IP to send them a request over the network... while it works on this small scale (the browser downloads the log file) it feels like an unreliable hack and Im hoping someone has a better idea.
JavaScript:
PHP:
Second question: Is this a reliable way to return the log file from SPIFFS over WiFi to the webserver, which in turn returns it to the browser as a file download? Is there a more reliable way? The file will grow over time, up to maybe 10Mb in size, so I'm worried about reliably getting the log file from the Wemos to the browser over WiFi. I'll also need to automate this so the webserver proactively polls each controller for its log file every minute maybe, as in case a controller dies I need to ensure I don't lose its onboard log. It would be nice if the controllers just sent their last 1 min worth of data, rather than the full file. Anyone know how to do that? i.e. return chunks of data from a SPIFFS txt file?
Third question: As there can be 100+ of these controllers on the WiFi network, is the above setup the best way to tackle this problem at scale? I was planning on have a set of Ubiquiti AP's chained around the warehouse to ensure each Wemos can connect to the WiFi network.
All input, opinions, feedback and ideas very much welcome. Thanks.
So to test this on a small scale, I setup 3 Wemos boards which were happily logging away. I have a mini PC running a simple PHP webserver which I plug into my wifi router. When each Wemos is turned on, it connects to the WiFi and then sends a request to the webserver (it has a fixed IP) to register itself with the server. The webserver stores its IP address. I can then connect to the webserver over WiFi from any browser (on the LAN, not over the internet) and the webserver displays a page showing all Wemos controllers and their IPs. I can click on a "get log" button next to any controller to send a request to a /getlog endpoint on the controller. I can also tick checkboxes next to multiple controllers on the page to send the "get log" request to all of them at once. The sketch returns the log file like this:
Code: Select all
ESP8266WebServer server(80);
...
...
File f = SPIFFS.open("/log.txt", "r");
server.sendHeader("Content-Type", "text/text");
server.sendHeader("Content-Disposition", "attachment; filename=log.txt");
server.sendHeader("Connection", "close");
server.streamFile(f, "application/octet-stream");
f.close();So first question: The communication from webserver to the Wemos controllers over WiFi is currently using curl commands... the html page is using JavaScript to post to a PHP script which builds curl commands for each wemos IP to send them a request over the network... while it works on this small scale (the browser downloads the log file) it feels like an unreliable hack and Im hoping someone has a better idea.
JavaScript:
Code: Select all
$.post("getLog.php", {controller: controllerIP}, function(theResponse) {
});PHP:
Code: Select all
$curlcmd = "curl " . $_POST["controller"] . "/logs";
passthru($curlcmd, $error);
exit();Second question: Is this a reliable way to return the log file from SPIFFS over WiFi to the webserver, which in turn returns it to the browser as a file download? Is there a more reliable way? The file will grow over time, up to maybe 10Mb in size, so I'm worried about reliably getting the log file from the Wemos to the browser over WiFi. I'll also need to automate this so the webserver proactively polls each controller for its log file every minute maybe, as in case a controller dies I need to ensure I don't lose its onboard log. It would be nice if the controllers just sent their last 1 min worth of data, rather than the full file. Anyone know how to do that? i.e. return chunks of data from a SPIFFS txt file?
Third question: As there can be 100+ of these controllers on the WiFi network, is the above setup the best way to tackle this problem at scale? I was planning on have a set of Ubiquiti AP's chained around the warehouse to ensure each Wemos can connect to the WiFi network.
All input, opinions, feedback and ideas very much welcome. Thanks.