Async webserver multipart response exceptions
Posted: Thu Feb 09, 2017 6:07 amHello everyone.
I am using an esp8266 as a web connected energy meter. As it will be in a switchboard it is a bit hard to get to and connect to a computer for a serial debugging. I am developing a replacement for the Serial module that can write to serial and also a file on the flash. The data (what was written to serial) i could then show on a page on the esp.
I am using the esp async web server to run the web serving part of it. For all other things (various pages, and web services that do not use the multipart response) it all seems OK.
The code to write to the file (appears at least) to be working OK, but the web service to retrieve the contents fails with an exception. Can anyone help me work out why it is failing?
Which exception is generated and which line of the code it is seems to vary, but a minimal application to trigger the error gives this stack trace:
the code to generate the exception is:
ino file:
header file:
cpp file
the buffer file which generated the error is here:
I am using an esp8266 as a web connected energy meter. As it will be in a switchboard it is a bit hard to get to and connect to a computer for a serial debugging. I am developing a replacement for the Serial module that can write to serial and also a file on the flash. The data (what was written to serial) i could then show on a page on the esp.
I am using the esp async web server to run the web serving part of it. For all other things (various pages, and web services that do not use the multipart response) it all seems OK.
The code to write to the file (appears at least) to be working OK, but the web service to retrieve the contents fails with an exception. Can anyone help me work out why it is failing?
Which exception is generated and which line of the code it is seems to vary, but a minimal application to trigger the error gives this stack trace:
Code: Select all
Exception 9: LoadStoreAlignmentCause: Load or store to an unaligned address
Decoding 34 results
0x4020cb96: fs::File::read() at E:\Users\Laura\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266/FS.cpp line 188
0x4020cb9d: fs::File::read() at E:\Users\Laura\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266/FS.cpp line 188
0x4020cc41: fs::File::seek(unsigned int, fs::SeekMode) at E:\Users\Laura\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266/FS.cpp line 188
0x40207374: operator() at E:\Users\Laura\Documents\Arduino\libraries\ProgressDevice/ProgressDevice.cpp line 207
: (inlined by) _M_invoke at c:\users\laura\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\1.20.0-26-gb404fb9-2\xtensa-lx106-elf\include\c++\4.8.2/functional line 2057
0x40209450: AsyncWebServerRequest::_onData(void*, unsigned int) at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncWebServer-master\src/WebRequest.cpp line 701
0x40209450: AsyncWebServerRequest::_onData(void*, unsigned int) at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncWebServer-master\src/WebRequest.cpp line 701
0x402099b5: AsyncChunkedResponse::_fillBuffer(unsigned char*, unsigned int) at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncWebServer-master\src/WebResponseImpl.h line 101
0x40103115: lmacProcessTXStartData at ?? line ?
0x40209f52: AsyncAbstractResponse::_ack(AsyncWebServerRequest*, unsigned int, unsigned int) at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncWebServer-master\src/WebResponseImpl.h line 101
0x401020f2: wDev_ProcessFiq at ?? line ?
0x40107984: vPortFree at E:\Users\Laura\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266/heap.c line 39
0x4022554a: pbuf_free at /Users/ficeto/Documents/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip/src/core/pbuf.c line 752
0x40207f5d: AsyncWebServerRequest::_onPoll() at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncWebServer-master\src/WebRequest.cpp line 701
0x4010086a: ppProcessTxQ at ?? line ?
0x40207f74: _M_invoke at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncWebServer-master\src/WebRequest.cpp line 701
0x402077ee: std::function ::operator()(void*, AsyncClient*) const at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncTCP-master\src/ESPAsyncTCP.cpp line 632
0x40207b1b: AsyncClient::_poll(tcp_pcb*) at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncTCP-master\src/ESPAsyncTCP.cpp line 632
0x40207b30: AsyncClient::_s_poll(void*, tcp_pcb*) at E:\Users\Laura\Documents\Arduino\libraries\ESPAsyncTCP-master\src/ESPAsyncTCP.cpp line 632
0x40226100: tcp_slowtmr at /Users/ficeto/Documents/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip/src/core/tcp.c line 967 (discriminator 1)
0x401004d8: malloc at E:\Users\Laura\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\umm_malloc/umm_malloc.c line 1664
0x402262f4: tcp_tmr at /Users/ficeto/Documents/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip/src/core/tcp.c line 127
0x40227744: tcpip_tcp_timer at /Users/ficeto/Documents/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip/src/core/timers.c line 87
0x4021fa72: sta_input at ?? line ?
0x402278e5: sys_check_timeouts at /Users/ficeto/Documents/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip/src/core/timers.c line 420
0x4022773c: tcpip_tcp_timer at /Users/ficeto/Documents/Arduino/hardware/esp8266com/esp8266/tools/sdk/lwip/src/core/timers.c line 81
0x401040f6: lmacTxFrame at ?? line ?
0x402318d8: ets_timer_handler_isr at ?? line ?
0x402318e5: ets_timer_handler_isr at ?? line ?
0x4023192a: ets_timer_handler_isr at ?? line ?
0x40231527: ets_snprintf at ?? line ?
the code to generate the exception is:
ino file:
Code: Select all
#include <ESP8266WiFi.h>
#include "ProgressDevice.h"
#include <ESPAsyncTCP.h>
#include <ESPAsyncWebServer.h>
#include <FS.h>
const char *ssid = "D+THome";
const char *pass = "Sponge75";
AsyncWebServer server(80);
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
SPIFFS.begin();
prog.start();
prog.println("joining ap");
WiFi.mode(WIFI_STA);
WiFi.begin(ssid,pass);
while (WiFi.status() != WL_CONNECTED){
prog.print('.');
delay(500);
}
prog.println("connected");
prog.println("IP address: ");
prog.println(WiFi.localIP());
server.on("/data/log", HTTP_ANY, &servicelog);
server.begin();
}
void loop() {
// put your main code here, to run repeatedly:
yield();
}
void servicelog(AsyncWebServerRequest *request){
/*
* Log data request
*/
String req=request->url();
if (req.endsWith("all")){
//we wnat all data
prog.getdata(request, false);
} else {
//just data since restart
prog.getdata(request, true);
}
}header file:
Code: Select all
#ifndef ProgressDevice_h
#define ProgressDevice_h
#include "Arduino.h"
#include <FS.h>
#include "LoggerUtils.h"
#include <ESPAsyncTCP.h>
#include <ESPAsyncWebServer.h>
//defines
//filelen is how big the buffer is. we make it 56kB
#define FILELEN 57344
//header len is how big the header is. make it 16 bytes to allow for future expansion
#define FHDRLEN 16
//address of current index
#define ADDRIDX 0
//address of enable flag
#define ADDRFLG 2
//progress file name
#define PROGFN "/progress/progress.buf"
//length of RAM buffer
#define FBUFLEN 160
//max number of NULLS when returning data until we assume file is empty
#define MAXNULL 32
class ProgressDevice : public Print {
public:
ProgressDevice();
void start(); //initialise the device.
//write methods
virtual size_t write(uint8_t); //write a char. this is called by the various print methods.
//other small macros to convert write
inline size_t write(unsigned long n) { return write((uint8_t)n); }
inline size_t write(long n) { return write((uint8_t)n); }
inline size_t write(unsigned int n) { return write((uint8_t)n); }
inline size_t write(int n) { return write((uint8_t)n); }
using Print::write; //use all the other print write commands
//get data
void getdata(
AsyncWebServerRequest *request,
bool sincerestart=true
); //get data from the file. sincerestart defines wheter it gets data since last restart (true) or all (false)
//config file
bool readcfg(
const char* settingname,
const char* settingvalue
); //takes a configuration setting and saves it if it is the one we wnat
void writecfg(File f); //write config settings to config file
void getcfg(File f); //get the config settings and put them in a JSON object
//file and settings to go here
private:
void initfile(); //initialise output file
unsigned int readui(File f, unsigned int address);
//read unsigned int from database
void writeui(File f, unsigned int address, unsigned int data);
//write unsigned int to database
void writebyte(uint8_t c); //write a byte to buffer (and file if required)
void writebuf(); //write contents of buffer to file
bool serial=true; //default to writing to serial
bool file=false; //default to no file (but check on start)
unsigned int pointer=0; //pointer to file position to write next byte
unsigned int startpointer=0; //where we were at the start of the process
char buffer[FBUFLEN]; //write buffer. temporary storage of data
//vars for data retrieval
bool readfile=true; //this flag is whether we are reading from a file or ram buffer
bool done=false; //flag for if we are done
unsigned int fp; //file pointer, where we are in the file
};
extern ProgressDevice prog;
#endifcpp file
Code: Select all
#include "Arduino.h"
#include "ProgressDevice.h"
#include <FS.h>
#include "LoggerUtils.h"
#include <ESPAsyncTCP.h>
#include <ESPAsyncWebServer.h>
ProgressDevice::ProgressDevice():Print(){
/*
initialiser
*/
//clear buffer
buffer[0]='\0';
}
void ProgressDevice::start(){
/*
Start the progress device. Basically what we want to do is detect
if we have file enabled.
we work this out by looking in the output file at address ADDRFLAG
to see if it is a '0' or something else.
*/
unsigned char flg;
File f;
//first, if the file doesn't exist, we assume flag remains false
if (SPIFFS.exists(PROGFN)){
//file exists, so check the flag
f=SPIFFS.open(PROGFN,"r");
//go to file location
f.seek(ADDRFLG, SeekSet);
//read flag
flg=f.read();
//set flag
if (flg!=0){
//we want to enable
file=true;
//now initialise file
initfile();
}
}
}
void ProgressDevice::initfile(){
/*
initialise file for write
what we do:
- if file exists:
- read pointer
- set restart pointer to current file position
- if file doesn't exits
- create file
- set pointers to beginning of file
*/
File f;
//first, if we have a pointer, we have already done this, so just return
if (pointer>0){ return; }
if (SPIFFS.exists(PROGFN)){
//file exists, so read
f=SPIFFS.open(PROGFN,"r");
//read pointer in
pointer=readui(f,ADDRIDX);
//can close file
f.close();
//we stated here too, os save that
startpointer=pointer;
} else {
//first, create the file
f=SPIFFS.open(PROGFN,"w+");
//now fill the file with nulls
for (unsigned int i=0; i<FILELEN; i++){
f.write('\0');
}
//set pointer to FHDRLEN, which is hte first byte we write to
writeui(f,ADDRIDX,FHDRLEN);
//set enable flag to '1'
f.seek(ADDRFLG, SeekSet);
f.write(1);
//close file
f.close();
//set pointers
pointer=FHDRLEN;
startpointer=FHDRLEN;
}
}
size_t ProgressDevice::write(uint8_t c){
/*
write a byte
*/
if (serial){
//pass byte on to serial
Serial.write(c);
}
if (file){
//write to file
writebyte(c);
}
return 1; //aways say we wrote. should probably fix that eventually
}
void ProgressDevice::writebyte(uint8_t c){
/*
write a byte to the file
Buffer text goes here******************************************
*/
//first, append to buffer
unsigned char x=strlen(buffer);
buffer[x]=c;
buffer[x+1]='\0';
//now there are two cases which might cause us to write:
// 1: The character is a newline (\n) or null (\0)
// 2: The buffer is full
//||(c=='\n')||(c=='\0')){
if ((x>=(FBUFLEN-1))){
writebuf();
}
}
void ProgressDevice::writebuf(){
/*
Write contents of buffer to file.
*/
File f;
if (SPIFFS.exists(PROGFN)){
//file exists, so we can use it
//open it
f=SPIFFS.open(PROGFN,"r+");
//write the character array
for (unsigned char i=0;i<strlen(buffer);i++){
//write byte
f.seek(pointer, SeekSet);
f.write(buffer[i]);
//new pointer
pointer++;
//now check if we need to rollover
if (pointer>FILELEN){
pointer=FHDRLEN;
}
//Now if the pointer is the same location as the start pointer
//we want to increment it
if (startpointer==pointer){
startpointer++;
if (startpointer>FILELEN){
startpointer=FHDRLEN;
}
}
}
//write pointer
writeui(f,ADDRIDX,pointer);
//close file
f.close();
//clear buffer
buffer[0]='\0';
}
}
void ProgressDevice::getdata(AsyncWebServerRequest *request, bool sincerestart){
/*
Get the data from the file
*/
//first, set the vars
//set flags
readfile=true;
done=false;
//now file pointer
if (sincerestart){
//we want values since restart
fp=startpointer;
} else {
//we want all of the file
fp=pointer+1;
//now check if we need to wrap around (we might happen to be on the first entry)
if (fp>FILELEN){ fp=FHDRLEN; }
}
AsyncWebServerResponse *response = request->beginChunkedResponse("text/html", [&](uint8_t *rbuffer, size_t maxlen, size_t index) -> size_t {
/*
write to response
*/
//return if done
if (done){ return 0; }
char b;
unsigned char nulls=0; //this is how many nulls in a row we have.
//if this exceeds MAXNULL we go to start of file
unsigned int rlen=0; //how much data we are rturning
File f;
//open file if we are in file mode
if (readfile){
//Serial.println(F("File mode"));
f=SPIFFS.open(PROGFN,"r+"); //open in read mode
if (!f){
//Serial.println(F("Error opening file!"));
//issue opening file
buffer[0]='\0';
return 1;
}
//Serial.println(F("File opened"));
}
f.seek(fp, SeekSet);//seek to correct point
for (unsigned int i=0; i<(maxlen-5); i++){
//loop until we run out of data or buffer space
if (readfile){
//file mode, so read data
b=f.read();
if (b!='\0'){
//ignore nulls
rbuffer[i]=b;
rbuffer[i+1]='\0';
nulls=0;
} else {
//determine if we go to start of file
nulls++;
if (nulls>MAXNULL){
Serial.println(F("too many nulls"));
fp=FHDRLEN;
f.seek(fp, SeekSet);//seek to correct point
}
}
//increment pointer
fp++;
if (fp>FILELEN){
fp=FHDRLEN;
f.seek(fp, SeekSet);//seek to correct point
Serial.println(F("Wrap around"));
}
//now determine if we are done with file
if (fp==pointer){
readfile=false;
fp=0;
Serial.println(F("buffer mode"));
}
rlen++;
} else{
//buffer mode
//return a character
rbuffer[i]=buffer[fp];
rbuffer[i+1]='\0';
fp++;
rlen++;
if (fp==strlen(buffer)){
//out of data
done=true;
//and break from loop
//Serial.println(F("Done"));
break;
}
}
}
//Serial.println(String(rlen)+" : "+String(maxlen));
if(f){ f.close(); }
//return
return rlen;
});
response->addHeader("Server","ESP Async Web Server");
request->send(response);
}
bool ProgressDevice::readcfg(const char* settingname, const char* settingvalue){
/*
Read the configuration setting in. returns true if it is a valid setting.
settings are:
- enable serial output
- enable file output
*/
bool set=false;
if (strcmp(settingname,"progenserial")==0){
if (strcmp(settingvalue,"true")==0){
serial=true;
} else {
serial=false;
}
set=true;
} else if (strcmp(settingname,"progenfile")==0){
if (strcmp(settingvalue,"true")==0){
file=true;
//we also need to init the file
initfile();
} else {
//TODO: Update fiule--------------------------------------
file=false;
}
set=true;
}
return set;
}
void ProgressDevice::writecfg(File f){
/*
write configuration file
*/
unsigned char i;
char buf[10];
f.print(F("progenserial:"));
i=0;
if (serial){
f.print(F("true"));
} else {
f.print(F("false"));
}
f.print(F("\nprogenfile:"));
i=0;
if (file){
f.print(F("true"));
} else {
f.print(F("false"));
}
}
void ProgressDevice::getcfg(File f){
/*
write settings in json format to a file
*/
f.print(F("\"Progress Device\":{"));
writesettingbool(f,"progenserial","bool",serial,"Enable serial port");
f.write(',');
writesettingbool(f,"progenfile","bool",file,"Enable file");
//close brackets
f.write('}');
}
unsigned int ProgressDevice::readui(File f, unsigned int address){
/*
read a unsigned int from the address
*/
//first, union lets the vars share memory. we read into this
unsigned char b;
union {
unsigned int o;
unsigned char bytes[2];
} rd;
//seek to location
f.seek(address, SeekSet);
//now read data
for (unsigned char i=0; i<2; i++){
b=f.read();
rd.bytes[i]=b;
f.seek(address+i+1, SeekSet); //need to seek between writes it seems
}
//return
return rd.o;
}
void ProgressDevice::writeui(File f, unsigned int address, unsigned int data){
/*
write an unsigned int to the database
*/
//first, union lets the vars share memory. we write this
int res, res1;
union {
unsigned int o;
unsigned char bytes[2];
} rd;
rd.o=data;
res=f.seek(address, SeekSet);
//now write data
for (unsigned char i=0; i<2; i++){
res1=f.write(rd.bytes[i]);
res=f.seek(address+i+1, SeekSet); //need to seek between writes it seems
}
}
//create a buffer for use elsewhere
ProgressDevice prog;the buffer file which generated the error is here: