-->
Page 1 of 1

Downloading Large File using HTTPClient and WiFiClient crash

PostPosted: Fri Mar 09, 2018 4:08 pm
by EmbeddedBob
I want to download files to my ESP8266 from a SSL/TLS protected site.

I am started from the HTTP over TLS (HTTPS) example sketch and got it working great downloading a tiny 3 byte file. When I step the file up to a larger file like around 100,000 bytes I get a crash. The crash seems to happen a tiny bit of time after the GET call. If I comment out all calls to access the HTTPClient object after the GET I don't get a crash. If I comment out just the http.getStreamPtr(); I still get a crash seemingly on the.End(). Nothing seems to help. I can call the http.getSize(); 5 times is a row and no crash but any other call after that crashes.

Would the contents of the file possibly cause the HTTPClient to crash?
Could I be running out of memory and the http objects gets killed then any attempt to call any http.xxx() causes the de-reference to blow? What can I try to narrow it down?

Code: Select all/*
    HTTP over TLS (HTTPS) example sketch
    This example demonstrates how to use
    WiFiClientSecure class to access HTTPS API.
    We fetch and display the status of
    esp8266/Arduino project continuous integration
    build.
    Limitations:
      only RSA certificates
      no support of Perfect Forward Secrecy (PFS)
      TLSv1.2 is supported since version 2.4.0-rc1
    Created by Ivan Grokhotkov, 2015.
    This example is in public domain.
*/

#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <ESP8266HTTPClient.h>

const char* ssid = "xxxx";
const char* password = "xxxx";
const char* host = "test.test.com";
const int httpsPort = 8080;

// Use web browser to view and copy
// SHA1 fingerprint of the certificate
const char* fingerprint = "35 85 74 EF 67 35 A7 CE 40 69 50 F3 C0 F6 80 CF 80 3B 2E 19";
uint8_t buff[128] = { 0 };
int len;
HTTPClient http;
WiFiClientSecure client;
WiFiClient* stream;
 
void setup() {
  Serial.begin(115200);
  Serial.println();
  Serial.print("connecting to ");
  Serial.println(ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

  len=0; 
  Serial.print("connecting to ");
  Serial.println(host);
  if (!client.connect(host, httpsPort)) {
    Serial.println("connection failed");
    return;
  }

  if (client.verify(fingerprint, host)) {
    Serial.println("certificate matches");
  } else {
    Serial.println("certificate doesn't match");
  }

  http.begin("https://test.test.com/app/upload/common/platform/recovery/2_1/test.bin" , fingerprint);

  String postMessage = "";
  http.setReuse(true);
  int httpCode = http.POST(postMessage);
  if(httpCode == 200) {
    Serial.print("http result:");
    Serial.println(httpCode);
    Serial.println("File Size: ");
    len = http.getSize();
    Serial.println(len);
    if(http.connected()) {
      Serial.println("Getting Stream");
      stream = http.getStreamPtr();     
      Serial.println("Got Stream");
    }
    Serial.println("After Stream");   
    http.end();
  }
  else {
    Serial.println("Call Failed");
  }
}

void loop() {
}



The output from this is:
.wifi evt: 2
...wifi evt: 0
......wifi evt: 3
.
WiFi connected
IP address:
192.168.1.91
connecting to xxxx
[hostByName] request IP for: xxxx
[hostByName] Host: xxxx IP: xxxx
certificate matches
[HTTP-Client][begin] url: https://test.test.com/app/upload/common ... 1/test.bin
[HTTP-Client][begin] host: xxxx port: 443 url: /app/upload/common/platform/recovery/2_1/test.bin
[HTTP-Client][begin] httpsFingerprint: xxxx
[hostByName] request IP for: test.test.com
[hostByName] Host: xxxx IP: xxxx
[HTTP-Client] connected to test.test.com:443
[HTTP-Client][handleHeaderResponse] RX: 'HTTP/1.1 200 OK'
[HTTP-Client][handleHeaderResponse] RX: 'Date: Fri, 09 Mar 2018 20:48:26 GMT'
[HTTP-Client][handleHeaderResponse] RX: 'Server: Apache/2.4.18 (Ubuntu)'
[HTTP-Client][handleHeaderResponse] RX: 'Last-Modified: Wed, 07 Mar 2018 15:59:13 GMT'
[HTTP-Client][handleHeaderResponse] RX: 'ETag: "1a36e-566d4a3f2a34e"'
[HTTP-Client][handleHeaderResponse] RX: 'Accept-Ranges: bytes'
[HTTP-Client][handleHeaderResponse] RX: 'Content-Length: 107374'
[HTTP-Client][handleHeaderResponse] RX: 'Content-Security-Policy: default-src 'none';form-action 'self'; ....[redacted]
[HTTP-Client][handleHeaderResponse] RX: 'X-Content-Type-Options: nosniff'
[HTTP-Client][handleHeaderResponse] RX: 'Strict-Transport-Security: max-age=63072000; includeSubDomains; preload'
[HTTP-Client][handleHeaderResponse] RX: 'X-Frame-Options: DENY'
[HTTP-Client][handleHeaderResponse] RX: 'X-XSS-Protection: 1; mode=block'
[HTTP-Client][handleHeaderResponse] RX: 'Keep-Alive: timeout=5, max=100'
[HTTP-Client][handleHeaderResponse] RX: 'Connection: Keep-Alive'
[HTTP-Client][handleHeaderResponse] RX: 'Content-Type: application/octet-stream'
[HTTP-Client][handleHeaderResponse] RX: ''
[HTTP-Client][handleHeaderResponse] code: 200
[HTTP-Client][handleHeaderResponse] size: 107374
http result:200
File Size:
107374

Exception (3):
epc1=0x4010044e epc2=0x00000000 epc3=0x00000000 excvaddr=0x40248254 depc=0x00000000

ctx: cont
sp: 3fff1230 end: 3fff1500 offset: 01a0

>>>stack>>>
3fff13d0: 4024a34e 00000000 3fff476c 4021022c
3fff13e0: 6c696166 74206465 7267206f 7020776f
3fff13f0: 6e69616c 66756220 0d726566 4020000a
3fff1400: 3fff143a 00000000 00000000 00000000
3fff1410: 3fff1460 39322039 00000005 3fff296c
3fff1420: 3fff0484 00000d4b 3fff697c 4010053a
3fff1430: 00000000 00000000 3fff476c 40100990
3fff1440: 00000000 3fff296c 3fff476c 4020ffd9
3fff1450: 000000c8 3fff296c fffffefb 40203671
3fff1460: 00000000 00000006 3fff04a4 40204ee5
3fff1470: 000000c8 3fff23fc 00000000 4020382d
3fff1480: 3ffe8b44 3fff01f0 00000000 4020387b
3fff1490: 000000c8 3fff01f0 3fff01f0 4020622a
3fff14a0: 3fff01d4 3fff01f0 3fff04a4 4020233c
3fff14b0: 00000000 00000000 00000000 3fff60e4
3fff14c0: 0000000f 00000000 feefeffe feefeffe
3fff14d0: feefeffe feefeffe feefeffe 3fff04d0
3fff14e0: 3fffdad0 00000000 3fff04c8 40205e17
3fff14f0: feefeffe feefeffe 3fff04e0 40100a20
<<<stack<<<

ets Jan 8 2013,rst cause:2, boot mode:(1,0)


ets Jan 8 2013,rst cause:4, boot mode:(1,0)

wdt reset

Re: Downloading Large File using HTTPClient and WiFiClient c

PostPosted: Fri Jun 08, 2018 8:47 am
by EmbeddedBob
So the answer was that I was running out of frame space. The larger the file, the more space it took and I ran out. Not a very graceful fail but once I cleaned up my code and made sure variables that were declared inside of functions were properly sized or cleaned up, it worked much better.