ESP stop parsing JSON, heap RAM keeps getting smaller
Posted: Thu Jan 24, 2019 10:05 amHi,
I am trying to use the twitter api library or any other social media api to receive stats.
Everything seems ok at the start but at some point it stops receiving the json message. It happens with different requests, not only twitter (Facebook, etc.).
I have checked if there is a RAM problem and it seems like every time there is a request the heap RAM keeps getting smaller to the point that it doesn't parse anymore. I have also checked if the ESP has been disconnected but it keeps connected.
I have also tried to use StaticjsonBuffer and clearing the jsonbuffer but it doesn't change.
This is my code:
This is what I get in the Serial Monitor:
According to an Arduino JSON assistant, the size of the JSON message is:
ESP32/ESP8266 1464+1994 = 3458 Bytes
I really don't know what else to do. The instagram library for Arduino works fine but it uses JsonStreamingParser.h. Would it be possible to use that library instead? Will it change the result?
It does seem like once the internal Free Heap RAM gets to around 7000, it doesn't work anymore.
I am also calling the function every 5 seconds. I have tried to increase the time but seems like it happens the same at some point. It also seems like the Free Heap RAM increases a bit too.
Thanks
I am trying to use the twitter api library or any other social media api to receive stats.
Everything seems ok at the start but at some point it stops receiving the json message. It happens with different requests, not only twitter (Facebook, etc.).
I have checked if there is a RAM problem and it seems like every time there is a request the heap RAM keeps getting smaller to the point that it doesn't parse anymore. I have also checked if the ESP has been disconnected but it keeps connected.
I have also tried to use StaticjsonBuffer and clearing the jsonbuffer but it doesn't change.
This is my code:
Code: Select all
#include <TwitterApi.h>
/*******************************************************************
* An Example of getting data back from the TwitterAPI *
* https://github.com/witnessmenow/arduino-twitter-api *
* *
* Written by Brian Lough *
*******************************************************************/
#include <WiFi.h>
#include <WiFiClientSecure.h>
//------- Replace the following! ------
char ssid[] = "ssss"; // your network SSID (name)
char password[] = "kkkkk"; // your network key
// Normally we would use these to generate the bearer token but its not working yet :/
// Use steps on the readme to generate the Bearer Token
#define BEARER_TOKEN "zzzzz"
//Using curl to get bearer token
// curl -u "$CONSUMER_KEY:$CONSUMER_SECRET" \
// --data 'grant_type=client_credentials' \
// 'https://api.twitter.com/oauth2/token'
WiFiClientSecure client;
TwitterApi api(client);
unsigned long api_mtbs = 5000; //mean time between api requests
unsigned long api_lasttime = 0; //last time api request has been done
bool firstTime = true;
//Inputs
String screenName = "xxx";
bool haveBearerToken = false;
void setup() {
Serial.begin(115200);
// Set WiFi to station mode and disconnect from an AP if it was Previously
// connected
WiFi.mode(WIFI_STA);
WiFi.disconnect();
delay(100);
// Attempt to connect to Wifi network:
Serial.print("Connecting Wifi: ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
Serial.print(".");
delay(500);
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
IPAddress ip = WiFi.localIP();
Serial.println(ip);
Serial.printf("Internal Total heap %d, internal Free Heap %d\n",ESP.getHeapSize(),ESP.getFreeHeap());
Serial.printf("SPIRam Total heap %d, SPIRam Free Heap %d\n",ESP.getPsramSize(),ESP.getFreePsram());
api.setBearerToken(BEARER_TOKEN);
haveBearerToken = true;
getTwitterStats(screenName);
// uint8_t* ps=(byte *)ps_malloc(900000);
}
void getTwitterStats(String name) {
Serial.println("Getting twitter stats for " + name);
String responseString = api.getUserStatistics(name);
Serial.println(responseString);
DynamicJsonBuffer jsonBuffer;
JsonObject& response = jsonBuffer.parseObject(responseString);
if (response.success()) {
Serial.println("parsed Json");
// Use Arduino Json to parse the data
int followerCount = response["followers_count"];
Serial.println(followerCount);
} else {
Serial.println("Failed to parse Json");
}
Serial.printf("Internal Total heap %d, internal Free Heap %d\n",ESP.getHeapSize(),ESP.getFreeHeap());
Serial.printf("SPIRam Total heap %d, SPIRam Free Heap %d\n",ESP.getPsramSize(),ESP.getFreePsram());
}
void loop() {
if(haveBearerToken){
if ((millis() > api_lasttime + api_mtbs)) {
getTwitterStats(screenName);
api_lasttime = millis();
if (WiFi.status() == WL_CONNECTED) {
Serial.println("Still connected");
}
}
}
}This is what I get in the Serial Monitor:
Code: Select all
Connecting Wifi: sssss
...
WiFi connected
IP address:
192.168.1.xxx
Internal Total heap 337244, internal Free Heap 282376
SPIRam Total heap 4194116, SPIRam Free Heap 4190924
Getting twitter stats for xxxx
{"......very long Json......."}
parsed Json
219
Internal Total heap 336328, internal Free Heap 234060
SPIRam Total heap 4194100, SPIRam Free Heap 4190788
ps=0x3f801288
Getting twitter stats for xxxx
Failed to parse Json
Internal Total heap 336900, internal Free Heap 247180
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxx
{"......very long Json......."}
parsed Json
219
Internal Total heap 336148, internal Free Heap 199752
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxx
Failed to parse Json
Internal Total heap 336764, internal Free Heap 213076
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxx
{"......very long Json......."}
parsed Json
219
Internal Total heap 336040, internal Free Heap 165672
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxx
..........
Internal Total heap 335664, internal Free Heap 63468
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxx
Failed to parse Json
Internal Total heap 336192, internal Free Heap 76104
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxx
Failed to parse Json
Internal Total heap 336196, internal Free Heap 76096
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxx
Failed to parse Json
Internal Total heap 336156, internal Free Heap 75676
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxxx
Failed to parse Json
Internal Total heap 336160, internal Free Heap 75668
SPIRam Total heap 4194084, SPIRam Free Heap 3290772
Still connected
Getting twitter stats for xxxx
........
According to an Arduino JSON assistant, the size of the JSON message is:
ESP32/ESP8266 1464+1994 = 3458 Bytes
I really don't know what else to do. The instagram library for Arduino works fine but it uses JsonStreamingParser.h. Would it be possible to use that library instead? Will it change the result?
It does seem like once the internal Free Heap RAM gets to around 7000, it doesn't work anymore.
I am also calling the function every 5 seconds. I have tried to increase the time but seems like it happens the same at some point. It also seems like the Free Heap RAM increases a bit too.
Thanks