-->
Page 1 of 2

Newb question: using and storing data object in eeprom

PostPosted: Wed Jul 31, 2019 2:43 pm
by cjohnweb
I am creating a simple temperature sensor device with http/s reporting. It has 2 modes: operating mode and access-point mode, and requires data such as internet access network ssid and password, broadcast network ssid and password, unique id number for identifying the device, etc. and these things are all editable via web server on the esp8266 chip.

What I want to do to keep things as simple and easy as possible is store all of this data in a single object so that when the device boots it reads this one object out of eeprom and runs using these values, and when I make a change through the web server it updates the one or 2 values in that object and saves the whole thing to eeprom. This way, if I add a new variable I don’t need to go edit a ton of stuff.

I have looked high and low and everything I’ve tried gives me errors or just doesn’t work.

Now I’ve been able to store, retrieve and update a single variable in eeprom, and that value survives a power cycle which is great, but getting some kind of multi-value array is confusing to me.

A friend told me I could do this with json but that didnt work the same way as the individual variable did. I’ve also seen examples using a struct, but that doesn’t work right and gives errors. I feel like there are things I just don’t understand about the way data is handled in this language and platform. What is a struct anyways? I thought it stood for “structure”, seemed to make sense from the code examples I’ve read.

Any help would be greatly appreciated, I think this is the last major hurdle I need to overcome for my first project to be finished. Thank you!

Re: Newb question: using and storing data object in eeprom

PostPosted: Wed Jul 31, 2019 3:30 pm
by btidey
EEPROM is just an array of bytes and you therefore have to encode the variables into byte format. You can write and retrieve the whole array in single operations.

There are a number of encoding strategies. Easiest is to put each variable at a fixed index in the byte array and leave enough room in the byte array to accommodate it. This means any strings need to have their maximum size predetermined. Numbers would take up 1,2,or 4 bytes in the array depending on their size. Best to avoid floats if at all possible.

Personally I don't use EEPROM for this purpose. Rather I use SPIFFS and store variables in a single config file. I normally make this a txt file with each variable occupying one line. This makes it really easy to maintain remotely from a browser. You then use a simple read routine at start up to read the config file a line at a time and store each variable. As each line will be read as String then it is simple to convert them to whatever variable type is needed (e.g. lineRead.toInt() )

If you are changing wifi credentials then consider using WifiManager instead of storing them in a config file. This makes it easy to set up in the first place and allows you to handle things like connecting bak in if router set up changes.

Re: Newb question: using and storing data object in eeprom

PostPosted: Wed Jul 31, 2019 3:33 pm
by torntrousers
This doesn't use RTC memory but it may give you the idea how to do it.

Define a struct (some code https://github.com/HarringayMakerSpace/ESP-Now/blob/master/espnow-sensor-bme280/espnow-sensor-bme280.ino#L27-L31):
Code: Select allstruct __attribute__((packed)) SENSOR_DATA {
    float temp;
    float humidity;
    float pressure;
} sensorData;


Then you can serialize that as a byte array (code]):
Code: Select alluint8_t bs[sizeof(sensorData)];
memcpy(bs, &sensorData, sizeof(sensorData));


and desrialize from bytes back into that struct (code):
Code: Select allswSer.readBytes((char*)&sensorData, len);


Does that help?

Re: Newb question: using and storing data object in eeprom

PostPosted: Fri Aug 02, 2019 6:33 pm
by cjohnweb
Last night I wrote a response but my phone died before i could post...ugh....here we go.

btidey - yes, exactly. But I'd like like not to add SPIFFs or anything extra at this time - I want to keep this as cheap and straight forward as possible. And honestly, I only need to save some 15 variables, so if the EEPROM can handle that small bits of data then that is the option I'd like to get working.

I believe one of the code examples I tried had a string saved to eeprom, and it would check the length of the string and then stuff the string in the eeprom. Then the device would power cycle, pull the string out, and write a different string to eeprom, power cycle and put the first string back in...back and forth back and forth. So I at least see that it is saving to eeprom and behaving accordingly when it boots.

I want to do exactly that, but not with a string, rather, a json object or an array of variables...something that will allow me to store multiple individual values by name under a single variable name, let call it my "data object". I stuff my data object variable holding all my data into EEPROM, power cycle, pull it all out, and map those values to variables that the application will actively use. When a piece of data wants to be edited, the web server grabs that specific data from the data object, overwrites the ones it needs, triggers the device to write the entire database object back to eeprom, and power cycle on success. I know it's not perfect but it's exactly what I need in this case.

This is where I think torntrousers is actually really really close to what I'm looking for! But I've got a few questions about this, would you mind clearing this up for me a little? Things I read just aren't making sense to me.

What is a struct?
What do the __attribute__((packed)) parts actually do?
What about SENSOR_DATA on the first line of the struct, and sensorData in the last line...what are they and why are they there?

I see that sensorData is the name of the struct. So how would I access an individual variable out of a struct? For example, using the float temp from your example torntrousers, can I pull that single value out? like ... sensorData.temp? or sensorData[temp] ?

And how would I over write that single value: sensorData.temp = 123.2 or sensorData[temp] = 123.2 . ?

If you could help me understand this, I can code it up this weekend and see what happens and I would be forever thankful!

Thank you, both of you, for your answers!