ESP8266 Community Forum

- Mon Mar 15, 2021 8:41 am #90840 Hi, all,
I'm new here.
I'm trying to write a small database class which can store/restore data to/from the esp8266's filesystem and can do the records search/find upon the database file(which is a compound of ascii and binary file, is it the problem?)
The LittleFs is used, SPIFFS is fine as well(same results to me).

My problem is, the find() behaves beyond my expected, the keyword could be found but succeeding same ones could not:
find() is success/true once found and could be identified by LittleFS.position() which is the current file pointer position pointing to the next one char of the found keyword. however I do the find() again, isn't it beginning from the current position? what I got was not found.

Simply put, the first one could be found but later ones could not.
Please kindly help me where I went wrong of the above idea or the code, I attached below.

the following is the snippet result:

21:13:08.651 -> [4 -1 4]
21:13:08.651 -> [AA 5A A5 55 0C 00 00 00 ]
21:13:08.651 -> [61 62 63 64 D7 11 00 00 ]
21:13:08.651 -> [66 E6 8A 44 AA 5A A5 55 ]
21:13:08.651 -> [0C 00 00 00 64 65 66 67 ]
21:13:08.651 -> [E6 1D 00 00 0A 1A 00 00 ]
21:13:08.651 -> [AA 5A A5 55 11 00 00 00 ]
21:13:08.651 -> [31 32 33 34 61 62 63 64 ]
21:13:08.651 -> [65 66 67 68 69 6A 6B 6C ]
21:13:08.651 -> [6D ]


keyword is AA 5A A5 55,
search instructions are:
printf("[%d %d %d]\r\n", s.FindNext(PREAMBLE), s.FindNext(PREAMBLE), s.FindNext(PREAMBLE));

the result as I thought might be [4 24 44] rather than [4 -1 4].

Thanks.

Attachments

sketch_feb01a.zip

find problem
(1.7 KiB) Downloaded 135 times

- Wed Mar 17, 2021 8:00 pm #90873 Hi All,
By my investigations, I change the "test file" which is mixed with ascii bytes and non-ascii bytes into all-ascii bytes, the find() hence success as expected.

So, find() can't successfully process the mixed file I used. I don't know whether I misused find() or the mixed file is inhibited to find() by rules, or yet other cases.

In some aspects, "mixed file" do needed. A way can readBytes then do the search by myself however second-copy the data cause less efficiency(FS loads a block each time) so I prefer find() directly.

Appreciates if comments.

Thanks.

- Sat Mar 20, 2021 10:30 pm #90889 Hi All,

At last, I fulfilled the class I required I mentioned at the beginning post, and put it here for freely share as attached.
Of course, the find() seems the current only way I have to do by myself.

Simply put, a class can store/load vars against a file in a simpler way.

I paste the main function and results for demo as follows.

Code: Select all    // -----------var assignments
    char strc[14];
    strcpy(strc, "nopqrstuvwxyz");
    unsigned strc_len=strlen(strc);
    double THE_PI=3.1415926f;
    String str="abcdefghijklm";

    // since size of String always changes,
    // here uses String type for demo is merely a demo.
    // namely, the cDataBase not applies to dynamic objects(classes do mem managements/nongranulated objs).
    unsigned str_length=sizeof(str);
    unsigned tmp_string[(str_length+3)/4]; //// for alignment.
    char *new_string=(char*)tmp_string; //// let new_string to fit String type.


    // -----------Open database file, create new if not exist
    cDataBase s;
    s="/database";
    s.Open();


    printf("3 vars are, pi=%f, str=%s, strc=%s\r\n", THE_PI, str.c_str(), strc);


    // -----------store vars to db file
    s.WriteBytes("ThisIsChars", 99, strc, strc_len);
    s.WriteWords("doubleTHE_PI", 99, THE_PI);
    s.WriteBytes("ThisIsAString1", 99, &str, 1);
    s.WriteBytes("ThisIsAString2", 99, (unsigned char*)&str, sizeof(str));
    s.WriteWords("[KeyWordHere]ThisIsADesignatorToPointOutTheId99IsWrittenAtTimestamp", 20210321, 99);


    // -----------take a look at the db file
    printf("the context of the database file is as follows(in ascii):\r\n");
    s.DumpAll(0);
    printf("the context of the database file is as follows(in hex):\r\n");
    s.DumpAll(1);

    printf("the records in the database file are as follows:\r\n");
    s.Close();
    s.LsRecords();


    // -----------zeroing the vars
    THE_PI=0;
    str="";
    strc[0]=0;
    printf("zeroing the vars, pi=%f, str=%s, strc=%s\r\n", THE_PI, str.c_str(), strc);


    // -----------restore the vars from db file
    s.Open();

    s.ReadBytes(strc, strc_len, "ThisIsChars");

    s.ReadWords(THE_PI, "doubleTHE_PI");

    ////s.ReadBytes(&str, str_length, "ThisIsAString1"); //// don't do that.
    s.ReadBytes(new_string, str_length, "ThisIsAString1"); //// use this instead.
    str=*(reinterpret_cast<String*>(new_string)); //// not work. just for demo.

    printf("after restore, pi=%f, str=%s, strc=%s\r\n", THE_PI, str.c_str(), strc);

    s.ReadBytes(new_string, str_length, "ThisIsAString2"); //// use this instead.
    str=*(reinterpret_cast<String*>(new_string)); //// not work. just for demo.
    printf("str=%s, str=%s\r\n", str.c_str(), (static_cast<String>(*new_string)).c_str());

    int a, b;
    s.ReadWords(a, b, "[KeyWordHere]ThisIsADesignatorToPointOutTheId99IsWrittenAtTimestamp");
    printf("timestamp=%d, id=%d\r\n", b, a);

    s.Close();


    // -----------delete a record in db file
    s.DeleteRecords("doubleTHE_PI");
    printf("after delete a record, the records in the db file are:\r\n");
    s.LsRecords();

    printf("after delete a record, the context(unchanged) of the database file(hex):\r\n");
    s.Open(); s.DumpAll(1); s.Close();


    // -----------compact the db file
    s.Renew();
    printf("after renew, the context(shrinked) of the database file(hex):\r\n");
    s.Open(); s.DumpAll(1); s.Close();

    cDataBase::Ls("/");


The results:

Code: Select all
11:00:38.335 -> filesystem formatted.
11:00:38.335 -> --------
11:00:38.335 -> The FileSystem:
11:00:38.335 -> entire space 2072576 bytes
11:00:38.335 -> 16384 bytes used
11:00:38.335 -> free space 2056192 bytes
11:00:38.335 -> block size 8192 bytes
11:00:38.335 -> page size 256 blocks
11:00:38.335 -> --------
11:00:38.335 -> list root directory "/":
11:00:38.369 -> 3 vars are, pi=3.141593, str=abcdefghijklm, strc=nopqrstuvwxyz
11:00:38.434 -> the context of the database file is as follows(in ascii):
11:00:38.434 -> [ .  Z  .  U  .  .  .  . ]
11:00:38.434 -> [ E  .  .  .  .  .  .  . ]
11:00:38.434 -> [ T  h  i  s  I  s  C  h ]
11:00:38.434 -> [ a  r  s  c  .  .  .  n ]
11:00:38.446 -> [ o  p  q  r  s  t  u  v ]
11:00:38.470 -> [ w  x  y  z  .  Z  .  U ]
11:00:38.470 -> [ .  .  .  .  T  .  .  . ]
11:00:38.470 -> [ .  .  .  .  d  o  u  b ]
11:00:38.470 -> [ l  e  T  H  E  _  P  I ]
11:00:38.470 -> [ c  .  .  .  .  .  .  @ ]
11:00:38.470 -> [ .  !  .  @  .  Z  .  U ]
11:00:38.470 -> [ .  .  .  .  =  .  .  . ]
11:00:38.470 -> [ .  .  .  .  T  h  i  s ]
11:00:38.470 -> [ I  s  A  S  t  r  i  n ]
11:00:38.470 -> [ g  1  c  .  .  .  .  . ]
11:00:38.470 -> [ .  ?  .  .  .  .  .  . ]
11:00:38.470 -> [ .  .  .  Z  .  U  .  . ]
11:00:38.470 -> [ .  .  >  .  .  .  .  . ]
11:00:38.470 -> [ .  .  T  h  i  s  I  s ]
11:00:38.501 -> [ A  S  t  r  i  n  g  2 ]
11:00:38.501 -> [ c  .  .  .  .  .  .  ? ]
11:00:38.501 -> [ .  .  .  .  .  .  .  . ]
11:00:38.501 -> [ .  Z  .  U  C  .  .  . ]
11:00:38.501 -> [ .  .  .  .  K  .  .  . ]
11:00:38.501 -> [ [  K  e  y  W  o  r  d ]
11:00:38.501 -> [ H  e  r  e  ]  T  h  i ]
11:00:38.501 -> [ s  I  s  A  D  e  s  i ]
11:00:38.501 -> [ g  n  a  t  o  r  T  o ]
11:00:38.501 -> [ P  o  i  n  t  O  u  t ]
11:00:38.501 -> [ T  h  e  I  d  9  9  I ]
11:00:38.501 -> [ s  W  r  i  t  t  e  n ]
11:00:38.501 -> [ A  t  T  i  m  e  s  t ]
11:00:38.534 -> [ a  m  p  .  b  4  .  c ]
11:00:38.534 -> [ .  .  . ]
11:00:38.534 -> the context of the database file is as follows(in hex):
11:00:38.534 -> [AA 5A A5 55 0B 00 00 00 ]
11:00:38.534 -> [45 04 00 00 1C 00 00 00 ]
11:00:38.534 -> [54 68 69 73 49 73 43 68 ]
11:00:38.534 -> [61 72 73 63 00 00 00 6E ]
11:00:38.534 -> [6F 70 71 72 73 74 75 76 ]
11:00:38.534 -> [77 78 79 7A AA 5A A5 55 ]
11:00:38.534 -> [0C 00 00 00 54 04 00 00 ]
11:00:38.534 -> [18 00 00 00 64 6F 75 62 ]
11:00:38.534 -> [6C 65 54 48 45 5F 50 49 ]
11:00:38.534 -> [63 00 00 00 00 00 00 40 ]
11:00:38.567 -> [FB 21 09 40 AA 5A A5 55 ]
11:00:38.567 -> [0E 00 00 00 3D 05 00 00 ]
11:00:38.567 -> [1E 00 00 00 54 68 69 73 ]
11:00:38.567 -> [49 73 41 53 74 72 69 6E ]
11:00:38.567 -> [67 31 63 00 00 00 14 FA ]
11:00:38.567 -> [FE 3F 0F 00 0D 00 FE EF ]
11:00:38.567 -> [EF 00 AA 5A A5 55 0E 00 ]
11:00:38.567 -> [00 00 3E 05 00 00 1E 00 ]
11:00:38.567 -> [00 00 54 68 69 73 49 73 ]
11:00:38.567 -> [41 53 74 72 69 6E 67 32 ]
11:00:38.567 -> [63 00 00 00 14 FA FE 3F ]
11:00:38.567 -> [0F 00 0D 00 FE EF EF 00 ]
11:00:38.567 -> [AA 5A A5 55 43 00 00 00 ]
11:00:38.600 -> [FD 19 00 00 4B 00 00 00 ]
11:00:38.600 -> [5B 4B 65 79 57 6F 72 64 ]
11:00:38.600 -> [48 65 72 65 5D 54 68 69 ]
11:00:38.600 -> [73 49 73 41 44 65 73 69 ]
11:00:38.600 -> [67 6E 61 74 6F 72 54 6F ]
11:00:38.600 -> [50 6F 69 6E 74 4F 75 74 ]
11:00:38.600 -> [54 68 65 49 64 39 39 49 ]
11:00:38.600 -> [73 57 72 69 74 74 65 6E ]
11:00:38.600 -> [41 74 54 69 6D 65 73 74 ]
11:00:38.600 -> [61 6D 70 91 62 34 01 63 ]
11:00:38.600 -> [00 00 00 ]
11:00:38.600 -> the records in the database file are as follows:
11:00:38.634 -> /database contents:
11:00:38.634 -> --------
11:00:38.634 -> [ThisIsChars    99]
11:00:38.634 -> [doubleTHE_PI    99]
11:00:38.634 -> [ThisIsAString1    99]
11:00:38.634 -> [ThisIsAString2    99]
11:00:38.634 -> [[KeyWordHere]ThisIsADesignatorToPointOutTheId99IsWrittenAtTimestamp    20210321]
11:00:38.634 -> --------
11:00:38.634 -> zeroing the vars, pi=0.000000, str=, strc=
11:00:38.634 -> after restore, pi=3.141593, str=, strc=nopqrstuvwxyz
11:00:38.634 -> str=, str=
11:00:38.634 -> timestamp=20210321, id=99
11:00:38.733 -> after delete a record, the records in the db file are:
11:00:38.733 -> /database contents:
11:00:38.733 -> --------
11:00:38.733 -> [ThisIsChars    99]
11:00:38.733 -> [ThisIsAString1    99]
11:00:38.733 -> [ThisIsAString2    99]
11:00:38.733 -> [[KeyWordHere]ThisIsADesignatorToPointOutTheId99IsWrittenAtTimestamp    20210321]
11:00:38.767 -> --------
11:00:38.767 -> after delete a record, the context(unchanged) of the database file(hex):
11:00:38.767 -> [AA 5A A5 55 0B 00 00 00 ]
11:00:38.767 -> [45 04 00 00 1C 00 00 00 ]
11:00:38.767 -> [54 68 69 73 49 73 43 68 ]
11:00:38.767 -> [61 72 73 63 00 00 00 6E ]
11:00:38.767 -> [6F 70 71 72 73 74 75 76 ]
11:00:38.767 -> [77 78 79 7A AB 5A A5 55 ]
11:00:38.767 -> [0C 00 00 00 54 04 00 00 ]
11:00:38.767 -> [18 00 00 00 64 6F 75 62 ]
11:00:38.800 -> [6C 65 54 48 45 5F 50 49 ]
11:00:38.800 -> [63 00 00 00 00 00 00 40 ]
11:00:38.800 -> [FB 21 09 40 AA 5A A5 55 ]
11:00:38.800 -> [0E 00 00 00 3D 05 00 00 ]
11:00:38.800 -> [1E 00 00 00 54 68 69 73 ]
11:00:38.800 -> [49 73 41 53 74 72 69 6E ]
11:00:38.800 -> [67 31 63 00 00 00 14 FA ]
11:00:38.800 -> [FE 3F 0F 00 0D 00 FE EF ]
11:00:38.800 -> [EF 00 AA 5A A5 55 0E 00 ]
11:00:38.800 -> [00 00 3E 05 00 00 1E 00 ]
11:00:38.800 -> [00 00 54 68 69 73 49 73 ]
11:00:38.800 -> [41 53 74 72 69 6E 67 32 ]
11:00:38.800 -> [63 00 00 00 14 FA FE 3F ]
11:00:38.800 -> [0F 00 0D 00 FE EF EF 00 ]
11:00:38.832 -> [AA 5A A5 55 43 00 00 00 ]
11:00:38.832 -> [FD 19 00 00 4B 00 00 00 ]
11:00:38.832 -> [5B 4B 65 79 57 6F 72 64 ]
11:00:38.832 -> [48 65 72 65 5D 54 68 69 ]
11:00:38.832 -> [73 49 73 41 44 65 73 69 ]
11:00:38.832 -> [67 6E 61 74 6F 72 54 6F ]
11:00:38.832 -> [50 6F 69 6E 74 4F 75 74 ]
11:00:38.832 -> [54 68 65 49 64 39 39 49 ]
11:00:38.832 -> [73 57 72 69 74 74 65 6E ]
11:00:38.832 -> [41 74 54 69 6D 65 73 74 ]
11:00:38.832 -> [61 6D 70 91 62 34 01 63 ]
11:00:38.832 -> [00 00 00 ]
11:00:38.932 -> after renew, the context(shrinked) of the database file(hex):
11:00:38.932 -> [AA 5A A5 55 0B 00 00 00 ]
11:00:38.932 -> [45 04 00 00 1C 00 00 00 ]
11:00:38.932 -> [54 68 69 73 49 73 43 68 ]
11:00:38.932 -> [61 72 73 63 00 00 00 6E ]
11:00:38.932 -> [6F 70 71 72 73 74 75 76 ]
11:00:38.965 -> [77 78 79 7A AA 5A A5 55 ]
11:00:38.965 -> [0E 00 00 00 3D 05 00 00 ]
11:00:38.965 -> [1E 00 00 00 54 68 69 73 ]
11:00:38.965 -> [49 73 41 53 74 72 69 6E ]
11:00:38.965 -> [67 31 63 00 00 00 14 FA ]
11:00:38.965 -> [FE 3F 0F 00 0D 00 FE EF ]
11:00:38.965 -> [EF 00 AA 5A A5 55 0E 00 ]
11:00:38.965 -> [00 00 3E 05 00 00 1E 00 ]
11:00:38.965 -> [00 00 54 68 69 73 49 73 ]
11:00:38.965 -> [41 53 74 72 69 6E 67 32 ]
11:00:38.965 -> [63 00 00 00 14 FA FE 3F ]
11:00:38.965 -> [0F 00 0D 00 FE EF EF 00 ]
11:00:38.965 -> [AA 5A A5 55 43 00 00 00 ]
11:00:38.965 -> [FD 19 00 00 4B 00 00 00 ]
11:00:38.998 -> [5B 4B 65 79 57 6F 72 64 ]
11:00:38.998 -> [48 65 72 65 5D 54 68 69 ]
11:00:38.998 -> [73 49 73 41 44 65 73 69 ]
11:00:38.998 -> [67 6E 61 74 6F 72 54 6F ]
11:00:38.998 -> [50 6F 69 6E 74 4F 75 74 ]
11:00:38.998 -> [54 68 65 49 64 39 39 49 ]
11:00:38.998 -> [73 57 72 69 74 74 65 6E ]
11:00:38.998 -> [41 74 54 69 6D 65 73 74 ]
11:00:38.998 -> [61 6D 70 91 62 34 01 63 ]
11:00:38.998 -> [00 00 00 ]
11:00:38.998 -> list directory "/":
11:00:38.998 -> --------
11:00:38.998 -> database    227 bytes
11:00:38.998 ->
11:00:38.998 -> entire space 2072576 bytes
11:00:39.031 -> 24576 bytes used
11:00:39.031 -> free space 2048000 bytes
11:00:39.031 -> --------




Attachments

var2file_v_0.2.zip

var2file
(4.78 KiB) Downloaded 137 times