ESP8266 Community Forum

- Mon Mar 30, 2015 9:55 am #13023 Incidentally, what I'm currently attempting to do is to use the OTA capabilities but tweak the size of the sections for the chunks of code. My code is currently building with 38kB for the 0x00000.bin file and 158kB for the 0x40000.bin file, so if you shrink the first section down to 44kB and the second to 164kB you gain 36kB of space. But since we also don't need the OTA device key and because there's 4kB blank at 0x40000 you can get an extra 12kB of space to add to that, which means you can squeeze in a 48kB chunk of user data after the user1.bin memory location. 48kB is loads of space to do interesting things with on a device like this so I'm pretty happy with that. You do also get 36kB of flash after user2.bin to play with, which I'm planning on using for other nefarious purposes.

I'm part way through this mission and have been battling with linker scripts to try and get this up and running - see here: viewtopic.php?f=6&t=2094

@prozac, if you've done any work in updating your version to run with boot_v1.3.bin I'd love to see it if you're able to push it

If we can make this an alternative way of building esp-httpd then I think it would be pretty useful, then we can easily add in the cgi handlers to be able to update the firmware via a web page

- Mon Mar 30, 2015 9:56 pm #13079 The way I would love to see this work is something like this:

user1.bin and user2.bin are completely different programs
user1.bin fits completely in iram. All it does is download user2.bin and flash it. Since it is loaded in iram at startup, it can overwrite itself.
when user1.bin finishes, it reboots into the new code (not an OTA reboot... just a standard one).
now basically the entire space is available for code and html.

There are several impediments to this goal.

My lack of knowledge around linker scripts.
Can we actually fit all the necessary espconn_* symbols, lwip, etc. along with any flash functions and other necessary symbols from the sdk in 64k of iram? I'm not 100% sure how that 64k works - is it 32k loaded at boot and another 32k paged in as needed, or is it 64k available at boot time and there's another 32k available as cached flash?
Can we even identify all the necessary symbols in the closed source sdk? Of course, the answer is yes, but is this something we can readily parse or will we have to slog through tons of assembly to find them?
If that is feasible, can we pick and choose only the symbols we need, or would we be stuck linking in a whole library.a file or none of it anyway?
Does the reverse engineered stage 1 bootloader actually compile and work, or would we basically have to write our own bootloader? (we may not need one, or it may be nothing but a single jump / function call).


So, given all those difficulties and the lack of free time that comes with a 10-month-old baby, I'd be completely happy with something that basically acts like the built-in OTA system. I've got one of the fancy 4 MByte versions of ESP-12 from ebay, so I'll want to use the first 1MB for code (two sections of almost 512KB), and the other 3 MB for the file system. But just thought I'd throw this idea out there in case someone with more time than me wants to run with it

P.S. Weren't they supposed to be adding an API to jump to any address after OTA reboot in bootv1.3? If so, that might get us most of the way there... at least to the point of having asymmetric bin files and having a small version for the "download-only" code and a large version for the real application.

EDIT: Yes, it looks like they did... just noticed the system_restart_enhance function that lets you reboot and run code from any location. This could let you tweak the build even more to squeeze in more room for code or files, whichever you need more of.

Last edited by dpwhittaker on Mon Mar 30, 2015 11:26 pm, edited 1 time in total.

- Mon Mar 30, 2015 11:22 pm #13083 Ok, well, I was in the middle of like a million changes, so the mods that just do OTA booting and flashing isn't quite clean yet. In the meantime, the most important part of this is the Makefile, since there are three different bootloaders to choose from depending on API version and there are two different ROM formats (old and new), which makes this all very clunky. I had also modded the makefile to use the esptool.py for flashing since it can flash all partitions in one go without reboot.

It also get's clunky as the code needs to know if this is an OTA FW or not to determine which partitions to write to (if using bootloader_1.1), so I have a define passed to the compiler to do that. To be honest, I just made the move myself to only use OTA only, API 1.0.0, and bootloader 1.3, but to try to get everything working, I cluged up the following makefile.
I can already see some areas that need to be cleaned up, but this should work to build OTA builds:

Code: Select allmake cloud
make flashcloud

Also, @dpwhitaker, I think the flash is mapped to memory addresses, not copied, so you can't touch the current flash partition while it is in use. That is why we update to partition 2 then reboot into the new FW. Could be wrong, though.
The new bootloader 1.3 does support booting to any ROM address

Code: Select allsystem_restart_enhance(uint8 bin_type, uint32 bin_addr)

- Tue Mar 31, 2015 5:24 am #13100 After spending most of yesterday chipping away at this I think I have a better understanding of what is actually happening with the different memory locations. Firstly, it doesn't help that the wiki page on the memory map is slightly wrong ( think - someone please correct me!)

The smaller 64kB chunk of data labelled as the user program is actually reserved for functions that have been allocated as being cached in RAM (e.g. using ICACHE_FLASH_ATTR). I verified this by moving one of my functions in and out of cached ram and it either went in the 0x00000 bin file if it was cached or in the 0x40000 bin file if not. I also listed the memory locations of the function from the elf intermediate output and this verified it. The 0x40000 file is all of the irom0 data.

I'm not sure what the RAM cacheing stage looks like (oh for the full source...) but I also tried another experiment where I set the start of the irom0 section to be 0x0C000 (48kB) and then flashed the files and everything ran fine, so at least we know that adjusting these sizes is possible.

I had got my head around the linker script which made sense for the "old" bootloader because it had the irom0 data starting at 0x40211000 or 0x40251000 which makes sense. But the "new" bootloader linker script has the irom0 data starting at 0x40201010 which is right in the middle of the flash data. Confused

I really think that one of the most important things we could do as a community is to build our own boot sector that can replace the closed binary version from Espressif so that we can do things like easily modify flash locations, etc. I tried to compile the reversed one from the wiki but at the moment it just errors in a loop. I confess though that I don't really know what I'm doing, so perhaps someone who does can help out? In any case I've put something up at https://github.com/bjpirt/esp-loader in case someone feels like taking a look.

(on another note, would the mods think about making a forum section for esp-httpd? One 29 page thread with lots of different questions is getting a little crazy!)