Post links and attach files for documentation here, also chat about these docs freely

User avatar
By wififofum
#877 Thanks Chris very helpful as always. I was going to ask about those linker scripts as it seemed to indicate what you said, but was contrary to a lot of the discussions here.

So:

Memory map
0x3ff00000 = peripheral registers base
0x3ffc8000 = stack base
0x3ffe8000 - 0x3fffffff = dram
0x40000000 - 0x4000ffff = mask rom
0x40100000 - 0x40107fff = iram
0x40200000 = external flash base

Perhaps some of that upper dram (16k?) is reserved for ROM functions since there are a few pointers to it in the linker script?

It will be interesting to find out more about the cache/execution from flash and performance.

The libraries must place most of the code in iram since the user applications have almost all functions in irom0?

Also curious how the OTA upgrade was implemented.

Since the start address of the irom section is variable, it must be specified in the header for the iram bin file? <- esptool related

User data is stored where again? Top page of flash?
User avatar
By mamalala
#878
wififofum wrote:Memory map
0x3ff00000 = peripheral registers base
0x3ffc8000 = stack base
0x3ffe8000 - 0x3fffffff = dram
0x40000000 - 0x4000ffff = mask rom
0x40100000 - 0x40107fff = iram
0x40200000 = external flash base


Well, yes and no. First, let's take a look at the memories defined in the core's config from the leaked VM. Of course, since this core is configurable, those values may or may not reflect what is in the actual chip at hand:

Code: Select all# Local memory info for ISS: 
# Get the count for each local memory type, and if
# the count is non-zero, return a list of parameters
# [ size, base_address, access_width, busy, dma, cbox, rcw, parity, ecc, enable_mask, enable_code ]
# for each instance of that memory type.
#
ISSUnifiedRAMCount = 0
ISSDataRAMCount = 2
ISSDataRAMInfo = [ 0x40000 0x3ffc0000 32 1 0 0 0 0 0 0 0 0x40000 0x3ff80000 32 1 0 0 0 0 0 0 0 ]
ISSDataROMCount = 1
ISSDataROMInfo = [ 0x40000 0x3ff40000 32 1 0 0 0 0 0 0 0 ]
ISSInstRAMCount = 2
ISSInstRAMInfo = [ 0x100000 0x40000000 32 1 0 0 0 0 0 0 0 0x100000 0x40100000 32 1 0 0 0 0 0 0 0 ]
ISSInstROMCount = 1
ISSInstROMInfo = [ 0x100000 0x40200000 32 1 0 0 0 0 0 0 0 ]
ISSDataPortCount = 1
ISSDataPortInfo = [ 0x40000 0x3ff00000 32 1 0 0 0 0 0 0 0 ]


So we have several segments in the address space used for various RAM/ROM areas. The question now is how all that maps to the flash, which is yet to be found out. Now, there are a few hints in the leaked linker scripts. First, i noticed that the linker script says:

Code: Select alldram0_0_seg :                         org = 0x3FFE8000, len = 0x14000


This defines a region with 80k. However, writing a simple test app it turns out that i can actually write&read from a 96k big segment starting at that address, that is, 0x18000 size. I think that the remaining 16k are probably used for stack & heap of the internal ROM functions, but i'm not sure about that.

Next segment is the iram, start at 0x40100000 with a size of 0x8000, which is 32k, and matches what my test-app could also write to.

Now the interresting part regarding the flash. The standard linker file says:

Code: Select allirom0_0_seg :                          org = 0x40240000, len = 0x32000


That is 200k in size. Also note that to upload that segment to the chip, the tools use 0x40000 as address, so i assume that the internal bootloader uses 0x40200000 as it's base. Another thing to note is that the addresses given to the uploader tool seem to be the addresses where it ends up in physical flash. For example, the assembled data chunk that goes to 0x00000 is really at 0x00000 in the flash, and the chunk that is usually uploaded to 0x40000 is really at that address in the flash.

Now lets look at what the other two linker files say about the irom0 segment. app1 says:

Code: Select allirom0_0_seg :                          org = 0x40211000, len = 0x2B000


And app2 says:

Code: Select allirom0_0_seg :                         org = 0x40251000, len = 0x2B000


So, going by that, the first _known_ address where code can be run from is 0x40211000, the last one is 0x4027B000-1. This is a 424k large region. Of course there are a few things that need to be figured out: Is the internal bootloader & OS able to start code from any of these addresses directly, or does is assume a small chunk of code always be present at 0x40240000? What else is in the flash in that address range, for example some config data or user-storable data? LIke, there is a esp_init_data_default.bin that is located at 0x7C000 and a blank.bin that goes at 0x7E000, so it's obvious that we can't use those addresses for random code.

Writing code that places a chunk of code at specific addresses is rather easy. Just add the required sections to the linker script, and place the propper attribute in front of a function. For example, there is tthe irom0_0_seg for the code from flash. In c_types.h we have:

Code: Select all#define ICACHE_FLASH_ATTR __attribute__((section(".irom0.text")))


If you look at the example code, you see things like:

Code: Select allvoid ICACHE_FLASH_ATTR blinky_init(void) { ... }


Which places the final code for that function into the address space given by the irom0_0_seg definition in the linker script. Once we found out what actual address space is available in the flash (avoiding regions that contain user- or other data), we can simply add them to the linker script, and then duplicate the section in the script that places such code, create our own attribute defines, and happily place code there.

BTW, if you want to fiddle around with reading random addresses in the address space, you could abuse the AT firmware as a base (basically drop all the AT related crap, we only want the UART printing routines). Create some functions to print hex-bytes and -words, for example:

Code: Select allconst char hextab[16] = { "0123456789ABCDEF" };

void printhex(unsigned char c)
{
   
    uart0_sendStr(" 0x");
    uart_tx_one_char(UART0, hextab[ ((c & 0xF0 ) >> 4) ]);
    uart_tx_one_char(UART0, hextab[ (c & 0x0F ) ] );
}

void print_hexaddr(unsigned long addr)
{
    uart0_sendStr(" 0x");
    uart_tx_one_char(UART0, hextab[ ((addr & 0xF0000000 ) >> 28) ]);
    uart_tx_one_char(UART0, hextab[ ((addr & 0x0F000000 ) >> 24) ]);
    uart_tx_one_char(UART0, hextab[ ((addr & 0x00F00000 ) >> 20) ]);
    uart_tx_one_char(UART0, hextab[ ((addr & 0x000F0000 ) >> 16) ]);
    uart_tx_one_char(UART0, hextab[ ((addr & 0x0000F000 ) >> 12) ]);
    uart_tx_one_char(UART0, hextab[ ((addr & 0x00000F00 ) >>  8) ]);
    uart_tx_one_char(UART0, hextab[ ((addr & 0xF00000F0 ) >>  4) ]);
    uart_tx_one_char(UART0, hextab[ ( addr & 0x0000000F )] );
    uart0_sendStr(" ");
   
}


Keep in mind that there is a crude, minimal OS running that handles a simple multitasking scheme. This also has a watchdog. You want to either disable it by first defining somewhere:

Code: Select allextern void ets_wdt_disable(void);


And then calling it in the main code. Alternatively you can define:

Code: Select allextern void wdt_feed(void);


And then call that peridically to keep it from resetting the chip. (Those are two of the functions in the internal ROM).

Now, to access an address:

Code: Select allunsigned char *ramaddr;

ramaddr = 0;
ramaddr += 0x40100000;

print_hexaddr((unsigned long)ramaddr);
printhex(ramaddr[0]);


This will create a pointer into memory, called "ramaddr". Then you set it to 0, and then add to it the address you want. In this example i set the pointer address, then have it print the address, and then the byte at that address. You could also write:

Code: Select allunsigned char *ramaddr;

ramaddr = 0;

printhex(ramaddr[0x40100000]);


to print the byte at that address.

You can use the same pointer stuff to actually write to any address you want. However, it will reset as soon as you attempt to write to an address that you are not allowed to. To write somewhere:

Code: Select allunsigned char *ramaddr;

ramaddr = 0;
ramaddr += 0x40100000;

ramaddr[0] = 0xAB;


Keep in mind that if you work with pointers that way, if you add/substract from them, it is done in sizes that corrospond to the type. So if you have a char* pointer at 0x1000, and do a "mypointer += 1", it will end up at address 0x1001. If you have a unsigned long* pointer at 0x1000, and again do "mypointer += 1", it will end up at 0x1004.

Greetings,

Chris
User avatar
By wififofum
#882 Thanks for the writeup.

So is the 0x40000 offset for irom hardcoded in the tools to match the linker script or does the tool discover the offset when it runs?

Edit: I guess neither since you have to manually flash the bins and specify the addresses? I was thinking it was automated for some reason ( because I haven't used it yet).

The size of the irom section may be limited as necessary, for example to save room for the OTA upgrade to use part of the flash?

Flash map
0x00000000 - ? = default iram/dram image
? - 0x0007bfff = free space for irom image or other use
0x0007c000 - 0x0007dfff = default config
0x0007e000 - 0x0007ffff = config

With ota updates:
0x00000000 - 0x00000fff = boot.bin
0x00001000 - 0x00010fff = app1 iram/dram ?
0x00011000 - 0x0003bfff = app1 irom
0x0003c000 - 0x0003dfff = app1 default config ?
0x0003e000 - 0x0003efff? = master_device_key.bin (shouldn't need more than 1 sector, used for iotbucket cloud)
0x0003f000 - 0x0003ffff = free space ?
0x00040000 - 0x00040fff = boot.bin copy ? (Probably not)
0x00041000 - 0x00050fff = app2 iram/dram ?
0x00051000 - 0x0007bfff = app2 irom
0x0007c000 - 0x0007dfff = app2 default config ?
0x0007e000 - 0x0007ffff = config

So you have to compile ota updates targeting both app1 and app2, the device picks the one not currently active and flashes it. Boot.bin presumably manages selecting between them.

So:
What's in the config sections?
Where's the boot.bin source?
What happens when you go to a 16M flash device?
User avatar
By Squonk
#889 OK, I think I found some important information regarding the Flash layout in the "document\Espressif IoT SDK 使用手册_v0.8.pdf" ("document\Espressif IoT SDK Manual_v0.8.pdf") from the leaked "esp_iot_sdk_v0.9.1" :ugeek:

I am not fluent In Chinese yet, but Google translated it for me ;)

The interesting part is Section 4.3, page 20:
4.3 Compile and Flash Method

In order to compile, please make sure to copy the "user", "include" subdirectories and the Makefile from the "esp_iot_sdk\examples" directory into the "esp_iot_sdk\app" directory.

For flashing, please refer to section "3.3 Download Tool". After each completed file downloaded and flashed from "bin", you need to close the serial port and then re-open the serial port connection before proceeding to flash the next file.

4.3.1 Without Support for the Cloud Upgrade

esp_iot_sdk_v0.7 and previous versions do not support the cloud upgrade.

Compile as follows:
1) Open the "xtensa" compiler, with a default compile path of "d:\esp_iot_sdk\app".
2) Enter the "make" command, press "Enter". The makefile execution will create the ".output" folder to store "lib" and "obj" files.
3) Execute the "gen_misc.bat" script, press "Enter" to generate the appropriate "bin" files.

The "bin" files are as follows:
1) blank.bin, provided by Espressif, burn to 0x7E000 address;
2) eagle.app.v6.flash.bin, as compiled, burn to 0x0000 address;
3) master_device_key.bin, from the Espressif server application, burn to 0x3E000 address;
4) eagle.app.v6.irom0text.bin, as compiled, burn to 0x40000 address;
5) esp_init_data_default.bin, provided by Espressif, burn to 0x7c000 address

Note that:
1) "blank.bin" needs to be flashed only when the sdk version is upgraded or when WiFi configuration parameters have to be erased;
2) "master_device_key.bin" has to be flashed once, or when the "master_device_key" needs to be changed;
3) Generally, only the two files "eagle.app.v6.flash.bin" and "eagle.app.v6.irom0text.bin" from the "bin" directory need to be flashed.

4.3.2 With Support for Cloud Upgrade

esp_iot_sdk_v0.8 and later software version support cloud upgrade. esp_iot_sdk_v0.8 is also compatible with the previous compilation and firmware flashing method, and after such upgrades without Cloud support, you can continue using the compiling and programming methods from section "4.3.1 Without Support for the Cloud Upgrade".

Regarding this specific Cloud upgrade feature, see the document "Cloud Upgrading Implementations".

Compile as follows:
1) Compile "user1.bin" by executing the command "make APP=1"
2) Execution of the "gen_misc_plus.bat user1" command will generate a file "user1.bin" into the directory "\esp_iot_sdk\bin\upgrade".
3) Execute "make clean" in order to clear information compiled previously;
4) Then compile "user2.bin" by executing the command "make APP=2"
5) Execution of the "gen_misc_plus.bat user2" command will generate a file "user2.bin" into the directory "\esp_iot_sdk\bin\upgrade".

The "bin" file are as follows:
1) "blank.bin", provided by Espressif, burn to 0x7E000 address;
2) "boot.bin", provided by Espressif, burn to 0x00000 address;
3) "user1.bin", as compiled, burn to 0x01000 address;
4) "master_device_key.bin", from the Espressif server application, burn to 0x3E000 address;

Note that:
Flash only in accordance with the above-mentioned first four steps, without flashing "user2.bin".

For subsequent software updates, the new version of "user1.bin" and "user2.bin" two "bin" files are uploaded to the server, the server sends a software update message to the user, and if the user chooses to update, the device will select to download "user1.bin" or "user2.bin" depending on its current state in order to complete the Cloud upgrade process.


To summarize, we have 2 different Flash memory configurations:

SDK <= v0.7:
Code: Select all0x00000-0x3DFFF (248KB) flash.bin
0x3E000-0x3FFFF (8KB)   master_device_key.bin
0x40000-0x7DFFF (248KB) irom0text.bin
0x7E000-0x7FFFF (8KB)   WiFi Configuration


SDK >= v0.8:
Code: Select all0x00000-0x00FFF (4KB)   boot.bin
0x01000-0x10FFF (64KB)  flash1.bin     \ user1.bin
0x11000-0x3DFFF (180KB) irom0text1.bin /
0x3E000-0x3FFFF (8KB)   master_device_key.bin
0x40000-0x40FFF (4KB)   ???
0x41000-0x50FFF (64KB)  flash2.bin     \ user2.bin
0x51000-0x7DFFF (180KB) irom0text2.bin /
0x7E000-0x7FFFF (8KB)   WiFi Configuration


If you wonder why I say that the "flashx.bin" files are limited to 64KB, please refer to the "gen_flashbin.py" script from leaked "esp_iot_sdk_v0.9.1", where this file is padded to 0x10000 with "0xFF" bytes :ugeek:

But why are "irom0text" sections separate from the "flash" sections in the beginning? My guess is that there are TWO separate cache regions, one for app, one for libs, but I have no clue why!