Comment about low-latency interrupts

ESPUSB - Chat about the software components
 Reply

User avatar

Re: Comment about low-latency interrupts

By cnlohr
- Wed Aug 10, 2016 4:13 pm #52669 No, the problem is in that: With 1.5MBit USB, I use ccount to stay on track. If there was a very small interrupt <10 cycles, it probably wouldn't notice and would keep going, however, if I do 12 MBit USB, then, there is no timer, I'll simply be relying on the time it takes to execute every instruction. If something interrupts for say even 5 cycles, that's a phase shift induced in the whole transaction and would likely cause a complete failure.
User avatar

Re: Comment about low-latency interrupts

By projectgus
- Wed Aug 10, 2016 9:07 pm #52679
cnlohr wrote:I am still trying to mull through this and find the least invasive mechanism to insert my own interrupts. Sadly, I can't even figure out which exception handles the GPIO (it is an exception, right?) I will hopefully poke and prod around in my ESP's memory soon.


Hey,

Happy to explain in more detail. :)

A ESP8266 peripheral interrupt (including GPIO, but also the other peripherals) will trigger UserException on the Xtensa CPU. So the normal flow for a GPIO interrupt (in the non-RTOS SDK) is something like this:

  • GPIO pin changes state, GPIO hardware sets its interrupt flag (assuming pin interrupt is configured)
  • GPIO peripheral interrupt line asserts. This causes the CPU's UserException to assert at the same time. It also sets the INUM_GPIO bit (bit 4) in the CPU's INTERRUPT special register.
  • On next CPU clock, CPU (assuming exceptions enabled) hits the exception and jumps to UserExceptionVector (VecBase+0x50) with exception level 1 set (check Xtensa ISA RM for gory details).
  • Current SDK's exception vector is implemented in libmain.a user-vector.o. The actual vector only has 0x20 bytes to work in (next exception vector is at VecBase+0x70, this is determined by hardware) so it's usually very short. In the SDK, _UserExceptionVector saves a0 to excsave1 and then uses a0 to call _UserExceptionVector_1 (implemented in the same object file.) Disassemble user-vector.o for details (it's only a dozen lines of assembler long.)
  • _UserExceptionVector_1 (also in user-vector.o) saves some more registers to the stack, loads EXCCAUSE special register (which has a "cause" for the UserException) and uses this as an index into a function pointer table starting at 0x3fffc000. For any peripheral interrupt, this register will have value 4 ("user"). (other UserException causes are all fatal faults like LoadStoreException, AFAIK.)
  • I don't actually know where this jump table entry lands (I think this part is in ROM for the non-RTOS SDK), but the routine there will read the special registers INTENABLE and INTERRUPT which is the actual interrupt line status (until now it just knew we had an UserException, but not which kind). INTENABLE & INTERRUPT gives the bitmask set of currently asserted and enabled interrupts. INUM_GPIO (4) is the index for a GPIO interrupt, and this bit will be set in INTERRUPT if a GPIO interrupt has occured.
  • When you called ETS_GPIO_INTR_ATTACH, it associated your GPIO interrupt handler with entry 4 in an array of interrupt handlers. So the exception handler code looks up the interrupt handlers indexed by the bits set in INTERRUPT & INTENABLE, and then executes those handlers.
  • After your interrupt handler exits, the exception handler pops register values back off the stack, restores a1 from excsave1, and leaves CPU exception context.

None of this is (unfortunately) available as source code at the moment. An unofficial recreation of the same code can be found in esp-open-rtos:
https://github.com/SuperHouse/esp-open- ... _vectors.S
https://github.com/SuperHouse/esp-open- ... terrupts.c

... this version is more complex because FreeRTOS also uses the interrupt mechanism for context switching, so we have to do things there like save the entire state of all registers (whereas non-RTOS SDK can just push working registers to the stack). But it gives you some idea of how the pieces fit together.

Now, you've probably realised from all the steps above that normal interrupt flow uses a lot of steps - lots of branching, lots of cycles.

So my suggestion is that if you want to prioritise USB GPIO interrupts over all other kinds of interrupts, you front-load those checks into the UserExceptionVector. You can either check the INTERRUPT special register immediately, or you can even read the GPIO registers immediately (this might be more useful as you can maybe then do something with them straight away?)

You can do this by replacing either _UserExceptionVector or (if you can't fit what you need in 0x20 bytes) you can replace _UserExceptionVector_1.

There are a few ways to replace an interrupt vector. The espgdbstub does this for DebugExceptionVector by just writing a new jump instruction into IRAM at the DebugExceptionVector address: https://github.com/esp8266/Arduino/blob ... try.S#L282

... this seems kind of hacky to me. I would recommend using the --wrap option of the linker to wrap _UserExceptionVector or _UserExceptionVector_1 at link time.

Probably the best thing you can do, from a cycle-count PoV, is re-implemented _UserExceptionVector_1 as your USB interrupt handler (link with --wrap=_UserExceptionVector_1). So yours will look something like (pseudocode):

Code: Select all__wrap_UserExceptionVector_1:
    save minimal set of working registers to stack
    if NOT a USB interrupt (checking as quickly as you can via exccause, INTERRUPT, GPIO registers, and/or some other way), callx0 __real_UserExceptionVector_1
    ... insert low latency USB stuff here (don't branch again, just do it inline)


You can probably use the same technique for "fast" CCOUNT CCOMPARE0 interrupts as well, these also trigger UserExceptionVector IIRC (with bit INUM_TICK (6) set in INTERRUPT). I don't know if the non-RTOS SDK relies on these at all (in the RTOS SDK they're used as the FreeRTOS tick that triggers context switching.)

Makes me wonder if there's some way I could make the GPIO interrupt a NMI - since they did say once an NMI fires, nothing else can interrupt it.


Like I said in the email, I don't think this is possible for hardware reasons. AFAIK the CPU NMI line is wired to the WiFi hardware and can be triggered by software, but that's it. I think you're better off using normal interrupts anyhow - you have a choice between losing WiFi connectivity due to timing, or losing USB frames due to timing - they are both pretty fault tolerant at the protocol level, but you have full control over the code on the USB side. :)

All of this is getting better in ESP32 - more configurable interrupts, more interrupt levels (so user interrupts can preempt each other), more official source code and some hardware-level documentation available. Soon. :)
Last edited by projectgus on Thu Aug 11, 2016 5:01 am, edited 1 time in total.
User avatar

Re: Comment about low-latency interrupts

By RichardS
- Thu Aug 11, 2016 10:54 am #52704 WOW that was a mouthful :-) welcome to real controllers :lol: (not AVRs)

RichardS
 Reply
Return to ESPUSB - Software