Entrant 11 : FORTH language implementation in native ESP8266
Posted: Sun Mar 13, 2016 1:35 pmUser
NiclasH
Description
FORTH is a unique programming language, with a remarkable small footprint, making it possible to embed the compiler into the target system and have a REPL on the target system. The additional overhead for the REPL and compiler is a few hundred assembler instructions in total.
FORTH is a programming language with two stacks, one for data and one for instruction return addresses. FORTH uses Reverse Polish Notation with operations acting on the data stack, so the arguments must be placed on the stack first. Example; 2 + 3 * 4 is written
2 3 4 * +
and the result will be on top of the data stack.
FORTH is extensible. In fact, it has almost no control structures built-in at its lowest level. IF, WHILE and even exceptions are coded in FORTH itself. 95 "words" (functions) has been implemented in assembler, but many are in assembler for performance reasons, and could be done in FORTH itself. About 20 words is the minimum set.
Google for more information about the FORTH language itself.
An interesting aspect of FORTH when it comes to the ESP8266 is that the entire machine code for execution of the FORTH program will fit into ~2-4 kBytes (~1000 assembler instructions) and never leave the instruction cache, no matter how large the FORTH program is. FORTHs total memory footprint is likely to be smaller than the equivalent C program and execute at similar speeds on ESP8266.
There are many implementation standards, and also many variants from the standards. Instead of trying to stick to the bloated ANS FORTH 1994 standard, I decided to use a non-standard variant, called JonesForth, due to Richard Jones' incredibly well documented source code, written for the i386 Linux.
Approximately 1000 lines of assembler needed to be hand translated from i386 assembler to the Xtensa LX106 used in ESP8266.
Instead of trying to implement every hardware aspect in FORTH from the beginning, I call C from FORTH for supporting functionality, such as UART, WiFi and more. Over time, I intend to gradually move some/all of these C libraries to FORTH.
The translation effort was quite challenging, as the Xtensa documentation is very "flexible" due to the nature of the Xtensa "customizable design" approach, and relatively little information of which options that Espressif included in the ESP8266. Trial and error, plus some feedback directly from Espressif sorted these issues out over a couple of weeks.
FORTH's approach to use very little subroutine calls at assembler level made the implementation very lean and efficient, but I suspect a few cycles/bytes here and there can be squeezed out (Performance tests has not been conducted yet)
Interpreter, Compiler, decompiler, standard word vocabulary occupy about 10kBytes of data RAM and about 2-4kBytes of instruction RAM. Additional work can be made to move much of the standard vocabulary from data RAM to instruction RAM to free up more data RAM memory.
Currently missing pieces;
* GPIO, PWM and ADC interfaces.
* Multitasking, perhaps as Actors in Erlang
* Debugger
* Higher level libraries.
Programming in FORTH might not become mainstream on ESP8266, but at least now there is one more language choice, and a choice that doesn't occupy the majority of the small memory space available.
Parts
Links
Github 1
Github 2
Video
Images
NiclasH
Description
FORTH is a unique programming language, with a remarkable small footprint, making it possible to embed the compiler into the target system and have a REPL on the target system. The additional overhead for the REPL and compiler is a few hundred assembler instructions in total.
FORTH is a programming language with two stacks, one for data and one for instruction return addresses. FORTH uses Reverse Polish Notation with operations acting on the data stack, so the arguments must be placed on the stack first. Example; 2 + 3 * 4 is written
2 3 4 * +
and the result will be on top of the data stack.
FORTH is extensible. In fact, it has almost no control structures built-in at its lowest level. IF, WHILE and even exceptions are coded in FORTH itself. 95 "words" (functions) has been implemented in assembler, but many are in assembler for performance reasons, and could be done in FORTH itself. About 20 words is the minimum set.
Google for more information about the FORTH language itself.
An interesting aspect of FORTH when it comes to the ESP8266 is that the entire machine code for execution of the FORTH program will fit into ~2-4 kBytes (~1000 assembler instructions) and never leave the instruction cache, no matter how large the FORTH program is. FORTHs total memory footprint is likely to be smaller than the equivalent C program and execute at similar speeds on ESP8266.
There are many implementation standards, and also many variants from the standards. Instead of trying to stick to the bloated ANS FORTH 1994 standard, I decided to use a non-standard variant, called JonesForth, due to Richard Jones' incredibly well documented source code, written for the i386 Linux.
Approximately 1000 lines of assembler needed to be hand translated from i386 assembler to the Xtensa LX106 used in ESP8266.
Instead of trying to implement every hardware aspect in FORTH from the beginning, I call C from FORTH for supporting functionality, such as UART, WiFi and more. Over time, I intend to gradually move some/all of these C libraries to FORTH.
The translation effort was quite challenging, as the Xtensa documentation is very "flexible" due to the nature of the Xtensa "customizable design" approach, and relatively little information of which options that Espressif included in the ESP8266. Trial and error, plus some feedback directly from Espressif sorted these issues out over a couple of weeks.
FORTH's approach to use very little subroutine calls at assembler level made the implementation very lean and efficient, but I suspect a few cycles/bytes here and there can be squeezed out (Performance tests has not been conducted yet)
Interpreter, Compiler, decompiler, standard word vocabulary occupy about 10kBytes of data RAM and about 2-4kBytes of instruction RAM. Additional work can be made to move much of the standard vocabulary from data RAM to instruction RAM to free up more data RAM memory.
Currently missing pieces;
* GPIO, PWM and ADC interfaces.
* Multitasking, perhaps as Actors in Erlang
* Debugger
* Higher level libraries.
Programming in FORTH might not become mainstream on ESP8266, but at least now there is one more language choice, and a choice that doesn't occupy the majority of the small memory space available.
Parts
Links
Github 1
Github 2
Video
Images