LLForth — Experimental implementation of Forth in LLVM
As a my personal project, I had recently written a Forth language processor targeting LLVM and finally published its source code — LLForth. In this blog, I’d like to introduce what it is, and why/how I did it briefly.
What is LLForth?
It has two tools llforthc and llforth . The former is compiler from Forth source code to LLVM IR but not supporting full features like if.
The latter is interpreter of Forth language which is written in Forth. This is compiled from a source code in Forth to LLVM IR by llforthc .
Then, you can try llforth as an interpreter of Forth like this:
$ ./llforth
> 1 1 + .
2
> : hi ." Hello world!" ;
> hi
Hello world!The detail of architecture is described here.
Why I did it?
It was for developing my understanding of computer architecture. I have never learned about compiler or virtual machine before, however, I have had strong desire to understand them since I started my career in this industry about 10 years ago. So I did this.
It involved so much stuff for me:
- Stack machine (Forth) implementation on register machine (LLVM)
- Compiler and interpreter
- Indirect threaded code
- LLVM API, C++17, Rust, CMake
Now I feel a little bit better around these area. Although it is kind of toy project, it works with a real Forth programming language somehow. I know my implementation is so poor because this is my first C++ and Rust project, but I’m proud of my first implementation of programming language processor.
How I did it?
I started researching about Forth about eight months ago by reading some books. Since I had no idea about Forth (actually it was the first time I heard that name when I met these books), everything was amazing, especially from two awesome books.
“Stack Computers: The New Wave” (published in 1989 – which is 30 years ago!) provided me why stack machine is important. “Low-Level Programming” has a concrete example of Forth implementation in assembly (x86_64), which is basic of LLForth.
Once I got basic knowledge, I started some Proof of Concept, especially Indirect Threaded Code (ITC) in LLVM IR and it worked!
After that, I suspended this project because I moved from Tokyo to Vancouver in June and started a new job, which consumed my whole energy. Then, I resumed it two months ago and completed now. Timeline was like below:
- Apr 2018: Researching by books
- May 2018: PoC for LLVM
- Jun 2018: (Moving to Vancouver)
- Nov 2018: Starting basic C++ implementation
- Dec 2018: Adding more words, Rust library
I had little knowledge about LLVM, C++, as well as Rust, FFI which are used to provide stdin interaction, so it took much time to get used to them. But I was able to achieve my first implementation of a language processor in two months.
Summary
It was possible to implement a Forth compiler/interpreter in two months and it provided me bunch of knowledge around computer and programming.
I recommend you to implement a Forth processor if you are interested in these area but don’t have enough knowledge like I was eight months ago. Forth is pretty simple language but has superpower. Enjoy!
