First of all, this is good, cranelift is good.

Second of all, it's really funny how LLVM started focused on JIT with emphasis on compilation speed, and now is quite slow. Cranelift started in the same way, and now is gaining more passes and IRs, etc.

I think more passes and IRs is good for clarity, correctness, and programming productivity, and single-pass designs are too brittle since the purpose of all software (the requirements) changes over time.

The solution is we redouble our focus on fusion techniques to try to allow algorithms specified in multiple passes to be fused into a single pass. Then libraries like cranelift and LLVM can prevent a variety of passes, and one has the option to either go for compiler performance, picking only those that fuse or speed of compilation code, picking additional ones that don't fuse.

Yeah this is a good observation. Some related papers I saved:

Miniphases: compilation using modular and efficient tree transformations, Odersky et al.

https://scholar.google.com/scholar?cluster=79468638640367065...

(I'd be interested if anyone has any knowledge of how this actually worked out in the Scala compiler; I don't have experience with it)

TreeFuser: a framework for analyzing and fusing general recursive tree traversals. This one said they prototyped it in Clang.

https://dl.acm.org/doi/abs/10.1145/3133900

Deforestation: Transforming programs to eliminate trees, Wadler

https://scholar.google.com/scholar?cluster=13826873296507613...

I think the key point is that some of the frameworks introduce restrictions that make it hard to write a big program like a compiler. There is a hard tradeoff between the amount of fusing that can be done and how expressive the metalanguage is.

---

related: https://github.com/oilshell/oil/wiki/Compact-AST-Representat... (compiling programs that operate on trees to operate on packed representations)

> Miniphases

To my knowledge the (to be released) Scala 3 compiler[1] implements this concept.

Interesting and related is also Odersky's "compilers are databases"[2] concept.

[1] https://dotty.epfl.ch/ [2] https://www.slideshare.net/Odersky/compilers-are-databases

His database concept can be seen already being discussed for C++, a couple of decades ago.

https://www.dreamsongs.com/Cadillac.html

Ironically modern C++ are only now catching up.

Chez Scheme is a good example on how many compiler passes can be extremely effective.

https://github.com/nanopass/nanopass-framework-scheme/

The earlier version of this article on Chris Fallin’s blog was discussed at https://news.ycombinator.com/item?id=24526088

Cranelift looks really promising, and this article is a great peek into how it generates machine code. +1

It is also faster to compile than using LLVM, ideally it will eventually become another cargo option.

In case anyone is wondering what’s happening with this, there was a progress report in Sept https://bjorn3.github.io/2020/09/28/progress-report-sep-2020...

I have a very powerful Rust transpiler back end that uses semantic programming. My profile has links to the programming engine and the details are put together across many semantically organized issues. The software still needs work and the code generator still needs Rust's lexemes to get to the next stage. The #wg-grammar issue and Discord channel has those plans