You're welcome. The thanks should go to haberman, though, who not only linked to the email but apparently also asked the questions that led to such a great explanation.
Reading it really made me want to do some assembly programming.
DynASM, which I used to write my JIT (also written by LuaJIT's author Mike Pall) is also an impressive piece of work. It makes writing a JIT so pleasant that I meant to write an article walking through the process of building a simple JIT. I was going to implement a JIT for the "Universal Machine" from ICFP 2006 as the example (http://www.boundvariable.org/), which is an absolutely delightful problem, but unfortunately discovered that it is heavily self-modifying, and thus would be hard to get a big performance boost from. But it looks like someone did it anyway: http://paul.bluereboot.net/um/
It makes writing a JIT so pleasant that I meant to write an article walking through the process of building a simple JIT.
Please do, if you have the time. DynASM really is a stroke of genius.
The way it works is that the JIT contains bits of pre-generated machine code, and then a little bit of bytecode stitches these bits together and emits them. This means the actual runtime piece of the JIT doesn't need to know all the wacky details of x86 instruction encoding. All that is done in a preprocessor written in Lua.
I never got around to finding out how to do dynamic register allocation though. It seemed to me that there was no way to parameterize the pre-generated machine code, but presumably I'm missing something.
I keep wanting to try it, too. But Lua + LuaJIT is so good that I find I never really need that last ninety-ninth percentile speedup.
One of the things that I have found marvelous and counterintuitive about LuaJIT is Mike Pall's recurring warning to unlearn optimization idioms you might have picked up to accelerate standard Lua (which is already quite fast).
For example, the Lua language has a keyword 'local' to give you a shortcut optimization for identifier lookups in the global namespace. Using a local alias for a global function in a tight loop inside a standard Lua program can and often does result in a nice, juicy speedup.
LuaJIT, however, has highly optimized heuristics for i386, x86_64 and PPC ASM that will perform far better optimizations for you if you leave your clever aliasing out and just run a naive, plain vanilla Lua version. See [0]:"No, please don't do this."
He has also provided a new mechanism to dynamically wire C libraries and ad-hoc data structures directly into the LuaJIT runtime [1].
If there is scope for me to use assembler in this ecosystem somewhere, I can't see it yet. :)
Mike did say on the list that he was considering at some point adding the ability to natively mix assembler code into Lua, that might give you an opportunity!
Reading it really made me want to do some assembly programming.