Well, it depends; if you can employ locality of code, cache lines etc. to your advantage it might pay off doing crazy things to optimize the innermost loop and similar. Though as you said, you'd structure whole blocks of functionality more higher-levelish. On some embedded systems it still pays off to modify code during execution, e.g. replacing a constant in innermost instruction by pre-computed value from higher level to avoid memory access penalty.