I thought this was about:

https://en.wikipedia.org/wiki/ML_(programming_language)

One approach is to build a compiler for a small but useful part of the model space.

For example, Fabrice Bellard's LibNC (https://bellard.org/libnc) handles transformers and LSTMs, but no convolutions.

NN-512 (https://NN-512.com) handles convolutions and is intended for, e.g., DenseNets, but doesn't support transformers.

Niche support, specialization.