Modern graphic card drivers contain full compilers, for the GPU's shading language. It seems reasonable that you can extract performance with better compiler technology.
On the other hand it also seems reasonable that a compiler for e.g. NVIDIA's hardware might be hard to re-target to another vendor's hardware while retaining the same advantage.
As I understand things (which I'm clearly not an expert at all), CPU opcode is like an ABI,
put this value into register A,
this value into register B,
and a magic opcode into register C,
and poof, the register D contains the value of A+B.
A driver will say that you put int X into memory point A,
Y into memory point A+1, and call function C with argument of A and D will contain the multiplied number.
Let's say your driver exports an API that computes f(x,y,z) where the math turns out such that g(h(x,y),i(y,z)) is an easier and/or numerically better way to compute it.
Let's also say your hardware implements g, h and i, but doesn't wire them together yet. So, your driver's code calls the three functions.
Unknown to you, the trick to use that decomposition is patented. if you release your driver's source code, you make it easier for the patent holder to figure out that you are violating their patent (your programmers might even have made it patently obvious by mentioning the paper describing the trick in a comment)
Unlikely? Maybe, but the way patents are written, who knows? For an almost random example, I searched for "driver code and hardware patents". The first link I clicked was https://www.google.ch/patents/US20090006831. Reading that, I wouldn't know what driver would _not_ infringe on it.
Also, there are many patents on ways to move data around efficiently. Avoiding all of them while still writing a performant driver at not be possible.
