Just to elaborate a bit for those not familiar to Nix (slightly simplified to exclude recent support for content addressing). Nix work with derivations, a derivation is basically a data structure that specifies how a package is built. Derivations are normally not created by hand but using a function (eg. stdenv.mkDerivation).
When you ask Nix to build a package, it hashes a normalized form of derivation data structure. This hash is useful in various ways, but one way it is used [1] is to look up whether the derivation is already in the Nix Store. Because if it is, there is no need to build it. So Nix looks up whether
/nix/<the_derivation_hash>
exists. If it exists, the build is done. If it doesn't exist and you have a binary cache configured (which by default is the binary cache provided by the NixOS project), Nix will look up the derivation hash in the binary cache. If it exists in the binary cache, Nix will download the path to the local Nix store. After that
/nix/<the_derivation_hash>
exists in the store and the build is done (without building anything). Only if that fails, Nix will actually build the derivation.
Now, one of the cool things about Nix is that it is derivations all the way down. So, it's not that just what we traditionally think of as packages is a derivation, but people wrap up all kinds of things as derivations, including configuration, etc. Since derivations are usually generated by functions, there are all kinds of useful functions that make derivations for eg.: single configuration files, scripts, etc.
In the end, building a NixOS system generation is just building a derivation. nixos-rebuild switches to a different generation by just setting a bunch of symlinks to an output path in the store containing that system generation (/nix/<system_config_derivation_hash).
At any rate, when you make a one-line change to a 200-line Nix configuration, Nix does have state to keep track of what it needs to rebuild or not. Nix will just try to build the derivation (and its dependencies), but it hashes the derivations, finds that their output paths are already in the store.
Some might argue that then the store is state. But it's not, at build time you are evaluating a pure function with memoization (the Nix Store).
[1] There is also a package name and version in the store path, but lets keep it simple.
I agree with everything but the last statement. This all comes down to: do you consider memoization to be state.
I predict people's answers to this question will come from experience with memoization. Here's mine: I kept trying to get nix to build tensorflow locally, so that I would get the avx512 benefits of the big, but gpu-less machine I had. I hadn't realized some other derivation had already downloaded tensorflow from online cache, so didn't have avx512 enabled. I kept making shells, trying tensorflow, seeing it doesn't have support. The solution was to tell nix to disregard the nix store, in order to force the local build. This experience has left me with the concrete feeling that the nix store is full-on state, and I the user must be aware of it.
> The solution was to tell nix to disregard the nix store, in order to force the local build.
If this actually led to avx512 being enabled in the package, then that's a bug. Nix builds should not be dependant on the machine doing the compilation, all such autodetection should be disabled via configure flag or patched out.
Then, the right way to enable avx512 would be to pass some 'enable avx512 please' flag to the package's configure flags. Which would then trigger recompilation, without any 'disregard the nix store, in order to force the local build' options.
It is, but the possibility of such bugs is a downside to the approach (not saying it's a showstopper but it is a negative). Some of the nastiest software problems to track down are the ones that cause some fundamental assumption everything rests on (often a cache keying assumption!) to be broken, making everything behave wrong, including the tools you're supposed to use to track down problems.
If you're going to build an entire system on an assumption of referential transparency you want to be able to guarantee that everything really is referentially transparent, and one valid criticism of nix is that it can't really enforce that in all cases.
There is work being done to address outputs by their content hash instead of xor-ing their input hashes, which in theory should eliminate this problem.
Yes, you have unfortunately discovered that sometimes the hardware itself is an input that isn't always captured explicitly, but also isn't controlled for with sandboxing. Ideally enabling avx512 would be an explicit input to the tensorflow package, but based on your experience it sounds like this feature is detected during the build automatically.
I hope that issues like this get better over time thanks to projects like Trustix, which would make non-reproducibility like this more apparent.
> Ideally enabling avx512 would be an explicit input to the tensorflow package, but based on your experience it sounds like this feature is detected during the build automatically
Looks like it sets e.g avx2 on the flags, forcing the package to be most compatible, thus removing the hardware state (the builder may or may not have avx512, ideally Nix packages should remove hardware autodetection to make it pure and consistent in face of cross-compiling).
It should indeed have an input in some way, e.g to add more flags. Then AIUI (still learning Nix) one would be able to call the package function with that input from the dependent package function, thus defining another package than the default one, which would be reified as its own specific derivation for that package to depend on.
I think part of the problem ist that derivation hashs sometimes don't fully cover the intermediate states of a derivation during the build process. This might lead to two hashs pointing to effectively two different configurations. I've had that experience in particular with non-reproducible derivations.
Even one of those in the store will make the store, or at least a subset of it, a state.
When you ask Nix to build a package, it hashes a normalized form of derivation data structure. This hash is useful in various ways, but one way it is used [1] is to look up whether the derivation is already in the Nix Store. Because if it is, there is no need to build it. So Nix looks up whether
exists. If it exists, the build is done. If it doesn't exist and you have a binary cache configured (which by default is the binary cache provided by the NixOS project), Nix will look up the derivation hash in the binary cache. If it exists in the binary cache, Nix will download the path to the local Nix store. After that exists in the store and the build is done (without building anything). Only if that fails, Nix will actually build the derivation.Now, one of the cool things about Nix is that it is derivations all the way down. So, it's not that just what we traditionally think of as packages is a derivation, but people wrap up all kinds of things as derivations, including configuration, etc. Since derivations are usually generated by functions, there are all kinds of useful functions that make derivations for eg.: single configuration files, scripts, etc.
In the end, building a NixOS system generation is just building a derivation. nixos-rebuild switches to a different generation by just setting a bunch of symlinks to an output path in the store containing that system generation (/nix/<system_config_derivation_hash).
At any rate, when you make a one-line change to a 200-line Nix configuration, Nix does have state to keep track of what it needs to rebuild or not. Nix will just try to build the derivation (and its dependencies), but it hashes the derivations, finds that their output paths are already in the store.
Some might argue that then the store is state. But it's not, at build time you are evaluating a pure function with memoization (the Nix Store).
[1] There is also a package name and version in the store path, but lets keep it simple.