Explaining what Nix is seems like one of its biggest barriers to adoption.
Part of the difficulty is it means different things to different people. My colleague spent a whole lot of time trying to answer this question and ended up with this:
My main take away after spending some time learning about Nix is that it embraces the functional programming concept of a pure function.
If I give a function a certain set of inputs, it will return the same result every time, no matter what. Nix is about building software the same way, whether it’s your own software, someone else’s software, or your entire OS: You declare all your inputs explicitly and it will be built the same way every time.
What really mattered to them, regardless of how they were using Nix, was its ability to bring purely functional programming concepts to computing areas that were previously off-limits.
From that single idea you get a whole ecosystem of tools. We mainly covered the Nix language, the Nix Package Manager, and NixOS, but there’s also a continuous build system called Hydra, nix shell, and a deployment and provisioning tool called NixOps. Probably, there’s even more.
The biggest barrier to adoption unfortunately is not people's inability to explain what the tool is in my opinion. It's that the tool is incredibly complicated, extremely hard to walk someone through compared to alternative projects, and honestly... In my opinion the problem it attempts to solve doesn't really exist.
I'll take an "impure" os or package manager over a pure one any day if complexity is a thousand fold less and the learning curve doesn't require half a decade. Got stuff to do!
> The biggest barrier to adoption unfortunately is not people's inability to explain what the tool is in my opinion.
That's simple: nix is a package manager and the language used by the package manager, NixOS is a Linux distro.
> It's that the tool is incredibly complicated, extremely hard to walk someone through compared to alternative projects, and honestly... In my opinion the problem it attempts to solve doesn't really exist.
From someone who is working as a DevOps Engineer for some years and managing Linux servers for a few years longer that thought is incredible naive. The problem of undefined and undocumented system state is a fundamental problem I encounter everywhere especially bad with legacy systems. I often do things on them blind and just pray for the best outcome, realising months later that some system was broken by one change I did and no one realised that for months.
> I'll take an "impure" os or package manager over a pure one any day if complexity is a thousand fold less and the learning curve doesn't require half a decade. Got stuff to do!
I thought the same first but unwedging Debian once a week on a different system is also not fun and a waste of time and having servers in some undefined state and no one who how the config is supposed to be and why or when it got changed, too.
The result in the end is that every system is different and unique and your Ansible playbook to run a common and good thought out task succeeds on 15 VMs and sometimes completely blows up the 16th because no one could have thought that the state of configuration there is so widely different.
Yes that is a fair comparison. In the latter you have to write down or remember what you did to reproduce it, and even if you make a script it could screw up and leave your system in a bad state.
As someone who seems to be going along a similar path (started as a dev, no one was around to do sysadmin so I did it, and now in trying to modernize a bunch of really old/unpatched servers running a legacy system I'm learning how to devops) I feel better knowing I'm not alone in this struggle.
Yes. Think if you coordinated a software project with email and a Sharepoint folder for the code.
Then you use git.
Current server state management is the former. Nobody knows what is running where and if some performance differences over time or between servers exist, how can it be bisected etc...
You basically have two choices: you can take the complexity upfront, and in a predictable fashion by learning Nix, or you can deal with the complexity after the fact when you’re dealing with dependency hell and a deadline is looming and your boss and/or client is mad.
The Nix language is basically JSON plus syntax sugar plus pure functions. A Nix derivation can be thought of as a super-powered lockfile that includes not just the versions of the dependencies, but also the build instructions and the environment in which to build them.
The argument for Nix is basically the same argument as the one for writing pure functions as much as possible, or not doing so. Any amount of experience doing the former will demonstrate that it is superior.
Now, Nix may be complex, and some of that may be reducible, but the fundamental idea of treating a build like a pure function is NOT reducible, and is well worth the effort of learning, because it will apply to ANY future pure build and dependency management tool
They do, but the people who spend time on Nix issues don't typically notice or count the time saved because it's harder to notice.
When reproducibility issues take 16 hours every 3-4 months/12 weeks and Nix maintenance like updating pins takes 8 hours per month most will feel like the first option is less work.
Imagine if you had the data showing with Nix your build is:
- 99% likely to work
- without Nix your build is 90% likely to work, but 16 hours to fix it when it breaks.
- The non-Nix build also has a 10% chance of it breaking randomly at any time.
- The nix build initially takes 8 hours per month to maintain for 6 months, 4 hours for the next 6 months, then 1 hour per month thereafter
Which do you feel would be better? What I describe above has been what the situation seems to be in my experience.
I can say from personal experience I've seen many days devoted exclusively to Nix upkeep and maintenance. That was from junior people to people who had spent half a decade or so deeply in the community and using Nix for their daily driver.
I've never had to do much for Nix itself, but packaging something to build from source can often require quite some effort. Applications that use a pretty unconstrained build/install process upstream may expect to do a lot of things that are not allowed in the Nix build sandbox, like unconstrained network access and or overwriting files in existing packages on the system. To deal with that you really have to dive in, learn how the sausage gets made in the upstream package, make some choices about if/where to compromise, and then spend some time tweaking and debugging. That can be a pain and can definitely take a day or two.
I've only had 'maintenance' issues with Nix itself on macOS, where OS upgrades routinely nuke Nix's hooks into the OS or add restrictions that break things. (But they do that to other package managers as well.)
I'm taking that approach with the package I've been working on, which has a somewhat pathological (by Nix standards) Gradle build which does things like
- manually download a copy of Elastic search outside of the normal Java dependencies scheme
- run NPM to fetch remote libraries to build web assets at build time
- *also* run Yarn, for some reason
- use Git at build time
The ways it does all of these things are actually fairly thoughtful (for example, it does checksum the artifacts it manually grabs at build time to verify their contents), but they don't play nice with running builds in offline mode or under a user that has no $HOME. But it's one of those freeform 'my build tool configuration is a weird DSL in an imperative, general purpose, Turing-complete language' situations, and I'm not very familiar with either the language (Groovy) or the DSL. So it's a lot of quirks to cope with.
I've made quite a bit of progress in building it from source by making a few small patches and eventually disabling the sandbox for now, but it's still dying on a weird test failure for reasons I don't yet understand. At this point I'm just back to munging the binaries provided by upstream because I was mostly building from source to learn about the project and how it's distributed/deployed anyway.
I messed a bit with gradle2nix for a better-behaved, old school FOD-based build with Gradle in offline mode, but that was pretty brittle as gradle2nix is unmaintained, and due to some design limitations it couldn't actually capture all dependencies. I'm kinda interested in working out something better but on the other hand, this is a third-party package and I don't myself use Gradle or Groovy for any kind of development, so mastering Gradle's quirks and wrangling it into the Nix sandbox for this package is more of a yak shave than a practical skills investment for me.
> In my opinion the problem it attempts to solve doesn't really exist.
Almost all of Docker use-cases are for solving that same problem, but badly and with partial completeness. The lack of adoption is really not caused by lack of value.
This matches my experience with it so far. Extremely complicated and hard to understand, projects that use it have builds fail anyway except now with very hard to debug errors.
Good point. I'm much more likely to help others because I know I can get to the exact state they are in and reproduce their issue with a simple `nix build`.
That is absolutely not true. If you start to get the hang of it and follow the way things are supposed to be done then things get easier over time. You need to invest upfront more time into your configuration but on the long run it pays off and saves you from an entire error class.
> projects that use it have builds fail anyway
The point of Nix/NixOS is not to have no failing builds but that those are reproducible and deterministic as much as possible and that those failures are noticed early and before the point of no return. A system build is supposed to fail early and not mid way through a major update and prompting you to merge some config under /etc by hand.
You say it's "absolutely not true" that it's "hard to understand", then go on to explain how you have to "invest upfront more time", and also that you have to learn "the way things are supposed to be done".
You are very clearly describing that people have to work more to understand it. The person you're replying to even tried! Denying the experience of other people does not make that go away. It just means that the problem you're pretending doesn't exist will never get fixed.
I've never seen it actually pay off in industry. I've seen it be used as good job security while other devs just wrote docker files and got things done.
As the only guy maintaining the flake.nix in my team's repo, I don't think it's really contributing to my job security. I'm just happy that they don't mind the extra files and commits here and there because I value the ability to contribute from different devices without worrying about which versions of what are installed.
Maybe it'll be job security if people start agreeing that downloading binary tools in in CI without a hash check is unacceptable attack surface, but until then it's just this weird thing I'm doing on the side.
I do catch a lot of bugs where people are relying on dependencies that they happen to have installed but have not declared. It's the kind of thing that prevents newcomers from being successful out of the gate, or makes taking a local process and putting it in CI difficult, but fixing those is not exactly high visibility.
Then you're not paying enough attention. There are plenty of companies using nix to distribute a reproducible environment (if you don't believe me, why not go search GitHub for "flake.nix" and see how many "industry" repos you find).
I think it would be more productive for you to sit down and give it a fair chance than posting little rebukes all over this thread.
I gave it a fair chance and it was a deciding factor in why I left a company believe it or not. Only one person could maintain and fix deployments. Not from lack of trying from seasoned experts and no new comers. It was the worst user experience I have probably ever encountered. Meanwhile I was able to pick up terraform and docker in a matter of days...
In my experience those "others" that "just wrote docker files" are exatly the ones that don't know how to build the system in a reproducible manner if their ci environment gets reset for some reason as they find out that stuff that was "supposed to be there, pinned and configured" wasn't.
In my experience the months required to get a handle on Nix is not worth the benefit(which is shakey in my opinion) compared to competing technologies. We don't have to agree, but that's my take...
If you have something easy to deploy like a go binary you can just write a dockerfile but for big python projects that start to compile dependencies that is quickly no longer true.
The dockerfile likely is also not matching the software you run and test on your local machine, so sometimes debugging is not as easy. Ofcourse you can debug inside the container but then you are missing all your tooling and need to bring that with you. And rebuilding a dockerfile is often not reproducible, so if you want the container back from 1 year ago and you no longer have the artifact you are probably out of luck.
With nix you can easily open a shell with the packages used in the docker image or go back in time and reproduce that image from a year ago with the flake.lock from a year ago.
Also applying patches to dependencies used in dockerfiles is not dead easy as with nix.
Most people would opt to not apply patches to their dependencies in my experience. Seems kinda sketchy if that's something you have to do on a regular basis. I'd chalk that up as a possibly serious business concern depending on the magnitude of the fixes, the importance of the dependency, and the frequency.
I have written software that would have been 100% not package-able any other way.
Also, let’s not lie to ourselves, there are plenty of ridiculous contraptions out there, like docker-images used for ML that take up some insane space, and are updated each day. Packaging is a hard problem, and there is finally a tool that can actually solve it.
You can continue spending your time messing around with your system then. I learned nix in a short amount of time and it has supercharged my development workflows and reduced the overall complexity. I have too much stuff to get done to not use it.
Based upon how I manage my system, Nix appears to be something that I could use productively. The problem isn't so much of them being able to explain what the tool is, but one of them being instilling confidence that it lives up to their claims. We exist, after all, in an industry of hyperbole. It also doesn't help that their solution is layered on top of an operating system that has traditionally been managed in a very different way.
A friend of mine said that he is currently using it instead of packer at his current gig. He can use the same code to build any type of output, AMI, docker image, VM etc. I dig that. But I'm still not gonna learn nix because I don't do enough of that stuff to warrant the pain of learning nix.
You don't have to manage your system with NixOS to reap the benefits of Nix. It solves very real problems that very much exist, it might not exist if you're a one-man show deploying WordPress to GoDaddy though.
Barrier to entry:
1. Run the nix installer
2. Enable flakes
3. cd project
4. nix run
This ensures you run the package with every dependency except the kernel pinned to a hashed version. If dependency hell is not a problem for you, be happy!
Ok and this requires root access, sets up some global directories under root, and a new user. Me as the administrator: why the hell do I need a new user and what is the nix store and what are the conditions that mutate it? (I know the answers to this question, but it's a barrier for people who give a shit).
> Enable flakes
What the fuck is a flake? Reads a bit... what the fuck is a derivation? (again: I know the answers to these questions already, but the invention of jargon by nix devs is a massive barrier to entry that shouldn't be overlooked, it's extremely confusing)
> cd project
Ok now I'm comfortable doing things I know
> nix run
Fine, but what about auto envs and nix shell? I don't use these with make or cmake. I need to attach a debugger, where does it go? How do I set up my IDE that has no idea nix exists?
My point is, nix has a lot bigger of a barrier than these four lines, and it's really naive to think that's it.
Okay, so you're required to read some documentation ahead of time, that's where your problem lies.
What's a Terraform module? What is Terraform? What is a provider? Why don't I just build all my infrastructure with the AWS Console? Why is it it's own weird language? What is this state thingy that just ended up in my folder? Do i give it to the devs?
I think it's pretty much consensus that Terraform is great for provisioning anything with an API. Nix does the same for your packages, partitions, OS, containers, shells and many more things in the same functional manner.
In a company not everyone has to be a Nix wizard either, if a small team knows Nix they can build the Nix infra, then developers can reap the benefits of not having to mess with it at all.
Just because people are unable to comprehend the benefits doesn't mean they do not exist. And if you wanna reap great benefits you might need to spend an hour or two reading things.
Yes it's a novel way of doing things, but it's also one of the most actively developed projects with one of the highest amounts of contributors in the world.
> Fine, but what about auto envs and nix shell? I don't use these with make or cmake. I need to attach a debugger, where does it go? How do I set up my IDE that has no idea nix exists?
The people that know Nix well enough will assist the ones that doesn't know, if you enter a nix shell and start vscode from there it'll be aware of $PATH which Nix sets, meaning it'll find all your dependencies.
On a tangent: I wonder why this is still the default.
The nixStatic binary has, since quite a while, support to as a non-root user create a "${XDG_DATA_HOME:-${HOME}/.local/share}/nix/root/nix" -> "/nix" unshare chroot before running the rest of the command if "/nix" is missing.
It's only a real issue if you really need to run something as root, or something else that needs unshare chroot itself, but in that case, I guess you could just have a /nix store folder anyways.
Almost everything here is valid, but iirc the last time I ran the nix installer I thought it offered a home directory based install now that doesn't require root.
You can use in theory something other than /nix, but then you have to recompile everything yourself. So not many people do if, not sure about it’s state.
I just dipped my toe into the Nix pool a couple of weeks ago, and there are instructions to use root to create a /nix and grant ownership rw to your user account. No further admin required, but everything else seems to work as if using the Nix root user method.
The installer used to have an option for this, but nowadays it's discouraged. How come you wanted a single-user install so much? And j/w, are you on macOS?
Ok cool! The reason I asked is that the multi-user setup has more benefits on Linux than on macOS. That's because on Linux, the full build sandbox is actually available, and the store's immutability is enforced by default.
The tradeoffs are obviously yours to consider. But the normal Nix build sandboxing helps protect you from nasty things like crypto miners in setup.py or whatever, as well as improving reproducibility.
That's less relevant on macOS where the sandboxing story is not so great.
Personally, using Nix with a daemon seems like a better setup to me but adding another highly privileged process unnecessarily is obviously a real security concern. There is some ongoing work, btw, to reduce the level of privileges that the Nix daemon needs.
Is Nix harder to learn for somebody who knows nothing about computers and OSes? Probably not. It might even be easier.
But that's not Nix's primary audience. It's targeted at people who already know a fair bit about the current paradigm and have plenty of skill with it. For those people, it's very different. Things that are easy for them will be hard until they learn the new paradigm.
I get that people who have already learned Nix and like it are in the new paradigm. Understanding it is not a problem for them. But pretending that work doesn't exist for others is unhelpful. And this sort of casual dismissal makes me suspect that even if I learned Nix, the experience still might be pretty bad. If Nix advocates can't take seriously the difficult noob experience, maybe the experience is painful all the way through.
I think the parent was focusing on the first point about the install. That particular argument against creating users and writing to directories would be a barrier to install any software.
Imo the harder part is learning bespoke build processes that you may not own in order to get software that assumes it can perform arbitrary network access or other naughtiness at build time to build successfully in a restricted sandbox.
The language is maybe a little strange at first but there's really not much to it.
You don't have to know how to write Nix to write 95% of all Nix, it's just "JSON with functions" after all. There are definitely advanced things that the Nix and NixOS developers maintain for me.
Just like I don't know how to implement any crypto, or how to implement efficient 3D pathfinding I don't know how to implement NixOS. But I can write a derivation using the helper functions for the language I want to package, which aren't many these days since nixpkgs is huge already.
> Part of the difficulty is it means different things to different people.
That's definitely how it seems to me. The pro-Nix stuff I see is generally about the theory much more than the practice. Which was also my experience with functional languages when their hype cycle was last on the rise.
On the one hand, that's fine. I like ideas, and I think taking an idea and running with it can be really interesting. You can clearly see that in history's various art movements, for example. On the other, for people who are just trying to get things done, it's often alienating and tedious, because the people in the grip of their Big Idea often seem heedless of other perspectives, and frequently can be quite evangelical about it.
Personally, my strategy with Nix, as with the various functional languages, is to keep a distance from it, waiting and seeing. Perhaps it will influence more mainstream projects, bringing the benefits to me without a lot of upheaval. Perhaps I'll have a project that really needs its particular benefits, and so I'll take on the cost of a paradigm switch. But in the meantime, I have stuff to do.
This is probably a pretty good read on the situation. People who have really thrived with Nix are often at the intersection of 'FP people' and 'extremely stubborn Linux people', and that's because sometimes it takes getting your hands dirty and fighting a build system that belongs to a package you've never used before to get the Big Ideas to pay out. When the footwork is already relatively familiar to you, it makes it easier to push through whatever obstacles there are to playing with those attractive Big Ideas in practice.
The Nix community's roots are definitely with FP people, partly due to the language and its inspiration and design, and partially also due to early success using Nix to solve particularly painful Haskell dependency hell problems years ago. All of the original 'marketing material' for Nix focused on principles and properties that would be attractive only to people who already knew and valued those things, which was mostly FP folks.
> Perhaps [Nix] will influence more mainstream projects, bringing the benefits to me without a lot of upheaval.
This is definitely already happening. Off the top of my head, Nix has served as inspiration for Guix, Habitat, and Spack, which are all respectable package managers that try to make things a little smoother than they are with Nix in terms of UX. The latter two are also more conventional, with a relaxed notion of 'purity', and so it may be easier to get packages to build in them when those packages are built in problematic ways. (Guix, if anything, is even stricter about packaging conventions than Nix, but it has a really nice CLI and the language might resonate more with some people, so if Nix has given you pain it's definitely still worth trying.
> Perhaps I'll have a project that really needs its particular benefits, and so I'll take on the cost of a paradigm switch. But in the meantime, I have stuff to do.
I love Nix and its fundamental design, and I want to see it flourish and grow, both in general and in my own professional life. But at work, I try to maintain the same attitude as you describe here. I use Nix for myself everywhere I can (with some escape hatches in place!), but for projects that others work with, I only use Nix where I feel that some specific aspect of the project calls for it.
All of that is just to say that even to some folks who really are drawn to Nix in substantial part due to the Big Ideas that power it, your pragmatic stance is quite understandable and entirely welcome.
Feel free to just play around with Nix in a low-stakes way and advance your usage as curiosity or new problems drive you to do so. You don't have to jump all the way in to benefit from Nix or get a taste of it.
Thanks! Really helpful comment. Especially this bit:
> People who have really thrived with Nix are often at the intersection of 'FP people' and 'extremely stubborn Linux people'
As an extremely stubborn Linux person, that makes total sense to me. But I also almost never recommend Linux to average folks, because I'm keenly aware how far along certain bell curves I am. I would love it if more advocates reflected on whether the personal characteristic that makes a technology great for them is one that makes it bad for others.
Thanks also for the pointers to projects inspired by Nix. I'll check them out.
I often find difficult to explain ideas the most interesting ones. It’s a new paradigm, naturally we need to develop “the neurons” to deal with it, to reason with it, to hold the paradigm in our minds.
I find the best way to get the paradigm is to dive in.
I see from people that did that they are always enthusiastic, it must be worth it.
It doesn’t help that the term is hugely overloaded. Nix might mean the package manager, the shell, the operating system. There are so many valid permutations of letters; why they chose to reuse Nix for everything is a mystery to me.
The other issue is complexity. If you can manage to figure out the jargon, you’re greeted with the requirement that you completely port an entire project to get any benefit, and that’s non trivial. It requires learning a whole new language, and when the (highly opinionated) language conflicts with other tools’ ideas, for example pip, the documentation generally bashes the other tool, boasts how much better it is, and then proceeds to have devs write dozens of lines in a new language they invented when one line of Python used to be enough.
Part of the difficulty is it means different things to different people. My colleague spent a whole lot of time trying to answer this question and ended up with this:
https://earthly.dev/blog/what-is-nix/Another way someone else described Nix to me was "Gentoo for Haskell devs".