I’ve been having a pretty good time digging in to NixOS over the last month or so. Got a bit bogged down trying to declaratively configure applications I seldom is actually use via home-manager, but that’s on me, not the tools.
I’m currently cranking through this upgrade and I’m not sure I want to run the bleeding edge release again any time soon, as it appears a number of large packages (like, WebKit-sized) don’t have cached binaries yet. My nixos-rebuild command for the upgrade has been chugging away for a couple of hours now and doesn’t appear to be all that close to done.
On the plus side, I know I can roll back at any time, which means I don’t feel at all stuck if I want to hit eject and keep running on 20.09 a while longer. So, yay NixOS, I think? Just maybe don’t try the upgrade on a machine with less than 4-6 fast cores unless you really like watching the output of configure, cmake, and g++. ;)
I use this trick to delegate builds on my X1C7 to the more powerful P71. What happens here is that a nixos-rebuild switch (or even per-project nix-build; like compiling GHCJS projects!) in X1C7 will use ssh to do the actual builds on P71, and then download the built binary assets from it.
My primary dev machine is actually a P1 Gen2, so it’s not so much a question of CPU power or RAM as it is needing the keep the machine plugged in and awake throughout the build. :)
That being said, I have a nice 8C/16T Xeon box in my office closet with gobs of RAM that would happily grind through these compilation cycles so I’ll definitely look at offloading builds.
Possibly naive question: If your OS configuration is already reproducible and declarative because it is all managed by Puppet or Ansible (assuming you aren’t abusing those tools to run non-idempotent scripts), does NixOS bring a lot of extra value?
Using Puppet, Ansible, or Salt can sort of replicate the declarative nature of NixOS, but they fall short of duplicating all the behavior.
For example, if you’re managing a file or package with Salt, and you decide you don’t need it anymore, you have to remember to do file.absent, package.absent, or similar to actually remove it from running systems. In NixOS you can just remove the declaration, and it will be removed from the active system. We’ve had many incidents at work caused by someone forgetting to remove a configuration file or systemd unit when no longer needed.
Puppet/Ansible/Salt can still display a “hysteresis” effect: because they rely on e.g. apk, their reproducibility is kinda “best effort”. Notably, installing some package and then uninstalling it with them can leave your system (say, the /etc dir) in a different state than before, even if your “declarative” config is identical as it was before. With NixOS, identical config guarantees (via hashes and readonly mounts) exactly identical OS directory tree and to a much wider extent, including /etc, and fully accounting for removed files. There’s still a few things that are hard to avoid keeping modifiable (/var, $HOME, hardware, …), but other than that, it’s a very different level of immutability/reproducibility. That said, it necessarily makes NixOS more “total”, i.e. you either fully go with it, or not. (There could be ways to make this more gradual, but no such approach has been developed enough to be publicly usable by others yet AFAIK.)
To paraphrase a former coworker, Puppet needs to be run repeatedly until the log is all green. Nix only needs to be run once. (I will admit that systemd can get hung up and need multiple attempts to figure out its units, but Nix only needs one run to change which units systemd is working on.)
The main benefit of NixOS, if you got everything else replicated in Ansible/Puppet (which is very hard), is that you can do atomic upgrade and switches between systems. Nix also allows you to use a systems current configuration and copy it 1:1 to another machine and switch to that atomically.
Once you managed to declare all your system configuration with Nix you use the same expressions (and the already generated files in your Nix store) to produce Installer ISOs, VM Images, … without much hassle. This goes as far as testing a new configuration of your system in a VM that doesn’t require building a full VM image. Instead if can just mount the subset of paths required for the system into the VM. This allows you to iterate on the level of seconds instead of whatever Puppet/Vagrant allow you to do. In my experience the whole Puppet workflow is at least tens of minutes and might then still install something different every time (due to timing, package sources being upgrade halfway, …).
Don’t forget the developer goodies. I mostly use the Nix package manager on non-NixOS linuxes to quickly get to a project of mine and open a shell with all dependencies set just the way I need them to start working on that project. If you’re familiar with python’s virtualenv – it’s a similar experience but with all the C/system dependencies that you need.
With Puppet or Ansible I’d need to maintain separate setup scripts and documentation for dev and non-dev environments.
Not really. Of course assuming that Puppet or Ansible configuration is truly idempotent which is very hard thing to do. In the other hand achieving idempotency with Nix is much easier.
I’ve been having a pretty good time digging in to NixOS over the last month or so. Got a bit bogged down trying to declaratively configure applications I seldom is actually use via home-manager, but that’s on me, not the tools.
I’m currently cranking through this upgrade and I’m not sure I want to run the bleeding edge release again any time soon, as it appears a number of large packages (like, WebKit-sized) don’t have cached binaries yet. My nixos-rebuild command for the upgrade has been chugging away for a couple of hours now and doesn’t appear to be all that close to done.
On the plus side, I know I can roll back at any time, which means I don’t feel at all stuck if I want to hit eject and keep running on 20.09 a while longer. So, yay NixOS, I think? Just maybe don’t try the upgrade on a machine with less than 4-6 fast cores unless you really like watching the output of configure, cmake, and g++. ;)
If you have a relatively powerful computer lying around, you can distribute the build on to it.
https://nixos.wiki/wiki/Distributed_build
I use this trick to delegate builds on my X1C7 to the more powerful P71. What happens here is that a
nixos-rebuild switch(or even per-projectnix-build; like compiling GHCJS projects!) in X1C7 will use ssh to do the actual builds on P71, and then download the built binary assets from it.If you do not have a powerful computer, there is also https://nixbuild.net/
That’s a great idea; thanks!
My primary dev machine is actually a P1 Gen2, so it’s not so much a question of CPU power or RAM as it is needing the keep the machine plugged in and awake throughout the build. :)
That being said, I have a nice 8C/16T Xeon box in my office closet with gobs of RAM that would happily grind through these compilation cycles so I’ll definitely look at offloading builds.
You can get cached binaries in unstable if you checkout one of the commits that got built by hydra: https://hydra.nixos.org/jobset/nixpkgs/trunk
You can also get the commit hash from https://status.nixos.org - which displays it for all active channels.
“Upgrading” then merely is a matter of changing this line in
flake.nixand runningnixos-rebuild switch.Possibly naive question: If your OS configuration is already reproducible and declarative because it is all managed by Puppet or Ansible (assuming you aren’t abusing those tools to run non-idempotent scripts), does NixOS bring a lot of extra value?
Using Puppet, Ansible, or Salt can sort of replicate the declarative nature of NixOS, but they fall short of duplicating all the behavior.
For example, if you’re managing a file or package with Salt, and you decide you don’t need it anymore, you have to remember to do
file.absent,package.absent, or similar to actually remove it from running systems. In NixOS you can just remove the declaration, and it will be removed from the active system. We’ve had many incidents at work caused by someone forgetting to remove a configuration file or systemd unit when no longer needed.Puppet/Ansible/Salt can still display a “hysteresis” effect: because they rely on e.g. apk, their reproducibility is kinda “best effort”. Notably, installing some package and then uninstalling it with them can leave your system (say, the /etc dir) in a different state than before, even if your “declarative” config is identical as it was before. With NixOS, identical config guarantees (via hashes and readonly mounts) exactly identical OS directory tree and to a much wider extent, including /etc, and fully accounting for removed files. There’s still a few things that are hard to avoid keeping modifiable (/var, $HOME, hardware, …), but other than that, it’s a very different level of immutability/reproducibility. That said, it necessarily makes NixOS more “total”, i.e. you either fully go with it, or not. (There could be ways to make this more gradual, but no such approach has been developed enough to be publicly usable by others yet AFAIK.)
To paraphrase a former coworker, Puppet needs to be run repeatedly until the log is all green. Nix only needs to be run once. (I will admit that systemd can get hung up and need multiple attempts to figure out its units, but Nix only needs one run to change which units systemd is working on.)
The main benefit of NixOS, if you got everything else replicated in Ansible/Puppet (which is very hard), is that you can do atomic upgrade and switches between systems. Nix also allows you to use a systems current configuration and copy it 1:1 to another machine and switch to that atomically.
Once you managed to declare all your system configuration with Nix you use the same expressions (and the already generated files in your Nix store) to produce Installer ISOs, VM Images, … without much hassle. This goes as far as testing a new configuration of your system in a VM that doesn’t require building a full VM image. Instead if can just mount the subset of paths required for the system into the VM. This allows you to iterate on the level of seconds instead of whatever Puppet/Vagrant allow you to do. In my experience the whole Puppet workflow is at least tens of minutes and might then still install something different every time (due to timing, package sources being upgrade halfway, …).
Don’t forget the developer goodies. I mostly use the Nix package manager on non-NixOS linuxes to quickly get to a project of mine and open a shell with all dependencies set just the way I need them to start working on that project. If you’re familiar with python’s virtualenv – it’s a similar experience but with all the C/system dependencies that you need.
With Puppet or Ansible I’d need to maintain separate setup scripts and documentation for dev and non-dev environments.
Not really. Of course assuming that Puppet or Ansible configuration is truly idempotent which is very hard thing to do. In the other hand achieving idempotency with Nix is much easier.