Do you use Nix or equivalent to manage projects and/or systems?

15

lucasew 4 months ago | link

I am a Computer Science undergrad Nix/NixOS user with over 2 years of experience in this [0], and I tried to apply Nix at $WORK.

We use it to set up dev environments to use Blender with some extra sauce and libraries, so we can replicate the same environment basically everywhere. We found a way to use Blender as a Jupyter Kernel based on someone else’s code [1].

Initially, we used Google Colab to run some stuff we need, and we were able to run it there by using a script I did [2].

Now we use a beefy machine for the number crunching, so we use Nix to build container images to run there. The same common code is reused, the difference is basically the entry point.

My advice is to not try to use it everywhere. Is it cool? Yes! But it can get overkill very quickly. It is giving very nice fruit for this Blender specific use case, but for training stuff it’s better to just stick with one of the many ready-made supported containers.

Another advice is that for iterative work just build the base container and when you are using that in the final machine just pass the code folder as a volume. We had a lot of iterations to set it up, but after it’s done we didn’t have to rebuild and upload the image anymore.

A not so cool thing about Nix containers is that it’s not integrated with Docker. When you run docker build it builds and consequently adds to the local storage. With Nix, you use buildLayeredImage to generate that script, then you must docker load it later. It also sucks if you want to set up stuff to run on Kubernetes. Another reason to stick with that code volume strategy.

If you need GPU stuff you will need NixGL and you need to be careful because the way NixGL is built is based on impure derivations, so if you build the image in your Intel you have to figure out how to generate the NixGL for that Nvidia card or your code will probably not find the GPU. The nixpkgs Blender has no support for CUDA by default, so you will want to stick with nix-warez’s Blender [3].

[0] https://github.com/lucasew/nixcfg/

[1] https://github.com/cheng-chi/blender_notebook

[2] https://github.com/lucasew/nix-on-colab

[3] https://github.com/edolstra/nix-warez/blob/master/blender/flake.nix

11

l0b0 edited 4 months ago | link

At Toitū Te Whenua LINZ we’re using Nix in a few places. Two open source repos include Geostore and emergency management tools.

Pros:

Reproducibility between developers and CI. Expecting everything which works locally to work elsewhere is a superpower.
Free build cache at Cachix (no affiliation).

Cons:

The learning curve is a cliff, as I’m sure you’re aware. I had to read and use it a fair bit before trusting it for work. This is probably the biggest point against it, so be prepared to mentor others on their journey.
Mixing traditional packages and Nix can be a pain. In the case of poetry2nix, because Python packaging is such a mess, it’s constantly having to add overrides to deal with packages not declaring their build dependencies, or build dependencies being non-Python and therefore not declared anywhere machine readable. In case of bundix, it’s built like a tool to generate a file which you need to keep in sync with your Gemfile, rather than a simple overlay like poetry2nix.

6

kj 4 months ago | link

As a kiwi, it’s very cool to see a government agency engaging with open source and using Nix!
1. 2
  
  l0b0 4 months ago | link
  
  Thanks, bruv :D
1

toastal 4 months ago | link

Is this more of a condemnation of the Python ecosystem or Nix?
1. 3
  
  l0b0 edited 4 months ago | link
  
  I’m very much in favour of Nix over any other packaging system I’ve used, with the only possible exception being Rust crates. Despite its strange syntax and lack of documentation, Nix is massively better than packaging using [about six unnamed mainstream packaging systems] then flip-flopping to the latest and greatest variation every couple years, having to learn a huge amount of new best practices and bug/usability workarounds to get to something vaguely stable and idiomatic.

10

friendlysock 4 months ago | link

Some friends and I have been quietly using NixOS for appliances and servers for…christ, coming up on four or so years now depending on which of us you talk to.

It’s made several things–for example, tomorrow’s OpenSSL3 vulns–fairly easy to patch and manage. It also has made, once the incantations are written, admin work tremendously boring.

I myself totally blew off Nix itself back in 2011 when somebody tried to educate me. I didn’t really understand the problem space and thought it rather silly–I’ve since realized the error of my ways. Just a reminder to always accept that today’s strongly held belief may end up looking silly a decade later.

5

LenFalken 4 months ago | link

what changed your mind? what exactly do you use it for?
1. 2
  
  friendlysock 4 months ago | link
  
  At the time, it was a Haskeller who was trying to sell us on reproducible builds and package management, and my comment was basically “I have apt-get and version control, this doesn’t really solve any issues I face and it seems extra complicated”.
  
  A decade later and with more experience under my belt, I found that its declarative syntax and speed of rebuilding things was something I enjoyed more than screwing around with Ansible and friends. It also produced really slim Linux images that were perfect for appliances, and its ZFS support was quite good. So, for things like writing a script I can use for a NAS + media and other software and then forget about except once or twice every two years, it was great.
  
  It also currently has the largest and most-up-to-date repository of Linux software right now iirc, even though sometimes packager’s choices for how to expose configs and bundle things leave a lot to be desired. In such cases, there are usually tricks you can do to make it work–and with overlays and overrides and whatnot you can usually bludgeon around anything you find too brain-damaged.
  
  I think it has a nice range between giving you something for fleet management (with a little work), something for reliable appliance stamping, and something for pets that just need modern tooling.
  1. 2
    
    friendlysock 4 months ago | link
    
    A further thought:
    
    The neat thing about Nix (here, I also mean NixOS) is that it results in what looks like a normal Linux system. Even when you login to a container root, it still looks like a normal system. This honestly feels very reassuring to those of us that are more comfortable and have a preference for older-style systems administration.
    
    You basically still have the option of doing the laying-on-hands that you’ve learned how to do over the decades, and it isn’t an afterthought. You will have to get comfortable with systemd, but otherwise a good NixOS setup results in just another boring Linux box–it just came from some much more modern code-as-configuration language.
    
    The other nice thing is that because of the way that the store/symlinks work–setting aside reproducibility which I don’t really care much to comment on–you can just use a bunch of software together (as was promised with Docker-style containers) without worrying about them slapfighting too much. Again, though, it still feels like a normal Linux system when you do that, albeit a system with a very strange view of how to setup symlinks and jails.
  2. 2
    
    LenFalken 4 months ago | link
    
    I found that its declarative syntax and speed of rebuilding things was something I enjoyed more than screwing around with Ansible and friends
    
    I wonder if you’d like what I made then git://len.falken.directory/code/replicare.git since I felt the same

5

wraithm 4 months ago | link

We use nix very conservatively. We only use it for managing local developer environments, ie. build toolchain and other cli tools (ansible, terraform, etc). That has worked out amazingly for us.

I’m in general a lot more skeptical about nix for production. You don’t clearly get the kind of support like you would from, for example, Ubuntu’s packages. There’s no “LTS” as far as I know for nix, merely the stable NixOS release. Though, that being said, nixpkgs tends to be way ahead of other package managers’ versions of software.

We’ve started messing around with using nixpkg’s docker tools for some web projects. That would be the first time that we’d be using nix in our production environment.

In general, it’s really easy to go overboard with nix and start using it really inappropriately. But if you use some discipline, it can be an amazing tool. It’s completely solved our python problems related to installing ansible. That’s invaluable.

6

zimbatm edited 4 months ago | link

LTS is something that comes up regularly, and I sincerely don’t know if it should exist or not.

On one hand, it seems like it’s something that corporations want to have. Looking at the reason deeper than “because that’s what other distros do”, it seems to be a mixed bag of reasons.

Upgrades are much less risky in NixOS. Most issues are generally caught at eval or build time. And if it fails at runtime, it’s easy to roll back. Something that was a milestone on another distro becomes a ticket.

The company has to pay the upgrade price more often, but it’s also a benefit to them not to be stuck behind old versions. In that regard, it might be possible that having NixOS LTS releases becomes a disservice to corporations.
1. 4
  
  wraithm edited 4 months ago | link
  
  This is a super interesting topic to me. I can give you some more context on the “corporations want it” part. We’re a federally regulated business. I need to be able to say to regulators, “yes, when this CVE comes out, I’ll be able to upgrade this package in no-time.” Often that implies that I need to be able to point at another company that violated their SLA if they didn’t upgrade the package (eg. Canonical for Ubuntu LTS). I’m very confident in practice that nixpkgs often will get the pkg upgrades faster than Canonical can bust it out, at least for unstable, but it’s really that legal infrastructure that I need. There are companies like Twaeg and such that provide support packages? But it still seems really shakey to me.
  
  I hope that provides some more insight into what’s going on. Honestly, we’re still exploring it, and maybe it’s a solved problem, but I just don’t know.
  
  Also, things like rollbacks would imply rolling back security updates. If everything gets changed with a rollback, then you’re taking away important changes. I often want to just rollback application code, but not dependent packages. This is pretty straightforward to setup with nix, afaik, but it’s still non-trivial.
  1. 3
    
    riking 4 months ago | link
    
    Can you point to the capability and reality of applying overrides and patches as evidence of rapid response capability?
    1. 1
      
      wraithm 4 months ago | link
      
      Good question. That certainly helps! But we’d be doing everything ourselves. Also, you can eventually accumulate a lot of overrides/overlays such that it’s quite hairy to mess with stuff.
      1. 2
        
        shazow edited 4 months ago | link
        
        It’s fairly ergonomic to pull some specific packages from a different release channel, this site gives convenient copypasta: https://lazamar.co.uk/nix-versions/
        
        One approach I’ve used in the past is maintain a separate generated TOML/JSON file with overrides, and pull that in from nix. The overrides file can be managed by some other script/process. In this particular case, I’d mainly want an expiration time for an override to get removed once it’s no longer valid.
        
        Also you might like this recent post about all the various ways to override a package, with your own patches and such: https://bobvanderlinden.me/customizing-packages-in-nix/
        
        And one more note that might bring comfort: You probably know this, but the only distinction between unstable and stable nixpkgs channels is that unstable is rolling releases but stable is discrete releases. Aside from that, I don’t think there’s any increased “stability” guarantee – they both pass the same automated testing suites. (Someone please correct me if I’m mistaken.)
2. 3
  
  viraptor 4 months ago | link
  
  If don’t company says “we support rhel 7” (so effectively lts), that’s what they support and can stay on those versions of dependencies for ages. It doesn’t matter if the upgrades are risky or not. It’s not a technical issue.
3. 2
  
  icefox 4 months ago | link
  
  On the subject of LTS: Upgrades can be less risky in Nix but I have been burned several times by now when upstream introduces a bug in a release which I proceed to hit the next day when installing a new system. I hate rolling-release systems as a consequence.
  
  I work in robotics, so for my work the benefit of riding the bleeding edge of software is pretty low, the potential costs of failure are very high (expensive hardware destroyed), and the cost of upgrading is also high. Even if the actual upgrade doesn’t take much work, there’s lots of integration, simulation, and real-world testing that has to be repeated. We also end up having to use hardware that has fairly limited software support, not all of it open-source, so in that case we are really stuck with the particular OS release that a vendor provides. (Fuck you, NVidia.)
  
  That said, we could potentially still benefit from Nix quite a lot, and I should play with it more someday. But we’d still end up essentially cutting our own LTS releases.

5

mankyKitty 4 months ago | link

Myself and a few colleagues all run NixOS. I use home-manager to manage the finer things. At $work we use nix for everything. All projects must be able to go from zero to running/built with a nix-build.

One of the important trade-offs (imo) is that everything a thing needs is laid bare by using Nix to wire it all together.

This can be super annoying (see other comment regarding the sad state of Python packaging), but the flip side is that you have what you need pinned down and can then rely on them being fixed.

Nothing side-loading crap without you knowing about it.
Easier to leverage upstream fixes because you can pin even dependencies of your dependencies.
Since everything is pinned (as long as you avoid diamond paths: <nixpkgs> etc) then Sally will see identical output to Bob. Modulo the ebb and flow of servers on the internet.

Other comments here make the point that the learning curve is a cliff and that’s not too far wrong. There is more and more information available to ease this process, but it’s still a process. The pay off in the end is well worth it however.

4

j3s 4 months ago | link

personally, I find nix far too complicated for most use-cases, to me it looks like an absolute support nightmare.

docker is much more understood if we’re interested in portability - and shell scripts are what I push when we’re talking about light dev tooling. it’s hard for me to find the right use-case for nix - i always wind up feeling like there’s something simpler.

3

toastal edited 4 months ago | link

At my job, we use Nix top to bottom for everything. The more samples I see from the folks that are really good at it, the more I’m convinced Nix is currently the best way to make reproducible software and can replace a lot of existing tooling. This isn’t to say there isn’t a learning a cliff or some Kool-Aid to drink, but it’s something worth chewing on instead of Docker or similar. What seems a real shame is only using Nix for its devShells because you take something that could be reproducible and instead do everything in a stateful shell which really doesn’t help you much.

As more people see the power, hopefully more will migrate and help maintain NixOS and nixpkgs.

At the present, my laptop is NixOS and all of my current projects have been converted or will be converted to Nix. Many have started with just the devShell, but as my own skills have increased, everything is getting package‘d up. I also maintain a few of the packages on nixpkgs (though I really wish they’d move off of GitHub).

3

AngryAnt edited 4 months ago | link

Switched from thirty years of macOS to Linux in 2019, landed on NixOS in 2021.

Personal use:

Laptop.
Tablet.
Family computers.
Home server.

Work use (gamedev, cross-platform including console, multiplayer servers, self-hosting team services, tooling, consulting):

Workstation.
Services.
Docker container hosts.
Deployment.
Live systems.

I also maintain a couple of nixpkgs and have too many I should clean up and push as well.

The nix syntax is really dumb. I’ve accepted the fact that it is just never going to stop being dumb.

Also if there’s an officially supported docker container for a service, I am unlikely to look elsewhere than just loading that up in podman on NixOS. I err on the side of pragmatism, which is why I went for NixOS in the first place.

I kept an eye on NixOS for a very long time, feeling it was definitely way too scary, until deciding to just rip off that bandaid, follow the short & simple step by step install instructions in the manual, and most importantly just go with a completely default generated initial config with a desktop environment rather than sitting in the installer forever, worrying about reaching a perfect setup in one go.

A core (to me) beauty of NixOS is the iterative nature of it. Just embrace that from the very start and realise that compared to any other OS there just isn’t any real punishment for “getting it wrong”.

I’m still hard smitten by NixOS for personal and work use. It gave me a solid sensation I’ve not had since Computer Science of “this is how computing is supposed to work”.

1

nextos 4 months ago | link

Do you use NixOS in your tablet? If so, is this a MS Surface?
1. 1
  
  AngryAnt 4 months ago | link
  
  Yes - a minimum spec Surface Go 2 (IIRC dual core w. 4GB ram).
  
  I use it for reading & other media consumption, light browsing, and occasional work with a terminal - relying on a Bluetooth keyboard & mouse.
  
  A big role is also gaming with a ps3 or steam controller over Bluetooth, streaming via the steam link app from a (NixOS) gaming VM on our home server. This is generally how we game at home - also streaming to the family computers and a physical steam link box.
  1. 1
    
    nextos 4 months ago | link
    
    Interesting, do you use GNOME? Is the touch functionality good enough in terms of low lag and sensitivity?
    1. 2
      
      AngryAnt 4 months ago | link
      
      For the specified use it works well. Obviously not designed for the experience, but not something I find myself fighting either.
      
      I would prefer to run plasma on it though and with the last few releases that plan is just waiting for me to get around to testing.
      
      The desire comes from the easier customisability and the delay from the lagging behind on implementing Wayland (do not even attempt touch operation on X).
      
      I originally made the switch from an iPad as a test and did not even consider ending that test. It’s been a year and a half at this point - several full story-driven games and books later.
      
      It is very much not an iPad, but it’s my tablet much more than any iPad has ever been.

2

yaymukund 4 months ago | link

I use nix to configure my three systems— 1 OSX, 1 NixOS, 1 Linux remote dev env. Each gets a flake, and there are flakes for common things such as neovim and zsh. I have some scripts to install, rebuild, and update. On NixOS, things are installed using nixos-rebuild as you would expect. On non-NixOS systems, I do nix print-dev-env > $HOME/.zshrc to construct a shell environment that contains everything from that Flake.

I’ve been using Nix for about two years, and this particular multi-system setup for 8 months (previously, I exclusively ran NixOS).

2

jackdk edited 4 months ago | link

I predominately write Haskell at home and work, so Nix has been an excellent fit. I have been on NixOS for personal machines for ~5 years now and it’s a dream once you set it up. The first time I saw Ansible (years before I saw NixOS), I remember thinking, “What the hell? Removing the requirement to have a package present just leaves it lying around?” It turns out that to do this right, you have to rethink everything from the ground up, and that requires an effort like NixOS.

For personal projects, it’s an absolute godsend: I can GC away things I don’t need, confident that they’ll reappear when I need them. My keyboard configuration needs a specific revision from the firmware repo, plus a couple of patches, plus a specific version of GCC, and writing Nix to get those has preserved my sanity.

At work, I’m the Nixpert, and probably the most conservative about its adoption. We use it a lot for development environments, and once people saw how smooth nix-shell was on the Haskell project, they started wanting it on the legacy Ruby code. Private gems mean we provide ruby and bundler via nix-shell, but let bundler fetch the Ruby dependencies. We only started using Nix for Haskell builds when forced; we had to start statically linking Haskell binaries with uncommon libraries, which is really only practical with haskell.nix.

As for your proposed push:

I wouldn’t push all the way to NixOS. Nix alone is a massive QoL improvement, and then let people get excited and help them adopt home-manager. If you have existing servers, porting over all the established monitoring etc., is a pain; we have continued with Ubuntu + Chef here.
The language really is a curve for people. Even though we have a lot of Haskell and Elm, and people familiar with “yeah it’s another pure functional ML-style language”, the idioms are still really different, and syntactic misfeatures (like space-separated lists) trip people up.
Good development shells are a great first step, and can often replace a lot of YAML in a CI setup. This is a good step towards bringing Nix from “parallel experiment” to “real infrastructure that’s relied upon”.
Probably more than even ordinary GNU/Linux, you need to know which hardware to get. A colleague purchased a dual GPU XPS laptop with a HiDPI screen, and it was one nightmare after another. I stick with lower-density screens, Intel GPUs, etc., and things have been great.

2

viraptor edited 4 months ago | link

I use it heavily in private (laptop, home server, random projects, building docker images), as well as in isolation at work (home-manager for my dev laptop).

Previously, over many years, I used puppet, chef and salt in large environments and in my opinion nix is ~2 generations of tooling ahead of them. I drank the koolaid and joined the nix project as a maintainer. If some place was building infrastructure from scratch, I’d definitely consider using nixos or building containers through nix. I’d be more cautious suggesting migration to that system though.

In practice, for server environments I see only positive things from the technical side. For desktop… it’s not great with how it requires workarounds for GPU drivers. (now a huge issue for occasional usage)

The big issues start with the non-technical side. While I think the difficulty of learning nix is a bit overblown if you can teach others and provide standards/examples, it is still an extra skill. The real difficulty is trying to figure it out on your own. (Then again, for example chef seems nice and approachable until you have to write your own resource ordering hacks) It’s also not what people are used to, so functional/lazy approach may not sit well with everyone. The debugging of issues is both hard and very rare in my experience once you have existing files you update over time. It’s the initial setup that may be tricky.

I’m a bit torn on the stability side. I believe the stable channels are really well managed and the security fixes land in the unstable faster than advisories are published for other distros. (I made a spreadsheet out of interest and for a month of CVEs the time to fix was mostly negative) And while things feel very much like a “best effort” situation in terms of backwards compatibility of updates - it’s the easiest system to revert one specific package, or apply specific patches. (for example managing today’s/tomorrow’s openssl vuln is easier on nixos than other systems https://discourse.nixos.org/t/openssl-3-0-7-update-2022-11-01-faq/22875) So while personally happy to explain the trust on a technical level, I really wouldn’t want to write a justification for higher level management.

2

rtpg edited 4 months ago | link

I use Nix (or rather, home-manager + Nix) to manage my own computer. NixOS scares me too much (I have tried to do the install on a VM a couple of times and don’t really have a high success rate of getting to boot).

I tried to get Nix to be useful for a Python-based thing at work, but at the end of the day Docker and docker-compose did what I needed and was more along the beaten path (with VS Code integration in particular being very powerful). If someone were able to make a “turn this Dockerfile into some nix expressions” thing that would be very fun and cool, though.

We got a looooot of value out of using Bazel for testing and the like, and I feel like the learning curve is easier (imagine that lol). I think if you have certain kinds of tech stacks it can be an easy onboarding process.

1
rprospero 4 months ago | link
I’m not sure whether I’ve understood your use case properly, but it is possible to write a Nix module that will take an existing Dockerfile and run it on your server. I use it for some packages that I’ve been too lazy to convert into proper nix expressions. For example, the FreshRSS aggregator was as simple as:
```
  virtualisation.oci-containers.containers = {
    freshrss = {
      image = "linuxserver/freshrss";
      ports = [ "8124:80" ];
      autoStart = true;
    };
  };
```

2

penguincoder 4 months ago | link

I use Nix + direnv at work. When I joined, they didn’t have a Dockerfile, so I set out to make an environment using Docker and docker-compose but it failed terribly because most of the other engineers are using Apple Silicon machines. In order to run our test suite, we have to download a prebuilt chromedriver and that build isn’t available for aarch64 in Linux. All of the M1 folks had to emulate the test suite in Docker. It was really slow and didn’t see a lot of adoption.

Fast forward a bit, I made a shell.nix file with all of the prerequisites to install dependencies. This change was very welcome, especially with the frontend devs. It took overall bootstrapping from something like (at least) 2 days of each engineer having a unique fiddling-with-the-stack setup to about 2 hours with a brand new machine having a working stack.

Needless to say, I’ve jumped from using Docker for side projects into isolated Nix environments. It’s way more ergonomic to not have the Docker layer getting in the way.

1

Aurelius 4 months ago | link

I use nix at both my company and personally.

At my company the nix is very bloated using way to many nix sources (revs set to old hashes) and involves quite a bit of impurity. Part of this is due to the repo being migrated to nix as well as the people setting up the project not knowing anything about nix before. This has lead to the decision to get rid of nix within the company.

Personally I use nix, this has proven to be more successful as the projects are often simpler with less complex pipelines and needs to have these different dependencies.

Going forward I plan to use nix entirely in my home projects.

1

abstract777 4 months ago | link

We manage one project in development with nix (node2nix), but it’s deployed with a dockerfile on flyio.

1

ptman 4 months ago | link

nix + docker (I don’t write nix packages for most services)

1

samhh 4 months ago | link

I run NixOS on my personal/dev machine and my homelab, and nix-darwin on my laptop: https://github.com/samhh/dotfiles. Happy to answer any questions.

We use Nix for dev environments on my team at work. I’d highly recommend it despite its immaturity.

1

puffnfresh 4 months ago | link

I’ve been using Nix and NixOS for around 8 years. I use it for everything. My work laptops have run NixOS for each of the jobs I’ve had in that time. I’ve had servers that I use for development environments and they run NixOS. I run a webservers, Matrix nodes, media servers, etc. all using NixOS. I use Mobile NixOS on my e-reader and I’ve put NixOS on a handheld game console.

I used to work on Atlassian Marketplace which uses nixpkgs’ dockerTools to create Docker images which are pushed and deployed to Atlassian’s internal PaaS. There’s around 5 or so production services using Nix packaged Docker images. Even repositories which aren’t deployed using Docker (e.g. Ansible scripts, Terraform code and our frontend JS) use a shell.nix to have a consistent environment. Outside of Atlassian Marketplace, Jira has used Nix in the past but got rid of it.

1

Corbin 4 months ago | link

I’ve used Nix:

On my personal machines, as NixOS
At my now-defunct small business, via nixops
In containers at work, using FROM nixos/nix for new Dockerfiles

Compared to RPM, APT, or Portage, Nix has no technical downsides. All anti-Nix sentiment is memetic, either from the difficulty of understanding capabilities or the difficulty of learning a new programming language. At the same time, NixOS is not perfect; GNU/Linux is a flawed platform, and UNIX was not meant for capabilities.

1

cmrfrd 4 months ago | link

The top use case of Nix for myself and coworkers is trying out new software via nix-shell.

Being able run and share commands like nix-shell -p simplescreenrecorder --command "simplescreenrecorder" and not have to worry about anything else is pretty sweet.

5
grahamc 4 months ago | link
You might like nix run, which uses Flakes and “apps” to simplify that a bit:
```
nix run nixpkgs#simplescreenrecorder
```
1. 1
  
  cmrfrd 4 months ago | link
  
  Nice! Will definitely try it. Feels a bit akin to comma