1. 14
  1. 8

    Speaking as somebody who has only briefly tried out Nix and never really stuck with it, the experience, this blogpost and the accompanying HN discussion reminds me of the Python packaging ecosystem: There is a low-level layer that is hard to understand (setuptools/distutils and to an extent pip itself). This is the only entrypoint that is actually “official” in any sense. Then there’s a variety of tools around this which provide abstractions on top of that layer to make the entire system easier to get started with (home manager, nix-darwin, niv).

    Having multiple layers like that is not generally a bad model, but if it turns out that some random community tool is needed to make the official base system usable in any way, something is wrong, and IME it is usually in governance.

    1. 3

      I believe this is pretty common. I mean, we’re likely compiling most software with gcc, which is extremely community based. (have you ever experienced the hell that’s a producer-provided compiler?) Lots of build systems are not related to the compiler either. We don’t have any non-community browser since W3. Etc etc. I don’t see a “random community tool” as a bad thing. Not everything needs to be controlled from the top. Sometimes that leads to amazing things like python requests when you don’t have strict rules limiting you.

      1. 4

        React and Flask are probably decent examples of projects where a lot of fairly important functionality isn’t in the core. I imagine these abound. But it does make it harder to find your sea legs.

        1. 2

          Just to make this perfectly clear: I am not advocating for creating large monolithic tools wherever possible, and I am talking mostly about governance structures rather than software architecture.

          Flask and React being lean and extensible is fine, though as a former maintainer of Flask I remember there always being a problem around discoverability of extensions, and a few important extensions are maintained by the same people who maintain Flask itself.

          I do however think it’s the wrong approach for packaging, dependency management, and things like that.

          1. 1

            Oh, for sure. Discovery is hard, and all of the different approaches differ for reasons or on qualities that newcomers are not at all equipped to evaluate.

            This is all on top of nix already having a lot of complexity to wrestle with. My nearby response to viraptor touches a little on why it’s tricky for the core to do all of this in nix’s case. I also agree that some things like hm and nix-Darwin should be moving toward the core, even if that just starts with standardizing the interface points and providing clear lists of tools that meet this need, picking a sane default, documenting when you’d use each, etc.

            (AFAICT Eelco has an ~admirable reluctance to pick winners and losers here, but I guess that may be indistinguishable from a governance problem.)

        2. 3

          A user experience that is “controlled from the top” is usually a more coherent one, and while I realize that this is highly subjective and at risk of devolving into Cathedral vs Bazaar, especially in the case of package management I heavily favor a well-integrated suite of tools produced by a group of well-coordinated people over the bazaar approach. A package manager is a tool for cobbling software together; I don’t want to cobble together the package manager itself.

          Note that this is not entirely the same thing as advocating for a monolithic software architecture: cargo and rustc are separate tools, but play really well together because they’re developed by (what I perceive to be) the same party.

          1. 3

            Fwiw, nix seems to be the most bazaar experience I’ve seen. It feels like it’s held together magically by a common belief in the same goal.

            1. 4

              Part of this magic, I think, is that nix is a bit like ice-nine.

              It has it’s own crystalline structure and chaotic external boundaries. It needs a lot of external change or compatibility shims/layers to align large swaths of the computing universe with its logic. There’s way too much world to build all of these interfaces in a top-down manner, but the need/value of some way to align the logic manufactures a lot of drive in the community to do this work.

        3. 3

          I feel like your complaint is content-free and generic, in the sense that it can apply to any pair of popular programming environments. Consider this list of environments, blessed low-level tools/interfaces which are confusing, and unofficial third-party tools/interfaces:

          • Debian: dpkg; aptitude
          • Ruby: gems; bundler
          • C: DSOs/DLLs and libtool; pkg-config
          • Java: JARs; Ant and Maven
          • ECMAScript: JS and WASM modules; Yarn, Webpack, Babel, etc.

          The list can go on, but five is a good number. Either we can think of low-level programming systems as being poorly governed as a rule, or we can find a better model for thinking about these abstractions.

          1. 5

            Eh, it doesn’t really feel like you’ve grasped their point; bundler has been in Ruby’s stdlib since Ruby 2.6. Aptitude is an official Debian project. These aren’t examples of a “random community tool” having to paper over deficiencies in the core tooling by adding an inconsistent hodge-podge of ever-shifting “upper layers” the way we see happen in Python or Nix or JS. Bundler might have started out outside the language, but the project converged on a solution rather than diverging into a mess of attempts at one.

            Some projects do get this right.

            1. 4

              Identifying this complaint and pointing out it affects other ecosystems is the point of my post, though I really don’t think that e.g. Debian or Ruby’s packaging UX can be compared to Python. Sure, sometimes apt-get doesn’t have a command I need and I need to copypaste some dpkg incantation from StackOverflow I don’t really understand, but for the most part apt-get feels like “one tool”. And I strongly doubt the governance model of dpkg vs apt-get can be compared to nix vs home-manager or nix-darwin, the latter of which isn’t even part of the nix-community org but just some rando’s GitHub user. And apt-get or aptitude is unofficial? Well surely that’s only so on paper.

              There’s certainly nuance I left out of my original post, that is, if there’s strong community consensus behind using an unofficial tool it alleviates the issue quite a bit because the various parties are forced to play nice with each other. I think however that this applies to bundler much more than it does for home-manager, for instance.

              or we can find a better model for thinking about these abstractions.

              I had to clarify elsewhere, but my point is not about software architecture. It’s about having a single coherent vision for what UX should be, not bazaar-style development where everybody wraps everybody else’s software and pushes into different directions.

              1. 4

                Aren’t you are proving GP’s point? Just because the mistake abounds doesn’t make it less of one.

                Eg, compare ruby’s bundler with Go, where the module tools are an official part of the ecosystem. Imo, as a language Go is just ok, but this single feature makes up for it and more – that everything you need for dep management, testing, even http servers, is officially supported as part of the core tooling or standard library.

                1. 1

                  Go modules are officially included only relatively recently, following years of community projects which provided similar solutions. And bundle is a part of ruby https://github.com/rubygems/rubygems/tree/master/bundle so neither is a great example here.

                  1. 2

                    Go modules are officially included only relatively recently, following years of community projects which provided similar solutions.

                    Isn’t this a point in favor of what I’m saying? They’ve done it now and it’s way better than it was before.

                    And bundle is a part of ruby https://github.com/rubygems/rubygems/tree/master/bundle

                    This is incorrect on multiple levels, afaict.

                    First, rubygems itself is not part of the core language or (again afaik) associated with the core language development:

                    The project was started in April 2009 by Nick Quaranto, and has since grown to include the contributions of over 350 Rubyists and millions of gem downloads. As of the RubyGems 1.3.6 release, the site has been renamed to RubyGems.org from Gemcutter to solidify the site’s central role in the Ruby community.

                    Rubygems About Page

                    That is, it’s a community initiative. Bundler itself has its own website, team, and means of support. Both rubygems and bundler have become “big” so they offer more stability than “random dev’s Github page,” but it is still a far cry from the single ownership of Go.

                    1. 2

                      You’re right, I should’ve said it’s part of rubygems.

                      1. 1

                        This is incorrect on multiple levels, afaict.

                        You’re actually no longer correct.

                        Rubygems became part of the Ruby standard library way back in 1.9.

                        Bundler was added to stdlib in 2.6.

                        These had to become an official part of the language as it pursues the long term goal of modularizing stdlib into separate gems

                        1. 1

                          Bundler was added to stdlib in 2.6.

                          Interesting, happened while I wasn’t looking… Thanks for the correction.

                          If you happen to know, how does the governance work, what is the relationship between the different entities? Bundler has its own web page and team page, as I noted – how does that interact with ruby core devs who are maintaining the stdlib? Is there an official relationship? Do ruby core devs review bundler code?

                          Rubygems became part of the Ruby standard library way back in 1.9

                          In what sense? I see in the docs there is a class Bundler::Source::Rubygems, but rubygems.org, the website and the repository where you publish and download gems – does that have an official relation to core ruby now and, if so, what is it?

                2. 1

                  I recently started using nix (NixOS and on macOS).

                  Nix is one of those things that is really hard to explain because the scope of what it enables one to do goes far beyond current paradigms (package / system management). It’s really easy to see it and think: “I have a package manager, and it sure doesn’t require me to write code to use!”.

                  Nix lets one manage a system top to bottom (bootloader, apps, app configurations, system configs like firewall.. etc), not just packages. More over, it can be used to manage entire fleets of systems. I can have a git repo that defines an entire datacenter (not exactly with nix on non-NixOS systems - but the option is still there, macOS just isn’t as integrated as NixOS).

                  Nix is one of those extremely powerful things that you really have to use (imo through NixOS) to gain a full appreciation for.

                  Other examples of this dramatic paradigm shift are plan9 and its namespaces. Looking at them, you think: “Oh, I can do that with symlinks!”.. but it goes way deeper! One of the AHHAA moments for me with plan9 was the realization that I can mount a remote hosts TCP stack onto my current system, effectively teleporting my traffic to that remote machine.

                  1. 1

                    I’m guessing this is meant to be a top-level comment?

                    1. 1

                      Yep :D

                3. 6

                  nix-darwin + flakes provide a much simpler setup IMO, and you can share it with Linux/WSL systems: https://github.com/srid/nixos-config

                  I don’t use homebrew at all. And Nix is what I use for developing Haskell projects (see template) on my M1.

                  1. 4

                    Is this satire??

                    People go through this willingly?!?

                    1. 13

                      Apparently the author used the techniques that someone else learned from just the docs (https://ianthehenry.com/posts/how-to-learn-nix/), avoiding any community tools. That makes the post not wrong, but close to “I needed to cut down a tree, so here’s how I made a saw for it”. It’s cool to read and understand, but it really needs a disclaimer.

                      For people without nix experience - use either flakes or niv for pining the version you want and home-manager (with optional nix-Darwin) for managing your installed packages / environment. There’s some learning curve still, but it’s 10x easier than each section of this blog post. If you want to just use the system and learn deeper as you go, I’d recommend completely ignoring this post for a few months. This may be better https://davidmyno.rs/blog/an-introduction-to-nix

                      why not use Home Manager? The short answer is Nix is already too complex.

                      This may be satire. I honestly can’t tell. (it’s like “Why not use C? The short answer is assembler is already too complex.”)

                    2. 4

                      This writeup really goes above and beyond to make things complicated that aren’t, it almost seems like a negative sell of nix as a whole. You don’t need any of this in 2022. Just install nix on your machine, set it up in your shell, enable “experimental” flakes, then just use direnv’s use flake for your projects. Give your project’s flake (flake.nix) a devShell output with what is needed to work with it, and direnv takes care of updating your PATH and building devShell. It really is an amazing system with some rough edges, I wish the community would coalesce around flakes and abandon the older ways of doing things, but that’s unlikely with the composition of users it seems. Your average dev spends multiple days just getting setup with a new laptop or on a new project, more if it’s stuck in the past. Nix +direnv makes this as easy as cd-ing into the directory.

                      Once you get comfortable with using nix flakes to manage your project’s devshell, you can work to accomplish greater goals like reproducible builds of your project itself, culminating in a Dockerfile replacement (nix can build container images from scratch, no Dockerfile needed). If you still have ambition, you should look into managing your system with nix-darwin.

                      Nix is a great tool, don’t take this post as representative of the complexity, it is there but the value is immense and there are easier ways to get work done in 2022 than tinkering endlessly. I’ve broken my entire machine a number of times learning so far, but back when I was learning Mac initially MacPorts was likely to do the same thing and now Homebrew often breaks your whole machine as well with its automatic system-wide upgrades. Nix is a tool who’s time has come.

                      1. 1

                        I have been putting off using Nix Package Manager for my M1 mac for a while now.

                        Given your points about the article is doing it the complicated way, do you have a recommendation on where would be the best place to get started today?

                        I have seen recommendation toward flake.nix, niv… but I’m still not sure how to get started with all these.

                      2. 3

                        I tried using Nix on macOS for a little while. The experience was pretty unpleasant. A lot of software was broken and failed to build (off the top of my head, Julia installation was broken on my Intel Mac. On my M1 Mac there were a lot more things that wouldn’t install). The user interface of Nix is notoriously obtuse and poorly documented. Integrating with build systems or projects that don’t use Nix and thus assume pkg-config or cmake or whatever is on your PATH becomes more painful and requires you to create a shell.nix file (often with some other poorly documented variables you must set to get things working). The “flakes” model was yet another layer and interface that was different from and incompatible with the old interface. Basic command line utilities like searching the package index were either slow or would suddenly break when Nix updated (searching is now considered an experimental feature apparently?) Simple things like “which package provides a given file or command” or “what packages depend on X?” are difficult or impossible. I ended up needing to understand yet another build system. And all of this for what? Whatever problems Nix intends to solve I have never personally experienced using traditional package managers. I switched back to Homebrew and have been much happier.

                        1. 5

                          And all of this for what? Whatever problems Nix intends to solve I have never personally experienced using traditional package managers. I switched back to Homebrew and have been much happier.

                          I think you made the right call switching back to Homebrew. I’m a little cautious about who I recommend Nix to for roughly this reason. It’s a very powerful lever, but it’s a lot of work to learn if you don’t need the lever.

                          All of these are pretty fair, but I want to add a little context to 2 of them:

                          A lot of software was broken and failed to build (off the top of my head, Julia installation was broken on my Intel Mac. On my M1 Mac there were a lot more things that wouldn’t install).

                          I wouldn’t say “a lot” (but of course that distinction is meaningless when it’s a lot of the software you need.)

                          Making software build on macOS tends to be work that takes hands and requires access to running macOS systems (which is rarer than access to general Linux systems, among the community of maintainers).

                          Apple’s also made a few decisions in the past few years that have all been independently pretty disruptive, and working them has soaked a lot of macOS energy/effort (both causing some package breaks, reducing the free time people are able to allocate to fixes, and blocking some things that people would fix until some trickier ~foundational work is done. The main ones I’d cite are:

                          • The change to a read-only root directory. This didn’t strictly hit whether individual packages work too much, but it ate up a lot of work to get fixed and caused a long rocky period that created a lot of uncertainty, squandered momentum on macOS, made it much harder to onboard new macOS users (and thus potentially a few new maintainers) for nearly a year, etc.
                          • macOS has made some architectural changes, like no longer has dynamic libraries on the filesystem (can see an HN thread in https://news.ycombinator.com/item?id=23612772) that’ve entailed work to catch up with before newer SDKs have clean support. Some higher-level background on this and more in https://discourse.nixos.org/t/nix-macos-monthly/12330/11
                          • M1/aarch64. (This obviously takes work to support, and also fractures the macOS maintainers a bit, since now a significant fraction of maintainers who have access to a macOS device will only have either an Intel or Apple Silicon system.

                          would suddenly break when Nix updated (searching is now considered an experimental feature apparently?)

                          This was a few mistakes that snowballed into a big butt fumble. Mainly:

                          1. It was a mistake to release the early “new” Nix CLI without guarding it behind the experimental flag.
                          2. The nix 2.3 series, which made this mistake, lasted way too long (years) before the project bit the bullet last fall and moved to releasing every 6 weeks.
                          3. Uses of the new CLI turned up in the official manuals.
                          4. A lot of unofficial community sources, which are important because of the well-trod documentation problems, were eagerly using the new CLI without putting up warning signs.
                          1. 2

                            Also there’s an issue where updating to a new version of Apple sdk is just harder than it should be. We’re close to 2 years now (https://github.com/NixOS/nixpkgs/issues/101229) and already ran out of time. Macos is really hostile to nix - not actively, it’s just not a consideration at all.

                            1. 1

                              I’d say that’s roughly all under the umbrella of what I mean by citing https://discourse.nixos.org/t/nix-macos-monthly/12330/11.

                              The community was technically put in touch with an Apple devrel back during the read-only-root period, but the go-between hasn’t had time to serve that role. I’m not sure it’ll help, but I’ve been trying to (as often as I remember and can bear being annoying) squeak enough to get toonn or I in touch with the devrel.

                          2. 1

                            Agreed regarding homebrew since I’ve fallen in love with simple Brewfiles and such for easy onboarding of development staff to a team.