There is something that has bothered me over time, especially in the world of DevOps tooling that is purported to be declarative and deterministic is that it is very hard to make things perfectly deterministic. There are a lot of things that almost get you there but there is still something unsatisfying about it. It seems you either need to live with bleeding edge updates and hope you can track it down when it breaks, or you live on older pinned versions and deal with the bugs and security.
Docker - Docker’s default tag is latest, which will run differently depending on whether you have a system with it already cached, how far apart in time the containers were deployed, etc. Easily solved with tags right? Secondly, many Dockerfiles do everything right except that when they source a base image with FROM, it can’t guarantee that’s the same base image. Even with “versioned” tags, you can’t guarantee that the older versions haven’t been updated over time.
Ansible - When installing or updating packages, pinning versions isn’t always straightforward and I’ve seen many systems which just use the bleeding edge version. You have things defined in code, but you end up with config drift across systems without realizing it.
NixOS - For a system that is supposed to immutable, declarative and deterministic, there is one major flaw that keeps it from achieving a dream of perfect determinism. It has no consistent version-pinning when it comes to the actual package names. Only in cases when the package maintainers have specifically decided to create a new attribute for a specific version of a package. On a local system it’s fine because it’s never going to change in the store until you garbage-collect it, but if you are working on ephemeral, constantly rebuilt systems, you can end up in the same kind of version-update config drift scenarios like before. Your configuration.nix file will not save you because it’s probably not made up entirely of hand-built derivations with hashes for everything, it’s probably built from the nixos package respository. Somebody created a work-around site for this, because the object store still has hashes from older things, but it’s a bit of a hack.
My dream would be seeing something between the Nix way of doing things, and allow version-pinning in the nixos config files. But also allow the ability to use the latest and greatest, but with a twist. The latest version still be pinned and incremented by some other system that checks for new versions of packages and updates them in git, then makes a PR that increments the package version and triggers CI to ensure that the update is sane, then merges. Declarative, deterministic, and auto-updating all in one. Some things would break down in that process but you’d write tests for things, pin packages if necessary, and live knowing that everything is valid and tracked in version control but you don’t have to constantly bump versions manually. I’m sure somebody has already done this idea but I’d love to see it realized in an approachable way.
But also allow the ability to use the latest and greatest, but with a twist.
Can’t you do this now just using another nixpkgs checkout? You can mix and match and name the nixpkgs definitions. On my mac my nixpkgs name is the stock nixpkgs, and anything else is just a git worktree of nixpkgs where i just setup things to be whatever version I need and add another named package setup and use that specific thing from it.
Its basically the same thing as doing local development, and the package at version N is built alongside/with the same other things at that same overall derivation so less chance of other things breaking.
There is something that has bothered me over time, especially in the world of DevOps tooling that is purported to be declarative and deterministic is that it is very hard to make things perfectly deterministic. There are a lot of things that almost get you there but there is still something unsatisfying about it. It seems you either need to live with bleeding edge updates and hope you can track it down when it breaks, or you live on older pinned versions and deal with the bugs and security.
Docker - Docker’s default tag is latest, which will run differently depending on whether you have a system with it already cached, how far apart in time the containers were deployed, etc. Easily solved with tags right? Secondly, many Dockerfiles do everything right except that when they source a base image with FROM, it can’t guarantee that’s the same base image. Even with “versioned” tags, you can’t guarantee that the older versions haven’t been updated over time.
Ansible - When installing or updating packages, pinning versions isn’t always straightforward and I’ve seen many systems which just use the bleeding edge version. You have things defined in code, but you end up with config drift across systems without realizing it.
NixOS - For a system that is supposed to immutable, declarative and deterministic, there is one major flaw that keeps it from achieving a dream of perfect determinism. It has no consistent version-pinning when it comes to the actual package names. Only in cases when the package maintainers have specifically decided to create a new attribute for a specific version of a package. On a local system it’s fine because it’s never going to change in the store until you garbage-collect it, but if you are working on ephemeral, constantly rebuilt systems, you can end up in the same kind of version-update config drift scenarios like before. Your configuration.nix file will not save you because it’s probably not made up entirely of hand-built derivations with hashes for everything, it’s probably built from the nixos package respository. Somebody created a work-around site for this, because the object store still has hashes from older things, but it’s a bit of a hack.
My dream would be seeing something between the Nix way of doing things, and allow version-pinning in the nixos config files. But also allow the ability to use the latest and greatest, but with a twist. The latest version still be pinned and incremented by some other system that checks for new versions of packages and updates them in git, then makes a PR that increments the package version and triggers CI to ensure that the update is sane, then merges. Declarative, deterministic, and auto-updating all in one. Some things would break down in that process but you’d write tests for things, pin packages if necessary, and live knowing that everything is valid and tracked in version control but you don’t have to constantly bump versions manually. I’m sure somebody has already done this idea but I’d love to see it realized in an approachable way.
Can’t you do this now just using another nixpkgs checkout? You can mix and match and name the nixpkgs definitions. On my mac my nixpkgs name is the stock nixpkgs, and anything else is just a git worktree of nixpkgs where i just setup things to be whatever version I need and add another named package setup and use that specific thing from it.
Kinda seems like nix has this covered to a degree with the nixpkgs setup tbh: https://nixos.wiki/wiki/Nixpkgs/Create_and_debug_packages
Its basically the same thing as doing local development, and the package at version N is built alongside/with the same other things at that same overall derivation so less chance of other things breaking.