Just, Nix Shell and Podman are a Killer Combo

9

toastal 5 months ago | link

for the purpose of this post, it is a way to easily create reproducible development environments

I don’t understand the advantage here versus all Nix and building a container, if necessary, through Nix. Using the Nix shell is quick/dirty, but it turns a stateless build tool into one that has the state of the shell. Why just have a reproducible development environment when the whole derivation top to bottom can be reproducible? Between packages and apps you can do just about everything and without the overhead of containers.

I understand Nix is a learning cliff to swallow, but it will, in the long run, simplify this entire setup to just one tool instead of multiple and this one-tool simplicity makes it a lot easier for teams to understand.

7

abhin4v 5 months ago | link

You are right. But it took me literally months to learn enough Nix to be able do everything in Nix only. This setup is easier, it works fine and I figured it out in less than a day. So I stuck to it.
2

knl edited 5 months ago | link

One of the reasons why I see not going full build in nix is when one needs to collaborate with others that do not use nix. For example, I have to occasionally contribute to internal python project, where each has a different setup (no instructions, use virtualenv with requirements.txt, use conda, use an internal tool). When I’m not a regular contributor, getting into nuances of each setup is a big overhead. That’s why I set up a shell.nix file that bring me almost everything I need to just start coding. Sometimes that works, sometimes it doesn’t, especially when external, non-python dependencies are needed (ironic, right?). That way, others can continue doing work using tools they are comfortable with, while I get my safety net.
1. 1
  
  toastal 5 months ago | link
  
  Sure, I bring a Flake to JS projects like this and don’t commit it, but I if it’s my own repo, I’m not going to try suggest folks use 3 tools (Nix, Just, Podman) when I could use 1 (Nix). Depending on the audience you can suggest a devShell or explain how to use the ‘normal’ community tooling, but even then, often the *2nix is overall a better experience and unlocks build sharing like Cachix.

4

pbsds 5 months ago | link

Wouldn’t it be easier to just make each justfile target call nix-shell --run "some command"?

4

andyc 5 months ago | link

FWIW you can also use shell for this instead of Just, since you’re writing shell anyway:

_run-in-nix-shell cmd *args:
    #!/usr/bin/env -S sh -eu
    if [ "{{ in_nix_shell }}" = "false" ]; then
        nix-shell "shell.nix" --run "just \"{{ root_dir }}/{{ cmd }}\" {{ args }}"
    else
        just "{{ root_dir }}/{{ cmd }}" {{ args }}
    fi

I would write this as:

_run-in-nix-shell() {
  if [[ $in_nix_shell = false ]]; then
    # original and new one have some quoting issues here
    nix-shell shell.nix --run "just \"$root_dir/$cmd\" $@"
  else
     # no quoting issues
     just "$root_dir/$cmd" "$@"
  fi
}

"$@"  # run function $1 with args $2 $3 $4 ...

The {{ }} syntax in Just seems unnecessary, in adition to the *args. Shell already has stuff like that!

This introduced more quoting problems than shell already has!

Probably should write a blog post about this … Related comment:

https://lobste.rs/s/sq9h3p/unreasonable_effectiveness_makefiles#c_bwfha2

1

carlmjohnson 5 months ago | link

Oh my god, yes, taskfiles all day long! You have to know shell anyway, why not use it for what it’s good at?
1. 2
  
  andyc 5 months ago | link
  
  Yup, sometimes I feel like we’re stuck in a loop where we keep inventing the same mistakes – e.g. make doing its own interpolation with $ on top of shell, various YAML-based languages and {{ }} embedding shell, and now Just and {{ }}.
  
  It’s 70’s string soup, but invented in 2020. (I am glad that memory safety has become a meme. But it’s weird that string hygiene isn’t a meme, despite those problems being arguably more common!)
  
  In meme format:
  
  Programmers; don’t use shell, it’s too complex and unsafe!
  
  Also programmers: check out this simple YAML-based config format! You can interpolate variables!
  
  But I also recognize that people want convenience, and defaults set up for them, and shell has stagnated, and has distribution problems.
  
  I mentioned in that comment that the Taskfile pattern in shell (ironically) doesn’t have autocompletion, and you want to generate help, etc. My former coworker made an attempt in pure shell, but it didn’t catch on: https://github.com/mbland/go-script-bash
  
  So there is no single way to do it in shell. Someone also mentioned “scripts to rule them all”, which is fundamentally the same, but somehow didn’t catch on, despite being used by Github (at least for a time).

1

matklad 5 months ago | link

Is there some nice way to add “task running” functionality directly to shell.nix? For example, I have the following shell.nix for my blog:

λ bat shell.nix
# rm ./vendor -rf && nix-shell --run 'bundle install'
# nix-shell --run 'bundle exec jekyll serve --livereload'
with import <nixpkgs> {}; mkShell {
  packages = [ ruby ];
}

The “task running” functionality are the two comments above, which I just copy-paste in the shell.

Is there some short, non-horrible way to allow me to type nix shell --run serve and have that to what I want? Preferably with something like nix shell --run list-available-tasks.

2
viraptor edited 5 months ago | link
I believe you could do something like this in a flake file:
```
 apps."<system>".task-name = { type = "app"; program = lib.writeScript... ; };
```
And then run it with “nix run task-name”. I can’t remember the actual definition, so may need some fixing.

Similar https://www.ertt.ca/nix/shell-scripts/
1. 2
  matklad 5 months ago | link
  Indeed!
  
  Now the shell.nix has
  
  with import <nixpkgs> {}; mkShell { packages = [ (writeShellScriptBin "serve" "bundle exec jekyll serve --livereload") (writeShellScriptBin "install" "rm ./vendor -rf && bundle install") ruby libffi pkg-config ]; }
  
  and nix-shell --run serve does what I need, many thank!
1

eterps 5 months ago | link

Why not just put those tasks as separate scripts in a subdirectory?

Like: https://github.blog/2015-06-30-scripts-to-rule-them-all/
1. 2
  
  matklad 5 months ago | link
  
  That can work, but I’d love too keep this scoped to a single file, to reduce fs clutter, and scoped to a single CLI entry point (nix shell) to reduce mental clutter (cc @grahamc, this actually sounds like a user pain potentially relevant to determinate systems).

1

MazeChaZer 5 months ago | link

Sure thing, you can just put these little scripts into derivations and make them available in the shell environment:

let pkgs = import <nixpkgs> {};
    installScript = pkgs.writeScriptBin "install" "rm ./vendor -rf && bundle install";
    serveScript = pkgs.writeScriptBin "serve" "bundle exec jekyll serve --livereload"
in with pkgs; mkShell {
  packages = [ installScript serveScript ruby ];
}

With this, you can use nix-shell --run install and nix-shell --run serve.

If you put a little more sophistication into it, I’m sure you can define the scripts in a way that you can also generate a list-available-tasks script. Something like:

let pkgs = import <nixpkgs> {};
    makeScripts = scriptDefinitions:
        let scripts = map ({name, source}: pkgs.writeScriptBin name source) scriptDefinitions;
            listTasksScript =
                pkgs.writeScriptBin "list-available-tasks"
                    (concatStringSep "" (map ({name}: "echo ${name}\n") scriptDefinitions));
        in scripts ++ [ listTasksScript ];
    scripts = makeScripts [
        {name = "install", source = "rm ./vendor -rf && bundle install"}
        {name = "serve", source = "bundle exec jekyll serve --livereload"}
    ];
in with pkgs; mkShell {
  packages = scripts ++ [ ruby ];
}

I didn’t test any of this, so there is probably a little bit of debugging required to make it work, but I’m confident it can work like this.

1

peterbourgon 5 months ago | link

this behavior is desirable when using make as a build system, but not when using it as a command runner

make is by definition a build system and not a command runner, right?