for the purpose of this post, it is a way to easily create reproducible development environments
I don’t understand the advantage here versus all Nix and building a container, if necessary, through Nix. Using the Nix shell is quick/dirty, but it turns a stateless build tool into one that has the state of the shell. Why just have a reproducible development environment when the whole derivation top to bottom can be reproducible? Between packages and apps you can do just about everything and without the overhead of containers.
I understand Nix is a learning cliff to swallow, but it will, in the long run, simplify this entire setup to just one tool instead of multiple and this one-tool simplicity makes it a lot easier for teams to understand.
You are right. But it took me literally months to learn enough Nix to be able do everything in Nix only. This setup is easier, it works fine and I figured it out in less than a day. So I stuck to it.
One of the reasons why I see not going full build in nix is when one needs to collaborate with others that do not use nix. For example, I have to occasionally contribute to internal python project, where each has a different setup (no instructions, use virtualenv with requirements.txt, use conda, use an internal tool). When I’m not a regular contributor, getting into nuances of each setup is a big overhead. That’s why I set up a shell.nix file that bring me almost everything I need to just start coding. Sometimes that works, sometimes it doesn’t, especially when external, non-python dependencies are needed (ironic, right?). That way, others can continue doing work using tools they are comfortable with, while I get my safety net.
Sure, I bring a Flake to JS projects like this and don’t commit it, but I if it’s my own repo, I’m not going to try suggest folks use 3 tools (Nix, Just, Podman) when I could use 1 (Nix). Depending on the audience you can suggest a devShell or explain how to use the ‘normal’ community tooling, but even then, often the *2nix is overall a better experience and unlocks build sharing like Cachix.
FWIW you can also use shell for this instead of Just, since you’re writing shell anyway:
_run-in-nix-shell cmd *args:
#!/usr/bin/env -S sh -eu
if [ "{{ in_nix_shell }}" = "false" ]; then
nix-shell "shell.nix" --run "just \"{{ root_dir }}/{{ cmd }}\" {{ args }}"
else
just "{{ root_dir }}/{{ cmd }}" {{ args }}
fi
I would write this as:
_run-in-nix-shell() {
if [[ $in_nix_shell = false ]]; then
# original and new one have some quoting issues here
nix-shell shell.nix --run "just \"$root_dir/$cmd\" $@"
else
# no quoting issues
just "$root_dir/$cmd" "$@"
fi
}
"$@" # run function $1 with args $2 $3 $4 ...
The {{ }} syntax in Just seems unnecessary, in adition to the *args. Shell already has stuff like that!
This introduced more quoting problems than shell already has!
Probably should write a blog post about this … Related comment:
Yup, sometimes I feel like we’re stuck in a loop where we keep inventing the same mistakes – e.g. make doing its own interpolation with $ on top of shell, various YAML-based languages and {{ }} embedding shell, and now Just and {{ }}.
It’s 70’s string soup, but invented in 2020. (I am glad that memory safety has become a meme. But it’s weird that string hygiene isn’t a meme, despite those problems being arguably more common!)
In meme format:
Programmers; don’t use shell, it’s too complex and unsafe!
Also programmers: check out this simple YAML-based config format! You can interpolate variables!
But I also recognize that people want convenience, and defaults set up for them, and shell has stagnated, and has distribution problems.
I mentioned in that comment that the Taskfile pattern in shell (ironically) doesn’t have autocompletion, and you want to generate help, etc. My former coworker made an attempt in pure shell, but it didn’t catch on: https://github.com/mbland/go-script-bash
So there is no single way to do it in shell. Someone also mentioned “scripts to rule them all”, which is fundamentally the same, but somehow didn’t catch on, despite being used by Github (at least for a time).
The “task running” functionality are the two comments above, which I just copy-paste in the shell.
Is there some short, non-horrible way to allow me to type nix shell --run serve and have that to what I want? Preferably with something like nix shell --run list-available-tasks.
That can work, but I’d love too keep this scoped to a single file, to reduce fs clutter, and scoped to a single CLI entry point (nix shell) to reduce mental clutter (cc @grahamc, this actually sounds like a user pain potentially relevant to determinate systems).
With this, you can use nix-shell --run install and nix-shell --run serve.
If you put a little more sophistication into it, I’m sure you can define the scripts in a way that you can also generate a list-available-tasks script. Something like:
I don’t understand the advantage here versus all Nix and building a container, if necessary, through Nix. Using the Nix shell is quick/dirty, but it turns a stateless build tool into one that has the state of the shell. Why just have a reproducible development environment when the whole derivation top to bottom can be reproducible? Between
packagesandappsyou can do just about everything and without the overhead of containers.I understand Nix is a learning cliff to swallow, but it will, in the long run, simplify this entire setup to just one tool instead of multiple and this one-tool simplicity makes it a lot easier for teams to understand.
You are right. But it took me literally months to learn enough Nix to be able do everything in Nix only. This setup is easier, it works fine and I figured it out in less than a day. So I stuck to it.
One of the reasons why I see not going full build in nix is when one needs to collaborate with others that do not use nix. For example, I have to occasionally contribute to internal python project, where each has a different setup (no instructions, use virtualenv with requirements.txt, use conda, use an internal tool). When I’m not a regular contributor, getting into nuances of each setup is a big overhead. That’s why I set up a shell.nix file that bring me almost everything I need to just start coding. Sometimes that works, sometimes it doesn’t, especially when external, non-python dependencies are needed (ironic, right?). That way, others can continue doing work using tools they are comfortable with, while I get my safety net.
Sure, I bring a Flake to JS projects like this and don’t commit it, but I if it’s my own repo, I’m not going to try suggest folks use 3 tools (Nix, Just, Podman) when I could use 1 (Nix). Depending on the audience you can suggest a
devShellor explain how to use the ‘normal’ community tooling, but even then, often the*2nixis overall a better experience and unlocks build sharing like Cachix.Wouldn’t it be easier to just make each justfile target call
nix-shell --run "some command"?FWIW you can also use shell for this instead of Just, since you’re writing shell anyway:
I would write this as:
The
{{ }}syntax in Just seems unnecessary, in adition to the*args. Shell already has stuff like that!This introduced more quoting problems than shell already has!
Probably should write a blog post about this … Related comment:
https://lobste.rs/s/sq9h3p/unreasonable_effectiveness_makefiles#c_bwfha2
Oh my god, yes, taskfiles all day long! You have to know shell anyway, why not use it for what it’s good at?
Yup, sometimes I feel like we’re stuck in a loop where we keep inventing the same mistakes – e.g. make doing its own interpolation with $ on top of shell, various YAML-based languages and {{ }} embedding shell, and now Just and {{ }}.
It’s 70’s string soup, but invented in 2020. (I am glad that memory safety has become a meme. But it’s weird that string hygiene isn’t a meme, despite those problems being arguably more common!)
In meme format:
But I also recognize that people want convenience, and defaults set up for them, and shell has stagnated, and has distribution problems.
I mentioned in that comment that the Taskfile pattern in shell (ironically) doesn’t have autocompletion, and you want to generate help, etc. My former coworker made an attempt in pure shell, but it didn’t catch on: https://github.com/mbland/go-script-bash
So there is no single way to do it in shell. Someone also mentioned “scripts to rule them all”, which is fundamentally the same, but somehow didn’t catch on, despite being used by Github (at least for a time).
Is there some nice way to add “task running” functionality directly to shell.nix? For example, I have the following shell.nix for my blog:
The “task running” functionality are the two comments above, which I just copy-paste in the shell.
Is there some short, non-horrible way to allow me to type
nix shell --run serveand have that to what I want? Preferably with something likenix shell --run list-available-tasks.I believe you could do something like this in a flake file:
And then run it with “nix run task-name”. I can’t remember the actual definition, so may need some fixing.
Similar https://www.ertt.ca/nix/shell-scripts/
Indeed!
Now the shell.nix has
and
nix-shell --run servedoes what I need, many thank!Why not just put those tasks as separate scripts in a subdirectory?
Like: https://github.blog/2015-06-30-scripts-to-rule-them-all/
That can work, but I’d love too keep this scoped to a single file, to reduce fs clutter, and scoped to a single CLI entry point (
nix shell) to reduce mental clutter (cc @grahamc, this actually sounds like a user pain potentially relevant to determinate systems).Sure thing, you can just put these little scripts into derivations and make them available in the shell environment:
With this, you can use
nix-shell --run installandnix-shell --run serve.If you put a little more sophistication into it, I’m sure you can define the scripts in a way that you can also generate a
list-available-tasksscript. Something like:I didn’t test any of this, so there is probably a little bit of debugging required to make it work, but I’m confident it can work like this.
makeis by definition a build system and not a command runner, right?