I think people mistake make for a task runner when it’s actually a (very basic) build system which has some options that make it somewhat viable as a task runner. I don’t have resources at hand for getting people started on how to use make “the right way”, but one neat trick that might get you started is knowing that you can do this without even having a Makefile:
$ printf '#include <stdio.h>\nint main(void) { puts("wah!"); return 0; }' > wah.c
$ make wah.o wah
cc -c -o wah.o wah.c
cc wah.o -o wah
$ ./wah
wah!
That said, make has a lot of footguns, and with most programming languages you’re gonna be using a language-specific build system that keeps track of all the complexities you’d run into if you were to write the Makefiles yourself, so using a shell script or an actual task runner will serve you better than make.
Rule of thumb: any make target that doesn’t become a no-op if you run it twice in a row is better expressed as a shell script. make run &c. is a clear red flag.
I would actually disagree on that. I use Makefiles as the default entry point to do anything with my software (building, packaging, running, provisioning, deploying, testing, releasing). The key problem is that Makefiles do not clearly make the distinction between asset-producing rules and automation rules (.PHONY). I find it makes it easy for others to use, even when they’re not familiar with the underlying stack. Make run, make build, make dist, make test, etc is pretty straightforward to use. I also use make to provision Nix development environments, and now have reached the holy grail of “make run” working on any Linux box after a fresh clone of the git repo and no extra configuration.
The key problem is that Makefiles do not clearly make the distinction between asset-producing rules and automation rules (.PHONY).
Well, I guess I would say that make is fundamentally a tool for asset producing rules, and while you can bend a few features (like .PHONY) to make it do other things, that’s not really what it’s been built for.
Yes. Underneath the arcane language lies a delightful paradigm – if it’s any point in using Make, it must be this: To just describe the dependency tree for the machine, let the machine figure out what needs to be rebuilt when inputs change, and profit massively by only rebuilding what needs to, with implicit parallellism even.
If you need to do this, you have made a mistake in your needs. Reconsider what you need done, and save future employees the trouble of a magic script that does all the things you need.
You might be tempted to protest. Let me summarize the other likely position. “It works on my machine.”
If you write it, it will be used for operations. If it’s used for operations, it will grow in scope and complexity. If it grows in scope and complexity, it will not know where to stop. It’s just a program, and the authors are too busy doing their jobs to worry about the scope creep of an unlimited scope program.
Eh this isn’t a useful comment without knowing the alternative …
I get why people are “afraid” of shell, but often the alternative is some sort of YAML monstrosity … that is not fully automated.
Also, this article is about replacing make. Shell is strictly better than make when you’re not using dependencies, i.e. just runing tasks. Because every line of a Makefile invokes the shell anyway.
Make is better than shell if you will be replacing the shell with automation later. It’s easier to go “yes this bit can be replaced” than trace a file to figure out if that exported variable changes any of the other commands being invoked.
The shell script approach leads to dangerous territory. A Makefile can be replaced. A bevy of single purpose scripts can be replaced. A magic do everything script is very hard to delete.
When running bin/oil, set -euo pipefail is on by default. And when running bin/osh, you can also do shopt -s oil:basic which includes that plus some more error handling.
It also has $_this_dir because I have the HERE thing in almost every shell script as well.
So you can do $_this_dir/mytool or cd $_this_dir.
The variables prefixed with _ are “special” globals that are automatically set by the interpreter
I recently converted a repo where “make” was more task runner than dependency tracker to using task, https://taskfile.dev/.
It’s yet another YAML system, but is clearly scoping to avoid doing programming in YAML: it’s structured and feels more like a Circle or GHActions flow. Except unlike GHActions, it’s shell syntax on every platform because of the use of https://github.com/mvdan/sh to parse the instructions and run them. Having recently had to debug and get GHActions working across a build matrix of platforms, including Windows, I really appreciate this design choice.
I have a project-level justfile that does more or less the same thing as the shell script but with all of the ergonomics of a Makefile. Using https://just.systems to automate common Rails, React, amd Postgres tasks is extremely convenient.
I went to click on this and got a full screen j u s t with no ability to scroll on Bromite or Fennec on Android. Was not giving the user any info intentional?
Make is a poor caching system that sometimes triggers on changed files but also has false positives and false negatives: https://apenwarr.ca/log/20181113
I think people mistake make for a task runner when it’s actually a (very basic) build system which has some options that make it somewhat viable as a task runner. I don’t have resources at hand for getting people started on how to use make “the right way”, but one neat trick that might get you started is knowing that you can do this without even having a Makefile:
That said, make has a lot of footguns, and with most programming languages you’re gonna be using a language-specific build system that keeps track of all the complexities you’d run into if you were to write the Makefiles yourself, so using a shell script or an actual task runner will serve you better than make.
+1. Make is not a task runner.
Rule of thumb: any make target that doesn’t become a no-op if you run it twice in a row is better expressed as a shell script.
make run&c. is a clear red flag.Yup agreed, another issue is that most such make targets should be
.PHONY, but this bug is easy to miss.i.e.
touch mytargetthen the Makefile will no longer “work” as expected. People rely on the file not existingI would actually disagree on that. I use Makefiles as the default entry point to do anything with my software (building, packaging, running, provisioning, deploying, testing, releasing). The key problem is that Makefiles do not clearly make the distinction between asset-producing rules and automation rules (.PHONY). I find it makes it easy for others to use, even when they’re not familiar with the underlying stack. Make run, make build, make dist, make test, etc is pretty straightforward to use. I also use make to provision Nix development environments, and now have reached the holy grail of “make run” working on any Linux box after a fresh clone of the git repo and no extra configuration.
Well, I guess I would say that
makeis fundamentally a tool for asset producing rules, and while you can bend a few features (like .PHONY) to make it do other things, that’s not really what it’s been built for.Yes. Underneath the arcane language lies a delightful paradigm – if it’s any point in using Make, it must be this: To just describe the dependency tree for the machine, let the machine figure out what needs to be rebuilt when inputs change, and profit massively by only rebuilding what needs to, with implicit parallellism even.
I work with an app using this technique.
If you need to do this, you have made a mistake in your needs. Reconsider what you need done, and save future employees the trouble of a magic script that does all the things you need.
You might be tempted to protest. Let me summarize the other likely position. “It works on my machine.”
If you write it, it will be used for operations. If it’s used for operations, it will grow in scope and complexity. If it grows in scope and complexity, it will not know where to stop. It’s just a program, and the authors are too busy doing their jobs to worry about the scope creep of an unlimited scope program.
Save yourself.
Eh this isn’t a useful comment without knowing the alternative …
I get why people are “afraid” of shell, but often the alternative is some sort of YAML monstrosity … that is not fully automated.
Also, this article is about replacing make. Shell is strictly better than make when you’re not using dependencies, i.e. just runing tasks. Because every line of a Makefile invokes the shell anyway.
Make is better than shell if you will be replacing the shell with automation later. It’s easier to go “yes this bit can be replaced” than trace a file to figure out if that exported variable changes any of the other commands being invoked.
The shell script approach leads to dangerous territory. A Makefile can be replaced. A bevy of single purpose scripts can be replaced. A magic do everything script is very hard to delete.
I do this so often that I have a hotkey that pastes the following header:
FWIW
bin/oil,set -euo pipefailis on by default. And when runningbin/osh, you can also doshopt -s oil:basicwhich includes that plus some more error handling.$_this_dirbecause I have the HERE thing in almost every shell script as well.$_this_dir/mytoolorcd $_this_dir.Testing/feedback is appreciated! https://www.oilshell.org/
I think you should quote
$0, since the path may contain a space. I’d be surprised if shellcheck didn’t complain about this, although I haven’t tested.No, as its zsh, so unless the
~/.zshrcfile sets theSH_WORD_SPLIToption, it won’t split on whitespace.(If only deploying to systems with a modern
env, then#!/usr/bin/env -S zsh -fwould avoid sourcing files and risks of custom options affecting this)I recently converted a repo where “make” was more task runner than dependency tracker to using
task, https://taskfile.dev/.It’s yet another YAML system, but is clearly scoping to avoid doing programming in YAML: it’s structured and feels more like a Circle or GHActions flow. Except unlike GHActions, it’s shell syntax on every platform because of the use of https://github.com/mvdan/sh to parse the instructions and run them. Having recently had to debug and get GHActions working across a build matrix of platforms, including Windows, I really appreciate this design choice.
Example:
https://github.com/nickjj/docker-flask-example/blob/main/run
Glad to see someone else using the same pattern I use. Reminds me I should write up these posts:
http://www.oilshell.org/blog/2020/02/good-parts-sketch.html#semi-automation-with-runsh-scripts
I have a project-level
justfilethat does more or less the same thing as the shell script but with all of the ergonomics of aMakefile. Using https://just.systems to automate common Rails, React, amd Postgres tasks is extremely convenient.I went to click on this and got a full screen
j u s twith no ability to scroll on Bromite or Fennec on Android. Was not giving the user any info intentional?The letters are clickable and link to the github repo.
An underline would maybe help users know this is a link and clickable eyeroll
I just incorporated this at work, would recommend it.
Not always a good idea as
makealso only triggers on changed files.Make is a poor caching system that sometimes triggers on changed files but also has false positives and false negatives: https://apenwarr.ca/log/20181113
“most of the times”?
I use make because most distributions have tab completions for makefiles