The “downsides” list is missing a bunch. I mean, I use Makefiles too, probably too much, but they do have some serious downsides, e.g.
$(SHELL)
, giving some awful escaping at timespossibly_failing_command | sed s/i/n/g > $@
(which will create $@
and trick make into thinking the rule succeeded because sed
exited with 0 even though the first command failed). And do all your devs know how to have multiple goals that each depend on a temp dir existing, without breaking -j
?and there’s probably lots more. OTOH, make been very useful to me over the years, I know its quirks, and it’s available on all kinds of systems, so it’s typically the first thing I reach for even though I’d love to have something that solves the above problems as well.
Your additional downsides makes it sound like maybe the world needs a modern make
. Not a smarter build tool, but one with less 40-year-old-Unix design sensibilities: a nicer, more robust language; a (small!) handful of missing features; and possibly a library of common functionality to limit misimplementations and cut down on the degree to which every nontrivial build is a custom piece of software itself.
i’ve also thought of that! for reference: https://9fans.github.io/plan9port/man/man1/mk.html
I think the same approach as Oil vs. bash is necessary: writing something highly compatible with Make, separating the good parts and bad parts, and fixing the bad parts.
Most of the “make replacements” I’ve seen make the same mistake: they are better than Make with respect to the author’s “pet peeve”, but worse in all other dimensions. So “real” projects that use GNU Make like the Linux kernel and Debian, Android, etc. can’t migrate to them.
To really rid ourselves of Make, you have to implement the whole thing and completely subsume it. [1]
I wrote about Make’s overlap with shell here [2] and some general observations here [3], echoing the grandparent comment – in particular how badly Make’s syntax collides with shell.
I would like for an expert in GNU Make to help me tackle that problem in Oil. Probably the first thing to do would be to test if real Makefiles like the ones in the Linux kernel can be statically parsed. The answer for shell is YES – real programs can be statically parsed, even though shell does dynamic parsing. But Make does more dynamic parsing than shell.
If there is a reasonable subset of Make that can be statically parsed, then it can be converted to a nicer language. In particular, you already have the well-tested sh parser in OSH, and parsing Make’s syntax 10x easier that. It’s basically the target line, indentation, and $() substitution. And then some top level constructs like define, if, include, etc.
One way to start would be with the “parser” in pymake [4]. I hacked on this project a little. There are some good things about it and some bad, but it could be a good place to start. I solved the problem of the Python dependency by bundling the Python interpreter. Although I haven’t solved the problem of speed, there is a plan for that. The idea of writing it in a high-level language is to actually figure out what the language is!
The equivalent of “spec tests” for Make would be a great help.
[1] https://lobste.rs/s/ofu5yh/dawn_new_command_line_interface#c_d0wjtb
[2] http://www.oilshell.org/blog/2016/11/14.html
We need a modern make, not make-style tools. It needs to be mostly compatible so that someone familiar with make can use “modern make” without learning another tool.
I think anything compatible enough with make
to not require learning the new tool would find it very hard to avoid recreating the same problems.
The world does, but
s/standards/modern make replacements/g
Do most of these downsides also apply to the alternatives?
The cross platform support of grunt and gulp can be quite variable. Grunt and gulp and whatnot have different features. The make world is kinda fragmented, but the “not make” world is pretty fragmented, too.
My personal experience with javascript ecosystem is nil, but during my foray into ruby I found tons of rakefiles that managed to be linux specific, or Mac specific, or whatever, but definitely not universal.
I recommend looking at BSD make as its own tool, rather than ‘like gmake but missing this one feature I really wanted’. It does a lot of things people want without an extra layer of confusion (automake).
Typical bmake-only makefiles rarely include shell script fragments piping output around, instead they will use ${VAR:S/old/new} or match contents with ${VAR:Mmything*}. you can use ‘empty’ (string) or (file) ‘exists’.
Deduplication is good and good mk fragments exist. here’s an example done with bsd.prog.mk. this one’s from pkgsrc, which is a package manager written primarily in bmake.
You really have to understand how make works to avoid doing things like
possibly_failing_command | sed s/i/n/g > $@
(which will create$@
and trick make into thinking the rule succeeded becausesed
exited with 0 even though the first command failed).
Two things you need to add to your Makefile to remedy this situation:
SHELL := bash -o pipefail
. Otherwise, the exit status of a shell pipeline is the exit status of the last element of the pipeline, not the exit status of the first element that failed. ksh
would work here too, but the default shell for make, /bin/sh
, won’t cut it – it lacks pipefail
..DELETE_ON_ERROR:
. This is a GNU Make extension that causes failed targets to be deleted. I agree with @andyc that this behavior should be the default. It’s surprising that it isn’t.Finally, for total safety you’d want make
to write to .$@.$randomness.tmp
and use an atomic rename if the rule succeeded, but afaik there’s no support in make
for that.
So yes, “you really have to understand how make works [to avoid very problematic behavior]” is an accurate assessment of the state of the things.
Hey! Original author here :). Thanks a bunch for this feedback. I’m pretty much a Make noob still, so getting this type of feedback from folks with more experience is awesome to have!
Your temp directories dependency problem makes me think a GUI to create and drag drop your rules around could be useful. It could have “branching” and “merging” steps that indicate parallelism and joining too.
Hey all! Original author here! Flattered this blog post found it’s way for discussion on this site. In fact, I’m ashamed to admit I wasn’t familiar with this site before. Seeing this comment thread (and other articles being posted), I love what I’m seeing. There is some great conversation here that will help me clear up my own ignorance and build on the knowledge of folks who have much more experience with this tooling for me. So thanks again :D.