1. 50
  1.  

  2. 43

    If you want to heavily invest into GNUisms (bash, RECIPEPREFIX, …) and make your makefile less portable, go ahead and follow the advice in this article. If you don’t, be my guest, use sh(1) instead of bash(1), read the make-standard and test your makefiles with other make-implementations than GNU make.

    This helped me tremendously with writing good, concise and simple makefiles. We need to stop writing unportable makefiles, and especially stop relying on the bloated GNU ecosystem.

    1. 11

      Alternatively, name your makefile GNUmakefile to mark it as “only tested with GNU”.

      1. 10

        and especially stop relying on the bloated GNU ecosystem.

        I know this site likes to hate on GNU, but its make has more useful features than any other make. % rules make writing pattern rules so much easier. The text-manipulation functions make working with variables so much nicer. These are “killer” features of GNU make for me.

        1. 11

          I used to be pretty harsh on GNU stuff until I realized how decrepit pure POSIX implementations are. For better or for worse, GNU tools do what people actually want, instead of some religious interpretation of Unix or minimum specification compliance.

          1. 5

            Seconded. POSIX make is pretty sparse. GNU Make has a bunch of warts but it has some conveniences that are nice to work with. Reading what automake dumps out isn’t a good way to judge it, either.

          2. 2

            I agree. Writing a portable (POSIX compatible) works when you do “regular” stuff, like building .c -> .o -> binaries, but anything more complex ends up having to lash out to shell script, either in them or around.

          3. 6

            I don’t agree with the article as well, although for a different reason (that might be added up to yours, really).

            My standpoint is that shell scripting and makefiles are poorly understood by many people in the industry. Spicing up a makefile like this is likely to end up in wrong makefiles when a colleague having a shallow experience with these tools is going to change something.

            To give an example, it is expected that individual failures in the recipes will make the whole build rule fail, but setting ONESHELL will change this sematics dramatically, with no change in syntax! Unless you also set SHELLOPTS += -e, error checking will work in an often unexpected way!

          4. 16

            Using > instead of \t is horrible imho - but the rest was pretty interesting.

            1. 2

              I think \t would be better replaced by ; than by ‘>’. It would let you join two consecutive lines together (vi J) or split a long line with multiple commands with little effort.

            2. 15

              IMO a text editor should ensure that makefiles use tabs. Or at least warn you if you don’t. This will just surprise everyone who doesn’t know it.

              1. 7

                GNU make is sufficiently smart to tell you when you mess up indentation also:

                $ cat makefile
                target:
                        echo bad indent
                $ make
                makefile:2: *** missing separator (did you mean TAB instead of 8 spaces?).  Stop.
                
                1. 2

                  Seems to only work if you use 8 spaces, but fails with the very verbose message

                  Makefile:2: *** missing separator. Stop.

                  if you use 4.

                  1. 1

                    Why would you not use 8 spaces anyway ? 😉

                2. 2

                  I don’t understand why editors don’t default to visualizing whitespace. The way Atom does it for example is extremely non-intrusive, yet makes it obvious when mixing is going on.

                  When multiple people are working on a code base, unless either they all use one standardized editor with a standardized config it they all use editors which visualize whitespace, unfortunate mixing of tabs and spaces seems inevitable. Standardizing on visualizing whitespace rather than standardizing on everything else sounds much better for programmer happiness.

                  I have even inadvertently accepted pull requests which introduce mixing because GitHub doesn’t visualize whitespace, so you have no immediately obvious way to know that an otherwise completely innocent change was written using spaces when the rest of the project uses tabs.

                  It’s just such a big problem which is so easy issue to solve, if editor vendors just dared to show their users what characters their whitespace actually consists of.

                  1. 0

                    I don’t understand why editors don’t default to visualizing whitespace. The way Atom does it for example is extremely non-intrusive, yet makes it obvious when mixing is going on.

                    When multiple people are working on a code base, unless either they all use one standardized editor with a standardized config it they all use editors which visualize whitespace, unfortunate mixing of tabs and spaces seems inevitable. Standardizing on visualizing whitespace rather than standardizing on everything else sounds much better for programmer happiness.

                    I have even inadvertently accepted pull requests which introduce mixing because GitHub doesn’t visualize whitespace, so you have no immediately obvious way to know that an otherwise completely innocent change was written using spaces when the rest of the project uses tabs.

                    It’s just such a big problem which is so easy issue to solve, if editor vendors just dared to show their users what characters their whitespace actually consists of.

                  2. 8

                    I think instead of trying to shoehorn bash into make, I will allow make to continue being make and delegate bash scripting to bash scripts.

                    1. 0

                      I think instead of trying to shoehorn bash into make,

                      How is this shoehorning bash into make? Every recipe used is under 3 commands. I don’t see what you would gain by splitting out parts of this makefile into a bash script. If anything, it would be more difficult to read because the relevant code would no longer be integrated.

                      1. 2

                        How is this shoehorning bash into make?

                        The article recommends setting a specific shell, and some specific shell flags. This is just to make it easier to write Bash inside a Makefile.

                        I don’t see what you would gain by splitting out parts of this makefile into a bash script. If anything, it would be more difficult to read because the relevant code would no longer be integrated.

                        I think a Makefile is useful as an ad-hoc command runner with a bundled directed acyclic graph resolver. I also think Bash is an excellent tool, though it comes with plethora caveats (enough so that I generally don’t write shell scripts without ShellCheck [and how would this work in a Makefile?]), and it seems unwise to introduce further caveats by having to account for the Makefile’s exceptional parsing rules. You think they’re essentially the same? Think again. Unless you’re suggesting that three commands ought be enough for anybody, in which case, I’m not sure what anyone can learn from this article except that with enough fettling you can make a tool do something it’s not exactly designed to in some trivial cases.

                        My approach basically is to keep everything as boring and simple as possible, so I can focus on making money with my business. Adding complexity to a Makefile is unlikely to be a wise investment for my business, IMO.

                        1. 1

                          Unless you’re suggesting that three commands ought be enough for anybody

                          But…. they really are enough for anyone. In order to use the “bundled directed acyclic graph resolver,” one needs to add the graph information. Usually recipes only need a few commands to transform the inputs into outputs. More than this and I agree that it’s probably better to split it off into its own script. However, the stuff that OP is adding also makes it easier to do inline bash programs. For example, .ONESHELL allows you to do something like

                          foo: bar baz
                                  if [ -z $(SOME_FLAG) ]; then
                                          frobnicate $^ -o $@
                                  fi
                          

                          which would require backslashes at the end of each line otherwise. Personally, I have no problem with bash-isms. At some point I remember needing a bash feature in my Makefile, but I can’t remember the context at the moment.

                          1. 1

                            In order to use the “bundled directed acyclic graph resolver,” one needs to add the graph information

                            Yeah. That comes from the part that make gives you, i.e., where you declare your targets and their dependencies. It doesn’t come from the recipe.

                            However, the stuff that OP is adding also makes it easier to do inline bash programs.

                            Yes. I know. This is the approach I have been disagreeing with the entire time. I do not think it’s a good idea to write Bash scripts inside a Makefile. I’ll ask you again: How do you run ShellCheck on a Makefile?

                            I have no problem with bash-isms

                            I also don’t have a problem with bashisms, but when people say “bashisms”, they don’t mean using Bash in a Makefile. They mean writing shell scripts specific to Bash when they ought to have written a more portable version, i.e., using features defined by POSIX.

                            1. 1

                              I’ll ask you again: How do you run ShellCheck on a Makefile?

                              You don’t. But I don’t really see the problem with that. It’s not like there’s hostile user input. At worst, the directory make is running in has spaces in it (or some weird character).

                              1. 1

                                It’s not like there’s hostile user input

                                Spoiler alert: The hostile user is you.


                                ShellCheck is not a user input validation library. It’s an invaluable shell script static analysis tool. It will catch things you hadn’t thought of. To think otherwise, I could only ascribe to hubris.

                    2. 5

                      And you will never again pull your hair out because some editor swapped a tab for four spaces and made Make do insane things.

                      This problem needs a wider solution than overriding your Makefile defaults to not make it look like the majority of other people’s Makefiles. I get pulled into different teams and companies frequently and not every team follows company convention or puts an editorconfig or something in their repo. I now just turn on “visualize (leading) whitespace” or whatever each IDE/editor calls it and have a lot easier time with tab/space consistency.

                      " e.g. vim
                      set listchars=tab:>.
                      set list
                      
                      1. 4

                        Changing the defaults is very slippery thing to do.

                        Once I’ve prepared a shell script with set -euo pipefail somewhere on the beginning to save time during future modifications. Later, another guy wanted to resolve some bigger problem, and changing this shell script was one step of many. He spent 2 hours fighting with the shell script because he wanted to do it quickly, and re-checking his OS because he thought the shell script “is behaving in a strange way”. He didn’t realize/didn’t notice the “set -euo pipefail” directive.

                        So, end result is that instead of doing something useful, I’ve made things worse. I think that using modifications from this blog post in a corporate setting can make things worse as well.

                        1. 6

                          I don’t agree that you made things worse. Bash in particular has terrible defaults for robust scripting, and making those failure modes explicit and immediate is a net improvement. “wanted to do it quickly” is the root of many programming errors that come back and bite you in the ass later.

                          1. 5

                            While I appreciate your point, if you don’t put set -eu in the shell script someone is going to lose two hours because of a typo or other small error leading to an undefined variable, or because the script keeps running after an error in ways that weren’t appreciated.

                            Never mind situations like rm -rf "$prefiix/usr" 😬

                          2. 6

                            And you will never again pull your hair out because some editor swapped a tab for four spaces and made Make do insane things.

                            One of first steps to be advanced user or even programmer is to master your tools. If the tools (like editors) are not under your control, you should fix this issue before doing anything else.

                            1. 2

                              This is pretty good. I did not know about .RECIPEPREFIX. Thanks for introducing me to that.

                              I have my own set of best practices here — It is not a blog post yet, but more for my own recall.

                              1. 1

                                I’ve done some creative substitution work to create good file-based targets in the past and take advantage of Make’s laziness. The sentinel file, though, is a very nice hack that gets some of the benefit without much work. I appreciate that tid-bit. As for the rest, I agree with the others that going all in on GNU and Bash can be helpful, but it cuts down on portability. People still do run BSD systems, after all.

                                1. 2

                                  Writing this post I was made aware of the empty target, which I now think is the traditional name for this pattern. Though it’s not documented as being useful for rules that output multiple files.

                                2. 1

                                  honestly, the .RECIPEPREFIX thing is a god send. I don’t have that many issues with make and tabs, but for the few times I generated makefiles to work with some other vendor’s makefile amalgamation, this looks much cleaner than something like printf -v tablit '\t'; cat << EOF .... ${tablit}some make stuff ... EOF

                                  thanks for the information.

                                  1. 1

                                    Since there is a deploy: test rule in the example. Will this actually depend on a recipe for test if that is made .PHONY? Or will a file name test still satisfy the rule?

                                    1. 1

                                      If you were to declare test phony, then any rule that declared test as a prerequisite would always be re-run, and would not look for a file named test.. I’m not sure if there’s a good use case for doing that, vs just using sentinel files for the test target.

                                      What we do if we want a “user friendly” name for a target like testing is to have a sentinel target that other rules depend on, and then a dedicated phony target for direct calls:

                                      tmp/.tests-ran: <source files>
                                      > touch out/.tests-ran
                                      
                                      # This is now just an alias to make it possible to run `make test` for human users
                                      test: tmp/.tests-ran
                                      .PHONY: test
                                      
                                      deploy: tmp/.tests-ran:
                                      > ..
                                      .PHONY: deploy