1. 82
  1.  

  2. 17

    I used to do this for years, a Makefile is like an executable README for tasks that developers sometimes need.

    And now my dayjob involves mostly C++ and suddently this doesn’t make any sense anymore because Makefiles are ignored by .gitignore (because qmake) and even if that wasn’t the case nobody is (rightfully) paying attention. Meh :P

    1. 15

      I’ve recently started using Makefiles for a lot more stuff; I use them in my notes directories to turn text files into PDFs, as well as whenever I’m programming. The only time I haven’t yet managed to get a decent workflow with make is when working with raw TeX (I try and avoid it where possible) - the workflow is so clunky (repeating commands and so on) that when I tried it still broke on me.

      Thanks for this!

      1. 18
        main.pdf:
        	latexmk -pdf $(basename $@)
        
        1. 3

          I didn’t know about latexmk, thanks!

        2. 3

          My current interesting (to me) Makefile use is, when I update underlying data tables, to automatically update all the dependent summaries and extracts for a public health research project at work.

          1. 3

            For commands that don’t have an output or need to be repeated multiple times, I found it useful to touch sentinel files, or use other artifacts outputted by the command that have a different semantic meaning, as checkpoints.

          2. 12

            Everyone who is interested in make(1) should check out the (superior) mk(1), a successor written by the original author, Andrew Hume, after 20 years of experience and use.

            There’s a good paper on it here http://www.vitanuova.com/inferno/papers/mk.html

            And there’s more on the cat-v website

            1. 7

              I’ve been using mk for my personal publishing workflows(and some JavaScript) for the past year or so. There’s a nice golang port of mk if it’s not in your system packages: https://github.com/dcjones/mk

              1. 2

                There’s also remake which has some nice extras, e.g.

                $ remake --tasks
                build
                fmt
                lint
                releases
                test
                test-integration
                test-memory
                todos
                update-version
                upload-releases
                vet
                

                Update: typo

                1. 2

                  Actually there are other more superior successors, called GNU Make and BSD Make. Both are pretty capable, and portable!

                  1. 9

                    Actually there are other more superior successors

                    [citation needed]

                    GNU Make is what I was referring to as the older version, it outright imports many of the problems that make(1) has including flaws that encourage bad Makefile usage.

                2. 9

                  I wonder how many projects requiring these trendier build systems like meson or ninja are using them as intended or to the capacity they allegedly facilitate. Meanwhile, make is unsexy but it’s everywhere.

                  I sort of get cmake but even that boils down to a Makefile. Do front ends like that really reduce enough boilerplate to justify making them a build requirement? They haven’t for my own projects but I’m generally not building in the large.

                  1. 6

                    I sort of get cmake but even that boils down to a Makefile. Do front ends like that really reduce enough boilerplate to justify making them a build requirement? They haven’t for my own projects but I’m generally not building in the large.

                    My experience with cmake is that it’s a worse wrapper than autotools/configure is, which is really saying something. I tried to get a i386 program to build on x86_64 and I had immense trouble just communicating to gcc what flags it should have, cmake actually lacked any architecture specific options that would have enabled me to do that.

                    1. 9

                      I’m not sure what you hit specifically, but cmake does provide those kind of options, namely CMAKE_SYSTEM_NAME, CMAKE_CROSSCOMPILING, and CMAKE_SYSTEM_PROCESSOR. I found this document really helpful in my own endeavours.

                      My personal experience with CMake is that it has a bear of a learning curve, but once you’ve seem some example setups and played around with different options a bit, it starts to click. Once you are more comfortable with it, it is actually quite nice. It has its rough edges, but overall I’ve found it to work pretty smoothly in practice.

                      1. 1

                        Ah, thanks! I’ll keep that in mind for next time I end up doing that

                    2. 4

                      CMake isn’t really an abstraction over Makefiles; in fact, there’s plenty you can probably do in Makefiles that would be cumbersome or perhaps impossible to do purely in CMake. It’s a cross platform build system that just uses Makefiles as one of it’s targets for doing the actual building step.

                      Where CMake tends to get it’s use (and what it seems to be intended for) is:

                      • Providing a build system for large, complex C++ (primarily) projects where lots of external dependencies exist and the project is likely to be distributed by a distribution or generally not controlled by the project itself
                      • Cross platform projects that are largely maintained for platforms where Makefiles are not well supported in a way that is compatible with GNU or BSD make, or where supporting ‘traditional’ IDEs is considered a priority (i.e. Win32/MSVC).
                      1. 2

                        Unfortunately, ninja is too simple for many tasks (e.g. no pattern rules) and building a wrapper is a more complex solution than Make.

                        Meson is too complex for simple tasks like LaTeX, data analysis, or plot generation. Make is great for these use cases but a few improvement are still possible:

                        • Hide output unless a command fails.
                        • Parallel by default.
                        • Do not use mtime alone, but in addition size, inode number, file mode, owner uid/gid. Apenwarr explained more.
                        • Automatic “clean” and “help” commands.
                        • Changes in Makefiles implicit trigger rebuilds where necessary.
                        • Continuous mode where it watches for file system changes.
                        • Proper solution for multi-file output.
                        1. 2

                          CMake can generate Makefiles, but I would hardly say its value simply boils down to what Make can provide. All of the value provided by CMake is what goes in to generating those files. It also benefits from being able to generate other build system files, i.e. using Ninja, or Visual Studio makefiles. A lot of complexity comes in when you need to adapt to locating dependencies in different places, detecting/auto-configuring compiler flags, generating code or other files based on configuration/target, conditionally linking things in, etc., and doing those things in plain Make is a huge pain in the ass, or not even possible in practice.

                          I wouldn’t say CMake is perfect, not by a long shot, but it works pretty well in my experience.

                        2. 9

                          If you don’t want to bother with .PHONY targets, tabs, weird syntax, and arcane built-in assumptions about build processes, check out Just, which is Make-inspired, but firmly focused on running commands, not producing files.

                          1. 9

                            I have a really hard time imagining what problems this solves that aren’t already solved by “a bin/ directory”?

                            1. 1

                              For example, just recursively walks up searching for justfile, so you can be deep in the project and still run just build, without clobbering PATH.

                              Consider the case where there are multiple projects, you’d have to use relative paths, or insist on unique name for scripts across all projects, or constantly reset PATH to project’s bin.

                              1. 2

                                I imagine you could use direnv for this, too—you could configure it so that whenever you enter a directory within ~/Projects/someproject it adds ~/Projects/someproject/bin to your PATH, and it would undo the change if you entered some other hierarchy. If you’re collaborating with others then I imagine that getting them to install Just would be easier than getting them to install and configure direnv, though.

                                1. 1

                                  I solve this problem a simpler way; I always have a shell open in the root of any project I’m ever working on, so bin scripts are very easy to use.

                            2. 7

                              tabs

                              yeah because that’s the problem with Make, it doesn’t use spaces.

                              1. 1

                                I have seen a few people mention Just. It looks like, while it does have a concept of dependencies, it doesn’t have a way to track if that dependency is satisfied or not, rather it just always runs all dependencies. Does it have a way to detect if the dependency is satisfied? In a Makefile, this is where the file timestamps play a role (and as I showed, we can use this even for tasks that don’t produce files).

                                1. 1

                                  AFAIR, Just always runs the dependencies, it is simpler mental model. This issue recommends using make in tandem with just when you want incremental runs.

                              2. 7

                                I’ve never really seen the point of these kind of Makefiles to be honest: it doesn’t use any of make’s dependency resolving features (“rebuild only a.c and b.c when b.c changes”) and is essentially just a shell script with extra steps and extra clunky syntax. I’d rather write just a shell script.

                                1. 1

                                  What kinds don’t use make’s dependencies? The example I gave does not re-resolve dependencies with bundle unless the file that declares the dependencies has changed. It also does not re-compile the javascript unless any of the javascript has changed. This is huge, in particular because the js and ruby tools on their own do not do this.

                                2. 4

                                  Yeah, I use Makefiles across languages and projects. It’s a classic. I wouldn’t mind some modern replacement but a structured Makefile might introduce complexity or something. There’s a lot of weird annotation syntax and exit code behavior or something … I don’t remember the details. Cmake, scons, I’ve never used them as a dev. It’d be cool to have some language’s flexible/easy/obvious/least-surprise/intuitive tool extracted out of the language ecosystem.

                                  1. 4

                                    I’ve fallen in love with Make again and again and use it in more contexts than I ever thought I would. It’s great for standardization of commands across codebases of a different language mix. At this very moment, I’m honing a Makefile that dozens if not hundreds of people in my organization will use in the next few months to write… white papers. I’ve got a solid Markdown to PDF pipeline constructed and people are chomping at the bit to use it plus GitHub PR reviews workflow to collaboratively edit beautifully typeset documents.

                                    I still get tripped up on paths with spaces. I wish that that would change. Using another tool is a less desirable option because Make is just… there, everywhere.

                                    1. 4

                                      Every time I start out with a Makefile, I end up instead with a bin/ directory full of shell scripts that work much better.

                                      1. 2

                                        But, how do you manage the dependencies between those scripts? You could still use a Makefile that calls those scripts, and then you get to define the dependency relationship between the scripts.

                                        1. 1

                                          It usually turns out that the dependencies aren’t usefully managed by a Makefile anyhow, and it’s less painful to just blow everything away and recompute each time if there are any.

                                      2. 3

                                        Makefiles are awesome! They’re one of those rare “learn it for life” tools. You may use meson, npm, or rake now, but who knows in 5 years. Make will likely be around forever.

                                        1. 2

                                          Yes! Exactly that!

                                        2. 3

                                          This HN comment talks about the problem of handling whitespace in Makefiles. I wrote safemount as a FUSE FS that URL encodes the filenames for another purpose (so you can mount the current directory in another directory but with filenames urlencoded). I wonder if it would be useful in Makefiles too.

                                          1. 2

                                            One of my favorite things about (GNU?) Make is that it already comes with a bunch of rules, so for a C project you can just do something like this:

                                            CFLAGS = -std=c11 -O2 -Wall -Wextra
                                            
                                            foo: foo.o bar.o
                                            

                                            I didn’t like the author’s Ruby Makefile at the bottom of the post, which was just filled with phony rules. I think that a small shell script would be nicer:

                                            #!/bin/sh
                                            
                                            case "$1" in
                                                "build")
                                            	echo "wowee building the program"
                                            	npm rake gulp grunt npx build
                                            	;;
                                                "deploy")
                                            	echo "deploying the cool program"
                                            	scp -r src/ remote:/srv/program
                                            	;;
                                                *) echo "Unknown command $1" ;;
                                            esac
                                            
                                            1. 7

                                              It’s a trap.

                                              Every makefile starts beautifully clean. And then you need dependencies. Which are slightly different between OSes. And then you want an out-of-tree build. And then you want to cross-compile. And then… all built-in rules work against you, and the whole thing is a mess.

                                              1. 2

                                                That’s true :( - ime cross-compiling works quite well, just setting CC and adding --sysroot=/blah/ to the CFLAGS, but I usually always end up switching to CMake.

                                                Maybe if you wanted to keep it simple you could write a configure script in shell or Python to generate your Makefile without conditionals and other funkiness.

                                                1. 3

                                                  Then you would reimplement what’s already written – CMake. But if someone thinks CMake is “too big”, then a good alternative would be Meson, which would be a configure-like (or CMake-like) script written in Python.

                                                  I don’t really see any real arguments for writing a custom configure script, unless the projects absolutely needs to have a custom one, but 99,9% of projects are not that custom.

                                              2. 3

                                                Most systems have tab completions for makefiles

                                                1. 1

                                                  That’s the main reason for me to use them.

                                                2. 3
                                                  1. Those rules are not tracking header dependencies for C files. So if you use any local headers in your project, changing them will not recompile the project, and will only produce confusion for people who expect dependency tracking to include headers.

                                                  2. Your shell script will ignore errors in commands, while Makefile script will fail entirely if 1 command from it will fail. To have your shell script behave similar to a Makefile, add “set -euo pipefail” directive at the top of the file. But then be prepared people will be confused how the shell script works, because they won’t notice this “set” directive ;)

                                                  1. 1

                                                    But this doesn’t manage the dependency relationship between those. If you just write the shell script, you will have to re-resolve the ruby dependencies, re-resolve the node dependencies, and re-compile the javascript every time you build, all of which will greatly increase the build time. With the Makefile, we detect if those steps are needed or not. Take another look.

                                                  2. 2

                                                    Just in case anyone else finds this useful… We’re doing C development on macOS using Make. One of the issues that we’ve run into, and may have now solved, is the awful performance of fork() on macOS, and the significant latency it adds when calling out to the shell from make files.

                                                    A new load directive was added in gmake v4.x which allows you to load a dynamic library, and have it register C functions to extend the expansion keywords available.

                                                    We’re currently splitting out the guts of our test harness utility into a separate library, and exposing the functionality as makefile expansions. We’ve found call overhead for a make expansion is about 7.8 microseconds, vs about 100ms for fork() (shell + ASAN + LSAN).

                                                    Even for non-macOS users, avoiding the overhead of calling out to a script to do text mangling that requires functionality not provided with the builtin expansions, would be a definite bonus in large build systems.

                                                    1. 2

                                                      See .ONESHELL for avoiding excessive fork. It essentially executes each recipe in a single shell script. If you set the shellflag -e, the failure behavior is almost the same as original makefile.

                                                    2. 2

                                                      The example is missing prerequisite:

                                                      file_1:
                                                          touch file_1
                                                      
                                                      file_2: <-- it should include "file_1" in here
                                                          touch file_2
                                                      

                                                      It seems like a typo.

                                                      1. 1

                                                        Thanks, I fixed this!

                                                      2. 2

                                                        Using Makefiles in order to automate you shell tasks is very fine, but please don’t use it to build software applications, or in other words: don’t use it in projects you intend others will use at some point.

                                                        Related article about it: http://anadoxin.org/blog/is-it-worth-using-make.html

                                                        1. 2

                                                          I’m not sure what it means for a Makefile to “support Visual Studio”. This type of thinking seems to put the requirement in the wrong place. Visual Studio, or whatever IDE, should be able to run an arbitrary command at least. But, the question that seems worth asking is why doesn’t Visual Studio support Makefiles? I mean, look, if someone is working in a .NET shop, of course just use the tooling that is available and out of the box. But this doesn’t seem to be an argument against makefiles in general.

                                                          1. 1

                                                            It means to be able to use the tool with Visual Studio.

                                                            Even if VS would support running a custom command, it’s still not enough. Running a custom command doesn’t inform the IDE about any definitions, include directories, relationships between files. It’s all hidden in a Makefile. So, VS would need to be able to interpret a Makefile directly in order to be able to extract this information. But, this still wouldn’t be enough, because then VS still wouldn’t be able to extract include directories, defined symbols information, linked system libraries, etc.

                                                            And CMake supports VS by having the ability to generate *.vcxproj. Those project files contain all necessary information so that VS can have all intellisense features enabled.

                                                            I agree that make and CMake are different tools, and I’m not requiring that make should support Visual Studio. I’m just saying that it doesn’t, and that for some people it’s a problem.

                                                            Would it be cool if VS would support Makefiles? Sure. But it doesn’t.

                                                        2. 1

                                                          I think Make is a poor choice for a simple wrapper script, and a disasterous choice for a sophisticated one. Your make tasks are shell code. If you don’t need the dependency invalidation, it is better to write a shell script so you only have to deal with one layer of arcane syntax, instead of two. You can e.g. run a linter on your Bash inside a bash script, but you don’t get any static analysis of the little snippets of bash you embed in your Makefile. Bash will let you break your code up into functions. The closest thing Make gives you to that are macros – which are an absolute nightmare. If you want to experiment, you can copy and paste your directly from your script into the bash prompt. With Make, there is no repl.

                                                          If you do need the dependency tracking, Make is also a bad tool. It has two modes: fail silently without any explanation of what tasks it performed and why – or extremely verbose mode where I will emit millions of lines and it’s up to you to weed out the interesting stuff. Again, you can’t write functions so there is no capacity for abstraction or code re-use. Also, using timestamps instead of checksums for invalidation will break spectacularly if you are trying to have any sort of build cache over a network.

                                                          Make is fine until it gets bigger than say 200 lines or until you have to reach for the manual. After that, i would rewrite in Shake, Bazel, or… anything else.

                                                          1. 1

                                                            Your points are valid. In my experience, having a dependency tree and a tool that is already available on all systems, is worth the trade-offs. Absolutely, their are poor Makefiles, and hugely complex Makefiles, but I think as a wrapper for the common case of node / ruby / elixir projects, where you are just calling out to docker and/or the build tool, maybe some ad-hoc commands, is the right amount of simplicity for a Makefile to be productive.

                                                            1. 1

                                                              Yeah there’s definitely a sweet spot. I just broke my rule and introduced a 50-line Makefile for a 48-hour hackathon project that relied on a dependent series of JSON files of data scraped from Wikipedia.

                                                              (I would have used shake but 48 hour project is not the time to thrust Haskell upon 2 teammates)

                                                              But I “has dependency graph, so Bash won’t do” and “not complicated/important enough that’s it’s worth sacrificing ubiquity and familiarity for a more disciplined tool” is a relatively small space.

                                                              1. 1

                                                                And I mean, I wasted probably 20 min total dealing with stupid mistakes putting together/working with even this 50-liner Makefile that uses nothing advanced – just because Make has no linter, and doesn’t allow the introduction of intelligent programming practices like functions and variable

                                                          2. 1

                                                            I used to drop a script called ‘build’ in every code directory - doing whatever was required for that particular code - and after a while I did alias make=./build

                                                            1. 1

                                                              Make isn’t great for running commands, because multi-line commands are difficult, and it can’t modify the environment (e.g. change directories or set environment variables)

                                                              So I made .go.sh, which I use heavily. It supports both shell functions and Python functions, and M4 macros. See here for a bunch of examples.

                                                              Make is great for what it’s designed for, though. I (ab)use it plenty; see any of the Makefiles in PoprC.

                                                              1. 3

                                                                See the .ONESHELL phony target. Is this what you want? (i.e multiline and cd)

                                                                1. 1

                                                                  That doesn’t work because it creates a new shell, but I want to modify the existing shell.

                                                                  1. 1

                                                                    I am not sure I understand, are you talking about setting environment variables? This can be done with export in GNU Make.

                                                                2. 2

                                                                  Yeah, it’s not the most intuitive tool, of course, and it does have some peculiarities, but I disagree on the point that you can’t modify the environment.

                                                                  If you need to cd into a directly, you can write that as part of the recipe. And for environment variables, this is also straightforward.

                                                                  mytarget:
                                                                    MY_ENV=$(MY_ENV) cd some/directory && ls -ahl
                                                                  
                                                                3. 1

                                                                  I continue to use hand-crafted makefiles for simple builds (driving a static blog, generating LaTeX, etc.), even throwing stub makefiles in Clojure (leiningen is the actual build tool) and Android/Java (gradle) projects as a convenience. For more complicated tasks that cross platforms (namely to Windows) I have been using waf which is written in Python. It’s very powerful and gives more than enough rope.

                                                                  1. 0

                                                                    There is actually a very nice example of this idea for PHP in Apache Ant: http://jenkins-php.org/automation.html