The problem with the idea of using a simple build system is that it imposes a big change later if the project grows. At some point you will need:
To build docs as well
To build tests, run them in some test harness, and report the results.
To find dependencies.
To generate a config header, pre-defined macros, or conditional build behaviour depending on the target.
Different debug and release configurations, possibly including sanitisers in debug builds.
You can build all of these on top of Make (bmake or GNU Make, not POSIX make) or Ninja, but you end up with a lot of complexity.
At some point, you realise that your build system is so bespoke that it’s completely incomprehensible to any outsider (see GNUstep and FreeBSD for examples). Now you have a choice of throwing away everything and starting again, or keeping adding complexity to a system with a rapidly shrinking bus factor.
A simple CMake file that generates the equivalent ninja file is simpler that this Ninja file is shorter than the Python program and CMake is a smaller dependency. Bazel looks simpler, but I can’t build any Bazel projects on FreeBSD/AArch64 at the moment because it depends on Java and OpenJDK appears to be broken. Meson looks to have a similar set of dependencies to this Python script.
If you start with a widely used build system, you will find it easy to find people that can modify your build system. If you do something bespoke, it will become a bottleneck for your project at some point in the future. The easier it looks to use. The more magic it will depend on until you discover that there is no one that understands the build system still involved and making a small change is impossible.
In general I agree with your comments. Configuration is the real killer non-feature of “simple Makefiles” (or “simple ninja files” too, really). So many things you just can’t do without a proper system probing and conditional build generator.
Make actually has a small advantage over ninja here, because with make you can use GNU-isms and get a complex and fragile build system that evaluates repeatedly every time you run make, uses environment variables for conditional build settings, and kinda works a bunch of the time. For ninja this is simply impossible: it shuns any form of pattern rules, $(shell …) based rule redefinition, if/else’d rules… the idea behind ninja was that all that stuff is a burden and a distraction from being a fast interdependency-ordered command runner and people should “just use a generator program like autotools has historically been for Makefiles”.
By the way: on the meson topic, meson is internally written in python, as is the python script, but it does have one less dependency: meson rigidly and pedantically limits itself to the python standard library to avoid dragging in a dependency on either additional python modules or the complex distribution method of “somehow get a python PyPI toolchain going with setuptools and pip or some kind of pep 517 build frontend, handling cyclic python dependencies, etc etc etc”.
Also, meson is intentionally designed to not be turing complete (it uses a custom DSL and does not permit using python or even adding python plugins), there’s a FAQ entry that notes a couple of reasons why we made that choice. One of the reasons is because it makes it impractical to ever port over to another language “should performance become a limiting factor”. And notably, this is a huge issue for autotools.
This has already paid off, because the community has taken the opportunity to write a c99 reimplementation of meson: https://sr.ht/~lattis/muon
We think that’s fantastic, both that people were able to scratch a personal itch like this (using and running meson builds on systems that don’t have python installed) and that we sometimes discover meson bugs or documentation ambiguities as a result of seeing how meson and muon occasionally behave differently.
At first I thought it was literally Python, but it looks like it’s a language written in Python, and then now as you say it’s ported to C.
C is definitely better for build systems because it’s easier to bootstrap on a new machine. (C is also horrible for string processing but that isn’t a dealbreaker)
As long as you use/make a reasonable string buffer type it is perfectly fine. String processing doesn’t benefit much from generics where C lacks some expressivity.
I needed to build something that uses Bazel a while ago. I didn’t have access to a pre-built version of Bazel, though I did have a functioning copy of OpenJDK installed. I was going to build Bazel myself, so I downloaded the provided bootstrapping archive. Inside was a shell script you were supposed to run, which I did. What did it do? Download an older version of Bazel (I think) and a copy of OpenJDK that was incompatible with my OS. Upon seeing this, I decided to never ever touch anything Bazel ever.
If you start with a widely used build system, you will find it easy to find people that can modify your build system
I broadly agree though I would raise that there is at least one widely used build systems which is so difficult that it negates this advantage. Specifically I’m thinking of autotools.
Python isn’t a great dependency, but I think it’s a lot more familiar and understandable than CMake.
I thought you were on the record saying CMake is ridiculously inscrutable.
I think CMake is OK when you’re doing “standard” well-trodden stuff, but as it gets more advanced – which happens as project grows – I think it gets more hairy.
Python is maybe more awkward since it’s not purpose-built, but it “scales” better and you can easily debug it.
One problem that was a bit tricky when writing my own Ninja generator is Python code generators, but it ended up working well.
I believe most people don’t even attempt this in GNU Make, but I’m not sure about CMake. (The more conventional thing seems to be to check in generated code)
That is, suppose the code gen tool is asdl_main.py, and it depends on util.py, and it produces foo_asdl.{h,cc}.
Then if util.py is changed, it has to invalidate foo_asdl and the binaries that depend on it (re-generated and re-compile)
With C++ code generators, the generator is one binary file, and its dependencies are known. With Python it’s less clear.
Bazel does this well, and my Ninja generator does too, but it required some custom plumbing / shell stubs.
Python isn’t a great dependency, but I think it’s a lot more familiar and understandable than CMake.
Familiar is subjective. Understandable is debatable. About ten years ago, I fixed a bug in some Python code that was due to their not understanding what an else clause on a while loop meant (my hatred of Python largely comes from a decade of having to write it only because packaged, release versions of Python things failed to launch because my computer wasn’t identical to that of the authors). Over the next few years, I asked people that told me that they liked Python what this meant. It was three years before someone got the answer right. That’s a fairly simple control-flow construct. Once you get into decorators, you discover that even C++ doesn’t have quite the extent of totally distinct languages that are subsets.
The real problem is stability. Maybe if you stick to the standard library and hope nothing like Python 3 will ever happen again, there’s a subset of the Python ecosystem for which you can write code and expect it to still work next year. I’d like to see an existence proof though.
I thought you were on the record saying CMake is ridiculously inscrutable.
The CMake language is awful and the abstractions that it provides are not great, but it’s stable. They have a policy mechanism that lets them change behaviours and give people a gradual migration path (which can include just ignoring the change and putting up with warnings for a really long time).
You can find a load of people that understand CMake and can hack on a CMake build system. You can find more people that understand Python, but far fewer that understand a custom set of abstractions built on top of Python for a single project. At least with Meson, the abstractions are shared across multiple projects, and someone else is responsible for handling differences with Python versions.
Well our Ninja generator is implemented in Python 2, which thanks to Python 3 is frozen in time forever :-P
But yeah I don’t recommend that solution widely. (It means people need to build Python 2 from source first, but we need that anyway for other reasons, similar to PyPy’s RPython)
I agree CMake is more a well-trodden path so I’m not really arguing against it.
But I will say that the Ninja generator I wrote does all 5 of the things you mentioned, and it’s not much code (though again I wouldn’t say it’s trivial, and wouldn’t recommend every one write it)
The build variants support is especially great, e.g. dbg / asan / opt / -D BUMP_ALLOC / etc. This has long been my pet peeve with open source build systems
I’ll also say that I think the whole point of a build system should be to help you write a correct parallel and incremental build. If you don’t care about those things, then you would just write a serial from-scratch shell script (and that’s actually what we started with, before it became too slow)
Make/autotools does essentially nothing about that problem – i.e. most build systems are full of bugs
It’s not clear to me that CMake is any different, fundamentally.
Ninja doesn’t solve it either because the answer is OS-specific (controlling the file system inputs to a process), but elsewhere in the thread I pointed to the Bazel technique I’d like to adapt, e.g. for Linux
It’s not clear to me that CMake is any different, fundamentally.
CMake has a serial stage, but the output (at least for Ninja) tracks all dependencies, though sometimes via the kinds of hacks that make-based build systems often use of touching a file to indicate that something is up to date. It also has some support for a copy-if-different operation, which can be used to avoid build steps if something hasn’t changed. LLVM uses this for a bunch of the TableGen files (if the input has formatting changes, you don’t want to rebuild everything).
Since LLVM moved from autotools to CMake, I have not had any problems with incremental builds (well, except ‘I touched this header file and now half of LLVM wants to be recompiled’ but that’s a C++ problem not a CMake problem, no incorrect builds) and happily saturated 32-core machines with parallel builds. Someone posted here a few years ago with a clang wrapper that ran as an AWS Lambda and was able to run all of the compile steps in parallel.
Sorry, typo. The following sentence was about Meson, my brain wasn’t working this morning and so typed Meson when I meant Bazel in the previous one. I’ve now fixed my comment, for anyone confused by what @sams is talking about.
Now you got me thinking about the configuration step. It’s a critical, yet underthought part of a build system beyond generation. Not helping people think about it is that autotools was the state of the art for years…
True, but only a machine has to read that and no human has to maintain it. If the time Ninja spends executing is under triple digit milliseconds, I don’t really care how long the ninja file is.
As other commenters point out, we are missing a build system adapted to FOSS needs.
Meson is the best we have going this direction, but it is still painful, especially for multi stage bootstrapping processes.
Hadrian with Shake is the other one but. Well. Haskell.
Nix tried but ended up elsewhere. Even if there is a lot to learn from it.
Bazell and Buck2 both go in that direction but from the other side of the spectrum. They target complex build systems that can have whole remote build farms and in house engineers.
What we need in FOSS is something a bit simpler than that, that can be maintained in a few hours every year, that can work cross platform, output to ninja or equivalent locally as a runner, not depend on Java (python is ok but urgh), etc. If possible something better to write than Starlark, which i still barely can understand. And need to be built from start for bootstrapping processes.
We have a real discrepancy between needs and what our tools provide and it is a real limiter for FOSS projects, eating a toooon of maintainer time.
As a bonus, if it could define API or contract with programming language specific tools, both package managers and builders, so that we could interoperate instead of reinventing the wheel…
There are pieces here and there, ideas going around, but noone with time, knowledge and funding.
I saw the needs first hand trying to document and clean up the Erlang/OTP build system. This is a massive drag on FOSS maintainers time, which have impact on everything else. Every hour spent fixing broken and FUBAR build systems are an hour not spent on fixing bugs or features. And FOSS maintainers have a really low budget.
I agree, and I view this as basically an OS problem – Windows, specifically.
For example I imagine Erlang/OTP does build on Windows (?) Or at the very least, all of Python / node.js / Ruby / Clang / Chrome / Firefox / etc. do.
CMake is not great, but it’s popular because it solves the Windows problem. It can be bootstrapped with just a C++ compiler, and it has libraries of rules for Windows tools.
The OS abstractions and the compiler are very different on Windows – no fork() and the compiler takes /I flags not -I flags, if I recall correctly.
Also GCC has the gcc -M hack to export dependencies to GNU Make – in the form of Makefile fragments that are essentially #included. And Clang copied this.
But what do you do on Windows? You don’t have that collaboration between the compiler and the build system.
Ninja solves those nasty Windows problems. It has both a gcc -M parser build in (yes it parses Makefile fragments), and it has a mechanism for msvc deps as well. It knows how to collaborate with the compiler.
So basically I think any solution that has the properties you want must solve the Windows problem (otherwise people won’t use it – they will still need some other solution for Windows.)
And Ninja basically did that hard work (because Chrome builds on Windows).
I think Meson is based on Ninja as well.
So yeah any such solution will save a lot of effort if built on Ninja (ideally, contributing any enhanecments upstream). Even so, the problem is still not simple – if someone is thinking, “I will just write a very simple tool to do cross-platform builds”, then they’ll be in for a rude awakening :) Even if they use Ninja, but especially if they don’t!
Just to emphasize the larger point, Guido van Rossum once told me that almost all CPython committers use Linux, but most CPython users use Windows (which seems pretty natural).
So I would say FOSS is Unix- and Linux-based, but Windows has a disproportionate influence on what FOSS gets used.
And Microsoft has been pretty sloppy about their toolchain IMO. “The day I gave up on Windows dev” around 2006-2007 actually has to do with trying to contribute a CPython patch that affected Windows (a thing that eventually led to the zipapp module).
The “giving up” involved me coming into work on a Saturday to install a Visual Studio DVD on a Windows box, which took hours, and a missing DLL …
And there was basically one “Windows build guy” on the CPython team at that point.
It is extremely ironic you say this, since Julia builds with a GNU makefile using mingw on Windows, and ships a unix userland emulator.
So does perl, by the way – and Raku is built on perl, which would matter if people actually used Raku :P
Using GNU Makefiles that require running on unix, but shipping a Unix userland on Windows, is NOT “a build system that works on Windows and is well adapted to Windows”. If it was, then virtually all linux-only software would be first class Windows software too – just install the msys2 package for it. Or… use WSL.
OK well I was technically correct – they don’t use a build system that doesn’t work on Windows :)
But yeah there is a qualitative difference between using GNU on windows and using CMake/ninja.
I’m a little surprised about Julia, since I thought there were more scientific computing users on Windows, but that’s besides the point.
I’ve never built Julia, but I have built CPython, Clang, Firefox, and Chrome, and I stand by what I said – these projects will always want a build system that works well on Windows
These days that seems to be either CMake or perhaps roll your own.
I think 10 years ago you “had to” use MSVC for a variety of reasons, but maybe that’s changed with Clang and GCC getting better on Windows
Nope, erlang/OTP does not build on windows. Being able to do it would be nice but is not a target.
And tbf. No. Windows is really not the root of our problems. Most of our problems come from the pov of build system being declarative, no understanding of packages, no understanding of multi steps builds, interactions between build and tests, the mess that are compilers options, etc etc.
We do not touch windows except for cross compilation and ninja and tool built on top of it barely would help us here.
There is a reason i point at Hadrian. Afaict, they are the only one that built a build system based on actual needs of multi language runtime big foss projects.
Cmake was evaluated for a rewrite and… It would basically solve none of our problems with out current mess of autotools and make.
OK yeah it does seem like Erlang is a special case in being Unix-only.
I don’t have any recommendations, besides saying I’m also maintaining a Unix-only language (a Unix shell), and
my solution was “build my own on top of Ninja”. It wasn’t trivial, but it was basically “done” over a year ago, and multiple contributors use it without problems. (There are other problems with our build, not related to Ninja)
But I still stand by what I said – if there were to be a big effort for a better F/OSS build system, it would see very little use and thus little contribution if it didn’t work on Windows.
But I don’t think such a big effort will happen. I think these things are built slowly and incrementally, and take 10 or more years – CMake was apparently first released in 2000, but I didn’t hear about it until maybe 2010. And I think Clang switched to it around that time.
So if there’s any “CMake successor” it probably already exists. Meson does look interesting if it can be re-implemented as Muon in C99. Ninja also has a C99 re-implementation apparently (original being C++)
I’ve never used it, but I would be interested in a report :)
Looks like it’s POSIX-only and doesn’t implement the msvc stuff I talked about. But for Oils that would be fine since we don’t build on Windows (I would like to someday, but there are bigger problems than the build system)
It works fairly well as a drop-in replacement for ninja in many workflows. I’m happy enough to use it, for example (meson will look for both ninja and samu binaries).
It has some advantages over ninja, such as supporting SAMUFLAGS=-j8. It also has some downsides, such as support for ninja “dyndeps” (their syntax term for handling the makedepend form of augmented dependency ordering discovery) is simply not yet implemented. This basically only affects Fortran, and in theory C++20 modules except that tooling for that is still… sparse.
(The author of samu is also willing to look at patches implementing support for GNU make jobserver/client support, something that ninja has proven oddly resistant to for a while – I’m not sure why…)
I mean. Haskell also has the same problems, Python too, PHP, etc etc.
The problem here is not really the Windows part. The problem is that all these build system work ok for a few years. But we are now nearly 30 years into them. The Haskell build system was rewritten at least 5 times, and started like yours. At this point we have enough data and case studies that we can anticipate the problems and we know what early projects need in order to increase adoption.
Hm I’m not sure what what “same problems” you’re referring to
I generally agree with the thrust of your original post – there’s a hole and it should be possible to do better – but I view this part as a bit contradictory:
we are missing a build system adapted to FOSS needs.
Meson … Hadrian … Nix … Bazel … Buck2 …
What we need in FOSS is something a bit simpler than that, that can be maintained in a few hours every year,
I think you want something that’s simple to use, which we all do, but that conversely means designing and implementing the build system itself is going to be harder.
I think any “real solution” for FOSS is actually MORE complex than Bazel/Buck2, because they assume a more heterogeneous monorepo, and thus have an EASIER problem.
So as someone who’s worked on builds for a long time too – it is a mess – but that’s also because it’s inherently a hard problem.
I’d certainly read a blog post about what you think the requirements are. (I think Erlang/OTP is probably a very good use case, but not completely represenative)
Also, I hate to say it, but there are only limited areas where FOSS “innovates”. The model is best at parallel labor where people don’t need to coordinate a new design.
That is, a “clean” solution is probably more likely to come as some adaption of the Bazel/Buck2 lineage than as something completely new. I don’t use either of them because I’m not interested in big monolithic software anymore (I prefer FOSS), but they’re certainly principled and have a theoretical basis
I think “easy to use” is fairly well correlated with being principled – CMake being fairly unprincipled IMO
Also it’s fairly well correlated with being small/simple, and that’s why I think Ninja is a good component of future solutions (although it’s definitely not perfect)
Which FLOSS needs did Nix fail to address? When it comes to availability and reproducibility, I think that nixpkgs is currently leading the pack. I like to share this graph to emphasize the degree to which Nix has successfully solved the typical packaging problems that a distro will face.
In terms of bootstrapping, GNU Guix appears to currently have the best story, although I acknowledge that bootstrapping is still an unsolved problem in general.
I think we are talking of different bootstrapping.
For what nix fails to do: actually output usable and installable results that do not depend on Nix, manage dependencies not from itself, not depend on bash scripts for all the actions themselves, and dozens other things.
That graph is slightly misleading I think. There’s loads of packages in nix which are extremely specialised, don’t get any development anymore and are maybe used by or two people. They inflate that count by quite a lot (including the freshness base level). I have a suspicion that both nix and aur (same issue) would end up very close to the rest of the pack if you filtered out packages with less than 100 downloads.
Also, I think there is one important item missing from your list: a FOSS solution needs to be “multi-repo first”. Most of the build systems you have listed (CMake, Bazel, etc) are monorepo-oriented.
Nearly all foss projects i know are basically mono repo. Other repos are seen as dependencies.
That’s exactly what I mean: most FOSS projects don’t bundle their dependencies in their own repository. Most build systems that you have listed expect you to (or at least your life will be a lot easier if you do). Most FOSS projects that have external dependencies use the distribution package manager to satisfy them.
Biggest “yeah nope” i see in build2 page “C/C++”. This already tells me it is not going to address most of my problems.
Ideally you want a single build system that can be used for every FOSS project, right? In that case you need it to be able to build C/C++, which is a large base of every single Linux distribution. I am talking about things like glibc, ICU, OpenSSL, Curl, GCC/LLVM, Qt, etc., these are all C or C++. There is also the Linux kernel itself.
I think it’s quite impractical to imagine a world where a single build system can be used for every FOSS project, because it assumes people have the same needs – which they don’t.
A good build system for glibc will look nothing like a good build system for Qt but fortunately this does not matter since glibc ships with your OS, it is not a dependency at all.
A good build system for OpenSSL is one that builds okay on OpenVMS, NonStop, tru64…
Frankly, you do not or should not care about this in the slightest. The distribution package manager is one way to satisfy these dependencies without worrying about external build systems. vcpkg/conan is another approach. A third approach is to wrap their build system in yours (e.g. calling ./configure && make && make install DESTDIR=$PWD/staging_root/).
Meson and CMake offer “External Project” drivers to automate the third approach for you, which allows you to use dependencies with minimal fuss while still actually being able to reliably consume the external project’s build system as intended by its developers.
(Both can also handle things a lot more nicely/integrated if you match up the build systems.)
A good build system for glibc will look nothing like a good build system for Qt
Why? Is there some fundamental constraint that a build system with a general-enough build model cannot be a good fit for both Qt and glibc?
which allows you to use dependencies with minimal fuss while still actually being able to reliably
I recon you’ve never tried to build OpenSSL on Windows.
To me your whole message sounds like it’s written by someone who’s only had experience working on a “sane” Linux distribution with a functioning package manager and who never actually had to develop any of the harrier bits of the stack like glibc, OpenSSL. While I personally haven’t touched these specific projects much, I’ve had the pleasure of dealing with GCC’s build system and the suggestion that “[I] do not or should not care about this in the slightest” definitely rubs the wrong way.
Why? Is there some fundamental constraint that a build system with a general-enough build model cannot be a good fit for both Qt and glibc?
If you want a build system with a “general-enough” build model, you’ve essentially locked yourself into a corner with autotools. And that’s constrained by not working very well on Windows if you don’t buy into the notion of using a mingw toolchain (cygwin, msys2).
To me your whole message sounds like it’s written by someone who’s only had experience working on a “sane” Linux distribution with a functioning package manager
This describes all linuxes other than slackware, but excludes Windows, naturally.
However, I will still call building OpenSSL on Windows after installing various arcane dependencies, a “minimal fuss” case compared to literally all the other methods that don’t include the OpenSSL project’s official support. (In this, I include https://github.com/mesonbuild/wrapdb/tree/master/subprojects/packagefiles/openssl which apparently works for at least one group’s use case, and all the more power to them, but still absolutely terrifies me even without taking into account the very big TODOs for Windows support.)
and who never actually had to develop any of the harrier bits of the stack like glibc, OpenSSL. While I personally haven’t touched these specific projects much, I’ve had the pleasure of dealing with GCC’s build system and the suggestion that “[I] do not or should not care about this in the slightest” definitely rubs the wrong way.
Don’t I wish it. (Ironically, your “ideally we should have one build system for all of FOSS” made me wonder something very similar…)
My experience tells me that you absolutely do not want to care about their build system in the slightest, only run it and assume it works and report a bug to them if it does not work…
… but again, if you are building either glibc or gcc as a vendored part of your codebase, you’re “doing it wrong” in too many ways to count. One of those ways is, again, you do not and should not want to know about their build systems, work on them, try to port them to this-weeks-fancy-new-buildsystem, or do anything with them whatsoever other than rely on:
your operating system layer to provide glibc, because glibc is absolutely part of your operating system and you’re not supposed to go around touching it or building private copies of it unless you want to get yourself into a lot of trouble. Or unless you’re building upgradeable components of the operating system itself, working within the constraints of your package manager and/or ports system.
your toolchain vendor (may be a linux distro, may be something like choco install rtools on Windows) for gcc or whichever other compiler you need. Especially since you cannot build a compiler unless you already have a compiler, and I’m pretty sure you’re not trying to solve the Bootstrappable Builds problem with build2 here.
Re dependency: no usually we do not use the distribution packaging system. We do this because the distribution bully us into it.
And we do vendor quite a lot by the sad reality of make.
For C/C++, yes we need it too. But something built with a focus on these will create a lot of problems for systems that mix it with multiple other things. We probably have half a dozen languages and built tools in the Erlang/OTP build and we would love to use more…
And we do vendor quite a lot by the sad reality of make.
Ok, I guess we mean different things by “FOSS”. I am referring to what you would normally find in a package repository of a distribution like Debian (and where you won’t be allowed to use your bundled dependencies in most cases).
But something built with a focus on these will create a lot of problems for systems that mix it with multiple other things.
Is there some fundamental reason that it must be so? I mean, I can see how by looking at CMake one can conclude that, but I don’t believe it’s inescapable. In fact, I would say if a build system is capable of building non-trivial C/C++ projects like, say, Qt, without hacks, then it’s a good sign that its build model is general enough to handle other languages.
We probably have half a dozen languages and built tools in the Erlang/OTP build and we would love to use more…
Well, build2 has support for C/C++, Bash, and Rust (preliminary/incomplete). All of them are implemented as build system modules so adding support for other languages should be a matter of just doing the legwork (you could also just use the equivalent of make pattern rules for some ad hoc/basic support).
You don’t need Starlark or any execution termination guarantees for 99% of projects. Starlark is used for parallel build config evaluation, and for us that takes about 100ms, so no real need to parallelize it.
Ninja is extremely parallel for the actual build. (FWIW I actually compared the Chrome/Android builds and Firefox like 9 years ago – Chrome/Android would use up 16 or 32 cores perfectly, but Firefox wouldn’t.)
It builds the main binary, 15-20 test binaries, and 20 binaries to test the mycpp translator, and does so with GCC and Clang, ASAN/UBSAN/dbg/opt, 64- or 32-bit, and custom builds for measuring the GC and allocators, like comparing GC vs bump alloc, etc.
It works well. The only thing is that sandboxing is extremely OS-dependent so we don’t have that yet. Bazel has a few different strategies on Linux, OS X, and maybe even Windows. It should be pretty easy to adapt those though, since they are fork-exec type wrappers.
I will say that it’s a small amount of code to write such a wrapper, but also not quite “trivial”. It’s a nice little exercise if you want to think about build graphs though! And it definitely got me more familiar with the toolchain.
I should note that this is only for devs and source code gen. The actual C++ binary can be built with just a shell and a compiler – no Make or Ninja necessary.
So our meta-build system lowers to either parallel/incremental Ninja or plain shell, which is a nice technique!
Ninja is enough… as long as you also have Python lying around.
The big problem with meta-build systems (or with using only Ninja) in C/C++ projects are auto-generated headers. You either have to specify inter-header dependencies manualy or you have to generate them as an ad hoc pre-build step. Neither of these options scale well.
At some point I wondered if we could replicate enough of Bazel or Buck syntax and semantics to be able to build small projects with this kind of hackery.
Working closely with Bazel, this is what I hear a lot nowadays.
Many bigger companies are moving from their in-house solution to Bazel, partly to reduce the complexity and in-house maintenance burden, and partly for performance gain. The Chromium project has started using “reclient”, a ninja wrapper that speaks Bazel’s remote execution API as the official supported build tool. So things are converging.
Bazel depends on both a working Python environment and a working Java environment, which is quite a painful set of dependencies for a build tool. One of the things I like about CMake and xmake is that they have a very small set of dependencies and so are easy to bootstrap. Xmake gives a rich Lua scripting environment, but embeds Lua (it’s tiny) and so doesn’t depend on any external scripting environment.
I couldn’t find the link this morning, but I saw a while ago that someone was doing a Bazel rewrite in Rust. Having a single statically-linked binary to deploy would be a big win (I don’t really care whether it’s Rust, C++, Go, or Lua with C bindings, I just want it to be small and fast).
Python dependencies is optional nowadays unless you want to build python stuffs with Bazel. The JRE is bundled with Bazel releases inside the executable package, with option to bring your own, so i don’t see why it would be a problem.
The JVM also comes with some of the most advanced runtime tooling in the industry that let you do very low level tunning and tracing at a maturity level that don’t exist in other ecosystems. For free! So I am not convinced that a “rewrite in rust” gona solve the problems enterprises are facing today.
The JRE is bundled with Bazel releases inside the executable package, with option to bring your own, so i don’t see why it would be a problem.
Right, the package… for which OS / architecture? Most of the code I write runs on any vaguely POSIXy platform, including Linux, macOS, *BSD, and Haiku, and runs on multiple architectures. Bazel is packaged officially for under half of those. OpenJDK runs on a subset of them. FreeBSD/AArch64 is only slightly off the mainstream, yet I can’t build anything that uses Bazel there because OpenJDK is broken. If you get any further away from the big three operating systems and x86-64 then you’re likely to hit problems.
The JVM also comes with some of the most advanced runtime tooling in the industry that let you do very low level tunning and tracing at a maturity level that don’t exist in other ecosystems
It also comes with startup times that are on a par with total incremental build times with Ninja. If you’re building something to build ten thousand source files in a distributed build environment, an OpenJDK dependency might make sense. I have worked on at least two orders of magnitude more projects that don’t need that level of complexity than I have on ones that do.
If you prefer X tool because they are easier to bootstrap vs Bazel, then you are just not the intended audience for the tool.
Folks often came to Bazel (or Buck) not for the simplicity and ease of use. They came after they have out grew the possibility of running X and Y build tool and need something that could helps solve the complexity of their millions LOC code base and thousands CI jobs pipelines. It’s a complex solution for a complex problems.
The good thing is: the ecosystem has been growing fast. That helps making difficult things easier.
Why would having a JDK available be painful? Dependency management is required either way. The proper solution is freezing these dependencies, for which nix (and guix) are the correct tools.
Part of the problem with Bazel is what you mention – it has an extremely complex bootstrap process, which may work if you’re on a “supported” platform, but is not particularly hackable to new situations
The second problem is that Bazel is meant for a monorepo.
That makes it less suitable for open source, which has diversity. Although some open source projects are a little monorepo-like, maybe Clang/LLVM and so forth. I think they control all their dependencies, simply because they have very few.
But there are definitely open source projects that don’t try to control their dependencies (vendor them, etc.)
i expect bazel to win for a while, in a worse-is-better sort of way
it’s current state is hell (at least partially because of the python and java dependencies), and i don’t see it changing any time soon. but it solves enough real problems that companies are switching to it and just facetanking the issues with it
i would love to see a bazel that can bootstrap its python/java dependencies, if not remove them entirely
At some point I wondered if we could replicate enough of Bazel or Buck syntax and semantics to be able to build small projects with this kind of hackery.
I just mean having parallel builds, nice output, and a small library of functions like cc_binary.
I think we talked about this like a year ago, but Oils has exactly that … there is a cc_binary() here
It’s attached to a Rules object ru , since I wanted explicit Python not global vars, but it behaves just like Bazel and so forth.
There is a loop, because reusing build config for 2 similar binaries in a loop is useful! (This kind of reuse/abstraction is extremely difficult or impossible with Make, especially once you start using its more advanced features, which do NOT compose like a real language.)
We also have asdl_library() which both Python and C++ binaries can depend on, to generate Python and C++ code. This is is exactly analogous to proto_library(), which is arguably the main feature of Bazel !
The problem with the idea of using a simple build system is that it imposes a big change later if the project grows. At some point you will need:
You can build all of these on top of Make (bmake or GNU Make, not POSIX make) or Ninja, but you end up with a lot of complexity.
At some point, you realise that your build system is so bespoke that it’s completely incomprehensible to any outsider (see GNUstep and FreeBSD for examples). Now you have a choice of throwing away everything and starting again, or keeping adding complexity to a system with a rapidly shrinking bus factor.
A simple CMake file that generates the equivalent ninja file is simpler that this Ninja file is shorter than the Python program and CMake is a smaller dependency. Bazel looks simpler, but I can’t build any Bazel projects on FreeBSD/AArch64 at the moment because it depends on Java and OpenJDK appears to be broken. Meson looks to have a similar set of dependencies to this Python script.
If you start with a widely used build system, you will find it easy to find people that can modify your build system. If you do something bespoke, it will become a bottleneck for your project at some point in the future. The easier it looks to use. The more magic it will depend on until you discover that there is no one that understands the build system still involved and making a small change is impossible.
In general I agree with your comments. Configuration is the real killer non-feature of “simple Makefiles” (or “simple ninja files” too, really). So many things you just can’t do without a proper system probing and conditional build generator.
Make actually has a small advantage over ninja here, because with make you can use GNU-isms and get a complex and fragile build system that evaluates repeatedly every time you run make, uses environment variables for conditional build settings, and kinda works a bunch of the time. For ninja this is simply impossible: it shuns any form of pattern rules, $(shell …) based rule redefinition, if/else’d rules… the idea behind ninja was that all that stuff is a burden and a distraction from being a fast interdependency-ordered command runner and people should “just use a generator program like autotools has historically been for Makefiles”.
By the way: on the meson topic, meson is internally written in python, as is the python script, but it does have one less dependency: meson rigidly and pedantically limits itself to the python standard library to avoid dragging in a dependency on either additional python modules or the complex distribution method of “somehow get a python PyPI toolchain going with setuptools and pip or some kind of pep 517 build frontend, handling cyclic python dependencies, etc etc etc”.
Also, meson is intentionally designed to not be turing complete (it uses a custom DSL and does not permit using python or even adding python plugins), there’s a FAQ entry that notes a couple of reasons why we made that choice. One of the reasons is because it makes it impractical to ever port over to another language “should performance become a limiting factor”. And notably, this is a huge issue for autotools.
This has already paid off, because the community has taken the opportunity to write a c99 reimplementation of meson: https://sr.ht/~lattis/muon
We think that’s fantastic, both that people were able to scratch a personal itch like this (using and running meson builds on systems that don’t have python installed) and that we sometimes discover meson bugs or documentation ambiguities as a result of seeing how meson and muon occasionally behave differently.
Yeah that’s definitely nice … I haven’t used Meson, it took me awhile to find the language docs
https://mesonbuild.com/Syntax.html
At first I thought it was literally Python, but it looks like it’s a language written in Python, and then now as you say it’s ported to C.
C is definitely better for build systems because it’s easier to bootstrap on a new machine. (C is also horrible for string processing but that isn’t a dealbreaker)
As long as you use/make a reasonable string buffer type it is perfectly fine. String processing doesn’t benefit much from generics where C lacks some expressivity.
I needed to build something that uses Bazel a while ago. I didn’t have access to a pre-built version of Bazel, though I did have a functioning copy of OpenJDK installed. I was going to build Bazel myself, so I downloaded the provided bootstrapping archive. Inside was a shell script you were supposed to run, which I did. What did it do? Download an older version of Bazel (I think) and a copy of OpenJDK that was incompatible with my OS. Upon seeing this, I decided to never ever touch anything Bazel ever.
I broadly agree though I would raise that there is at least one widely used build systems which is so difficult that it negates this advantage. Specifically I’m thinking of autotools.
Python isn’t a great dependency, but I think it’s a lot more familiar and understandable than CMake.
I thought you were on the record saying CMake is ridiculously inscrutable.
I think CMake is OK when you’re doing “standard” well-trodden stuff, but as it gets more advanced – which happens as project grows – I think it gets more hairy.
Python is maybe more awkward since it’s not purpose-built, but it “scales” better and you can easily debug it.
One problem that was a bit tricky when writing my own Ninja generator is Python code generators, but it ended up working well.
I believe most people don’t even attempt this in GNU Make, but I’m not sure about CMake. (The more conventional thing seems to be to check in generated code)
That is, suppose the code gen tool is
asdl_main.py, and it depends onutil.py, and it producesfoo_asdl.{h,cc}.Then if
util.pyis changed, it has to invalidatefoo_asdland the binaries that depend on it (re-generated and re-compile)With C++ code generators, the generator is one binary file, and its dependencies are known. With Python it’s less clear.
Bazel does this well, and my Ninja generator does too, but it required some custom plumbing / shell stubs.
Familiar is subjective. Understandable is debatable. About ten years ago, I fixed a bug in some Python code that was due to their not understanding what an else clause on a while loop meant (my hatred of Python largely comes from a decade of having to write it only because packaged, release versions of Python things failed to launch because my computer wasn’t identical to that of the authors). Over the next few years, I asked people that told me that they liked Python what this meant. It was three years before someone got the answer right. That’s a fairly simple control-flow construct. Once you get into decorators, you discover that even C++ doesn’t have quite the extent of totally distinct languages that are subsets.
The real problem is stability. Maybe if you stick to the standard library and hope nothing like Python 3 will ever happen again, there’s a subset of the Python ecosystem for which you can write code and expect it to still work next year. I’d like to see an existence proof though.
The CMake language is awful and the abstractions that it provides are not great, but it’s stable. They have a policy mechanism that lets them change behaviours and give people a gradual migration path (which can include just ignoring the change and putting up with warnings for a really long time).
You can find a load of people that understand CMake and can hack on a CMake build system. You can find more people that understand Python, but far fewer that understand a custom set of abstractions built on top of Python for a single project. At least with Meson, the abstractions are shared across multiple projects, and someone else is responsible for handling differences with Python versions.
Well our Ninja generator is implemented in Python 2, which thanks to Python 3 is frozen in time forever :-P
But yeah I don’t recommend that solution widely. (It means people need to build Python 2 from source first, but we need that anyway for other reasons, similar to PyPy’s RPython)
I agree CMake is more a well-trodden path so I’m not really arguing against it.
But I will say that the Ninja generator I wrote does all 5 of the things you mentioned, and it’s not much code (though again I wouldn’t say it’s trivial, and wouldn’t recommend every one write it)
The build variants support is especially great, e.g. dbg / asan / opt / -D BUMP_ALLOC / etc. This has long been my pet peeve with open source build systems
I’ll also say that I think the whole point of a build system should be to help you write a correct parallel and incremental build. If you don’t care about those things, then you would just write a serial from-scratch shell script (and that’s actually what we started with, before it became too slow)
Make/autotools does essentially nothing about that problem – i.e. most build systems are full of bugs
It’s not clear to me that CMake is any different, fundamentally.
Ninja doesn’t solve it either because the answer is OS-specific (controlling the file system inputs to a process), but elsewhere in the thread I pointed to the Bazel technique I’d like to adapt, e.g. for Linux
CMake has a serial stage, but the output (at least for Ninja) tracks all dependencies, though sometimes via the kinds of hacks that make-based build systems often use of touching a file to indicate that something is up to date. It also has some support for a copy-if-different operation, which can be used to avoid build steps if something hasn’t changed. LLVM uses this for a bunch of the TableGen files (if the input has formatting changes, you don’t want to rebuild everything).
Since LLVM moved from autotools to CMake, I have not had any problems with incremental builds (well, except ‘I touched this header file and now half of LLVM wants to be recompiled’ but that’s a C++ problem not a CMake problem, no incorrect builds) and happily saturated 32-core machines with parallel builds. Someone posted here a few years ago with a clang wrapper that ran as an AWS Lambda and was able to run all of the compile steps in parallel.
Meson shouldn’t need Java at all. Could you elaborate? @eschwartz would be interested to know more.
Sorry, typo. The following sentence was about Meson, my brain wasn’t working this morning and so typed Meson when I meant Bazel in the previous one. I’ve now fixed my comment, for anyone confused by what @sams is talking about.
No worries, thank you! If there is a problem, do feel free to let me or Eli know though.
Now you got me thinking about the configuration step. It’s a critical, yet underthought part of a build system beyond generation. Not helping people think about it is that autotools was the state of the art for years…
fwiw, the generated Ninja from CMake is not short. A two-line executable in CMake leads to ~200 lines of Ninja.
True, but only a machine has to read that and no human has to maintain it. If the time Ninja spends executing is under triple digit milliseconds, I don’t really care how long the ninja file is.
As other commenters point out, we are missing a build system adapted to FOSS needs.
Meson is the best we have going this direction, but it is still painful, especially for multi stage bootstrapping processes.
Hadrian with Shake is the other one but. Well. Haskell.
Nix tried but ended up elsewhere. Even if there is a lot to learn from it.
Bazell and Buck2 both go in that direction but from the other side of the spectrum. They target complex build systems that can have whole remote build farms and in house engineers.
What we need in FOSS is something a bit simpler than that, that can be maintained in a few hours every year, that can work cross platform, output to ninja or equivalent locally as a runner, not depend on Java (python is ok but urgh), etc. If possible something better to write than Starlark, which i still barely can understand. And need to be built from start for bootstrapping processes.
We have a real discrepancy between needs and what our tools provide and it is a real limiter for FOSS projects, eating a toooon of maintainer time.
As a bonus, if it could define API or contract with programming language specific tools, both package managers and builders, so that we could interoperate instead of reinventing the wheel…
There are pieces here and there, ideas going around, but noone with time, knowledge and funding.
I saw the needs first hand trying to document and clean up the Erlang/OTP build system. This is a massive drag on FOSS maintainers time, which have impact on everything else. Every hour spent fixing broken and FUBAR build systems are an hour not spent on fixing bugs or features. And FOSS maintainers have a really low budget.
There is an opportunity for high impact there.
I agree, and I view this as basically an OS problem – Windows, specifically.
For example I imagine Erlang/OTP does build on Windows (?) Or at the very least, all of Python / node.js / Ruby / Clang / Chrome / Firefox / etc. do.
CMake is not great, but it’s popular because it solves the Windows problem. It can be bootstrapped with just a C++ compiler, and it has libraries of rules for Windows tools.
The OS abstractions and the compiler are very different on Windows – no fork() and the compiler takes
/Iflags not-Iflags, if I recall correctly.Also GCC has the
gcc -Mhack to export dependencies to GNU Make – in the form of Makefile fragments that are essentially#included. And Clang copied this.But what do you do on Windows? You don’t have that collaboration between the compiler and the build system.
Ninja solves those nasty Windows problems. It has both a
gcc -Mparser build in (yes it parses Makefile fragments), and it has a mechanism for msvc deps as well. It knows how to collaborate with the compiler.https://ninja-build.org/manual.html#_deps
So basically I think any solution that has the properties you want must solve the Windows problem (otherwise people won’t use it – they will still need some other solution for Windows.)
And Ninja basically did that hard work (because Chrome builds on Windows).
I think Meson is based on Ninja as well.
So yeah any such solution will save a lot of effort if built on Ninja (ideally, contributing any enhanecments upstream). Even so, the problem is still not simple – if someone is thinking, “I will just write a very simple tool to do cross-platform builds”, then they’ll be in for a rude awakening :) Even if they use Ninja, but especially if they don’t!
Just to emphasize the larger point, Guido van Rossum once told me that almost all CPython committers use Linux, but most CPython users use Windows (which seems pretty natural).
So I would say FOSS is Unix- and Linux-based, but Windows has a disproportionate influence on what FOSS gets used.
Reminds me of “dictatorship of the minority” – https://medium.com/incerto/the-most-intolerant-wins-the-dictatorship-of-the-small-minority-3f1f83ce4e15 (although it depends which POV you take – my point is that people who don’t care about Windows are very affected by Windows)
And Microsoft has been pretty sloppy about their toolchain IMO. “The day I gave up on Windows dev” around 2006-2007 actually has to do with trying to contribute a CPython patch that affected Windows (a thing that eventually led to the zipapp module).
The “giving up” involved me coming into work on a Saturday to install a Visual Studio DVD on a Windows box, which took hours, and a missing DLL …
And there was basically one “Windows build guy” on the CPython team at that point.
Why should a build system for FLOSS needs be well-adapted to Windows?
Well I gave some examples … CPython, node.js, Ruby, Clang, Firefox, etc. Also PHP, Raku, Julia, etc.
I’m 100% sure that none of those projects will ever use a build system that doesn’t work on Windows.
The lower level projects tend to influence the high levels, e.g. how git started at the kernel, and influenced the whole world
And there’s a similar logic for new projects, not just existing ones
It is extremely ironic you say this, since Julia builds with a GNU makefile using mingw on Windows, and ships a unix userland emulator.
So does perl, by the way – and Raku is built on perl, which would matter if people actually used Raku :P
Using GNU Makefiles that require running on unix, but shipping a Unix userland on Windows, is NOT “a build system that works on Windows and is well adapted to Windows”. If it was, then virtually all linux-only software would be first class Windows software too – just install the msys2 package for it. Or… use WSL.
OK well I was technically correct – they don’t use a build system that doesn’t work on Windows :)
But yeah there is a qualitative difference between using GNU on windows and using CMake/ninja.
I’m a little surprised about Julia, since I thought there were more scientific computing users on Windows, but that’s besides the point.
I’ve never built Julia, but I have built CPython, Clang, Firefox, and Chrome, and I stand by what I said – these projects will always want a build system that works well on Windows
These days that seems to be either CMake or perhaps roll your own.
I think 10 years ago you “had to” use MSVC for a variety of reasons, but maybe that’s changed with Clang and GCC getting better on Windows
Nope, erlang/OTP does not build on windows. Being able to do it would be nice but is not a target.
And tbf. No. Windows is really not the root of our problems. Most of our problems come from the pov of build system being declarative, no understanding of packages, no understanding of multi steps builds, interactions between build and tests, the mess that are compilers options, etc etc.
We do not touch windows except for cross compilation and ninja and tool built on top of it barely would help us here.
There is a reason i point at Hadrian. Afaict, they are the only one that built a build system based on actual needs of multi language runtime big foss projects.
Cmake was evaluated for a rewrite and… It would basically solve none of our problems with out current mess of autotools and make.
OK yeah it does seem like Erlang is a special case in being Unix-only.
I don’t have any recommendations, besides saying I’m also maintaining a Unix-only language (a Unix shell), and my solution was “build my own on top of Ninja”. It wasn’t trivial, but it was basically “done” over a year ago, and multiple contributors use it without problems. (There are other problems with our build, not related to Ninja)
But I still stand by what I said – if there were to be a big effort for a better F/OSS build system, it would see very little use and thus little contribution if it didn’t work on Windows.
But I don’t think such a big effort will happen. I think these things are built slowly and incrementally, and take 10 or more years – CMake was apparently first released in 2000, but I didn’t hear about it until maybe 2010. And I think Clang switched to it around that time.
So if there’s any “CMake successor” it probably already exists. Meson does look interesting if it can be re-implemented as Muon in C99. Ninja also has a C99 re-implementation apparently (original being C++)
Do you have a link to the C99 ninja?
https://github.com/michaelforney/samurai
I’ve never used it, but I would be interested in a report :)
Looks like it’s POSIX-only and doesn’t implement the
msvcstuff I talked about. But for Oils that would be fine since we don’t build on Windows (I would like to someday, but there are bigger problems than the build system)It works fairly well as a drop-in replacement for ninja in many workflows. I’m happy enough to use it, for example (meson will look for both ninja and samu binaries).
It has some advantages over ninja, such as supporting
SAMUFLAGS=-j8. It also has some downsides, such as support for ninja “dyndeps” (their syntax term for handling themakedependform of augmented dependency ordering discovery) is simply not yet implemented. This basically only affects Fortran, and in theory C++20 modules except that tooling for that is still… sparse.(The author of samu is also willing to look at patches implementing support for GNU make jobserver/client support, something that ninja has proven oddly resistant to for a while – I’m not sure why…)
I mean. Haskell also has the same problems, Python too, PHP, etc etc.
The problem here is not really the Windows part. The problem is that all these build system work ok for a few years. But we are now nearly 30 years into them. The Haskell build system was rewritten at least 5 times, and started like yours. At this point we have enough data and case studies that we can anticipate the problems and we know what early projects need in order to increase adoption.
Hm I’m not sure what what “same problems” you’re referring to
I generally agree with the thrust of your original post – there’s a hole and it should be possible to do better – but I view this part as a bit contradictory:
I think you want something that’s simple to use, which we all do, but that conversely means designing and implementing the build system itself is going to be harder.
I think any “real solution” for FOSS is actually MORE complex than Bazel/Buck2, because they assume a more heterogeneous monorepo, and thus have an EASIER problem.
So as someone who’s worked on builds for a long time too – it is a mess – but that’s also because it’s inherently a hard problem.
I’d certainly read a blog post about what you think the requirements are. (I think Erlang/OTP is probably a very good use case, but not completely represenative)
Also, I hate to say it, but there are only limited areas where FOSS “innovates”. The model is best at parallel labor where people don’t need to coordinate a new design.
That is, a “clean” solution is probably more likely to come as some adaption of the Bazel/Buck2 lineage than as something completely new. I don’t use either of them because I’m not interested in big monolithic software anymore (I prefer FOSS), but they’re certainly principled and have a theoretical basis
I think “easy to use” is fairly well correlated with being principled – CMake being fairly unprincipled IMO
Also it’s fairly well correlated with being small/simple, and that’s why I think Ninja is a good component of future solutions (although it’s definitely not perfect)
Which FLOSS needs did Nix fail to address? When it comes to availability and reproducibility, I think that
nixpkgsis currently leading the pack. I like to share this graph to emphasize the degree to which Nix has successfully solved the typical packaging problems that a distro will face.In terms of bootstrapping, GNU Guix appears to currently have the best story, although I acknowledge that bootstrapping is still an unsolved problem in general.
I think we are talking of different bootstrapping.
For what nix fails to do: actually output usable and installable results that do not depend on Nix, manage dependencies not from itself, not depend on bash scripts for all the actions themselves, and dozens other things.
You can take a nix closure and shove that into a container image all by itself. Or a tarball.
I think that doesn’t meet your criterion for “installable result” but I’m not sure.
That graph is slightly misleading I think. There’s loads of packages in nix which are extremely specialised, don’t get any development anymore and are maybe used by or two people. They inflate that count by quite a lot (including the freshness base level). I have a suspicion that both nix and aur (same issue) would end up very close to the rest of the pack if you filtered out packages with less than 100 downloads.
Well, that’s essentially our goal with
build2.Also, I think there is one important item missing from your list: a FOSS solution needs to be “multi-repo first”. Most of the build systems you have listed (CMake, Bazel, etc) are monorepo-oriented.
Nearly all foss projects i know are basically mono repo. Other repos are seen as dependencies. That problem is not one i have encountered.
Biggest “yeah nope” i see in build2 page “C/C++”. This already tells me it is not going to address most of my problems.
That’s exactly what I mean: most FOSS projects don’t bundle their dependencies in their own repository. Most build systems that you have listed expect you to (or at least your life will be a lot easier if you do). Most FOSS projects that have external dependencies use the distribution package manager to satisfy them.
Ideally you want a single build system that can be used for every FOSS project, right? In that case you need it to be able to build C/C++, which is a large base of every single Linux distribution. I am talking about things like glibc, ICU, OpenSSL, Curl, GCC/LLVM, Qt, etc., these are all C or C++. There is also the Linux kernel itself.
I think it’s quite impractical to imagine a world where a single build system can be used for every FOSS project, because it assumes people have the same needs – which they don’t.
A good build system for glibc will look nothing like a good build system for Qt but fortunately this does not matter since glibc ships with your OS, it is not a dependency at all.
A good build system for OpenSSL is one that builds okay on OpenVMS, NonStop, tru64…
Frankly, you do not or should not care about this in the slightest. The distribution package manager is one way to satisfy these dependencies without worrying about external build systems. vcpkg/conan is another approach. A third approach is to wrap their build system in yours (e.g. calling ./configure && make && make install DESTDIR=$PWD/staging_root/).
Meson and CMake offer “External Project” drivers to automate the third approach for you, which allows you to use dependencies with minimal fuss while still actually being able to reliably consume the external project’s build system as intended by its developers.
(Both can also handle things a lot more nicely/integrated if you match up the build systems.)
Why? Is there some fundamental constraint that a build system with a general-enough build model cannot be a good fit for both Qt and glibc?
I recon you’ve never tried to build OpenSSL on Windows.
To me your whole message sounds like it’s written by someone who’s only had experience working on a “sane” Linux distribution with a functioning package manager and who never actually had to develop any of the harrier bits of the stack like glibc, OpenSSL. While I personally haven’t touched these specific projects much, I’ve had the pleasure of dealing with GCC’s build system and the suggestion that “[I] do not or should not care about this in the slightest” definitely rubs the wrong way.
If you want a build system with a “general-enough” build model, you’ve essentially locked yourself into a corner with autotools. And that’s constrained by not working very well on Windows if you don’t buy into the notion of using a mingw toolchain (cygwin, msys2).
This describes all linuxes other than slackware, but excludes Windows, naturally.
However, I will still call building OpenSSL on Windows after installing various arcane dependencies, a “minimal fuss” case compared to literally all the other methods that don’t include the OpenSSL project’s official support. (In this, I include https://github.com/mesonbuild/wrapdb/tree/master/subprojects/packagefiles/openssl which apparently works for at least one group’s use case, and all the more power to them, but still absolutely terrifies me even without taking into account the very big TODOs for Windows support.)
Don’t I wish it. (Ironically, your “ideally we should have one build system for all of FOSS” made me wonder something very similar…)
My experience tells me that you absolutely do not want to care about their build system in the slightest, only run it and assume it works and report a bug to them if it does not work…
… but again, if you are building either glibc or gcc as a vendored part of your codebase, you’re “doing it wrong” in too many ways to count. One of those ways is, again, you do not and should not want to know about their build systems, work on them, try to port them to this-weeks-fancy-new-buildsystem, or do anything with them whatsoever other than rely on:
your operating system layer to provide glibc, because glibc is absolutely part of your operating system and you’re not supposed to go around touching it or building private copies of it unless you want to get yourself into a lot of trouble. Or unless you’re building upgradeable components of the operating system itself, working within the constraints of your package manager and/or ports system.
your toolchain vendor (may be a linux distro, may be something like
choco install rtoolson Windows) for gcc or whichever other compiler you need. Especially since you cannot build a compiler unless you already have a compiler, and I’m pretty sure you’re not trying to solve the Bootstrappable Builds problem with build2 here.Re dependency: no usually we do not use the distribution packaging system. We do this because the distribution bully us into it.
And we do vendor quite a lot by the sad reality of make.
For C/C++, yes we need it too. But something built with a focus on these will create a lot of problems for systems that mix it with multiple other things. We probably have half a dozen languages and built tools in the Erlang/OTP build and we would love to use more…
Ok, I guess we mean different things by “FOSS”. I am referring to what you would normally find in a package repository of a distribution like Debian (and where you won’t be allowed to use your bundled dependencies in most cases).
Is there some fundamental reason that it must be so? I mean, I can see how by looking at CMake one can conclude that, but I don’t believe it’s inescapable. In fact, I would say if a build system is capable of building non-trivial C/C++ projects like, say, Qt, without hacks, then it’s a good sign that its build model is general enough to handle other languages.
Well,
build2has support for C/C++, Bash, and Rust (preliminary/incomplete). All of them are implemented as build system modules so adding support for other languages should be a matter of just doing the legwork (you could also just use the equivalent of make pattern rules for some ad hoc/basic support).FWIW Oils is using basically a little mini-Bazel in a few hundred lines of Python, on top of Ninja
https://www.oilshell.org/blog/2022/10/garbage-collector.html#declarative-ninja-mini-bazel
Example build config which may look familiar - https://github.com/oilshell/oil/blob/master/bin/NINJA_subgraph.py
You don’t need Starlark or any execution termination guarantees for 99% of projects. Starlark is used for parallel build config evaluation, and for us that takes about 100ms, so no real need to parallelize it.
Ninja is extremely parallel for the actual build. (FWIW I actually compared the Chrome/Android builds and Firefox like 9 years ago – Chrome/Android would use up 16 or 32 cores perfectly, but Firefox wouldn’t.)
It builds the main binary, 15-20 test binaries, and 20 binaries to test the mycpp translator, and does so with GCC and Clang, ASAN/UBSAN/dbg/opt, 64- or 32-bit, and custom builds for measuring the GC and allocators, like comparing GC vs bump alloc, etc.
It works well. The only thing is that sandboxing is extremely OS-dependent so we don’t have that yet. Bazel has a few different strategies on Linux, OS X, and maybe even Windows. It should be pretty easy to adapt those though, since they are fork-exec type wrappers.
I will say that it’s a small amount of code to write such a wrapper, but also not quite “trivial”. It’s a nice little exercise if you want to think about build graphs though! And it definitely got me more familiar with the toolchain.
“Missing manual” that was published a few months ago - https://fabiensanglard.net/dc/
I should note that this is only for devs and source code gen. The actual C++ binary can be built with just a shell and a compiler – no Make or Ninja necessary.
So our meta-build system lowers to either parallel/incremental Ninja or plain shell, which is a nice technique!
Ninja is enough… as long as you also have Python lying around.
The big problem with meta-build systems (or with using only Ninja) in C/C++ projects are auto-generated headers. You either have to specify inter-header dependencies manualy or you have to generate them as an ad hoc pre-build step. Neither of these options scale well.
https://ninja-build.org/manual.html#ref_headers
https://github.com/build2/HOWTO/blob/master/entries/handle-auto-generated-headers.md
Working closely with Bazel, this is what I hear a lot nowadays.
Many bigger companies are moving from their in-house solution to Bazel, partly to reduce the complexity and in-house maintenance burden, and partly for performance gain. The Chromium project has started using “reclient”, a ninja wrapper that speaks Bazel’s remote execution API as the official supported build tool. So things are converging.
Bazel depends on both a working Python environment and a working Java environment, which is quite a painful set of dependencies for a build tool. One of the things I like about CMake and xmake is that they have a very small set of dependencies and so are easy to bootstrap. Xmake gives a rich Lua scripting environment, but embeds Lua (it’s tiny) and so doesn’t depend on any external scripting environment.
I couldn’t find the link this morning, but I saw a while ago that someone was doing a Bazel rewrite in Rust. Having a single statically-linked binary to deploy would be a big win (I don’t really care whether it’s Rust, C++, Go, or Lua with C bindings, I just want it to be small and fast).
Facebook’s Buck2?
Buck 2 would also be my guess for what the “Bazel rewrite in Rust” was, and not being a Bazel rewrite would explain why it wasn’t found. It was discussed at https://lobste.rs/s/b0fkuh/build_faster_with_buck2_our_open_source.
Python dependencies is optional nowadays unless you want to build python stuffs with Bazel. The JRE is bundled with Bazel releases inside the executable package, with option to bring your own, so i don’t see why it would be a problem.
The JVM also comes with some of the most advanced runtime tooling in the industry that let you do very low level tunning and tracing at a maturity level that don’t exist in other ecosystems. For free! So I am not convinced that a “rewrite in rust” gona solve the problems enterprises are facing today.
Right, the package… for which OS / architecture? Most of the code I write runs on any vaguely POSIXy platform, including Linux, macOS, *BSD, and Haiku, and runs on multiple architectures. Bazel is packaged officially for under half of those. OpenJDK runs on a subset of them. FreeBSD/AArch64 is only slightly off the mainstream, yet I can’t build anything that uses Bazel there because OpenJDK is broken. If you get any further away from the big three operating systems and x86-64 then you’re likely to hit problems.
It also comes with startup times that are on a par with total incremental build times with Ninja. If you’re building something to build ten thousand source files in a distributed build environment, an OpenJDK dependency might make sense. I have worked on at least two orders of magnitude more projects that don’t need that level of complexity than I have on ones that do.
If you prefer X tool because they are easier to bootstrap vs Bazel, then you are just not the intended audience for the tool.
Folks often came to Bazel (or Buck) not for the simplicity and ease of use. They came after they have out grew the possibility of running X and Y build tool and need something that could helps solve the complexity of their millions LOC code base and thousands CI jobs pipelines. It’s a complex solution for a complex problems.
The good thing is: the ecosystem has been growing fast. That helps making difficult things easier.
Why would having a JDK available be painful? Dependency management is required either way. The proper solution is freezing these dependencies, for which nix (and guix) are the correct tools.
OpenJDK currently doesn’t build on FreeBSD/AArch64. As soon as you get away from the big three platforms, Java support drops off rapidly.
Part of the problem with Bazel is what you mention – it has an extremely complex bootstrap process, which may work if you’re on a “supported” platform, but is not particularly hackable to new situations
The second problem is that Bazel is meant for a monorepo.
That makes it less suitable for open source, which has diversity. Although some open source projects are a little monorepo-like, maybe Clang/LLVM and so forth. I think they control all their dependencies, simply because they have very few.
But there are definitely open source projects that don’t try to control their dependencies (vendor them, etc.)
i expect bazel to win for a while, in a worse-is-better sort of way
it’s current state is hell (at least partially because of the python and java dependencies), and i don’t see it changing any time soon. but it solves enough real problems that companies are switching to it and just facetanking the issues with it
i would love to see a bazel that can bootstrap its python/java dependencies, if not remove them entirely
Here is a lot more deep article on using ninja+python: https://lwn.net/Articles/821367/
Since people are chiming in with their favorite build tools, I’ve been enjoying please.build.
No it’s not, it does escaping, use datalisp.
Oh neat, thanks for posting. Curious to hear what you all have to say.
I think we talked about this like a year ago, but Oils has exactly that … there is a
cc_binary()herehttps://github.com/oilshell/oil/blob/master/bin/NINJA_subgraph.py
It’s attached to a Rules object
ru, since I wanted explicit Python not global vars, but it behaves just like Bazel and so forth.There is a loop, because reusing build config for 2 similar binaries in a loop is useful! (This kind of reuse/abstraction is extremely difficult or impossible with Make, especially once you start using its more advanced features, which do NOT compose like a real language.)
We also have asdl_library() which both Python and C++ binaries can depend on, to generate Python and C++ code. This is is exactly analogous to proto_library(), which is arguably the main feature of Bazel !
More details in my comment - https://lobste.rs/s/qnb7xt/ninja_is_enough_build_system#c_tutskb
And also if anyone wants to adapt this,
ninja_lib.pyis fairly reusablehttps://github.com/oilshell/oil/blob/master/build/ninja_lib.py
Though of course any build system inherently has “tentacles” into all parts of a system – it’s much like a shell script.
I would be really interested in an enhancement with the sandboxing features, which are described here
https://bazel.build/docs/sandboxing
Someone should extract these hard-won, platform-specific, low-dependency sandboxing tools so other build systems can use them :-)
https://github.com/bazelbuild/bazel/tree/69359279d4ff40b25b6a69219f0576bae19f0297/src/main/tools