I think this misses one approach: write the build script in the main language of the project (Java in this case). This requires a bit of code to kick of bootstrap spiral, which I think is a #! for Java11+ and a tiny shell script to compile and then invoke before that.
More generally, I wish language build systems had the following capabilities:
building language’s notion of libraries/binaries/other artifacts with a set of rigid conventions
CLI affordance for running an arbitrary custom program in the language as a part of the build
Which is restricted to the end projects (such that dependencies can’t have custom builds)
It should be easy to bootstrap arbitrary custom build almost out of nothing; when that is solved, you don’t need a second language to orchestrate the build.
Building things that mix c++ and java is a little hairy no matter how you slice it, and I’d never throw rocks at someone for using python to drive that kind of build. It’s certainly not silly to do so…
But last time I needed to do this, I used cmake. I was, like the author of this post, already using cmake to build the C++ side, and getting it to build the java pieces was not a tall order. My project doesn’t have a public repo I can point to, but this one takes an approach that is similar to what we did.
gradle is a fine build tool that can do everything you want. I find this hand-wavy “oh its magic” talk off-putting. You have to learn the isms of every tool, yet when you roll your own you are going to create an even bigger mess than any other build tool
I had an easier time getting cmake to handle my java things than getting gradle to handle my c++ things. I suspect that for different projects that could break very much the other way.
gradle is a fine build tool that can do everything you want. I find this hand-wavy “oh its magic” talk off-putting.
Gradle feel very “magic” to many me and I expect to many other people.
Therefore, using something else that you build yourself is already less magic from one’s point of view.
I maintain a handful of Gradle plugins. It isn’t magic, but it is complex. Half its API is deprecated, the other half is unstable.
I have been using it since 2013 and I have contributed patches to plugins and def. don’t want to go back to maven. (I had to for a job in between then and now, but please never again). It may indeed be fast moving, yet still better than hand-rolling something.
Here’s an easy one: what is the stable way to change the Java language target from version 8 (the default) to version 11 (the late LTS)?
I think SCons was onto something very similar: use Python as build system for other languages, because it is a full programming language and it serves well as glue between components … In fact, SCons supports Java projects and maybe it could be a drop-in replacement for the author’s custom scripts (I myself have never used it like that).
I’ve used Ruby’s ‘rake’ in similar situations before and it works very well. the main downside is adding a ruby dependency to your project, but that’s true for python and even bash, as the article points out.
It means infra/tools ops people migrating between the companies and bringing their favorite tools with them.
Which has nothing to with whether it’s a good choice or not. Enterprise companies are notorious in copying each other approaches, so whichever gets popular at any point tends to be entrenched for some years. “Google does it” is always going to win against solitary voices saying “but it’s complicated, and there’s a better way”.
I work in an enterprise, a pretty large one and all I see is maven and some gradle. No bazel in sight. I also have been following the project a bit for year, but never met anyone who used it (outside Google, but they have the original called blaze IIRC)
Hey, let me make a list of enterprise who is using Bazel and then share it with Lobste.rs
There is definitely an upward trend in adoption of bazel in the last 3 years, so much that I don’t think I can adequately dropping all the names in 1 go. But if you must know, I will make a small list with some big names here who have publicly said that they are adopting bazel (not just 1-2 projects):
Google
Twitter (Java, Scala)
VMWare (Erlang, C/C++)
Adobe Cloud (Go / Docker / K8s)
Tinder (Kotlin / Java)
Pinterest (Java, Go, Python, NodeJS)
Dropbox (C, Python, Go)
Huawei (C/C++)
Lyft (iOS)
Uber (Golang)
Grab (Android)
Stripe (Ruby, Python)
Apple (Go, Java)
Robinhood
Wix
SpaceX
TwoSigma
Etsy
NVDIA (C/C++, CUDA, Go, Scala)
So you can see that a large portion of FAANGMULA and equivalents unicorns are adopting Bazel. Among them, Facebook is rebuilding Bucks 2.0 and has snatched some key hires from Bazel/Build community in the past years. Microsoft MSBuild is also adopting Bazel’s RBE spec. There are also bigger companies / startups who are experimenting with Bazel on 1 or 2 smaller projects:
Redhat
Gitlab
Brave
BMW
LinkedIn
Tweag
There are a number of startups / consulting companies in this space, some include members of the Blaze team leaving Google, to address the upcoming market demand:
BuildBuddy
EngFlow
FlareBuild
Aspect Dev
OasisDigital
The above are not an exhaustive list, it’s what in my head after 15 mins of writing this post.
Google people are trying to push Bazel into everything that they touch. It’s a fine build system if you can guarantee that you will only ever want to ship things on one of the three or four platforms that Google thinks are important. Since ‘Enterprise’ is often a synonym for ‘expensive lock-in’, I don’t have any reason to doubt the original claim.
Does Python have anything built in to construct and resolve the dependency graph and order compilation accordingly? Since that’s the primary job of a build tool.
I think this misses one approach: write the build script in the main language of the project (Java in this case). This requires a bit of code to kick of bootstrap spiral, which I think is a
#!for Java11+ and a tiny shell script to compile and then invoke before that.More generally, I wish language build systems had the following capabilities:
It should be easy to bootstrap arbitrary custom build almost out of nothing; when that is solved, you don’t need a second language to orchestrate the build.
Building things that mix c++ and java is a little hairy no matter how you slice it, and I’d never throw rocks at someone for using python to drive that kind of build. It’s certainly not silly to do so…
But last time I needed to do this, I used cmake. I was, like the author of this post, already using cmake to build the C++ side, and getting it to build the java pieces was not a tall order. My project doesn’t have a public repo I can point to, but this one takes an approach that is similar to what we did.
gradle is a fine build tool that can do everything you want. I find this hand-wavy “oh its magic” talk off-putting. You have to learn the isms of every tool, yet when you roll your own you are going to create an even bigger mess than any other build tool
I had an easier time getting cmake to handle my java things than getting gradle to handle my c++ things. I suspect that for different projects that could break very much the other way.
Gradle feel very “magic” to many me and I expect to many other people. Therefore, using something else that you build yourself is already less magic from one’s point of view.
I maintain a handful of Gradle plugins. It isn’t magic, but it is complex. Half its API is deprecated, the other half is unstable.
Here’s an easy one: what is the stable way to change the Java language target from version 8 (the default) to version 11 (the late LTS)?
I have been using it since 2013 and I have contributed patches to plugins and def. don’t want to go back to maven. (I had to for a job in between then and now, but please never again). It may indeed be fast moving, yet still better than hand-rolling something.
Toolchains https://docs.gradle.org/current/userguide/toolchains.html
agreed.
Gradle is an equal but different bad to Bazel. I agree, both are better than hand-rolled for a non-trivial project.
Not really: https://docs.gradle.org/current/userguide/building_java_projects.html#sec:java_cross_compilation
And even those docs aren’t the whole story. 😪
I think SCons was onto something very similar: use Python as build system for other languages, because it is a full programming language and it serves well as glue between components … In fact, SCons supports Java projects and maybe it could be a drop-in replacement for the author’s custom scripts (I myself have never used it like that).
I’ve used Ruby’s ‘rake’ in similar situations before and it works very well. the main downside is adding a ruby dependency to your project, but that’s true for python and even bash, as the article points out.
The enterprise community is uniting around Bazel right now.
Citation needed! No seriously, which enterprise community is that supposed to be?
It means infra/tools ops people migrating between the companies and bringing their favorite tools with them.
Which has nothing to with whether it’s a good choice or not. Enterprise companies are notorious in copying each other approaches, so whichever gets popular at any point tends to be entrenched for some years. “Google does it” is always going to win against solitary voices saying “but it’s complicated, and there’s a better way”.
I work in an enterprise, a pretty large one and all I see is maven and some gradle. No bazel in sight. I also have been following the project a bit for year, but never met anyone who used it (outside Google, but they have the original called blaze IIRC)
Hey, let me make a list of enterprise who is using Bazel and then share it with Lobste.rs
There is definitely an upward trend in adoption of bazel in the last 3 years, so much that I don’t think I can adequately dropping all the names in 1 go. But if you must know, I will make a small list with some big names here who have publicly said that they are adopting bazel (not just 1-2 projects):
So you can see that a large portion of FAANGMULA and equivalents unicorns are adopting Bazel. Among them, Facebook is rebuilding Bucks 2.0 and has snatched some key hires from Bazel/Build community in the past years. Microsoft MSBuild is also adopting Bazel’s RBE spec. There are also bigger companies / startups who are experimenting with Bazel on 1 or 2 smaller projects:
There are a number of startups / consulting companies in this space, some include members of the Blaze team leaving Google, to address the upcoming market demand:
The above are not an exhaustive list, it’s what in my head after 15 mins of writing this post.
https://github.com/bazelbuild/bazel/search?q=blaze always makes me laugh.
Google people are trying to push Bazel into everything that they touch. It’s a fine build system if you can guarantee that you will only ever want to ship things on one of the three or four platforms that Google thinks are important. Since ‘Enterprise’ is often a synonym for ‘expensive lock-in’, I don’t have any reason to doubt the original claim.
Do you mean Google? Or anyone else?
(I’ve read that Bazel has been designed especially for building Google projects, on Google architecture, using Google style of source management)
Square.
As well as Twitter, Dropbox & Pinterest (all to varying degrees).
Does Python have anything built in to construct and resolve the dependency graph and order compilation accordingly? Since that’s the primary job of a build tool.
Yes: https://docs.python.org/3/library/graphlib.html#module-graphlib