Not the author. But, such a simple, battle tested solution, would require the author not to write all this Kubernetes stuff. Which, the fun of it, is my guess, is probably why he did so. Once you’re in Kubernetes land, simple things are out of the question…
For years, every time I installed emacs in a new box I spent hours mapping the extended keyboard to various keymaps, but I haven’t bothered lately. Now to get to end of line I press End, C-g, C-e.
I put in an online order at a local restaurant for pickup. In the comments I wrote “Embrace whimsy.” When my wife picked it up they all came out of the kitchen to say how much they loved that.
Whatever else you agree with or disagree with in this article, I think they should include Jeff Bezos along with Trump and Musk. I hear a lot of people store their data on Jeff Bezos’s online book retailer’s servers, of all places. And he has thrown his lot in with the other two.
Sounds great, except I don’t want my browser to stop me from connecting to devices inside my threat perimeter. It’s a major fucking pain in the ass to connect to older hardware on my own fucking network. No one seems to get that.
We used to wish we could say DWIM to computers and the computer would do what you mean. We knew it to be a pipe dream ofc. Unfortunately, this problem is part of a larger class of problems where someone has tried to program a computer to DWIM and getting it wrong and deciding to do something else.
I understand that he isn’t trying to address the problems with LLMs, but he does list them. So I am going to respond that his list does not include energy usage. In the energy sector, there is a term called EROI, Energy Return on Energy Invested. Basically, if you use more than a barrel oil oil to pump a barrel of oil out of the ground, it’s a net loss for the planet, unless that is you’re using someone else’s barrel of oil in the first place. That’s what’s going on here, as all the compute energy is being subsidized for your benefit by the big companies to try to enter the market. Until someone does a thermodynamic analysis of LLMs I remain unimpressed.
The energy needed to run a prompt through an LLM has dropped by an order of magnitude over the past 12-24 months.
You can tell this from the pricing. Running a prompt through GPT-3 Da Vinci (at the time the best available model) back in November 2021 cost 6 cents per 1,000 tokens, which works out as $60/million. Today, OpenAI’s most expensive model is o1 at $15.00/million tokens - 1/4 of Da Vinci - but their more commonly used models are GPT-4o ($2.50/million) and GPT-4o mini ($0.15/million). GPT-4o mini is a massively more useful model than GPT-3 Da Vinci, and costs 400x less to use.
Not all of that price reduction is down to energy efficiency, but I’m confident that a whole lot of it is. The same trend is visible across every other model provider as well.
The new Mistral Small 3, released this morning, claims to be equivalent to GPT-4o mini and I can run that comfortably on my MacBook Pro.
Energy usage for training is still large, but energy usage for most inference is a rounding error these days.
I agree that money is a rough proxy for energy. How many billions have been spent on training models? For this investment, how much work has been saved? I don’t see billions there but maybe?
I don’t see that as compelling at all. That’s not how pricing works in general. Is your AWS bill a rough proxy for their energy costs?
I think the market-wide price decrease is exactly what we’d expect from ramping up competition in a new niche for services that are generally still trying to prove out their value propositions.
Yeah, I’m not about to claim that the huge cash investment around all of this has come close to paying off. The real expense isn’t even the training runs, it’s the enormous amount of hardware and data centers that have been built for this stuff.
I compared that to the buildout of the railways in the 19th century a few weeks ago.
This happens every day with emails. People want to email me about something new so they find an old email from me and I get a message with a subject line “Re: Some Dumb Old Thing” and the entire thread attached.
(Have you tried telling them to type into your name into the “To:” section and clicking the first thing that autocompletes, like a functional human being?)
It’s good to have a name for this phenomenon. I wound up learning Make in college and I don’t understand why it in particular has this reputation when it is so simple. I assume part of it is the wrongheaded way it runs rules line-by-line through the shell; Plan 9’s version is a major improvement, or GNU Make if you turn on certain features. But it seems weird that I know many developers who understand DAGs and shell scripts and don’t understand Make.
Maven is another great example of this, if anyone is still using Java and stuck with it. Despite knowing many Java engineers, I never met anyone who actually went to the trouble to learn how to write a pom file, they were always copy-paste jobs full of strange stuff nobody could explain. A certain amount of this must be due to XML-as-configuration, but Maven also created a pretty specific and complex build pipeline and expected you to attach plugins to the fixed steps and order it understood.
I would like not to be one of these people with gitlab pipelines, but unfortunately, I don’t know how to run a pipeline locally to debug it. If I can’t run it locally, the debug loop is necessarily going to be a long one, making changes and committing them and then looking to see what effect they have, which eventually makes me hate the tool and want to replace it with something simpler like shell scripts or even Make.
I don’t understand why it in particular has this reputation when it is so simple.
Oh, I know this one–it’s because of GNU Autotools! That’s most of it anyway, make takes a tremendous amount of reputational splash damage due to standing next to autotools.
Virtually nobody understands how that system works.
Yeah autotools is much, much worse because not only does no one understand how it works, but most of the people editing the autotools config files literally do not have the ability to asses whether making a certain change will break the reason it was added, because it was added to support some bizarre Solaris configuration or only triggers on MIPS or whatever, and they don’t have access to the system that manifested the problem supposedly fixed by that change.
This is true but it is also a symptom of a deeper problem. We have too many tools that call themselves make without much overlap in terms of functionality. If you limit yourself to the functionality that is common to all major make implementations you essentially have to use something like autotools. If you are able and willing to just use one full-featured implementation like GNU make then you can avoid autotools and similar tools in most projects.
We have too many tools that call themselves make without much overlap in terms of functionality.
Agreed; another big factor leading to people hating make is bad experiences from trying to stick to the POSIX subset of make, which is so limiting as to be useless. Standards like POSIX make a lot more sense when the program itself is not portable or easy to install across a wide variety of machines, but for a program like make, they do more harm than good.
I wound up learning Make in college and I don’t understand why it in particular has this reputation when it is so simple
It’s simple for very simple things. It can’t express some things, such as:
A single rule that generates multiple outputs (you can fudge this by having bodiless rules that add dependencies).
Dynamic dependencies (various include hacks use .d files generated by gcc / clang, but they depend on GNU or BSD extensions, which are not compatible).
A single rule that generates multiple outputs (you can fudge this by having bodiless rules that add dependencies).
GNU make has this now.
Dynamic dependencies (various include hacks use .d files generated by gcc / clang, but they depend on GNU or BSD extensions, which are not compatible).
Don’t use BSD make. GNU make is available everywhere and is strictly more featureful.
Building in different locations.
Doable, but definitely much more painful than it could be.
GNU make is available everywhere and is strictly more featureful.
I don’t believe this is true. Last time I looked, there were things in bmake that gmake had no equivalent for. And gmake is GPLv3, which means that putting it in a dev container that you distribute, for example, comes with legal risks.
And now try using either to build on Windows. Make (all flavours) expects a POSIX shell to exist, so it ends up bringing all of MinGW along for the ride just to get a build system. And dealing with the differences in linkers, dependency discovery, and so on, are a complete nightmare.
I’ve worked on large projects that used gmake and bmake as their build systems. They were both horrible, barely maintainable, fragile messes. Only one built on Windows (and required Cygwin or MinGW, no native build - a third party wrote their own CMake build system to be able to build on Windows with less pain and proper support for using native libraries).
EDIT:
The reason that Make is so bad for maintainable build systems is that it is not designed for building abstractions. You define a make rule for turning .c files into .o files, for example (and there’s even a default one, hurray!). But this rule lacks any context. What happens if you have some tools that you need to build for the target and some for the host? Now you need to define a new rule for .c files that are going to be built into host tools and .c files that are going to be built into target binaries. Only these can’t just forward to the generic ones because the rules don’t take arguments, they rely on global state. So you end up defining special rules that copy and paste the .o.c rule.
And then you discover that the .c, .S, and .cc rules all replace the exception with .o, so you have a problem if you have foo.c and foo.s in the same program: both will generate foo.o. So you rewrite the built-in rule to generate output called foo.c.o and foo.s.o instead. Only you forget that only your target builds are using that, so now you need to copy and paste the solution to your host-tool builds.
And then you discover you need more flavours. For example, all of the things in FreeBSD libc need to be built as normal .o files for libc.a, position-independent ones for libc.so, and each of these needs variants with and without profiling. And the only way to differentiate these is by extension so you end up with hacks like .po and .pico for things with profiling enabled and PIC variants. And then you want to support statically linked PIEs, so you need to rewrite your libc.a target to use .pico not .o files.
And then you cry in a corner because the build system is a total mess and no one understands it.
Higher-level build systems provide hooks for how you do a compile phase and how you do a link phase, but they allow building abstractions. I am building a shared library, which will be either a Windows .dll or a UNIX .so, here are the source files. Someone can build a reusable recipe for ‘compile a bunch of source files and build a shared library’. If I am building ten shared libraries and I now need to build shared libraries a different way, there’s a single place that has the ‘build a shared library’ abstraction. This knows to pass the -fpic or fPIC (depending on the platform) flag during the build phase. It knows to pass -shared during the link phase. If I want to build a shared library for a host tool, I can create a new abstraction that’s a host shared library that wraps the shared library but sets some of its properties (such as target triple and sysroot locations) differently.
The key feature that I want from any language is the ability to build reusable abstractions. Build-system languages are no different.
There are lots of things I grumble about with xmake, but I really like that it has an abstract map-reduce model. Each target has a bunch of inputs, it applies some function to all of them, and then some function to the output of the compile phases. Both phases are optional, so something like Go or Java might use the reduce phase for the entire compile job, an interpreted language might just use the map phase to generate cached bytecode files but not do linking. Each phase is a Lua function and can be inherited from a rule, or can explicitly call another rule to delegate the behaviour. These can be nested. For example, with CHERIoT, we have shared library and compartment target types, which can be built from source files and are linked as relocatable objects, then a firmware image that takes these, plus a load of metadata (which generates the linker script) and links them together.
I never met anyone who actually went to the trouble to learn how to write a pom file, they were always copy-paste jobs full of strange stuff nobody could explain.
It is OK to copy the headers and namespaces or some common parts, but you should be able to explain every single line you copied. Unlike Makefile, I would not be able to write a pom.xml from scratch, but I can tell you, why every single plugin or dependency is needed.
The XML is actually helpful, because there is a schema and editor/IDE will usually tell you, what configuration options are available and will show you the documentation. It usually shows you even the groupIds or artifactIds of dependencies. (example). People do not have to remember or copy everything, they can write it with documentation available in their IDE or editor.
I do not say that people never copy code without understanding. Of course, they do. I sometimes see even whole big codebase being copied for new customer, then some parts are deleted, some rewritten… and project is done. This is terrible. But it is not a thing of particular language. It is rather caused by people and (work) environment (see also my other comment). It is this simple implication:
lack of knowledge + no time to learn + complex product demanded + mananager asking every now and then, when it will be finished → people copy code without understanding and hope that it will work
It does not matter whether it is Java, PHP, Python, Scheme, Haskell, Rust, Lua, whatever.
There is also related problem - the last couple of years - popularity of generative AI and LLM - instead of copying carefully handcrafted code from their colleagues, people copy garbage from an AI model. If you do code reviews, you probably seen some mysterious code, that looked somehow relevant, somehow too complex and not working well.
if anyone is still using Java and stuck with it
I am hearing these boring jokes for decades and it is unlikely to change in the future.
$ gradle init --type java-application --dsl groovy
Select test framework:
1: JUnit 4
2: TestNG
3: Spock
4: JUnit Jupiter
Enter selection (default: JUnit Jupiter) [1..4]
Project name (default: gradle-demo):
Source package (default: gradle.demo):
> Task :init
Get more help with your project: https://docs.gradle.org/7.2/samples/sample_building_java_applications.html
BUILD SUCCESSFUL in 5s
2 actionable tasks: 2 executed
$ find -type f
./.gradle/7.2/fileHashes/fileHashes.lock
./.gradle/7.2/fileHashes/fileHashes.bin
./.gradle/7.2/fileChanges/last-build.bin
./.gradle/7.2/dependencies-accessors/dependencies-accessors.lock
./.gradle/7.2/dependencies-accessors/gc.properties
./.gradle/7.2/executionHistory/executionHistory.lock
./.gradle/7.2/executionHistory/executionHistory.bin
./.gradle/7.2/gc.properties
./.gradle/vcs-1/gc.properties
./.gradle/checksums/checksums.lock
./.gradle/buildOutputCleanup/buildOutputCleanup.lock
./.gradle/buildOutputCleanup/cache.properties
./.gradle/buildOutputCleanup/outputFiles.bin
./gradle/wrapper/gradle-wrapper.properties
./gradle/wrapper/gradle-wrapper.jar
./gradlew
./gradlew.bat
./settings.gradle
./.gitignore
./.gitattributes
./app/src/main/java/gradle/demo/App.java
./app/src/test/java/gradle/demo/AppTest.java
./app/build.gradle
(however, Maven can generate project files from templates too – it is called Archetypes – not a bad thing, you can prepare templates of particular project types you use).
Even passionate Gradle enthusiasts I have met do not initialize a project off the top of their heads and use this or some web „initializr“ or copy some older project…
Compared to this, Makefiles are handcrafted more often, IMHO.
Friend, I think you’re reading my comments pretty uncharitably. I’m not trying to create a fight here; I agree with what you said about XML. My experience with Maven and people who used it is just my experience. When I was moving away from Java, people were moving to Gradle. I never used it myself.
I am sorry if that sounded unfriendly to you. I am still thinking about the original article and why Make was chosen as the „biggest villain“ and why the „effect“ is named after it. Because according to my experience, Make is not the best example of copy&paste habit.
I am also a bit allergic to people who do not miss a single opportunity to take a dig at Java. Maybe it was a subconscious reaction and not directed at you.
You can’t create control structures that span multiple lines (conditionals, loops) inside a make recipe because make passes each line separately to a different subshell. This is part of why you see such a proliferation of && and || in make recipes. GNU Make’s documentation discusses this in more depth and provides a .ONESHELL target for circumventing the problem.
For simplicity, I try to turn any recipe where I would want a control structure into its own script (often in /ops or /bin); this lets me avoid a level of indentation, get editor assistance from shellcheck, and test it independently.
Oftentimes, it becomes useful to call the scripts from places other than the makefile as well, and they can even form the basis of CLI tools for the project.
Yes, there are benefits to having to work around the misfeature, but it’s still a misfeature. Either Make should have always done the right thing here, or it should have insisted on single-line recipes. There are benefits to doing it the way you describe, but there are benefits to calling into Make instead too, like knowing that dependent steps are going to get executed.
I think the fundamental reason it’s not actually a misfeature is that separate subshells allows the exit status of each line to be checked independently regardless of whether the shell you’ve chosen has implicit errexit behaviour. It also allows make to perform better logging of commands and their outputs.
I would also note that using shell loops in Makefiles has been a serious red flag basically every time I’ve seen it. It’s much more idiomatic to produce a word list in a variable and then have that list end up as a dependency of something else. Make can then understand the dependencies and give you parallelism, etc.
Of course all this is nice for the reader, but notice that avoiding these annoyances makes it pretty much impossible to make any money from a website. So there’s a certain aspect of privilege here — we all make our living by higher means and don’t have to sully our web pages with tawdry commercial concerns.
I agree! What I’m criticizing is the implicit shaming of sites that do need revenue and are basically forced to use some of the annoyances listed here (because people hate paywalls too.)
Fixing these bad incentives would be the one of the most broadly beneficial things one could hope to do on the web. I don’t have any good suggestions, sorry. Micropayments clearly haven’t worked. Substack-style subscription models are an improvement, in my view, but I think they have some limits.
Ur amazing
“Ignore FEDERAL-SPYCAM; that is just my phone.”
My phone’s hotspot is called Warning Expired Certificate. Not sure if that would deter hackers or lure them.
My home SSID is “FBI Surveillance Van”. I’ve got the same question.
Sorry if you are not the author, or this is just obvious and wrong because I know nothing about k8s, but have you tried https://www.noip.com ?
Not the author. But, such a simple, battle tested solution, would require the author not to write all this Kubernetes stuff. Which, the fun of it, is my guess, is probably why he did so. Once you’re in Kubernetes land, simple things are out of the question…
For years, every time I installed emacs in a new box I spent hours mapping the extended keyboard to various keymaps, but I haven’t bothered lately. Now to get to end of line I press End, C-g, C-e.
I notice that the pint in the picture is full to the brim. Good! If you get a pour that’s a half inch below, the bartender has docked you 10%.
+1 instead of flagging because the world needs more whimsy right now :)
I put in an online order at a local restaurant for pickup. In the comments I wrote “Embrace whimsy.” When my wife picked it up they all came out of the kitchen to say how much they loved that.
Exactly why I posted this <3
Whatever else you agree with or disagree with in this article, I think they should include Jeff Bezos along with Trump and Musk. I hear a lot of people store their data on Jeff Bezos’s online book retailer’s servers, of all places. And he has thrown his lot in with the other two.
Sounds great, except I don’t want my browser to stop me from connecting to devices inside my threat perimeter. It’s a major fucking pain in the ass to connect to older hardware on my own fucking network. No one seems to get that.
I appreciate the focus on energy.
I’d change it to Gulf of Despair or Gulf of Social Disorder.
Can this be made to work against Microsoft’s Copilot and/or Recall? I’d pay for that.
We used to wish we could say DWIM to computers and the computer would do what you mean. We knew it to be a pipe dream ofc. Unfortunately, this problem is part of a larger class of problems where someone has tried to program a computer to DWIM and getting it wrong and deciding to do something else.
I understand that he isn’t trying to address the problems with LLMs, but he does list them. So I am going to respond that his list does not include energy usage. In the energy sector, there is a term called EROI, Energy Return on Energy Invested. Basically, if you use more than a barrel oil oil to pump a barrel of oil out of the ground, it’s a net loss for the planet, unless that is you’re using someone else’s barrel of oil in the first place. That’s what’s going on here, as all the compute energy is being subsidized for your benefit by the big companies to try to enter the market. Until someone does a thermodynamic analysis of LLMs I remain unimpressed.
The energy needed to run a prompt through an LLM has dropped by an order of magnitude over the past 12-24 months.
You can tell this from the pricing. Running a prompt through GPT-3 Da Vinci (at the time the best available model) back in November 2021 cost 6 cents per 1,000 tokens, which works out as $60/million. Today, OpenAI’s most expensive model is o1 at $15.00/million tokens - 1/4 of Da Vinci - but their more commonly used models are GPT-4o ($2.50/million) and GPT-4o mini ($0.15/million). GPT-4o mini is a massively more useful model than GPT-3 Da Vinci, and costs 400x less to use.
Not all of that price reduction is down to energy efficiency, but I’m confident that a whole lot of it is. The same trend is visible across every other model provider as well.
The new Mistral Small 3, released this morning, claims to be equivalent to GPT-4o mini and I can run that comfortably on my MacBook Pro.
Energy usage for training is still large, but energy usage for most inference is a rounding error these days.
I agree that money is a rough proxy for energy. How many billions have been spent on training models? For this investment, how much work has been saved? I don’t see billions there but maybe?
I don’t see that as compelling at all. That’s not how pricing works in general. Is your AWS bill a rough proxy for their energy costs?
I think the market-wide price decrease is exactly what we’d expect from ramping up competition in a new niche for services that are generally still trying to prove out their value propositions.
Yeah, I’m not about to claim that the huge cash investment around all of this has come close to paying off. The real expense isn’t even the training runs, it’s the enormous amount of hardware and data centers that have been built for this stuff.
I compared that to the buildout of the railways in the 19th century a few weeks ago.
“Open Source and Free Software are inherently political”
Friendly reminder that you may not be into politics, but politics is always into you.
This happens every day with emails. People want to email me about something new so they find an old email from me and I get a message with a subject line “Re: Some Dumb Old Thing” and the entire thread attached.
Re: Re: Re: Fwd: Re: Re: ...For some reason, my mother has never figured out what Subject is for, so every email I get from her has the sent-on date typed manually in that field.
That’s actually adorable! She just wants to help you stay organised ♥
That’s horrifying.
(Have you tried telling them to type into your name into the “To:” section and clicking the first thing that autocompletes, like a functional human being?)
“Larry, I’m on Duck Tales”
It’s good to have a name for this phenomenon. I wound up learning Make in college and I don’t understand why it in particular has this reputation when it is so simple. I assume part of it is the wrongheaded way it runs rules line-by-line through the shell; Plan 9’s version is a major improvement, or GNU Make if you turn on certain features. But it seems weird that I know many developers who understand DAGs and shell scripts and don’t understand Make.
Maven is another great example of this, if anyone is still using Java and stuck with it. Despite knowing many Java engineers, I never met anyone who actually went to the trouble to learn how to write a pom file, they were always copy-paste jobs full of strange stuff nobody could explain. A certain amount of this must be due to XML-as-configuration, but Maven also created a pretty specific and complex build pipeline and expected you to attach plugins to the fixed steps and order it understood.
I would like not to be one of these people with gitlab pipelines, but unfortunately, I don’t know how to run a pipeline locally to debug it. If I can’t run it locally, the debug loop is necessarily going to be a long one, making changes and committing them and then looking to see what effect they have, which eventually makes me hate the tool and want to replace it with something simpler like shell scripts or even Make.
Oh, I know this one–it’s because of GNU Autotools! That’s most of it anyway, make takes a tremendous amount of reputational splash damage due to standing next to autotools.
Autotools are an even better example of this effect. Virtually nobody understands how that system works.
Yeah autotools is much, much worse because not only does no one understand how it works, but most of the people editing the autotools config files literally do not have the ability to asses whether making a certain change will break the reason it was added, because it was added to support some bizarre Solaris configuration or only triggers on MIPS or whatever, and they don’t have access to the system that manifested the problem supposedly fixed by that change.
So, not only no one understand how it works, few understand what «it works» is…
(It’s impressive that those hidden horrors often do achieve their original aims…)
This is true but it is also a symptom of a deeper problem. We have too many tools that call themselves
makewithout much overlap in terms of functionality. If you limit yourself to the functionality that is common to all major make implementations you essentially have to use something like autotools. If you are able and willing to just use one full-featured implementation like GNU make then you can avoid autotools and similar tools in most projects.Agreed; another big factor leading to people hating make is bad experiences from trying to stick to the POSIX subset of make, which is so limiting as to be useless. Standards like POSIX make a lot more sense when the program itself is not portable or easy to install across a wide variety of machines, but for a program like make, they do more harm than good.
I don’t think Makefile Effect is a good name though. This is simple copy/paste error with a side of cargo cult.
It’s simple for very simple things. It can’t express some things, such as:
GNU make has this now.
Don’t use BSD make. GNU make is available everywhere and is strictly more featureful.
Doable, but definitely much more painful than it could be.
I don’t believe this is true. Last time I looked, there were things in bmake that gmake had no equivalent for. And gmake is GPLv3, which means that putting it in a dev container that you distribute, for example, comes with legal risks.
And now try using either to build on Windows. Make (all flavours) expects a POSIX shell to exist, so it ends up bringing all of MinGW along for the ride just to get a build system. And dealing with the differences in linkers, dependency discovery, and so on, are a complete nightmare.
I’ve worked on large projects that used gmake and bmake as their build systems. They were both horrible, barely maintainable, fragile messes. Only one built on Windows (and required Cygwin or MinGW, no native build - a third party wrote their own CMake build system to be able to build on Windows with less pain and proper support for using native libraries).
EDIT:
The reason that Make is so bad for maintainable build systems is that it is not designed for building abstractions. You define a make rule for turning .c files into .o files, for example (and there’s even a default one, hurray!). But this rule lacks any context. What happens if you have some tools that you need to build for the target and some for the host? Now you need to define a new rule for .c files that are going to be built into host tools and .c files that are going to be built into target binaries. Only these can’t just forward to the generic ones because the rules don’t take arguments, they rely on global state. So you end up defining special rules that copy and paste the .o.c rule.
And then you discover that the .c, .S, and .cc rules all replace the exception with .o, so you have a problem if you have foo.c and foo.s in the same program: both will generate foo.o. So you rewrite the built-in rule to generate output called foo.c.o and foo.s.o instead. Only you forget that only your target builds are using that, so now you need to copy and paste the solution to your host-tool builds.
And then you discover you need more flavours. For example, all of the things in FreeBSD libc need to be built as normal .o files for libc.a, position-independent ones for libc.so, and each of these needs variants with and without profiling. And the only way to differentiate these is by extension so you end up with hacks like .po and .pico for things with profiling enabled and PIC variants. And then you want to support statically linked PIEs, so you need to rewrite your libc.a target to use .pico not .o files.
And then you cry in a corner because the build system is a total mess and no one understands it.
Higher-level build systems provide hooks for how you do a compile phase and how you do a link phase, but they allow building abstractions. I am building a shared library, which will be either a Windows .dll or a UNIX .so, here are the source files. Someone can build a reusable recipe for ‘compile a bunch of source files and build a shared library’. If I am building ten shared libraries and I now need to build shared libraries a different way, there’s a single place that has the ‘build a shared library’ abstraction. This knows to pass the -fpic or fPIC (depending on the platform) flag during the build phase. It knows to pass -shared during the link phase. If I want to build a shared library for a host tool, I can create a new abstraction that’s a host shared library that wraps the shared library but sets some of its properties (such as target triple and sysroot locations) differently.
The key feature that I want from any language is the ability to build reusable abstractions. Build-system languages are no different.
There are lots of things I grumble about with xmake, but I really like that it has an abstract map-reduce model. Each target has a bunch of inputs, it applies some function to all of them, and then some function to the output of the compile phases. Both phases are optional, so something like Go or Java might use the reduce phase for the entire compile job, an interpreted language might just use the map phase to generate cached bytecode files but not do linking. Each phase is a Lua function and can be inherited from a rule, or can explicitly call another rule to delegate the behaviour. These can be nested. For example, with CHERIoT, we have shared library and compartment target types, which can be built from source files and are linked as relocatable objects, then a firmware image that takes these, plus a load of metadata (which generates the linker script) and links them together.
It is OK to copy the headers and namespaces or some common parts, but you should be able to explain every single line you copied. Unlike Makefile, I would not be able to write a
pom.xmlfrom scratch, but I can tell you, why every single plugin or dependency is needed.The XML is actually helpful, because there is a schema and editor/IDE will usually tell you, what configuration options are available and will show you the documentation. It usually shows you even the
groupIds orartifactIds of dependencies. (example). People do not have to remember or copy everything, they can write it with documentation available in their IDE or editor.I do not say that people never copy code without understanding. Of course, they do. I sometimes see even whole big codebase being copied for new customer, then some parts are deleted, some rewritten… and project is done. This is terrible. But it is not a thing of particular language. It is rather caused by people and (work) environment (see also my other comment). It is this simple implication:
lack of knowledge + no time to learn + complex product demanded + mananager asking every now and then, when it will be finished → people copy code without understanding and hope that it will work
It does not matter whether it is Java, PHP, Python, Scheme, Haskell, Rust, Lua, whatever.
There is also related problem - the last couple of years - popularity of generative AI and LLM - instead of copying carefully handcrafted code from their colleagues, people copy garbage from an AI model. If you do code reviews, you probably seen some mysterious code, that looked somehow relevant, somehow too complex and not working well.
I am hearing these boring jokes for decades and it is unlikely to change in the future.
Stuck with Maven. Gradle seems to be supplanting it.
Gradle? It has integrated copy&paste!
(however, Maven can generate project files from templates too – it is called Archetypes – not a bad thing, you can prepare templates of particular project types you use).
Even passionate Gradle enthusiasts I have met do not initialize a project off the top of their heads and use this or some web „initializr“ or copy some older project…
Compared to this, Makefiles are handcrafted more often, IMHO.
Friend, I think you’re reading my comments pretty uncharitably. I’m not trying to create a fight here; I agree with what you said about XML. My experience with Maven and people who used it is just my experience. When I was moving away from Java, people were moving to Gradle. I never used it myself.
I am sorry if that sounded unfriendly to you. I am still thinking about the original article and why Make was chosen as the „biggest villain“ and why the „effect“ is named after it. Because according to my experience, Make is not the best example of copy&paste habit.
I am also a bit allergic to people who do not miss a single opportunity to take a dig at Java. Maybe it was a subconscious reaction and not directed at you.
Thank you. I agree completely!
What options change running the recipes line by line?
You can’t create control structures that span multiple lines (conditionals, loops) inside a make recipe because make passes each line separately to a different subshell. This is part of why you see such a proliferation of
&&and||in make recipes. GNU Make’s documentation discusses this in more depth and provides a.ONESHELLtarget for circumventing the problem.For simplicity, I try to turn any recipe where I would want a control structure into its own script (often in
/opsor/bin); this lets me avoid a level of indentation, get editor assistance from shellcheck, and test it independently.Oftentimes, it becomes useful to call the scripts from places other than the makefile as well, and they can even form the basis of CLI tools for the project.
Yes, there are benefits to having to work around the misfeature, but it’s still a misfeature. Either Make should have always done the right thing here, or it should have insisted on single-line recipes. There are benefits to doing it the way you describe, but there are benefits to calling into Make instead too, like knowing that dependent steps are going to get executed.
I think the fundamental reason it’s not actually a misfeature is that separate subshells allows the exit status of each line to be checked independently regardless of whether the shell you’ve chosen has implicit errexit behaviour. It also allows make to perform better logging of commands and their outputs.
I would also note that using shell loops in Makefiles has been a serious red flag basically every time I’ve seen it. It’s much more idiomatic to produce a word list in a variable and then have that list end up as a dependency of something else. Make can then understand the dependencies and give you parallelism, etc.
Absolutely! Didn’t mean to disagree, just a bit of a tangent
Of course all this is nice for the reader, but notice that avoiding these annoyances makes it pretty much impossible to make any money from a website. So there’s a certain aspect of privilege here — we all make our living by higher means and don’t have to sully our web pages with tawdry commercial concerns.
“I have a full time job and make enough money so I share my knowledge to you for free because teaching is its own reward” is a fine use of privilege.
I agree! What I’m criticizing is the implicit shaming of sites that do need revenue and are basically forced to use some of the annoyances listed here (because people hate paywalls too.)
Fixing these bad incentives would be the one of the most broadly beneficial things one could hope to do on the web. I don’t have any good suggestions, sorry. Micropayments clearly haven’t worked. Substack-style subscription models are an improvement, in my view, but I think they have some limits.