So much AI discourse takes as a premise that you can extrapolate from the post rate of growth and progress to some hypothetical future state.
And maybe I’m just old and curmudgeonly but it’s not at all obvious to me that this should be presumed correct. Yes it is impressive what these things can do on specific circumstances. I’m just not entirely convinced that future progress is guaranteed.
I’d say the same pace of improvement is not guaranteed but it’s very unlikely we’ll hit hard limit next Tuesday. There definitely will be improvement. More training data, novel approaches, efficiency improvements—something will turn up. AI, even in its current state, is somewhat useful at least in some contexts. We should take that into account. I’m pretty sure that in a decade there will be a lot (if not majority) of developers who user AI all the time. Maybe not exclusively but it’s going to be a standard feature like syntax highlighting. There will be developers who never worked without AI. We can speculate on the quality of their work but their existence is pretty much a given at this point.
For sure, I don’t think LLM’s are going away. I was commenting more on the “We have proven we know how to do AGI now” narrative. A mythical super AGI entity is mentioned as if it’s inevitable with our current tools and understanding in a lot of these articles. I’m not so sure that is a true assumption.
I saw a comment on the “what software do you dream about” thread about teaching assembly programming, and made a sudden connection with this article. The article seems to assume that LLM code generation will get good enough and reliable enough that manually writing high-level code will become equivalent to manually writing assembly. We don’t currently have evidence for this claim, and I think the claim qualifies as extraordinary.
LLMs are trained in existing code, written by humans. I have yet to see a good response to what’s going to happen when they start training on LLM-generated code and errors are compounded. Perhaps “programmers” in the future will have as a necessary skill “fitness testing” to see if the LLM-generated code actually does what it’s supposed to do, or the derived model actually models the use case, or whatever…
…but spicy autocomplete trained on existing corpora will never, from an information-theoretic perspective, contain more information than those initial corpora. To say that LLMs will do all the coding is to say there will never be truly new coding ever again.
(I suspect that there may be some sea change in the future where we join LLMs with modeling or something to produce formal specifications more quickly and comprehensively, and then pass those models to a (non-LLM) code generator, but it would still need a human to check that the derived specifications actually match the domain. Basically, short of true AGI, we’re not gonna be completely removing humans from coding for a while.)
…but spicy autocomplete trained on existing corpora will never, from an information-theoretic perspective, contain more information than those initial corpora. To say that LLMs will do all the coding is to say there will never be truly new coding ever again.
I don’t understand the information-theoretic argument, seeing as you can come up with example dumb programs no one has written that LLMs will be able to produce (“Write me a program that prints Hello World!! to the console in blue if it is 11:11 PM or AM in UTC +0 otherwise in green”). I suspect what you’re saying is that transfer learning or whatever tactics these models use to generalize their training data has some limit – and presently whatever limit of human developers is beyond it.
I think an interesting question is “how many of the programs we care about have been written [and are on the internet to train]?” I would guess that part of the flashiness of LLMs is that we tend to evaluate them on small programs/functions which may have already been written before. So the answer to that question would be “most of the simple programs have been written,” thus explaining the hype when you tell the LLM to write my sample dumb program and it succeeds.
If the hardest problems in programming that senior devs do, like architecting and maintaining large pieces of software, have as wide a range of options to solve as we think, then I would agree that the comparative dearth of training data could pose a challenge for LLMs. Especially if their reasoning is tantamount to selecting and modifying some existing piece of code that they have seen before.
I think I agree with the general sentiment of your parenthetical, though, which is that we’ll need something more than just LLMs before machines can hope to replace human developers.
I think you highly overestimate amount of novel programming out there. We do not invent new sorting algorithms every day. Neither we solve travelling salesperson problem every week. Most of the work is doing the same CRUD e-commerce site over and over again. Sure, you might not have coded exactly the same button to the character twice but you surely did a whole lot of buttons that do very similar things.
Another point is that code is not the only corpus that goes into training data. Is it a new code if it comes from a paper that describes an algorithm but doesn’t provide a reference implementation? Is it a novel code if it recreates a system from the documentation/specification of a system, whose code is not in the training corpus? Is it a new code if it’s a translation from another language and there’s no implementation in the target language? Is it a new code if it’s an OOP rewrite of an FP implementation?
There’s a lot can be done with non-code component of the training corpus that might produce useful code. After all, at one point all of science was pushed forward by polymaths drawing on connections between disciplines. It’s still a major source of inspiration for novel discoveries. Why can’t LLMs do the same, especially since they are encoded patterns.
What amazes me about LLM code is the incredible subtleness with which it is so often confidently incorrect regarding anything even remotely specific. I’ll worry about my job being stolen by the infinite monkeys once they finish with Shakespeare tyvm.
Maybe the author is a selfish asshole with no appreciation for others, but I value humanity over productivity. GenAI guys are hard for slop and consume for consumption’s sake, like a capitalist psychosis.
A piece is left out. It’s the expectation that there will be a market for these hand crafted computational artifacts, like there is a market for hand-crafted luxury physical items and people are willing to pay more.
There won’t be a market.
Yeah, fuck you, pal. I’ve paid for Free Software and I’ll happily pay for hand-crafted software.
It was pretty harsh, I’ll admit that was an unfair thing to say. His post made me angry and I commented heedlessly. My point still stands - he doesn’t value creation, he values creations: the work of individuals is incidental and can be dismissed. I can’t elaborate just how vile a worldview this article presents to me.
OK, I’ll bite. I agree with what the author says about “artisanal software.” I don’t see what’s vile about viewing it as not inherently more valuable.
I think any hard work a human does is commendable. I respect people who memorize digits of pi or can solve a Rubik’s cube fast by hand. Even if machines can do both with more accuracy or speed, I don’t see why the human achievement shouldn’t be lauded.
But supposing I wanted several thousand digits of pi or my Rubik’s cube to be solved – and that a machine would do it cheaper – I wouldn’t hesitate to pick the machine. I don’t think it’s unreasonable for these people to then be out of a job if a machine obviates them.
BUT, this is in a hypothetical where I am asked to choose abstractly between machine-created and human-created software.
If in the present you asked me whether I would pay more for “handmade software” then I absolutely would because I would trust it far more.
If the reliability problem of AI-generated code is solved, then the question for me is an ethical one – assuming (not unreasonably) that the AI-generated software of the future continues to amass ethical concerns. In that case, I would argue that the author should not call it “artisanal” but rather “ethically-sourced”. And I should hope that we as consumers try to prefer the latter.
I’m the first person who’d love for AI to massively help me in writing code. But I’m just not seeing it.
It’s been years now, years of people telling me about this revolutionary new AI future that’s going to completely change everything. And AIs can be useful for small things, an improvement on search engines to be sure.
But I’ve not once seen AI do anything in the programming space that makes me think “wow, this is a game changer”.
The beauty in the craft lies not in the coding itself but the design of the software architecture. LLMs are good at churning out code, sparing the programmer menial tasks, but in the end the programmer has to check the output to prevent hallucinations. Even though this saves time over programming by hand, it does not affect the craft too much, in my eyes. I have no problem with an LLM ‘filling out’ a provided software architecture.
If you allow me an analogy, one can sharpen a sword by hand just with sand rubbed into the edge for hours, or one can use a sharpening machine. Despite the latter being optimised for decades, it still requires a human operator for proper sharpening, and we will always pay more for knifes that have, even in highly automated settings, have been inspected by a human in the end.
The necessity for human inspection is not overcome by current LLMs. If I may speculate, I even see a boom in the areas of contract programming and general formal methods in the near future to have mathematically-proven constraints on AI generated code. But in the end it’s still a human architect.
And fellow lobsters, please relax. This article is on the verge of being ragebait. :)
Isn’t this the main argument of the OP? The source code and architecture is not the end product. Binaries are. And they hide the source code and architecture. The end users don’t see or care about the craft as long as the end product is sufficiently functional.
It’s early days but it’s fun to speculate. The first question will be whether LLM-generated software is not just higher quality on average, but whether its failure modes are no worse than what human programmers do. If an LLM has a 0.00001% chance of hallucinating that an ordinary user should have a button to become an admin, then humans and all their faults might be in business for a while yet.
The other open question is to what degree software developers will atrophy in their problem solving and working knowledge set, affecting their ability to solve problems that their LLMs can’t reason about. This is by no means a given but I’ve read anecdotes about “time wasted” by developers trying futilely to get the right prompt to solve an issue and this suggests to me that it’s already displacing time that could be spent reading structured documentation, etc. If this turns out to be a real phenomenon, old school coders might be in demand even if their specific artifacts are not.
Or TFA may be right and I’ll either get with the times or end up cleaning the burger-flipping machines. We will see.
If this turns out to be a real phenomenon, old school coders might be in demand even if their specific artifacts are not.
My worry is that senior developers will remain in demand (to ride herd on the LLMs), but there will be no need for junior developers. And therefore no pipeline to create senior developers.
Really interesting take, and likely to be prophetically correct in a lot of ways I think. I do see professional software engineering becoming a much more niche profession as many if not most people become software literate. The cold fact of it is that everyone needs to be software literate in today’s world, because it’s the only way to ensure that our technology creates value for us instead of trying to suck value from us. I just disagree that it won’t be people writing and reading code. Code is the law of our lives. There will always be need and desire for people to engage with code in practical manner.
It would also be wise to remember that the power of LLMs is entirely derived from the power of the programming languages they can model. It is the expressiveness of languages that allows them tame fundamental complexity and bring even massively complex systems (an OS, a processor, web browser) within our ability to reason about and build correctly. Put another way, a LLM asked to generate code in a logic programming language will have very different problem solving abilities than one asked to generate a program in assembly language.
So, while language modeling is a huge new capability, programming languages are not going away nor is the need for the formation of political alliances to distribute the costs of building and maintaining languages, protocols, libraries, and systems.
Also while sometimes there’s a lot of code that needs to be written, for mature projects I would still expect slow development to be the enlightened way. Indeed models will be able to do the rushing around, but that means that quickly all the value that can be captured by rushing will be captured and people will again be forced to think carefully about which direction represents progress.
So much AI discourse takes as a premise that you can extrapolate from the post rate of growth and progress to some hypothetical future state.
And maybe I’m just old and curmudgeonly but it’s not at all obvious to me that this should be presumed correct. Yes it is impressive what these things can do on specific circumstances. I’m just not entirely convinced that future progress is guaranteed.
I’d say the same pace of improvement is not guaranteed but it’s very unlikely we’ll hit hard limit next Tuesday. There definitely will be improvement. More training data, novel approaches, efficiency improvements—something will turn up. AI, even in its current state, is somewhat useful at least in some contexts. We should take that into account. I’m pretty sure that in a decade there will be a lot (if not majority) of developers who user AI all the time. Maybe not exclusively but it’s going to be a standard feature like syntax highlighting. There will be developers who never worked without AI. We can speculate on the quality of their work but their existence is pretty much a given at this point.
For sure, I don’t think LLM’s are going away. I was commenting more on the “We have proven we know how to do AGI now” narrative. A mythical super AGI entity is mentioned as if it’s inevitable with our current tools and understanding in a lot of these articles. I’m not so sure that is a true assumption.
Someone is choking in LLM Kool-Aid
I saw a comment on the “what software do you dream about” thread about teaching assembly programming, and made a sudden connection with this article. The article seems to assume that LLM code generation will get good enough and reliable enough that manually writing high-level code will become equivalent to manually writing assembly. We don’t currently have evidence for this claim, and I think the claim qualifies as extraordinary.
LLMs are trained in existing code, written by humans. I have yet to see a good response to what’s going to happen when they start training on LLM-generated code and errors are compounded. Perhaps “programmers” in the future will have as a necessary skill “fitness testing” to see if the LLM-generated code actually does what it’s supposed to do, or the derived model actually models the use case, or whatever…
…but spicy autocomplete trained on existing corpora will never, from an information-theoretic perspective, contain more information than those initial corpora. To say that LLMs will do all the coding is to say there will never be truly new coding ever again.
(I suspect that there may be some sea change in the future where we join LLMs with modeling or something to produce formal specifications more quickly and comprehensively, and then pass those models to a (non-LLM) code generator, but it would still need a human to check that the derived specifications actually match the domain. Basically, short of true AGI, we’re not gonna be completely removing humans from coding for a while.)
I don’t understand the information-theoretic argument, seeing as you can come up with example dumb programs no one has written that LLMs will be able to produce (“Write me a program that prints Hello World!! to the console in blue if it is 11:11 PM or AM in UTC +0 otherwise in green”). I suspect what you’re saying is that transfer learning or whatever tactics these models use to generalize their training data has some limit – and presently whatever limit of human developers is beyond it.
I think an interesting question is “how many of the programs we care about have been written [and are on the internet to train]?” I would guess that part of the flashiness of LLMs is that we tend to evaluate them on small programs/functions which may have already been written before. So the answer to that question would be “most of the simple programs have been written,” thus explaining the hype when you tell the LLM to write my sample dumb program and it succeeds.
If the hardest problems in programming that senior devs do, like architecting and maintaining large pieces of software, have as wide a range of options to solve as we think, then I would agree that the comparative dearth of training data could pose a challenge for LLMs. Especially if their reasoning is tantamount to selecting and modifying some existing piece of code that they have seen before.
I think I agree with the general sentiment of your parenthetical, though, which is that we’ll need something more than just LLMs before machines can hope to replace human developers.
[Comment removed by author]
I think you highly overestimate amount of novel programming out there. We do not invent new sorting algorithms every day. Neither we solve travelling salesperson problem every week. Most of the work is doing the same CRUD e-commerce site over and over again. Sure, you might not have coded exactly the same button to the character twice but you surely did a whole lot of buttons that do very similar things.
Another point is that code is not the only corpus that goes into training data. Is it a new code if it comes from a paper that describes an algorithm but doesn’t provide a reference implementation? Is it a novel code if it recreates a system from the documentation/specification of a system, whose code is not in the training corpus? Is it a new code if it’s a translation from another language and there’s no implementation in the target language? Is it a new code if it’s an OOP rewrite of an FP implementation?
There’s a lot can be done with non-code component of the training corpus that might produce useful code. After all, at one point all of science was pushed forward by polymaths drawing on connections between disciplines. It’s still a major source of inspiration for novel discoveries. Why can’t LLMs do the same, especially since they are encoded patterns.
What amazes me about LLM code is the incredible subtleness with which it is so often confidently incorrect regarding anything even remotely specific. I’ll worry about my job being stolen by the infinite monkeys once they finish with Shakespeare tyvm.
Maybe the author is a selfish asshole with no appreciation for others, but I value humanity over productivity. GenAI guys are hard for slop and consume for consumption’s sake, like a capitalist psychosis.
Yeah, fuck you, pal. I’ve paid for Free Software and I’ll happily pay for hand-crafted software.
That is unnecessarily harsh. FYI, the author has pretty much devoted his career to helping others learn ML and Python. (I am not the author.)
It was pretty harsh, I’ll admit that was an unfair thing to say. His post made me angry and I commented heedlessly. My point still stands - he doesn’t value creation, he values creations: the work of individuals is incidental and can be dismissed. I can’t elaborate just how vile a worldview this article presents to me.
OK, I’ll bite. I agree with what the author says about “artisanal software.” I don’t see what’s vile about viewing it as not inherently more valuable.
I think any hard work a human does is commendable. I respect people who memorize digits of pi or can solve a Rubik’s cube fast by hand. Even if machines can do both with more accuracy or speed, I don’t see why the human achievement shouldn’t be lauded.
But supposing I wanted several thousand digits of pi or my Rubik’s cube to be solved – and that a machine would do it cheaper – I wouldn’t hesitate to pick the machine. I don’t think it’s unreasonable for these people to then be out of a job if a machine obviates them.
BUT, this is in a hypothetical where I am asked to choose abstractly between machine-created and human-created software.
If in the present you asked me whether I would pay more for “handmade software” then I absolutely would because I would trust it far more.
If the reliability problem of AI-generated code is solved, then the question for me is an ethical one – assuming (not unreasonably) that the AI-generated software of the future continues to amass ethical concerns. In that case, I would argue that the author should not call it “artisanal” but rather “ethically-sourced”. And I should hope that we as consumers try to prefer the latter.
Agreed, but the vile-ness doesn’t make it less likely.
But encouraging more people to recognize it as vile might.
Good. You’re awesome. You’re also in a very small minority.
It’s a minority that only grows, join us if you haven’t yet.
I’m the first person who’d love for AI to massively help me in writing code. But I’m just not seeing it.
It’s been years now, years of people telling me about this revolutionary new AI future that’s going to completely change everything. And AIs can be useful for small things, an improvement on search engines to be sure.
But I’ve not once seen AI do anything in the programming space that makes me think “wow, this is a game changer”.
The beauty in the craft lies not in the coding itself but the design of the software architecture. LLMs are good at churning out code, sparing the programmer menial tasks, but in the end the programmer has to check the output to prevent hallucinations. Even though this saves time over programming by hand, it does not affect the craft too much, in my eyes. I have no problem with an LLM ‘filling out’ a provided software architecture.
If you allow me an analogy, one can sharpen a sword by hand just with sand rubbed into the edge for hours, or one can use a sharpening machine. Despite the latter being optimised for decades, it still requires a human operator for proper sharpening, and we will always pay more for knifes that have, even in highly automated settings, have been inspected by a human in the end.
The necessity for human inspection is not overcome by current LLMs. If I may speculate, I even see a boom in the areas of contract programming and general formal methods in the near future to have mathematically-proven constraints on AI generated code. But in the end it’s still a human architect.
And fellow lobsters, please relax. This article is on the verge of being ragebait. :)
Isn’t this the main argument of the OP? The source code and architecture is not the end product. Binaries are. And they hide the source code and architecture. The end users don’t see or care about the craft as long as the end product is sufficiently functional.
[Comment removed by author]
It’s early days but it’s fun to speculate. The first question will be whether LLM-generated software is not just higher quality on average, but whether its failure modes are no worse than what human programmers do. If an LLM has a 0.00001% chance of hallucinating that an ordinary user should have a button to become an admin, then humans and all their faults might be in business for a while yet.
The other open question is to what degree software developers will atrophy in their problem solving and working knowledge set, affecting their ability to solve problems that their LLMs can’t reason about. This is by no means a given but I’ve read anecdotes about “time wasted” by developers trying futilely to get the right prompt to solve an issue and this suggests to me that it’s already displacing time that could be spent reading structured documentation, etc. If this turns out to be a real phenomenon, old school coders might be in demand even if their specific artifacts are not.
Or TFA may be right and I’ll either get with the times or end up cleaning the burger-flipping machines. We will see.
My worry is that senior developers will remain in demand (to ride herd on the LLMs), but there will be no need for junior developers. And therefore no pipeline to create senior developers.
Really interesting take, and likely to be prophetically correct in a lot of ways I think. I do see professional software engineering becoming a much more niche profession as many if not most people become software literate. The cold fact of it is that everyone needs to be software literate in today’s world, because it’s the only way to ensure that our technology creates value for us instead of trying to suck value from us. I just disagree that it won’t be people writing and reading code. Code is the law of our lives. There will always be need and desire for people to engage with code in practical manner.
It would also be wise to remember that the power of LLMs is entirely derived from the power of the programming languages they can model. It is the expressiveness of languages that allows them tame fundamental complexity and bring even massively complex systems (an OS, a processor, web browser) within our ability to reason about and build correctly. Put another way, a LLM asked to generate code in a logic programming language will have very different problem solving abilities than one asked to generate a program in assembly language.
So, while language modeling is a huge new capability, programming languages are not going away nor is the need for the formation of political alliances to distribute the costs of building and maintaining languages, protocols, libraries, and systems.
Also while sometimes there’s a lot of code that needs to be written, for mature projects I would still expect slow development to be the enlightened way. Indeed models will be able to do the rushing around, but that means that quickly all the value that can be captured by rushing will be captured and people will again be forced to think carefully about which direction represents progress.