Same here…kids those days complaining about kids these days. :)
I was telling someone recently about what “browsing the literature” was like when I was working on compilers at Apple in 1990: going to the internal library down the road to read some ACM proceedings, copying references from the bibliography in the paper, asking the librarian to call Stanford to ask them to photocopy the papers Apple didn’t have, and waiting for the copies to show up in the interoffice mail. That was just 35 years ago! And getting answers to your questions on demand…if you were lucky enough to know somebody who might know the answer, you used the phone. (The lucky few in the know could try Usenet.)
Same – I remember when Stack Overflow was brand new[0], and it was introduced as some mixture of (1) a better expertsexchange and (2) a pseudo-wiki to gather and summarize knowledge that was otherwise spread across personal blogs and academic homepages.
At some point it transformed from a Q&A site aimed at skilled professionals into a code snippets repository for people new to the industry. Nowadays whenever I’m doing something that leads me to an SO question, usually the question is exactly what I want answered but it’s locked as a duplicate of some completely unrelated topic.
[0] Fun(?) story: I created the first Wikipedia article about Stack Overflow, but it got speedy-deleted by a Wikipedia administrator after about ten minutes. A lot of the early SO content was written by people who wanted to contribute computer science information to the public square, but didn’t want to deal with Wikipedia’s opaque deletionist bureaucracy.
Also, if you really want to feel old, write some C++ with a whole class of new hires who pronounce #include as “hashtag include”…
I took a couple classes in the pre-hashtag era, but I don’t recall if/how it was referred to verbally.
I suspect I’d never heard it called anything but a number/pound sign when I took those classes. I’m pretty sure I didn’t know octothorp(e) until years later, and I doubt I would have thought to call it a “hash” until encountering shebangs or hashtags years later.
When I was in school the # character was universally pronounced “pound”, to the point that both students and professors would (jokingly) refer to C# as “C Pound”.
Oddly it wasn’t pronounced at all when naming C processor directives: #include was simply “include”. And the justification for that is “the leading pound is silent, like the leading ‘p’ in pterodactyl”. To this day I am not sure whether that professor was serious.
This doesn’t match my experience working with junior developers. It just seems to be that new folks always have a lot to learn, and the amount of knowledge that’s needed to be productive is going up over time.
There are reasons to be concerned about the effect that tools have on the way folks learn, and how raising the level of abstraction reduces the need to learn lower-level details, but this post just seems to be overstating things significantly.
Our last intern was excellent. He dug into things, was super technical and effective, and never pushed code he didn’t understand. AFAIK he was fine to use AI but did so appropriately.
What’s described here sounds like a lot of engineers I’ve known, especially back in college. A lot of young, new engineers don’t know why their code works and don’t have the ability to investigate why either - it’s pretty damn common.
I suspect AI will make those engineers worse but, frankly, they were never very good and I’ve never had to work with anyone that bad/ they would have been fired eventually. They aren’t going to be bad because of AI they’re going to be bad because they never put the work in.
But, there’s still one reason that StackOverflow was superior:
I remember posting a lot on C++. I had some high ranked questions about various things like lvalues and xvalues etc. I was lucky to get a bombard of idiots and days later one person going “I think others are misunderstanding” and expertly weighing in with real insights if I was very lucky, most of the time I got little response or “why are you doing that?” etc. SO was extremely limited and I quickly migrated into Slacks with people I could ask directly, which was infinitely more effective for learning but few benefited from my conversations. This is just the nature of a world where “document stores” aren’t ideal for conversations but conversations are ideal for learning. AI is actually good at indexing conversations so it may help here.
Regarding “What an we do?” IMO/IME the good developers who were going to be good without AI are already just doing these things.
Do code reviews differently. Instead of just checking if the code works, start a conversation with your team. What other approaches did they consider? Why did they pick this one? Make understanding the process as important as the end result.
Oof, this (the fact that this is considered “different”) hurts. This has always been how I’ve thought Code Review should run - “checking the code for correctness” is almost secondary to “ensuring that understanding of the problem domain, and the reasoning behind the chosen solution, is spread around the team”. Indeed, you can’t do the former without the latter, because Code Only Says What It Does - you can’t validate that an approach is a good fit for the problem until you understand what the constraints of the problem are.
IME there’s two points in a project lifecycle that have lots of code reviews, neither of which are amenable to the review style you describe:
Early on there’s going to be prototyping, where the boundaries of the problem aren’t well understood and code needs to be written to check assumptions / validate implementation strategies / wring out the ambiguity introduced by natural-language design discussions. Checking for “understanding of the problem domain” (etc) only slows down progress at this point as people argue about the experimental code’s outcome in the review comments, instead of getting the code merged in so they can run the experiment.
After some sort of rough consensus has been reached in the design doc and implementation has started, allowing code reviewers to object to PRs on design grounds is basically just allowing obstinate people to re-litigate their pet theory of how the design “should have” been decided. If the design calls for the component to use SQLite and Go, then review comments like “Why aren’t you using MongoDB here? The standard language for server-side projects is Ruby, justify your decision to use Go” are (at best) noise.
When I do code reviews, I generally check that the code is doing what the author intends. If there’s an error in the code that causes its actual behavior to diverge from author intent, or if the author seems to have misunderstood the design doc, then that’s worth blocking on. Or if the code is unclear enough that the intent can’t be discerned without reference to the design doc, I might ask for it to be restructured (or get more inline commentary added). Either way the code review doesn’t really involve a “conversation”.
Fascinating - I must have expressed myself poorly, because I don’t think those are counter-examples to my approach.
In the first case - I don’t see how “checking for ‘understanding of the problem domain’ only slows down progress at this point” is consistent with “I generally check that the code is doing what the author intends”. How can you know what the author intends (in order to check that the code does it) unless they have told you? Maybe we’re using the term “problem domain” differently - I don’t mean it in a Domain-Driven Design sense of “business unit”, but rather just the scope of the Code Review. Why is this change being made? What do we hope it’s going to achieve? If not trivially-obvious; how do we hope it will achieve that? These questions are just as relevant at the fast-and-scrappy beginning of a project as during maturity - just that the scope of the answers will be way smaller. My overall point is that it is not possible to usefully review code in isolation - it must be reviewed with a knowledge of the change-in-behaviour in the system that it is intended to bring about. Is the line const sum = 1 + 2 correct or not? Without knowing what sum is meant to hold, there is no way to tell (you can certainly recognize some forms of incorrectness in isolation - or at least invalidity or inconsistency - but recognizing correctness requires knowledge of what “correct” is)
In the second case - I fully agree that those kinds of comments are unproductive to landing a change.
They are, however, still helpful if made in earnest, because they indicate that the commenter wasn’t aware of the design doc - in which case, that noise can be negated by a link to the design doc, and hey presto! You have improved knowledge and context among the team; precisely the aim I intended.
If those comments are being made unearnestly, to sulkily complain about a particular choice, then yeah you certainly have a problem - but I suspect that kind of situation will lead to problems regardless of your code review process, that such a person will find ways to cause problems no matter what “proper process” says. I tend to find it easier and more productive to assume that my coworkers are earnestly trying to do their best to collaborate.
Either way the code review doesn’t really involve a “conversation”.
This is indeed the ideal to be striven for! A commit should stand alone, with a commit message describing the context and motivation for the change; a reviewer should be able to review the commit without needing to ask questions, because the context has already been recorded into the system. A conversation is the failure case that we fall back to if the metadata of commit message and comments has failed to provide that context.
I don’t see how “checking for ‘understanding of the problem domain’ only slows down progress at this point” is consistent with “I generally check that the code is doing what the author intends”. How can you know what the author intends (in order to check that the code does it) unless they have told you?
What the author intends might not be what someone who is an expert in the area would intend, and might not be what that same author would do after a few months of experimentation have enabled the problem to be better understood.
In a code review the only information you have to go on is the code itself (including comments) and the commit description (or PR description, if GitHub). This is enough to provide some basic summary of what the author is trying to do in that specific change, but it doesn’t necessarily indicate understanding in the more typical sense.
To be more concrete, here are three of my open-source changes:
https://github.com/bazelbuild/bazel/pull/16052 Bazel is a build system that downloads upstream source archives, and my change allows it to rename the files in those archives during the extraction process. I’m not anything near being an expert on source archive formats, or compression, or build systems – it’s just a quick-n-dirty patch to unblock a project I was working on (getting the Linux kernel to build on macOS).
https://github.com/rust-lang/rust/pull/117156 moved some unstable APIs around in the Rust standard library. It was related to a larger project (reworking the Rust’s APIs for UNIX socket ancillary data) that ended up stalling out due to lack of upstream review bandwidth. I’m nowhere near an expert on either the Rust standard library or low-level socket options in NetBSD, and if the reviewer had tried to “start a conversation” by asking me what other approaches I’d considered, I would probably have just closed it and moved on.
https://github.com/bazelbuild/rules_rust/pull/1600 is a case where the reviewer behaves as I understand you recommend – they dug deep into the change to question whether alternative approaches had been considered, spent a lot of time on “conversation”, and had me document in several ways the reasoning behind it. In the end I ended up closing that PR rather than try to continue pushing it forward.
In an employment setting there can be other motivations involved (i.e. want to afford rent => don’t get fired => don’t close PRs if the reviewer tries to turn them into a design review), but that’s only a motivation to endure frustration, not a justification to create it.
I don’t mean it in a Domain-Driven Design sense of “business unit”, but rather just the scope of the Code Review. Why is this change being made? What do we hope it’s going to achieve? If not trivially-obvious; how do we hope it will achieve that? These questions are just as relevant at the fast-and-scrappy beginning of a project as during maturity - just that the scope of the answers will be way smaller.
My overall point is that it is not possible to usefully review code in isolation - it must be reviewed with a knowledge of the change-in-behaviour in the system that it is intended to bring about.
I don’t think it’s necessary to go beyond that sort of plain commit message when doing experimental work early in a project’s lifecycle.
Documenting the reason that a particular approach was chosen, and the alternatives considered, are more likely to be part of a design document that is created after the initial experiments have identified which direction to go in. Or even after the project has launched, as part of the retrospective.
In the second case - I fully agree that those kinds of comments are unproductive to landing a change.
[…]
They are, however, still helpful if made in earnest, because they indicate that the commenter wasn’t aware of the design doc - in which case, that noise can be negated by a link to the design doc, and hey presto! You have improved knowledge and context among the team; precisely the aim I intended.
I’m struggling with this one, because all of the places I’ve worked would have the design doc be linked from the commit description, and sometimes from the comments (linking to specific sections with functional requirements).
If a code reviewer commented to ask if I’d investigated alternative designs, and I responded by linking to the design doc’s ~10 page “Appendix D: Alternatives considered” section, then that would probably be … the corporate equivalent of linking to Let Me Google That For You.
If those comments are being made unearnestly, to sulkily complain about a particular choice, then yeah you certainly have a problem - but I suspect that kind of situation will lead to problems regardless of your code review process, that such a person will find ways to cause problems no matter what “proper process” says. I tend to find it easier and more productive to assume that my coworkers are earnestly trying to do their best to collaborate.
To my displeasure, and occasionally great distress, when employed I have not had the option to choose who gets hired to be a coworker. Contributing to open-source is much easier because walking away is as simple as closing the PR.
I agree with nearly everything you’re saying, and - to me - you appear to agree with the point I’m trying to make, so I don’t know how we’re still managing to feel like we’re in opposition :)
What the author intends might not be what someone who is an expert in the area would intend, and might not be what that same author would do after a few months of experimentation have enabled the problem to be better understood.
That’s totally fine! I don’t claim that it should be. I simply claim that a reviewer must have an idea of what the author intended with a change at the time if they hope to review the code for how well it meets that intention.
The author’s intention might be out-of-line with business goals or higher-level technical designs, and that’s fine (not desirable, but a natural part of experimentation). Hopefully, as part of outlining their intention, that misalignment will come to light and be corrected. Even if it doesn’t, though - the reviewer still needs to know that intention in order to evaluate the change against that intention.
I don’t think it’s necessary to go beyond that sort of plain commit message when doing experimental work early in a project’s lifecycle.
I think this might be the root of the disagreement. I’m not advocating for conversations-for-conversation’s-sake, but rather saying that a reviewer needs to have this context in order to give a helpful review. Yes, the information you describe should be in a plain commit message. If it’s not - i.e. if “the only information you have to go on […] the code itself (including comments) and the commit description” is insufficient to provide a reviewer with the information they need in order to give a meaningful review - then the responsibility falls on the reviewer to request it.
That is - I agree with you about the amount of information that needs to be in (or “linked from”) a commit message. I am further stating that, if that information is lacking, a reviewer is not only justified, but obliged, to request it.
To my displeasure, and occasionally great distress, when employed I have not had the option to choose who gets hired to be a coworker
Fair, and relatable, but not a counter-example to my point. I, too, have coworkers who I dislike and who operate differently than I do. Nevertheless, I have still found it more productive when designing processes or setting standards for collaboration to start from a baseline assumption that collaborators earnestly want to collaborate, rather than starting from a perspective of defending myself against bad actors. The benefits of a smoother, simpler, process that works well for most people, most of the time, far outweigh the cost of setting up a “Zero Trust” process that is safe against bad actors, in my experience.
That said, I do think it’s notable that you’re talking about Open Source contributions rather than paid ones, where the incentives are naturally different. A contributor to Open Source is in a meaningful sense a volunteer with very little “skin in the game” for the ongoing maintenance of the system, whereas a paid employee has both a duty (via employment) and a personal obligation (making their own future lives easier) to think about maintainability. I empathize with your frustration about the requests for extra work in your third PR, and agree that that was probably unreasonable: an Open Source maintainer does still have a responsibility to ensure that that documentation exists, but - unlike a professional setting - they probably don’t have justification for requesting it from the author (who is an unpaid volunteer). They can ask for it, sure - but if they get pushback, as the actual owner or the code, the maintainer should probably step up and write it themselves, based on whatever context the author can provide.
This is just one side of the coin, the other side are exceptional juniors with so much knowledge because it’s so much more available these days. I’ve seen this quite a lot with Haiku. Some of our best contributors at the moment are very young.
I also see positive effects. LLMs can get junior devs quickly unstuck.
There are tools and languages with error messages that are just “computer says: no”, and you just have to know to pass some -dont-mess-it-up flag. Long-lived languages/libraries/OSes are full of legacy APIs footguns, and hacks required to make them work. Front-end web development requires keeping up with what is buggy in which browser, and which quirks are from IE/Netscape era and must not be touched. A lot of this stuff is not even worth knowing.
If it makes you feel any better, 10 years ago the big “problem” was people copying and pasting code directly from Stack Overflow (as others have mentioned in the comments here). Stack Overflow even made it the point of an April Fool’s joke in 2021. xkcd joked about “Stacksort” in the alt-text of “ineffective sorts” in 2013. As a random example I found, Scott Hanselman worried about a version of this problem in 2013 as well. “Remember that there was a time we programmed without copying our work” is pretty perennial advice.
Nostalgia – whether experienced or manufactured – has a tendency to cherry-pick good or bad examples depending on what you want the result to be. This might be a dated reference itself but I remember someone asking why Saturday Night Live wasn’t as good as it used to be, and the answer is that it hasn’t changed – there were always 9 bad skits for every one that was good, but you only remember the good ones.
Also, “junior engineers can’t code” is a genre of blog post older than timeitself.
The author of this piece wrote a month ago a piece called AI is Creating a Generation of Illiterate Programmers which enjoyed some success and even got featured by ThePrimeagen. In that article he mentioned that once ChatGPT was down he couldn’t code anymore and went on to say:
this is the new reality for software developers
No, it’s not. This may be true for a group of developers who are overusing these tools for mainstream languages and frameworks, but using your singular experience and claiming it to be the reality of us all is purely selection bias.
Now, the author writes a rather identical similar piece, lacking nuance and empathy, this time throwing under the bus a whole group of developers and the ones who struggle the most to find a job nowadays. But hey did you already subscribed to his newsletter? Did you know Elon Musk reads his articles? Or that he’s also working on an AI tool?
Agreed. One of the great features of LLMs is that you can ask it to elaborate, clarify, give examples, etc. If you like, you can learn much more from an LLM than from a single StackOverflow answer. The author mentions a particular detailed answer. But now every answer can be like that.
As always, some people put in the work, and others don’t.
I don’t have a great deal of experience with LLMs, but typically as soon as I ask them more than two follow-up questions, they begin hallucinating.
I was recently working with go-git and decided to ask ChatGPT how I could upload packs via SSH as the documentation didn’t make it seem immediately obvioys, it kept trying to use nonexistent HTTP transport functions over SSH even though I explicitly provided the entire godoc documentation for the SSH transports and packfiles. Granted, the documentation was lacking, but all I needed to do at last was to thoroughly digest the documentation, which ChatGPT is evidently unable to do. In another scenario, it also suggested ridiculous things like “Yes, you can use sendfile() for zero-copy transfers between a pipe and a socket”.
Anyhow, at least for the fields that I encounter, ChatGPT is way worse than asking on SO or just asking in an IRC channel.
Unfortunately, one needs to develop some kind of intuition of what an LLM is capable of. And with that, I don’t just mean LLMs in general and the types of questions they can handle, but also the different models and the way you feed them your documentation.
Some are better at technical questions than others, some are better at processing large text input. I prefer to use local models, but they can’t handle long conversations. I almost automatically start a new one after two follow-ups because I know Qwen will get confused. On the other hand, I know that if I were to use Gemini on NotebookLM, I could throw a whole book in it, and it would find the right part without breaking a sweat.
Using the right LLM-equipped tool is just as an important choice as the model itself. For understanding codebases, aider is the best in my experience. It adds the information git has about your project automatically to the context. For more general learning and research, I like to create Knowledge Stacks in Msty.
Fair point. If I’m ever slightly in doubt about its competence, I push back on LLMs vigorously. I include in my pre-instructions that I’m not looking for a “yes man”. (At the same time, I don’t want pedantic disagreement. Ah, the complexity of it all!)
I am currently doing an apprenticeship and what I am witnessing is, that there seems to be a change in motivation. Many don’t go into development, because they are curios or eager to learn and understand, but to make money and remote working.
I don’t want to say that this is bad, but just a change I’ve seen
It’s not a change. When I studied computer science, most of my peers were motivated by wanting a high grossing job and did not have a lot of ambition to “go beyond” during studies. That was around 2003.
That’s fine, it’s not a bad attitude, just one that needs to end up at the right place. A lot of them ended up really reliable IT workers that can really accurately do a job by spec, much better than I can.
Some others found their love for hacking later and do really absurd stuff now.
I think a lot of this is that you don’t need to be a hacker (https://en.wikipedia.org/wiki/Hacker_culture not cyber security hacker) to be a good software engineer, just like you don’t have to be a good software engineer to be a hacker. It’s more of a difference in culture rather than any sort of indicator as to job performance in my opinion.
Right, but there was an understanding that only good devs could make a high salary, so the money motivation worked. Today there’s the idea that there’s plenty of work LLM copyists and many people would do it for a living wage.
I’ve been observing that trend for at least five years. Maybe ten, though the interest in remote work has obviously changed. I think it’s due both to the influx of money and the lowering of various barriers to entry. Newcomers no longer have to climb the kinds of learning cliffs only dyed-in-the-wool geeks will attempt.
I’m 33, and I started coding when I was 11/12, in C — git wasn’t even a thing. I haven’t touched copilot or anything like that up to this day because I just enjoy writing code and understanding what I do, and also, I’m pretty sure there’s a « behind the scene » issue with copylefts that those models violate daily to train their data on and then sell via license.
The headline could use some qualifiers. Badly. (and I’m not touching what “actually code” is supposed to mean)
That said, I asked my llama the same +/Number/parseFloat question and its response, while a bit contradictory, seemed at least as good as the SO reply. Was it lifted from SO? Well, my llama didn’t mention anything like that … and on the bright side, I only have to wait a couple minutes for a lengthy reply. (and it’ll never close my question as duplicate)
Does it really matter if I pull up my 1990s search engine or if I pull it out of the fair-use-laundromat ? It’s more or less the same content, private messaging aside
Anywhere but private messaging. Blog posts, Lobsters/HN, good old PHP forums, MDN, public documentation, published APIs, maybe public lecture notes, perhaps copyrighted technical books, open source code, Github issues, Gists, ya know … whatever they feed the thing
But maybe SO is the only place on the internet people ask fresh questions. Maybe
It feels like this article is so focused on the AI aspect that it misses the most important point. Whether the junior developer uses AI or StackOverflow, the important thing for the junior developer to learn is that if they want to get better they must NOT use the code it provides without first fully understanding HOW it works.
Sure, it’s important to understand, but even more important in my opinion is to practice. I would bet that many programmers understand how binary search or quick sort work, but couldn’t implement them (it’s very easy to get the bounds wrong.) Practice is the essential ingredient for bridging the gap between theory and practice, for improving, for gaining experience, and for maintaining competence. Unfortunately, practice is not glamorous nor fun; it requires time and it requires doing the same boring things over, and over, and over.
LLMs will exploit the natural tendency of people to be lazy and many programmers will deprive themselves of the opportunity to get things wrong, to struggle on their own to fix the problems, and to grow. We might start seeing a lot of programmers who seem very productive on the happy path, but become completely helpless when things go slightly wrong.
I don’t think I agree. Sure, if one uses an LLM to generate a few lines at a time of code then even if one stops to make sure one understands the syntax, one’s ability to write syntax from scratch won’t develop as strongly because one isn’t practicing that as much. But I don’t think of that as a loss. When I started regularly using IDEs that used syntax highlighting and bracket-matching I’m sure my ability to read over a block of code and match parentheses by eye began to atrophy – I used to spend LOTS of time working on that skill. But honestly, it’s not much of a loss… the IDE does a way better job. I think auto-generating snippets of well-understood code is similar to that: it loses a skill, but not one that was essential.
What I wouldn’t give up is an understanding of how the code WORKS – that is always relevant to writing good code.
My suggestion is that you treat code they generated from a LLM the same way you would treat code an older developer copied off StackExchange or an even older developer copied from a textbook (remember those?). You would tell them “It’s great to get an example from a source like that, but you really should not use it until you UNDERSTAND everything it is doing. This serves two purposes: it makes you a better programmer (by learning something) and it also ensures that this IS the correct thing to be doing for your specific use case.”
As a junior dev myself, I’d stand in the “LLMs are only useful in very particular conditions” crowd.
I’ve never seen an LLM understand complex concurrent systems.
I’ve tried GitHub copilot’s nvim integration for a while. Perhaps I’m using it wrong, but it feels like a glorified snippets generator. Somewhat helpful to complete tedeous things like error handling in Go, I suppose.
By far the most useful way I’ve used LLMs in programming: I’d give it a large chunk of code from multiple source files and let it identify duplicate code that I should refactor out. I’m not sure whether this is useful to others though, in particular because I have a more ad-hoc approach to functions.
The core issue is still, as always, people copying code that they don’t understand. I don’t think the complaints about junior devs have changed, although by the time I started programming nobody was really actively complaining about junior devs copying blindly from SO anymore, as people seemed more or less aware that other people’s code likely doesn’t apply to your situation. But perhaps people have this illusion that using LLMs definitely generate relevant and correct code if you give it sufficient context.
When I was learning how to code I copied code from a Basic programming manual with only a bare understanding of what was going on, or even English. “syntax” in Syntax Error, ‘function” in “Illegal Function Call” or all words in “Type Mismatch” were errors Basic would throw at me. I had no idea what these words meant; now I do. It didn’t help that I was also a kid. I wrote in my blog about this
The internet provided a very different new path to learning how to program not available to me until much later.
In this era of LLMs there are going to be new paths to learning how to program too. Are they better or worse paths? I think the jury is still thoroughly out on that one; this topic is only being explored right now. I know a lot of people have a strong opinion that it’s definitely bad but who knows what learning trajectories are being figured out by intelligent, motivated people right now?
I see a lot of these sort of arguments that from where I am sitting mostly revolving around the difference in culture between hackers and software engineers in general, as well as the prolification of software on our day to day lives in both more and less important areas from before. Straight from the wikipedia entry on hacker culture you have:
those who enjoy—often in collective effort—the intellectual challenge of creatively overcoming the limitations of software systems or electronic hardware (mostly digital electronics), to achieve novel and clever outcomes.”
I don’t think you have to be in this sub-culture to make a good software engineer. Sure maybe the proportion of hackers who are good software devs is higher than normal, but its not an exlusive or (XOR) relationship.
There are those who just want to take a paycheck and that’s ok and there is also a large difference between a junior developer who is in the pipeline to work on kernel code, space-rocket code etc…. than on a CRUD web-app.
It’s like comparing the engineers who work on the space shuttle, to a bentley, to a mercedes, to a bike mechanic. I don’t need as good a bike mechanic as I do car mechanic, as the consequences of failure are less costly. It doesn’t make them any less a mechanic, just that the bike mechanic is likely to focus on quantity over quality, as opposed to the space shuttle mechanic. Different priorities for different requirements.
The What Can We Do section takeaways (learning mindset, find a discussion community, use code reviews for team communication, and build things from scratch), could have been written after StackOverflow became prominent. Or after using internet search engines became prominent. That’s not a knock on the article (to the contrary, durable advice is good), but just to say that a plea for fundamentals doesn’t require the arm-waving of Yet Another Sliderule-to-Calculator moral panic. Curious and intrinsically motivated junior devs will one day become competent senior devs, and the others will become… um… hopefully something positive; and that’s applicable to past, present, and future, regardless of the coding aid of the day.
People writing articles reminiscing about when junior devs just copy-and-pasted code from Stack Overflow is making me feel old.
Same here…kids those days complaining about kids these days. :)
I was telling someone recently about what “browsing the literature” was like when I was working on compilers at Apple in 1990: going to the internal library down the road to read some ACM proceedings, copying references from the bibliography in the paper, asking the librarian to call Stanford to ask them to photocopy the papers Apple didn’t have, and waiting for the copies to show up in the interoffice mail. That was just 35 years ago! And getting answers to your questions on demand…if you were lucky enough to know somebody who might know the answer, you used the phone. (The lucky few in the know could try Usenet.)
I think the op is literally 23
[Comment removed by author]
Same – I remember when Stack Overflow was brand new[0], and it was introduced as some mixture of (1) a better expertsexchange and (2) a pseudo-wiki to gather and summarize knowledge that was otherwise spread across personal blogs and academic homepages.
At some point it transformed from a Q&A site aimed at skilled professionals into a code snippets repository for people new to the industry. Nowadays whenever I’m doing something that leads me to an SO question, usually the question is exactly what I want answered but it’s locked as a duplicate of some completely unrelated topic.
[0] Fun(?) story: I created the first Wikipedia article about Stack Overflow, but it got speedy-deleted by a Wikipedia administrator after about ten minutes. A lot of the early SO content was written by people who wanted to contribute computer science information to the public square, but didn’t want to deal with Wikipedia’s opaque deletionist bureaucracy.
Also, if you really want to feel old, write some C++ with a whole class of new hires who pronounce
#includeas “hashtag include”…Out of curiosity, how do you pronounce it? :)
I took a couple classes in the pre-hashtag era, but I don’t recall if/how it was referred to verbally.
I suspect I’d never heard it called anything but a number/pound sign when I took those classes. I’m pretty sure I didn’t know octothorp(e) until years later, and I doubt I would have thought to call it a “hash” until encountering shebangs or hashtags years later.
When I was in school the
#character was universally pronounced “pound”, to the point that both students and professors would (jokingly) refer to C# as “C Pound”.Oddly it wasn’t pronounced at all when naming C processor directives:
#includewas simply “include”. And the justification for that is “the leading pound is silent, like the leading ‘p’ in pterodactyl”. To this day I am not sure whether that professor was serious.This doesn’t match my experience working with junior developers. It just seems to be that new folks always have a lot to learn, and the amount of knowledge that’s needed to be productive is going up over time.
There are reasons to be concerned about the effect that tools have on the way folks learn, and how raising the level of abstraction reduces the need to learn lower-level details, but this post just seems to be overstating things significantly.
Our last intern was excellent. He dug into things, was super technical and effective, and never pushed code he didn’t understand. AFAIK he was fine to use AI but did so appropriately.
What’s described here sounds like a lot of engineers I’ve known, especially back in college. A lot of young, new engineers don’t know why their code works and don’t have the ability to investigate why either - it’s pretty damn common.
I suspect AI will make those engineers worse but, frankly, they were never very good and I’ve never had to work with anyone that bad/ they would have been fired eventually. They aren’t going to be bad because of AI they’re going to be bad because they never put the work in.
I remember posting a lot on C++. I had some high ranked questions about various things like lvalues and xvalues etc. I was lucky to get a bombard of idiots and days later one person going “I think others are misunderstanding” and expertly weighing in with real insights if I was very lucky, most of the time I got little response or “why are you doing that?” etc. SO was extremely limited and I quickly migrated into Slacks with people I could ask directly, which was infinitely more effective for learning but few benefited from my conversations. This is just the nature of a world where “document stores” aren’t ideal for conversations but conversations are ideal for learning. AI is actually good at indexing conversations so it may help here.
Regarding “What an we do?” IMO/IME the good developers who were going to be good without AI are already just doing these things.
Oof, this (the fact that this is considered “different”) hurts. This has always been how I’ve thought Code Review should run - “checking the code for correctness” is almost secondary to “ensuring that understanding of the problem domain, and the reasoning behind the chosen solution, is spread around the team”. Indeed, you can’t do the former without the latter, because Code Only Says What It Does - you can’t validate that an approach is a good fit for the problem until you understand what the constraints of the problem are.
IME there’s two points in a project lifecycle that have lots of code reviews, neither of which are amenable to the review style you describe:
Early on there’s going to be prototyping, where the boundaries of the problem aren’t well understood and code needs to be written to check assumptions / validate implementation strategies / wring out the ambiguity introduced by natural-language design discussions. Checking for “understanding of the problem domain” (etc) only slows down progress at this point as people argue about the experimental code’s outcome in the review comments, instead of getting the code merged in so they can run the experiment.
After some sort of rough consensus has been reached in the design doc and implementation has started, allowing code reviewers to object to PRs on design grounds is basically just allowing obstinate people to re-litigate their pet theory of how the design “should have” been decided. If the design calls for the component to use SQLite and Go, then review comments like “Why aren’t you using MongoDB here? The standard language for server-side projects is Ruby, justify your decision to use Go” are (at best) noise.
When I do code reviews, I generally check that the code is doing what the author intends. If there’s an error in the code that causes its actual behavior to diverge from author intent, or if the author seems to have misunderstood the design doc, then that’s worth blocking on. Or if the code is unclear enough that the intent can’t be discerned without reference to the design doc, I might ask for it to be restructured (or get more inline commentary added). Either way the code review doesn’t really involve a “conversation”.
Fascinating - I must have expressed myself poorly, because I don’t think those are counter-examples to my approach.
In the first case - I don’t see how “checking for ‘understanding of the problem domain’ only slows down progress at this point” is consistent with “I generally check that the code is doing what the author intends”. How can you know what the author intends (in order to check that the code does it) unless they have told you? Maybe we’re using the term “problem domain” differently - I don’t mean it in a Domain-Driven Design sense of “business unit”, but rather just the scope of the Code Review. Why is this change being made? What do we hope it’s going to achieve? If not trivially-obvious; how do we hope it will achieve that? These questions are just as relevant at the fast-and-scrappy beginning of a project as during maturity - just that the scope of the answers will be way smaller. My overall point is that it is not possible to usefully review code in isolation - it must be reviewed with a knowledge of the change-in-behaviour in the system that it is intended to bring about. Is the line
const sum = 1 + 2correct or not? Without knowing whatsumis meant to hold, there is no way to tell (you can certainly recognize some forms of incorrectness in isolation - or at least invalidity or inconsistency - but recognizing correctness requires knowledge of what “correct” is)In the second case - I fully agree that those kinds of comments are unproductive to landing a change.
This is indeed the ideal to be striven for! A commit should stand alone, with a commit message describing the context and motivation for the change; a reviewer should be able to review the commit without needing to ask questions, because the context has already been recorded into the system. A conversation is the failure case that we fall back to if the metadata of commit message and comments has failed to provide that context.
What the author intends might not be what someone who is an expert in the area would intend, and might not be what that same author would do after a few months of experimentation have enabled the problem to be better understood.
In a code review the only information you have to go on is the code itself (including comments) and the commit description (or PR description, if GitHub). This is enough to provide some basic summary of what the author is trying to do in that specific change, but it doesn’t necessarily indicate understanding in the more typical sense.
To be more concrete, here are three of my open-source changes:
https://github.com/bazelbuild/bazel/pull/16052 Bazel is a build system that downloads upstream source archives, and my change allows it to rename the files in those archives during the extraction process. I’m not anything near being an expert on source archive formats, or compression, or build systems – it’s just a quick-n-dirty patch to unblock a project I was working on (getting the Linux kernel to build on macOS).
https://github.com/rust-lang/rust/pull/117156 moved some unstable APIs around in the Rust standard library. It was related to a larger project (reworking the Rust’s APIs for UNIX socket ancillary data) that ended up stalling out due to lack of upstream review bandwidth. I’m nowhere near an expert on either the Rust standard library or low-level socket options in NetBSD, and if the reviewer had tried to “start a conversation” by asking me what other approaches I’d considered, I would probably have just closed it and moved on.
https://github.com/bazelbuild/rules_rust/pull/1600 is a case where the reviewer behaves as I understand you recommend – they dug deep into the change to question whether alternative approaches had been considered, spent a lot of time on “conversation”, and had me document in several ways the reasoning behind it. In the end I ended up closing that PR rather than try to continue pushing it forward.
In an employment setting there can be other motivations involved (i.e. want to afford rent => don’t get fired => don’t close PRs if the reviewer tries to turn them into a design review), but that’s only a motivation to endure frustration, not a justification to create it.
These seem like things that would be part of the change description, either directly (like in the Linux commits) or linked to in some sort of ticket tracker (examples: https://github.com/NationalSecurityAgency/ghidra/pull/4546, https://github.com/rust-lang/rust/pull/102445).
I don’t think it’s necessary to go beyond that sort of plain commit message when doing experimental work early in a project’s lifecycle.
Documenting the reason that a particular approach was chosen, and the alternatives considered, are more likely to be part of a design document that is created after the initial experiments have identified which direction to go in. Or even after the project has launched, as part of the retrospective.
I’m struggling with this one, because all of the places I’ve worked would have the design doc be linked from the commit description, and sometimes from the comments (linking to specific sections with functional requirements).
If a code reviewer commented to ask if I’d investigated alternative designs, and I responded by linking to the design doc’s ~10 page “Appendix D: Alternatives considered” section, then that would probably be … the corporate equivalent of linking to Let Me Google That For You.
To my displeasure, and occasionally great distress, when employed I have not had the option to choose who gets hired to be a coworker. Contributing to open-source is much easier because walking away is as simple as closing the PR.
I agree with nearly everything you’re saying, and - to me - you appear to agree with the point I’m trying to make, so I don’t know how we’re still managing to feel like we’re in opposition :)
That’s totally fine! I don’t claim that it should be. I simply claim that a reviewer must have an idea of what the author intended with a change at the time if they hope to review the code for how well it meets that intention.
The author’s intention might be out-of-line with business goals or higher-level technical designs, and that’s fine (not desirable, but a natural part of experimentation). Hopefully, as part of outlining their intention, that misalignment will come to light and be corrected. Even if it doesn’t, though - the reviewer still needs to know that intention in order to evaluate the change against that intention.
I think this might be the root of the disagreement. I’m not advocating for conversations-for-conversation’s-sake, but rather saying that a reviewer needs to have this context in order to give a helpful review. Yes, the information you describe should be in a plain commit message. If it’s not - i.e. if “the only information you have to go on […] the code itself (including comments) and the commit description” is insufficient to provide a reviewer with the information they need in order to give a meaningful review - then the responsibility falls on the reviewer to request it.
That is - I agree with you about the amount of information that needs to be in (or “linked from”) a commit message. I am further stating that, if that information is lacking, a reviewer is not only justified, but obliged, to request it.
Fair, and relatable, but not a counter-example to my point. I, too, have coworkers who I dislike and who operate differently than I do. Nevertheless, I have still found it more productive when designing processes or setting standards for collaboration to start from a baseline assumption that collaborators earnestly want to collaborate, rather than starting from a perspective of defending myself against bad actors. The benefits of a smoother, simpler, process that works well for most people, most of the time, far outweigh the cost of setting up a “Zero Trust” process that is safe against bad actors, in my experience.
That said, I do think it’s notable that you’re talking about Open Source contributions rather than paid ones, where the incentives are naturally different. A contributor to Open Source is in a meaningful sense a volunteer with very little “skin in the game” for the ongoing maintenance of the system, whereas a paid employee has both a duty (via employment) and a personal obligation (making their own future lives easier) to think about maintainability. I empathize with your frustration about the requests for extra work in your third PR, and agree that that was probably unreasonable: an Open Source maintainer does still have a responsibility to ensure that that documentation exists, but - unlike a professional setting - they probably don’t have justification for requesting it from the author (who is an unpaid volunteer). They can ask for it, sure - but if they get pushback, as the actual owner or the code, the maintainer should probably step up and write it themselves, based on whatever context the author can provide.
This is just one side of the coin, the other side are exceptional juniors with so much knowledge because it’s so much more available these days. I’ve seen this quite a lot with Haiku. Some of our best contributors at the moment are very young.
I also see positive effects. LLMs can get junior devs quickly unstuck.
There are tools and languages with error messages that are just “computer says: no”, and you just have to know to pass some
-dont-mess-it-upflag. Long-lived languages/libraries/OSes are full of legacy APIs footguns, and hacks required to make them work. Front-end web development requires keeping up with what is buggy in which browser, and which quirks are from IE/Netscape era and must not be touched. A lot of this stuff is not even worth knowing.I understand the concern, but has real “recorded music will cause us to evolve to have no vocal chords” vibes
I’m 24 years old - I am NOT old enough to reminisce about the good old days before kids got everything handed to them.
And yet here I am.
If it makes you feel any better, 10 years ago the big “problem” was people copying and pasting code directly from Stack Overflow (as others have mentioned in the comments here). Stack Overflow even made it the point of an April Fool’s joke in 2021. xkcd joked about “Stacksort” in the alt-text of “ineffective sorts” in 2013. As a random example I found, Scott Hanselman worried about a version of this problem in 2013 as well. “Remember that there was a time we programmed without copying our work” is pretty perennial advice.
Nostalgia – whether experienced or manufactured – has a tendency to cherry-pick good or bad examples depending on what you want the result to be. This might be a dated reference itself but I remember someone asking why Saturday Night Live wasn’t as good as it used to be, and the answer is that it hasn’t changed – there were always 9 bad skits for every one that was good, but you only remember the good ones.
Also, “junior engineers can’t code” is a genre of blog post older than time itself.
The author of this piece wrote a month ago a piece called AI is Creating a Generation of Illiterate Programmers which enjoyed some success and even got featured by ThePrimeagen. In that article he mentioned that once ChatGPT was down he couldn’t code anymore and went on to say:
No, it’s not. This may be true for a group of developers who are overusing these tools for mainstream languages and frameworks, but using your singular experience and claiming it to be the reality of us all is purely selection bias.
Now, the author writes a rather
identicalsimilar piece, lacking nuance and empathy, this time throwing under the bus a whole group of developers and the ones who struggle the most to find a job nowadays. But hey did you already subscribed to his newsletter? Did you know Elon Musk reads his articles? Or that he’s also working on an AI tool?I’m glad I’m not the only one who raised eyebrows at those things…
If used in certain ways, sure.
To be fair, there are people who also copy-pasta from Stack Overflow.
If you use your tools mindfully, actively seek understanding, and pay attention to self-improvement, language models can serve a useful role.
Agreed. One of the great features of LLMs is that you can ask it to elaborate, clarify, give examples, etc. If you like, you can learn much more from an LLM than from a single StackOverflow answer. The author mentions a particular detailed answer. But now every answer can be like that.
As always, some people put in the work, and others don’t.
I don’t have a great deal of experience with LLMs, but typically as soon as I ask them more than two follow-up questions, they begin hallucinating.
I was recently working with go-git and decided to ask ChatGPT how I could upload packs via SSH as the documentation didn’t make it seem immediately obvioys, it kept trying to use nonexistent HTTP transport functions over SSH even though I explicitly provided the entire godoc documentation for the SSH transports and packfiles. Granted, the documentation was lacking, but all I needed to do at last was to thoroughly digest the documentation, which ChatGPT is evidently unable to do. In another scenario, it also suggested ridiculous things like “Yes, you can use
sendfile()for zero-copy transfers between a pipe and a socket”.Anyhow, at least for the fields that I encounter, ChatGPT is way worse than asking on SO or just asking in an IRC channel.
Unfortunately, one needs to develop some kind of intuition of what an LLM is capable of. And with that, I don’t just mean LLMs in general and the types of questions they can handle, but also the different models and the way you feed them your documentation.
Some are better at technical questions than others, some are better at processing large text input. I prefer to use local models, but they can’t handle long conversations. I almost automatically start a new one after two follow-ups because I know Qwen will get confused. On the other hand, I know that if I were to use Gemini on NotebookLM, I could throw a whole book in it, and it would find the right part without breaking a sweat.
Using the right LLM-equipped tool is just as an important choice as the model itself. For understanding codebases, aider is the best in my experience. It adds the information git has about your project automatically to the context. For more general learning and research, I like to create Knowledge Stacks in Msty.
As a test, I cloned the
go-gitrepository and asked aider your question. It pointed me to this file: https://github.com/go-git/go-git/blob/main/plumbing/transport/ssh/upload_pack_test.go and then proceeded to write come code.I am not familiar with go-git, but would that have been a helpful answer to you?
It’s much closer than what I got, but the actual solution is slightly more complex. (Though I stopped using go-git for remote operations due to bugs)
Fair point. If I’m ever slightly in doubt about its competence, I push back on LLMs vigorously. I include in my pre-instructions that I’m not looking for a “yes man”. (At the same time, I don’t want pedantic disagreement. Ah, the complexity of it all!)
I know Juniors who can code but can’t get jobs
Haha me :)
I am currently doing an apprenticeship and what I am witnessing is, that there seems to be a change in motivation. Many don’t go into development, because they are curios or eager to learn and understand, but to make money and remote working. I don’t want to say that this is bad, but just a change I’ve seen
It’s not a change. When I studied computer science, most of my peers were motivated by wanting a high grossing job and did not have a lot of ambition to “go beyond” during studies. That was around 2003.
That’s fine, it’s not a bad attitude, just one that needs to end up at the right place. A lot of them ended up really reliable IT workers that can really accurately do a job by spec, much better than I can.
Some others found their love for hacking later and do really absurd stuff now.
I think a lot of this is that you don’t need to be a hacker (https://en.wikipedia.org/wiki/Hacker_culture not cyber security hacker) to be a good software engineer, just like you don’t have to be a good software engineer to be a hacker. It’s more of a difference in culture rather than any sort of indicator as to job performance in my opinion.
Right, but there was an understanding that only good devs could make a high salary, so the money motivation worked. Today there’s the idea that there’s plenty of work LLM copyists and many people would do it for a living wage.
I’ve been observing that trend for at least five years. Maybe ten, though the interest in remote work has obviously changed. I think it’s due both to the influx of money and the lowering of various barriers to entry. Newcomers no longer have to climb the kinds of learning cliffs only dyed-in-the-wool geeks will attempt.
I’m 33, and I started coding when I was 11/12, in C — git wasn’t even a thing. I haven’t touched copilot or anything like that up to this day because I just enjoy writing code and understanding what I do, and also, I’m pretty sure there’s a « behind the scene » issue with copylefts that those models violate daily to train their data on and then sell via license.
They certainly use all open-source (including GPL’ed) code they can get their hands on, but I unfortunately doubt that’s going to matter much.
The headline could use some qualifiers. Badly. (and I’m not touching what “actually code” is supposed to mean)
That said, I asked my llama the same +/Number/parseFloat question and its response, while a bit contradictory, seemed at least as good as the SO reply. Was it lifted from SO? Well, my llama didn’t mention anything like that … and on the bright side, I only have to wait a couple minutes for a lengthy reply. (and it’ll never close my question as duplicate)
Does it really matter if I pull up my 1990s search engine or if I pull it out of the fair-use-laundromat ? It’s more or less the same content, private messaging aside
If your llama got it from SO, and your llama kills SO, where will it get the answers for new questions going forward?
If it isn’t able to produce the answer you’ll go to SO, Discord, Slack, etc, at which point llama can get it from those.
Anywhere but private messaging. Blog posts, Lobsters/HN, good old PHP forums, MDN, public documentation, published APIs, maybe public lecture notes, perhaps copyrighted technical books, open source code, Github issues, Gists, ya know … whatever they feed the thing
But maybe SO is the only place on the internet people ask fresh questions. Maybe
Funnily, they haven’t used LLM in this case.
It feels like this article is so focused on the AI aspect that it misses the most important point. Whether the junior developer uses AI or StackOverflow, the important thing for the junior developer to learn is that if they want to get better they must NOT use the code it provides without first fully understanding HOW it works.
Sure, it’s important to understand, but even more important in my opinion is to practice. I would bet that many programmers understand how binary search or quick sort work, but couldn’t implement them (it’s very easy to get the bounds wrong.) Practice is the essential ingredient for bridging the gap between theory and practice, for improving, for gaining experience, and for maintaining competence. Unfortunately, practice is not glamorous nor fun; it requires time and it requires doing the same boring things over, and over, and over.
LLMs will exploit the natural tendency of people to be lazy and many programmers will deprive themselves of the opportunity to get things wrong, to struggle on their own to fix the problems, and to grow. We might start seeing a lot of programmers who seem very productive on the happy path, but become completely helpless when things go slightly wrong.
Interesting perspective.
I don’t think I agree. Sure, if one uses an LLM to generate a few lines at a time of code then even if one stops to make sure one understands the syntax, one’s ability to write syntax from scratch won’t develop as strongly because one isn’t practicing that as much. But I don’t think of that as a loss. When I started regularly using IDEs that used syntax highlighting and bracket-matching I’m sure my ability to read over a block of code and match parentheses by eye began to atrophy – I used to spend LOTS of time working on that skill. But honestly, it’s not much of a loss… the IDE does a way better job. I think auto-generating snippets of well-understood code is similar to that: it loses a skill, but not one that was essential.
What I wouldn’t give up is an understanding of how the code WORKS – that is always relevant to writing good code.
The main question I have is how to mentor younger people when my process is so different than their process?
My suggestion is that you treat code they generated from a LLM the same way you would treat code an older developer copied off StackExchange or an even older developer copied from a textbook (remember those?). You would tell them “It’s great to get an example from a source like that, but you really should not use it until you UNDERSTAND everything it is doing. This serves two purposes: it makes you a better programmer (by learning something) and it also ensures that this IS the correct thing to be doing for your specific use case.”
As a junior dev myself, I’d stand in the “LLMs are only useful in very particular conditions” crowd.
The core issue is still, as always, people copying code that they don’t understand. I don’t think the complaints about junior devs have changed, although by the time I started programming nobody was really actively complaining about junior devs copying blindly from SO anymore, as people seemed more or less aware that other people’s code likely doesn’t apply to your situation. But perhaps people have this illusion that using LLMs definitely generate relevant and correct code if you give it sufficient context.
When I was learning how to code I copied code from a Basic programming manual with only a bare understanding of what was going on, or even English. “syntax” in Syntax Error, ‘function” in “Illegal Function Call” or all words in “Type Mismatch” were errors Basic would throw at me. I had no idea what these words meant; now I do. It didn’t help that I was also a kid. I wrote in my blog about this
The internet provided a very different new path to learning how to program not available to me until much later.
In this era of LLMs there are going to be new paths to learning how to program too. Are they better or worse paths? I think the jury is still thoroughly out on that one; this topic is only being explored right now. I know a lot of people have a strong opinion that it’s definitely bad but who knows what learning trajectories are being figured out by intelligent, motivated people right now?
I see a lot of these sort of arguments that from where I am sitting mostly revolving around the difference in culture between hackers and software engineers in general, as well as the prolification of software on our day to day lives in both more and less important areas from before. Straight from the wikipedia entry on hacker culture you have:
I don’t think you have to be in this sub-culture to make a good software engineer. Sure maybe the proportion of hackers who are good software devs is higher than normal, but its not an exlusive or (XOR) relationship.
There are those who just want to take a paycheck and that’s ok and there is also a large difference between a junior developer who is in the pipeline to work on kernel code, space-rocket code etc…. than on a CRUD web-app.
It’s like comparing the engineers who work on the space shuttle, to a bentley, to a mercedes, to a bike mechanic. I don’t need as good a bike mechanic as I do car mechanic, as the consequences of failure are less costly. It doesn’t make them any less a mechanic, just that the bike mechanic is likely to focus on quantity over quality, as opposed to the space shuttle mechanic. Different priorities for different requirements.
The What Can We Do section takeaways (learning mindset, find a discussion community, use code reviews for team communication, and build things from scratch), could have been written after StackOverflow became prominent. Or after using internet search engines became prominent. That’s not a knock on the article (to the contrary, durable advice is good), but just to say that a plea for fundamentals doesn’t require the arm-waving of Yet Another Sliderule-to-Calculator moral panic. Curious and intrinsically motivated junior devs will one day become competent senior devs, and the others will become… um… hopefully something positive; and that’s applicable to past, present, and future, regardless of the coding aid of the day.