huge +1 to this article. in the same way that it’s a huge step up to go from print debugging to an interactive debugger, it’s magical to go from a general purpose debugger to a bespoke one.
I feel like this is isolated demands for environmentalism. I’ve never heard of anyone say that the taylor swift eras tour is “incinerating the planet for fun”, even though it plausibly has resulted in a similar order of magnitude of CO2 emissions and fun. How many emissions have come from all nascar races? If you look at the energy consumption of youtube’s data centers, I bet it’s a lot.
A Taylor Swift concert (or a Rolling Stones one, or any other big act) is a huge logistical enterprise. Potentially hundreds of thousands of people are relying on the star showing up on time and in the right place. Having them use a private jet is not just an affectation, it’s a sound business decision compared to relying on commercial flights.
Edit speaking about carbon emissions, in the recent TS concert in Stockholm there were audience members who literaly flew in from the US because the tickets were cheaper compared to any American venue.
I saw an estimate for the CO2 impact of the NeurIPS 2024 conference - given the thousands of people who flew in for the event - which suggested it may have had significantly more CO2 impact than a training run for one of the larger models.
Llama 3.3 70B reported using 11,390 tons CO2eq for training and the model itself runs on my laptop.
As far as I can tell, 11,390 tons CO2eq is equivalent to twenty fully loaded passenger jets from NYC to London.
That number is for just that one model though - it doesn’t include many other research training runs Meta’s AI team would have performed along the way - or the impact of many AI labs competing with each other.
I still think it’s a useful point of comparison for considering the scale of the CO2 impact of this technology.
There are papers out there talking about 10% of total emissions are for training, and 90% for inference (and the longer you wait, the more training cost is dwarfed). Also the inference cost could vary from 1 to 100 orders of magnitude apparently, so that would make a lot of twenty-fully-loaded-nyc-london-flights.
Like I said, the Llama 3.3 70B model runs on my laptop. It’s hard for me too get too worried about the CO2 impact of software that runs on the laptop that’s already sitting on my desk - I guess it runs a little hotter?
I also have trouble imagining that an LLM running on a server in a data center where it’s shared with hundreds of thousands of other users is less efficient than one that runs on my own laptop and serves only me.
I’m using Cursor and I’m finding this to be an incredibly charitable take an how AI can help a programmer. Maybe I’m just bad at using these tools?
I know people who say they have generated tens of thousands of lines of code with LLMs that they are happily using but I guess the old adage comes in: “We can write tons of code for you if all the code is allowed to be shit.”
Have you tried prompting non-IDE-integrated LLMs directly?
I do most of my AI-assisted code work directly in Claude or ChatGPT, providing instructions for what I want built and then copying out the bits that are useful.
Personally I like the chat-and-copy approach as well, although there are decent IDE-integrated options for that like Zed or NeoVim’s CodeCompanion plugin so that you can more easily share file context and avoid even needing to switch windows.
Although for web UI specifically, it’s hard to beat the iterative speed of working with Claude Artifacts and seeing the UIs render immediately next to the chat view…
huge +1 to this article. in the same way that it’s a huge step up to go from print debugging to an interactive debugger, it’s magical to go from a general purpose debugger to a bespoke one.
This is actually the first use of LLMs in programming I’ve seen and thought, “oh, neat”.
incinerating the planet for fun.
I feel like this is isolated demands for environmentalism. I’ve never heard of anyone say that the taylor swift eras tour is “incinerating the planet for fun”, even though it plausibly has resulted in a similar order of magnitude of CO2 emissions and fun. How many emissions have come from all nascar races? If you look at the energy consumption of youtube’s data centers, I bet it’s a lot.
swift burning entire forests is a common meme on instagram https://knowyourmeme.com/memes/events/taylor-swifts-private-jet-emissions-controversy
“Private jets are bad” is not a hot take. “Going to a concert is bad” is a bizarre take.
A Taylor Swift concert (or a Rolling Stones one, or any other big act) is a huge logistical enterprise. Potentially hundreds of thousands of people are relying on the star showing up on time and in the right place. Having them use a private jet is not just an affectation, it’s a sound business decision compared to relying on commercial flights.
Edit speaking about carbon emissions, in the recent TS concert in Stockholm there were audience members who literaly flew in from the US because the tickets were cheaper compared to any American venue.
I saw an estimate for the CO2 impact of the NeurIPS 2024 conference - given the thousands of people who flew in for the event - which suggested it may have had significantly more CO2 impact than a training run for one of the larger models.
you may underestimate the footprint. There has been an editorial in the CACM https://cacm.acm.org/opinion/genai-giga-terawatt-hours-and-gigatons-of-co2/ that does some math and demands energy to be considered with every IT decision.
Llama 3.3 70B reported using 11,390 tons CO2eq for training and the model itself runs on my laptop.
As far as I can tell, 11,390 tons CO2eq is equivalent to twenty fully loaded passenger jets from NYC to London.
That number is for just that one model though - it doesn’t include many other research training runs Meta’s AI team would have performed along the way - or the impact of many AI labs competing with each other.
I still think it’s a useful point of comparison for considering the scale of the CO2 impact of this technology.
There are papers out there talking about 10% of total emissions are for training, and 90% for inference (and the longer you wait, the more training cost is dwarfed). Also the inference cost could vary from 1 to 100 orders of magnitude apparently, so that would make a lot of twenty-fully-loaded-nyc-london-flights.
Like I said, the Llama 3.3 70B model runs on my laptop. It’s hard for me too get too worried about the CO2 impact of software that runs on the laptop that’s already sitting on my desk - I guess it runs a little hotter?
I also have trouble imagining that an LLM running on a server in a data center where it’s shared with hundreds of thousands of other users is less efficient than one that runs on my own laptop and serves only me.
Which papers?
Oh don’t be so dramatic. We’re also generating a lot of shareholder value!
Dupe of https://lobste.rs/s/90djfu/ai_generated_tools_can_make_programming
I’m using Cursor and I’m finding this to be an incredibly charitable take an how AI can help a programmer. Maybe I’m just bad at using these tools?
I know people who say they have generated tens of thousands of lines of code with LLMs that they are happily using but I guess the old adage comes in: “We can write tons of code for you if all the code is allowed to be shit.”
Have you tried prompting non-IDE-integrated LLMs directly?
I do most of my AI-assisted code work directly in Claude or ChatGPT, providing instructions for what I want built and then copying out the bits that are useful.
Personally I like the chat-and-copy approach as well, although there are decent IDE-integrated options for that like Zed or NeoVim’s CodeCompanion plugin so that you can more easily share file context and avoid even needing to switch windows.
Although for web UI specifically, it’s hard to beat the iterative speed of working with Claude Artifacts and seeing the UIs render immediately next to the chat view…