god I can’t wait for this bubble to burst. they have spent billions of dollars and generated tons of greenhouse gasses to, what, build a bullshit machine that is slightly better than the one before it? an LLM is inherently untrustworthy, it does not “understand” anything. it cannot reason, it can only generate an output that looks like the inputs it was trained on
not only that, those inputs were stolen and if you happen to have a blog they were probably stolen from you.
Your cynicism stops you from actually evaluating what these models can do, and how this one progresses the state of the art.
For example, this model has lower latency, and they’ve priced it at half of the previous model’s price, despite it having comparable or better performance. If this is not a dumping price, this suggests it’s significantly cheaper to run, and therefore wastes less energy than the previous one.
The single thing LLMs can do with high reliability is generate spam.
If you can evaluate an LLM with half the energy, all you’ve done is double the rate at which SEO parasites can bury the web in sludge. It’s like celebrating a breakthrough that makes leaded gasoline cheaper, or a fantastic new technique for making disposable diapers out of asbestos.
I’ve been using LLMs on almost a daily basis for two years now, and I’ve found plenty of useful things to do with them that aren’t generating spam (or generating text for publication).
Unreliable tools can still be incredibly useful if you figure out how to use them responsibly in a way that takes their weaknesses into account.
I have not yet taken the time to blog about it, but your series of posts persuaded me, 6 or 8 weeks ago, to stop dismissing LLMs as useless. While I do not yet find them as useful as you do, I have noticed a marked improvement in how far I get on an exploratory project within my time box.
Thank you for sharing your experiences. It’s been helpful for me to read them, and I’m sure I’m not the only one.
I’d be surprised if the reduced price is due to it being cheaper to run or more efficient. Seems more likely that they’re seeing increased competition from very large competitors with significant distribution advantages.
There is so much low hanging fruit in making these things more efficient - the openly licensed model community keeps on coming up with new tricks for running better models on smaller hardware (see Phi-3 for example), and OpenAI can read their papers and apply those tricks, while also coming up with their own.
GPT-4o is definitely faster than 4-Turbo - you can measure that yourself - and I think it’s likely that performance and efficiency are linked.
Cool, it’s wasting less energy! I’ll make sure the people of Rio Grande do Sul and of future flooding zones are aware, so they don’t look back in anger at the massive waste put into literally generating crap. It was less resource wastage than there could’ve been, for… reduced latency when generating Twitter spam. Hopefully GPT-4o will be able to handle the millions of climate migrants over the next 20 years thanks to this reduced wastage! 😊
You need to put some perspective on this. OpenAI doesn’t publish their numbers, but back-of-the-envelope estimates I could find are around 300MWh to train a model. Same ballpark as rendering Toy Story 1. USA produces over 4,200,000MWh of electricity per year, so that’s about 0.007% of USA’s production. You may argue it should be 0%, but it’s not that far off 0 in the grand scheme of things, and there are much bigger non-decarbonized industries that deserve such anger.
The cost of training is amortized, and inference doesn’t use much more energy than playing a high-end PC game. People playing Cyberpunk on Ultra settings are killing Rio Grande do Sul citizens at the same rate as people chatting with GPT.
LLM’s are not doomed to be so wasteful forever. There is progress in making more efficient hardware and software. NVidia’s Blackwell is said to reduce energy use for training by 25x. Moore’s law exponentially improves performance per watt too (thanks to this todays smartphones are faster than supercomputers 25 years ago).
It’s used for muskX spam, but so was everything else. It does have productive uses. I’ve seen first-hand how it helps non-programmers write programs to solve their problems (with “bad” code, but bad code is still empowering people), and I think that’s important given that software has been eating the world. Real-time multi-modal models have huge potential to help visually impaired people. It can produce spam, but we also have a better tool to recognize spam and phishing than we’ve ever had. Their image recognition abilities are far beyond even dedicated models that we’ve had before. Translation has become really good. They’re very good at text processing, even with their flaws and limitations, are still far ahead of bags of words.
Regarding copyright: I’m annoyed by implementations like GitHub copilot, because it’s code in and code out but slightly changed to dodge licensing. But even copyright has exceptions for transformative uses, and turning web-scraped data into “her” is pretty transformative to me (I feel I have to add: I’m not saying the world needs more e-waifus, but rather that in the last decade our ability to process speech in a coherent manner went from being science fiction to being a simple API).
Are there any stats on the energy usage to train these models?
I was never a fan of crypto due to the fact that energy waste was a feature and not a bug of those systems. Here it seems like something the providers are incentivized to solve at least.
The energy needed to teach an AI model like ChatGPT could power an average American’s home for hundreds of years — this is how Gizmodo sums up a research paper by Stanford’s AI division. Other estimations measure the amount of energy used to train ChatGPT at 1.3 gigawatt hours, or the amount of energy that 120 U.S. homes use in a year.
Not every AI is similar in this regard. The paper compares the training of four major AI models, including ChatGPT, and ranks the energy-efficiency of the data centers used to train the four AI models. ChatGPT’s GPT-3 released the most carbon into the atmosphere, 500 metric tons worth. Compare this to the AI model BLOOM which is similar in size to ChatGPT but emitted 22.7 metric tons of carbon dioxide. Much less in comparison, but still sizable — training BLOOM consumed more energy than the average American does in one year.
I tend to find that the CO2 needed to train the largest of LLMs looks a whole lot less shocking if you compare it to the CO2 used by a single fully loaded passenger airliner flying from New York to London.
If these numbers are correct, then AI costs aren’t actually that big of a deal compared to other normalized carbon footprints. 500 metric tons is something like a dozen fully loaded flights between New York and Los Angeles, and there’s hundreds of those flights a week.
Relatively speaking that doesn’t seem so bad. There are millions of homes in the US so this is a relatively miniscule amount of energy by comparison. So long as they continue to make improvements to training and execution, I don’t know if I see a problem.
That said, if there are diminishing returns with training then that could mean that increasing amounts of energy will be required to train more advanced models. I think we would need to understand what the relative scaling rules are there to understand the energy consumption concerns.
Well I should have added this in my previous post, for judging the sustainability of AI, looking only at energy usage for training on its own isn’t enough. The article goes on
Lorena points out that creating the computers themselves comes at a cost too. “The extraction of minerals like lithium, cobalt, and rare earth elements for technology components often occurs in regions with environmental and social concern,” says Lorena.
And just like any other industry, we need to lower our emissions, not increase them. Our emissions are on par with the aviation industry
Yeah, it does take a lot of resources to build the compute, but how much of those resources are dedicated to these models? Meta supposedly bought their GPUs for videos to compete with tiktok and that enabled them to do their AI training. It’s not like we throw the GPUs away when we’re done training with them either. They can be reused for other tasks.
There are also the resources to actually run the models. I’m sure those are substantial as well. How do they compare to Netflix streaming video or high end video games?
There is definitely potential for harm by generating spam and people are probably going to lose their jobs to these models. But they don’t seem to be wholly bad. For example, they seem like they could be very good at tutoring students. An infinitely patient language tutor that fits in your pocket seems like an incredible tool.
Yeah, that’s concerning. I wouldn’t want to see that be the trend for the next few years.
What’s striking is that people were willing to burn a lot of resources on crypto for years when there was very little concrete benefit for most people. (Granted, I know some people in developing countries benefited from money transfers.) AI seems to actually have some benefits and a track record of improving. I imagine people will burn a lot more resources on AI than they did on crypto.
AI data centers alone are expected to add about 323 terawatt hours of electricity demand in the U.S. by 2030, according to Wells Fargo. The forecast power demand from AI alone is seven times greater than New York City’s current annual electricity consumption of 48 terawatt hours. Goldman Sachs projects that data centers will represent 8% of total U.S. electricity consumption by the end of the decade.
[…]
Goldman Sachs forecast carbon emissions from data centers could more than double by 2030 to about 220 million tons, or 0.6% of global energy emissions, assuming natural gas provides the bulk of the power.
As for my personal point of view, 7 times NYC’s electricity demand is hell of a lot for SEO and spam generation, drowning the web in bad content, and making some hobbyists slightly more productive.
There’s a special place in hell for news sites that have links, and mention a report, but do not link to the report. I’ve been trying to find the Wells Fargo report for the past 20 minutes and could find zip, zilch, nada. Just a bunch of other websites writing the same thing, but not providing the gosh-darned link.
(I’m sure it exists, but could they at least make it possible to find?)
Anyway the Goldman Sach’s prediction is from here. It’s quite interesting and I haven’t fully digested it yet.
Yeah, that would change the equation quite a bit. The equivalent of adding an extra 140 million people’s worth of electricity consumption is pretty substantial. Adding in the other risks associated with AI generated content seems less than optimal.
Computers are insanely energy efficient compared to the value added. Like, some random plastic toy/garbage whatever that traveled from the other side of the world will harm the Earth countless times for the couple of pennies “economic value”. LLMs fair much much better, the only IT thing that is similarly terrible than our general lifestyle from an added value/environmental cost ratio are cryptocurrencies.
It’s okay, people can just build endless stretches of wall. Problem solved. /s
But in all seriousness, LLMs/AI can also help automate so many processes that currently uses more energy. Food for office workers also costs energy. And if LLMs/AI can run byrocracies (and whole countries) more efficiently, there are potential energy savings there too.
How much more efficiently? Concrete numbers are important for comparisons. I estimate that ending capitalism would result in a 3-4x efficiency improvement across all resource usage, and I’m not sure LLMs come close.
I think the amount of wasted resources on Earth that could be handled more intelligently is pretty high. If new technology could help handle just some of this, it would help a lot. We are both only estimating here, but I would imagine it would be greater than the energy cost of LLMs, especially over time.
In addition to this, LLMs are in their infancy, and performance and energy optimizations have just started.
I am also hoping the bubble bursts soon but because companies then will stop lying and overhyping. The tech has potential uses but people seriously suggesting getting diagnosis and therapy from LLMs does not seem healthy.
“stolen inputs” is exactly what is expected from a usual individual contributor by its manager (hence the popularity of slogans like “Uber for X” — steal the model and apply it everywhere where it may potentially stick). So I don’t think it will fail due to ineffectiveness — we’re “en mass” are ineffective too. BUT the leverage of the thing makes it quite dangerous — imagining some kind of fund that uses LLM for strategy and thus bancrupts let’s say some country. While it’s kinda OK when humane-led government does it, it may become quite a bust if it will be “AI-led” decision.
I don’t agree that reapplying a business model is the same as using huge swathes of writing that other people have done as an input to train a statistical model. In my view, business models are reapplied all the time, and that’s a working part of society (temporarily ignoring whether the business models themselves are good!). But what’s going on here is that the writing work needed to make the seemingly miraculous writing machine work has been taken without even any real recognition that it’s essential to making the writing machine work. It’s treated like writing on the Web is a given, not something people have spent hours toiling to make.
A quick search seems to indicate that this would be a drop in a very, very large bucket. Plus it could easily be offloaded to a green datacenter. Training seems to be about equivalent to ~60 cars per year. There are 284 million registered cars in the US. It seems quite absurd to compare the two, especially considering that cars burn gasoline whereas computers can hook up to the grid directly.
build a bullshit machine
Seems a bit absurd. It’s obviously quite useful to many people. Reducing it to a “bullshit” machine is hyperbolic and nonsensical.
an LLM is inherently untrustworthy, it does not “understand” anything. it cannot reason, it can only generate an output that looks like the inputs it was trained on
That’s true about LLMs, although I don’t know why it matters - lots of things don’t “understand”, can’t reason, etc and yet we consider them valuable. But it’s also specific to LLMs. ChatGPT can, for example, shell out to Python, allowing it to generate output that it has never seen before (for example, reversing a uuid).
not only that, those inputs were stolen and if you happen to have a blog they were probably stolen from you.
Okay.
Sorry but this “bubble” isn’t bursting. Or at least, “AI” isn’t going anywhere. It’s use cases are so varied and so obvious that it feels like you have to be extremely biased to not see that.
I’m not sure monorepos are that common. Having worked at two and a half companies using monorepos, I’ve never seen any particular benefits.
The half was a half-assed attempt by a manager to unify all the codebases in his division using git and gitlab. Using tools not designed to support a monorepo for a monorepo is just terribly painful. Tbh traditional CI tooling (more even than the VC) really assumes a single project to build.
Amazon and Bloomberg use monorepos (or did in the 2000s and 2010s) and generally it was mostly painless, but not beneficial in any identifiable way. (Tbh Bloomberg was a bit bad but that was because it took 45 minutes to build their monolith and the source control was based on cvs).
Which is all to say, monorepos need special tooling but I can’t work out why anyone wants one.
Note I’ve worked at Meta on Source Control at the time of the merges and adoption of Mercurial. Monorepositories and polyrepositories enable very different workflows that solve very different problems. Both of them have tradeoffs and one is not better than the others. Which one to choose highly depends on a variety of factors such as the product you are developing and the engineering culture you are fostering.
How you benefit from a monorepositories, depends on the integration you have. A monorepository by itself, a single place to put code, solves few problems, and only allows for easy discoverability. Most importantly however, it’s a platform to build a strict vertical integration such as a common build system, a common ci integration, etc to it. If you build a full monorepo, including a single source build system such as buck or bazel, a common ci system integration, and release workflow you get the real value add:
One-Way-To-Do-Things: If you start a new project or jump into an existing project, it always looks the same. There is one way to format the code, there is one way to build the code, there is one way to do dependency management, one way to test, and one way to release. It’s true for all projects in the codebase. In an open engineering culture, this can be very desirable. You generally have a lot of velocity just from being able to pull in any part of the existing codebase you want, develop, push it and how to build, how to test, and how to deploy is all taken care for you.
Flexibility: You can evolve projects from binaries to libraries, from a library for a specific project to a common library for all of your engineering projects. Since you wrok in one big tree, moving, separating, recombining becomes easier. You dont’ have to adhere to the boundries of repositories. So you see more often than not that a library developed inside a project becomes a common library that people depend on. This is because its discoverable and easy to make common. It also means if you need changes upstream, it is very easy to just commit those yourself.
Single-Versioning: Most Monorepos go with a single version. This ensures your tree always builds against the most recent version. This makes it possible for upstream developers to understand their changes downstream easier, and make sure downstream is always able to build against the most recent version for security or performance reasons. (This also has drawbacks, putting tons of pressure of upstream, but it’s a choice). For instance if you must upgrade openssl in your tree, you can guarantee all projects build with the new version and will get released easily. You don’t have to worry about release schedules of different teams, figuring out how to get all downstreams to finally upgrade your package, etc.
Easy dependencies: A monorepo means, that all libraries are available to everyone. This means you have easy cross-organisational access to code without complicated package management, ownership, etc problems, or even worse, access problems. In a good monorepo, getting ANY dependency is just a one-liner in buck/bazel.
Debugging: If anything goes wrong, you can debug the whole stack. You have all the source code and can easily rebuild it.
Common CI System: You don’t have to add anything to get CI. All integrations, including for example static analysis, unit testing, integration testing, fuzzing, etc are coming out of the box. Your CI system can also make a lot of assumptions, such as keeping the monorepo hot and the build caches warm since 95% of it will serve the monorepo and no other repo.
There are more along those lines. It boils down: If you are large scale, have a centralised, open engineering culture and develop few, fairly related products that depend on a large set of common libraries, then a monorepository is a great approach. A lot of the problems that come from access problems, dependency management, ci systems, and so on dissapear. The cost is that you have to invest into source control and build systems instead. At certain scale, shifting that cost is useful.
To give you a bit more of an experience view: I want to develop a new backend service at FB: I checkout the monorepo, create a new directory, add a BUCK file. I now pull in service discovery, protocol buffer and the newest computer vision library into it and code away. When I am done, i commit it all, and click push. Now my new project instantly builds, gets code coverage, source code indexing, static analysis, formatting, unit tests, integration tests, and packaging for free, and I get access to the newest CV library by adding two lines to my config. I code away. 5-weeks later i notice a new commit in my project, changing the call of the CV library because they updated their underlying library. Now the CV team doesn’t have to wrry about maintaing the old version, since there is none, they just change the “world” instead.
Of course, and i’ll end my ramble here, there A LOT of drawbacks. Monorepo vs Polyrepo is not a “this one is better than the other”, they are deliberate workflow choice that come at costs both way, and you gotta choose the one that’s right for you.
One nice example of the usefulness of a monorepo: At some point the Hack team (FB’s PHP), decided to get rid of some of the more annoying footguns within PHP and change some syntax around: So instead of having to go to each team, build a plan, get commitments and chase them to the work and then hope it works out, instead they could literally just make the changes over a few days themselves. Change millions of lines in one go to remove certain constructs completely from both language and codebase in a big swoop, delete it a few weeks later from the Hack compiler itself and move on.
So instead of having to go to each team, build a plan, get commitments and chase them to the work and then hope it works out, instead they could literally just make the changes over a few days themselves
They would have been able to do the same thing with multiple repos, as long as they had access to them. The important difference is that they were empowered to make this change to other people’s code.
Yes and no. The problem here is not only human/political. It’s about tooling and visibility: are all the repos using the same tools? Can you automatically detect what CI system is being used? Can you be sure that it all works together? When do you know it’s all been deployed safely and the old code be deleted? Are you going to hit dependency versioning problems?
I think theoretically what you suggest works, but in practice it doesn’t.
It’s easier to have a tool that can know what commits are in what running binaries if you don’t have to maintain knowledge of where binaries are built and what the right commit SHA was for each deployment.
Again, you can do all this. There’s nothing that stops writing the tools outside monorepos. It’s just much harder. Why spend time just to have multiple repos when you can just have a monorepo and be done with it?
Hah yes, this reminds me of a recent post from Rachel Kroll reminiscing about a huge Facebook outage, involving a bug that had been fixed but the fix was not deployed http://rachelbythebay.com/w/2024/03/05/outage/
You don’t control what version of your compiler they are using. You don’t control the pace of upgrade. This is much easier in a single repository where everyone is on the latest compiler.
Plus the logistics of producing commits across a large number of repositories has complexity explosion: N pull requests, N commits, etc.
If you’ve changed many millions of lines of code in one repository and you’ve done it across a company that doesn’t have one repository you will understand the pain.
Plus the logistics of producing commits across a large number of repositories has complexity explosion: N pull requests, N commits, etc.
If you’ve changed many millions of lines of code in one repository and you’ve done it across a company that doesn’t have one repository you will understand the pain.
I think it’s an unfair comparison because there is a big tool investment in monorepos, but not similarly in handling many repos with coordinated changes.
Also the comment I replied to explicitly talked about having to get other people to do the work. That’s what I replied to.
I dunno how generalizable that is; I’ve never heard of any other monorepo where the compiler for the language the application used was actually being developed in the same repository as the application.
Google does this. Running against the Google monorepo is the first step for Go to ensure a new version is backwards-compatible. I would guess, but do not know, the same is true for Starlark.
Their Angular team team does it too, kinda. When they make backwards incompatible changes to the framework, they update all 100+ Google Angular applications at the same time.
From what I’ve read there is some special treatment to Go, Android and like 1 other product. Everything else is a monorepo.
I think @cflewis is referring to the fact that changes to Go are tested against the monorepo - and while those aren’t open, if there’s a problem they share with the Go team.
This is correct. Go’s source of truth is the public GitHub repo. It is regularly imported into Google’s internal monorepo and tested to catch regressions.
$work doesn’t have the ruby/go/node sourcetrees unpacked into the repository, but it does have tooling to ensure you’re using the same version (down to the patchlevel) as production does.
I can create a single commit that upgrades the runtime and also changes the code to remove use of deprecated features / add use of new features. Even without being a compiler contributor, that’s a real benefit.
The LLVM open source project uses a monorepo. The MLIR project was initiated by Google, and originally had its own repo. When it reached a sufficient level of maturity, ownership was transferred to the LLVM project, and that separate repo was migrated into the monorepo: https://discourse.llvm.org/t/mlir-landing-in-the-monorepo/53700
The migration must have been non-trivial. Why did LLVM do it?
I think it’s due to the difficulty of atomically commiting a change that updates both the LLVM core library and the MLIR code, if those are in two separate git repos. That’s a use case that the git tooling doesn’t have good support for. The coordination costs of managing tightly integrated code split across multiple repos slows down development.
I think it’s due to the difficulty of atomically commiting a change that updates both the LLVM core library and the MLIR code, if those are in two separate git repos.
Dealing with a similar pain in my dayjob. Even though I work on a “devops” team, no one knows how to use git effectively and only knows the basics of pushing and pulling commits and blowing up the tree with merge commits instead of squashing or rebasing anything.
This makes it especially painful when I have a project where I’m using submodules because it’s the best solution short of monorepos and no one knows how to properly clone or initialize that repo, so I get stuck working on it and helping whoever works on it every. time.
Submodules when described sounds like a solution to many problems. Submodules as implemented, either because the authors didn’t know what they were trying to achieve or that the Git model made it impossible to create effectively, are a nightmare.
I think there’s a lot of back and forth about this because of at least two things:
The strengths and weaknesses of polyrepo vs monorepo really depend on your use case; some people will argue with each other about which to use without realizing their use cases differ
Both approaches, if well implemented, can be used to solve most problems (and vice versa)
With that said, I switched from a monorepo setup (the internal Neo4j cloud codebase) to a polyrepo setup (the current micro services codebase at my current employer Tibber), and I really miss the monorepo, on average. The main reason for that is: I found it a lot easier to stop accidental breaking changes at the Pull Request layer in a monorepo, because it was easier to write tests like “if you change this library rebuild all services that depend on it in the PR”. Now we often have mistakes where someone merges a change to a library, it passes PR tests, and then as soon as it’s merged in downstream CI builds in totally different teams and repos start breaking.
I agree, I notice the trend of “massive monorepos are fine because Google/Facebook do it” and it saddens me that people believe that nonsense; it is a huge pain to maintain such monstrous repos. What Google does not inform people about is the massive logistical and technological weight they must bear as a result of that decision, which they will most likely not reverse due to the sink cost fallacy and the desire to save face. Amazon and many other tech companies no longer use monorepos; instead, everything is deployed individually, allowing teams to deliver faster, which is a much superior way of doing software. Git is an incredible tool, and by far the greatest option; otherwise, everyone would use something else.
I wouldn’t call something that contains a single project/application, even a big one, a monorepo. The distinction of a monorepo is that everything in an organization is in one repo, notably both the clients of an API and the implementation of the API.
Otherwise you could argue that Git was always built for monorepos, since it was built for the Linux kernel, which is one large-ish repo.
That’s not a monorepo in the same sense. A real corporate monorepo would include Rust (including Cargo and all the related infra stuff), LLVM, all of the Mozilla/Firefox codebase (for historical reasons) and thousands of other small and large projects not directly related to Rust.
It’s nice to do one PR instead of N PRs in N repos to ship a change. If you never worked across polyrepo boundaries then polyrepo is fine. I’ve found the toil from polyrepo workflows much more frustrating than the toil from monorepo workflows
The upside of monorepos is pretty huge–your entire system is essentially a single git commit, and all tests run so an upstream change that breaks a downstream system is much more likely to be caught before it is merged to master. The problem is that the monorepo build tooling almost universally sucks (yes, this includes Nix–not because Nix’s core philosophy is bad, but because the implementation misses the mark) such that it’s not really worth it for small organizations (you end up needing a dedicated team to operate the monorepo build tooling).
I desperately want a CI platform that ships some sort of reasonable monorepo support. Bazel and its prodigies are a scourge on the industry. They’re necessary but also just so hard to get people onboarded onto. There’s really gotta be a more user-friendly way to get incremental builds, and I think if a CI provider shipped with support for it many people would flock to “this is going to be faster for you and cheaper”
Amazon moved away from Perforce to Git (IIRC, this was sometime around 2014) and generally do not use monorepos. (I’m a former Amazon employee who worked on some pretty old systems - e.g. one with almost 20 years of history - and occasionally had to spelunk into the depths of history to understand why something worked the way it did)
The build side of things has some similarities with monorepos though (I recall seeing others writing articles about the brazil build system which explains this in more detail)
At my last job, the team made a new repo with git flow for each data pipeline. This had horrible ergonomics. Some days I would write 30-40 pull requests to add a new tool to the dev branch and then another 30-40 pull requests to merge it to the main branch. If I was feeling saucy I would use the github CLI to automate things, but it still sucked. There ended up being over 100 repos for a 3 person team.
I eventually made a monorepo and moved most of the jobs over. Everyone was happy. The end.
Meanwhile, outside of Google, JavaScript also continued to evolve and surprisingly even became popular. In part to work around those IE garbage collection bugs we built Chrome which led to v8 which led to nodejs which means most web tools today are themselves written in JavaScript, unlike the Java used to build this sort of tool at Google.
This is not the right history. Google built Chrome because they became frustrated with having to cooperate with Mozilla; previously they had assigned engineers to work on Firefox. Chrome was almost certainly built so that Google would not have to worry about Mozilla changing their default search engine, or about Mozilla implementing/refusing to implement features that Google didn’t want/wanted.
The author was an early engineer on the Chrome project and the former TL of the Linux port. He says “in part”, fully understanding that Chrome was built for a number of reasons and that some of them are relevant to his post while others, frankly, aren’t.
Are you seriously the Program Manager for Uber’s Security Division, with a 2013 psych degree and zero relevant industry experience other than technical recruiting?
The real impact with this vulnerability is the lack of rate limiting and/or IP address blacklisting for multiple successive failed authentication attempts, both issues of which were not mentioned within your summary dismissal of the report. Further, without exhaustive entropy analysis of the PRNG that feeds your token generation process, hand waving about 128 bits is meaningless if there are any discernible patterns that can be picked up in the PRNG.
I mean, it’s a lot better than, say, promising a minimum of 500 for unlisted vulnerabilities and then repeatedly not paying it. Also, that’s not an unfair critique–if you’re a program manager in a field, I’d expect some relevant experience. Or, maybe, we should be more careful about handing out titles like program manager, project manager, product manager, etc. (a common issue outside of security!).
At the core of it, it seems like the fellow dutifully tried to get some low-hanging fruit and was rebuffed, multiple times. This was compounded when the issues were closed as duplicate or known or unimportant or whatever…it’s impossible to tell the difference from the outside between a good actor saying “okay this is not something we care about” and a bad actor just wanting to save 500 bucks/save face.
Like, the correct thing to have done would have been to say “Hey, thanks for reporting that, we’re not sure that that’s a priority concern right now but here’s some amount of money/free t-shirt/uber credits, please keep at it–trying looking .”
The fact that the company was happy to accept the work product but wouldn’t compensate the person for what sounded like hours and hours of work is a very bad showing.
Also, that’s not an unfair critique–if you’re a program manager in a field, I’d expect some relevant experience.
No-one deserves to be talked to in that way, in any context, but especially not in a professional one.
Or, maybe, we should be more careful about handing out titles like program manager, project manager, product manager, etc. (a common issue outside of security!).
There is no evidence that the title was “handed out”, especially since we don’t even know what the job description is.
I don’t presume to know what her job entails or whether or not she’s qualified, but titles should reflect reality or they lose their value. She certainly has a lot of endorsements on linkedin, which often carry more value than formal education.
There is no evidence that the title was “handed out”, especially since we don’t even know what the job description is.
There’s no evidence that it wasn’t–the point I’m making is that, due to practices elsewhere in industry, that title doesn’t really mean anything concrete.
Go doesn’t have a dispatch table, but interface values. It’s literally more of a freestyle dispatcher mechanism that requires some work during interface value assignment — it generates a tiny lookup hash-table for the concrete type it’s pointing to. The assignment is not insanely expensive, so it’s a fair exchange for a more pleasant type system. Ian Lance Taylor has a great blog post about the internals if you’re looking for further reading.
Yes it does have a dispatch table, and the linked post says it too:
The table of functions is stored sorted by the name of the method, although the name does not appear in the table. Calling a method on an interface means calling a specific entry in this table, where the entry to call is known at compile time. Thus the table of functions is essentially the same as a C++ virtual function table, and calling a method on an interface requires the same series of steps as calling a C++ virtual function: load the address of the virtual table, load a specific entry in the table, and call it.
The post should have said Go doesn’t have a traditional dispatch table, the ivalue mechanism I’m explaining is also dispatching. I made it more obvious.
The point of the Go’s type system is the way it organizes the table, a method via an interface has its own entry that’s not associated with a parent type.
The design docs briefly mention Calvin (“This is another great paper.”), but without a close comparison. I’d be interested[1] in hearing the Cockroach team’s views on the tradeoffs they make differently etc. Are the replication and consistency guarantees the same? Throughput and latency?
andybons from Poptip, here. You may have seen me in the contributor list as the top committer, but if you look closely at the stats, Spencer has written the most code and wrote the original design document linked to in the README. He is the mad scientist behind Cockroach.
Poptip has been supportive, but it was not born within our walls and I do not work on it full-time.
From the list of authors, everyone works at Square, Inc. now and we are all ex-Google (except for one person). It is more than a side-project, but I, personally, don’t have the domain expertise that the other authors have within this space, so it has largely been a fun learning experience for me.
I hope this clarifies things a bit. I don’t want to speak for the other authors regarding some of the questions raised, but I can say they are brilliant people who I am lucky to work with.
Thanks Andrew: yes, it clarifies things a lot! This is an ambitious project: kudos for tackling this. Now that I know most authors work at Square, I understand better the emphasis on the ACID properties which, I guess, are quite useful to them.
god I can’t wait for this bubble to burst. they have spent billions of dollars and generated tons of greenhouse gasses to, what, build a bullshit machine that is slightly better than the one before it? an LLM is inherently untrustworthy, it does not “understand” anything. it cannot reason, it can only generate an output that looks like the inputs it was trained on
not only that, those inputs were stolen and if you happen to have a blog they were probably stolen from you.
Your cynicism stops you from actually evaluating what these models can do, and how this one progresses the state of the art.
For example, this model has lower latency, and they’ve priced it at half of the previous model’s price, despite it having comparable or better performance. If this is not a dumping price, this suggests it’s significantly cheaper to run, and therefore wastes less energy than the previous one.
The single thing LLMs can do with high reliability is generate spam.
If you can evaluate an LLM with half the energy, all you’ve done is double the rate at which SEO parasites can bury the web in sludge. It’s like celebrating a breakthrough that makes leaded gasoline cheaper, or a fantastic new technique for making disposable diapers out of asbestos.
I’ve been using LLMs on almost a daily basis for two years now, and I’ve found plenty of useful things to do with them that aren’t generating spam (or generating text for publication).
Unreliable tools can still be incredibly useful if you figure out how to use them responsibly in a way that takes their weaknesses into account.
So could we have some examples, then, please, to back up your point?
Simon has been documenting his experiences and work with LLMs absolutely uniquely and awesomely, giving many, many examples.
Just a few pointers.
Look at his blog you’ll find much more.
I have a series of posts about this here: https://simonwillison.net/series/using-llms/
I have not yet taken the time to blog about it, but your series of posts persuaded me, 6 or 8 weeks ago, to stop dismissing LLMs as useless. While I do not yet find them as useful as you do, I have noticed a marked improvement in how far I get on an exploratory project within my time box.
Thank you for sharing your experiences. It’s been helpful for me to read them, and I’m sure I’m not the only one.
I’d be surprised if the reduced price is due to it being cheaper to run or more efficient. Seems more likely that they’re seeing increased competition from very large competitors with significant distribution advantages.
I expect it’s more efficient.
There is so much low hanging fruit in making these things more efficient - the openly licensed model community keeps on coming up with new tricks for running better models on smaller hardware (see Phi-3 for example), and OpenAI can read their papers and apply those tricks, while also coming up with their own.
GPT-4o is definitely faster than 4-Turbo - you can measure that yourself - and I think it’s likely that performance and efficiency are linked.
Cool, it’s wasting less energy! I’ll make sure the people of Rio Grande do Sul and of future flooding zones are aware, so they don’t look back in anger at the massive waste put into literally generating crap. It was less resource wastage than there could’ve been, for… reduced latency when generating Twitter spam. Hopefully GPT-4o will be able to handle the millions of climate migrants over the next 20 years thanks to this reduced wastage! 😊
You need to put some perspective on this. OpenAI doesn’t publish their numbers, but back-of-the-envelope estimates I could find are around 300MWh to train a model. Same ballpark as rendering Toy Story 1. USA produces over 4,200,000MWh of electricity per year, so that’s about 0.007% of USA’s production. You may argue it should be 0%, but it’s not that far off 0 in the grand scheme of things, and there are much bigger non-decarbonized industries that deserve such anger.
The cost of training is amortized, and inference doesn’t use much more energy than playing a high-end PC game. People playing Cyberpunk on Ultra settings are killing Rio Grande do Sul citizens at the same rate as people chatting with GPT.
LLM’s are not doomed to be so wasteful forever. There is progress in making more efficient hardware and software. NVidia’s Blackwell is said to reduce energy use for training by 25x. Moore’s law exponentially improves performance per watt too (thanks to this todays smartphones are faster than supercomputers 25 years ago).
It’s used for muskX spam, but so was everything else. It does have productive uses. I’ve seen first-hand how it helps non-programmers write programs to solve their problems (with “bad” code, but bad code is still empowering people), and I think that’s important given that software has been eating the world. Real-time multi-modal models have huge potential to help visually impaired people. It can produce spam, but we also have a better tool to recognize spam and phishing than we’ve ever had. Their image recognition abilities are far beyond even dedicated models that we’ve had before. Translation has become really good. They’re very good at text processing, even with their flaws and limitations, are still far ahead of bags of words.
Regarding copyright: I’m annoyed by implementations like GitHub copilot, because it’s code in and code out but slightly changed to dodge licensing. But even copyright has exceptions for transformative uses, and turning web-scraped data into “her” is pretty transformative to me (I feel I have to add: I’m not saying the world needs more e-waifus, but rather that in the last decade our ability to process speech in a coherent manner went from being science fiction to being a simple API).
Are there any stats on the energy usage to train these models?
I was never a fan of crypto due to the fact that energy waste was a feature and not a bug of those systems. Here it seems like something the providers are incentivized to solve at least.
This article from Mozilla talks about the footprint of AI.
I tend to find that the CO2 needed to train the largest of LLMs looks a whole lot less shocking if you compare it to the CO2 used by a single fully loaded passenger airliner flying from New York to London.
If these numbers are correct, then AI costs aren’t actually that big of a deal compared to other normalized carbon footprints. 500 metric tons is something like a dozen fully loaded flights between New York and Los Angeles, and there’s hundreds of those flights a week.
Relatively speaking that doesn’t seem so bad. There are millions of homes in the US so this is a relatively miniscule amount of energy by comparison. So long as they continue to make improvements to training and execution, I don’t know if I see a problem.
That said, if there are diminishing returns with training then that could mean that increasing amounts of energy will be required to train more advanced models. I think we would need to understand what the relative scaling rules are there to understand the energy consumption concerns.
Well I should have added this in my previous post, for judging the sustainability of AI, looking only at energy usage for training on its own isn’t enough. The article goes on
And just like any other industry, we need to lower our emissions, not increase them. Our emissions are on par with the aviation industry
According to that link most of that is streaming video though, not AI or server builds or anything 99.9% of us engineers are involved in.
Well it’s contributing substantially at Microsoft
https://au.pcmag.com/news/105283/microsofts-emissions-spike-29-as-ai-gobbles-up-resources
Yeah, it does take a lot of resources to build the compute, but how much of those resources are dedicated to these models? Meta supposedly bought their GPUs for videos to compete with tiktok and that enabled them to do their AI training. It’s not like we throw the GPUs away when we’re done training with them either. They can be reused for other tasks.
There are also the resources to actually run the models. I’m sure those are substantial as well. How do they compare to Netflix streaming video or high end video games?
There is definitely potential for harm by generating spam and people are probably going to lose their jobs to these models. But they don’t seem to be wholly bad. For example, they seem like they could be very good at tutoring students. An infinitely patient language tutor that fits in your pocket seems like an incredible tool.
https://au.pcmag.com/news/105283/microsofts-emissions-spike-29-as-ai-gobbles-up-resources
Well it seems that for Microsoft’s energy usage went up by quite a bit due to Ai
Yeah, that’s concerning. I wouldn’t want to see that be the trend for the next few years.
What’s striking is that people were willing to burn a lot of resources on crypto for years when there was very little concrete benefit for most people. (Granted, I know some people in developing countries benefited from money transfers.) AI seems to actually have some benefits and a track record of improving. I imagine people will burn a lot more resources on AI than they did on crypto.
As an extra to the ongoing thread: https://www.cnbc.com/2024/05/05/ai-could-drive-natural-gas-boom-as-utilities-face-surging-electric-demand.html
As for my personal point of view, 7 times NYC’s electricity demand is hell of a lot for SEO and spam generation, drowning the web in bad content, and making some hobbyists slightly more productive.
There’s a special place in hell for news sites that have links, and mention a report, but do not link to the report. I’ve been trying to find the Wells Fargo report for the past 20 minutes and could find zip, zilch, nada. Just a bunch of other websites writing the same thing, but not providing the gosh-darned link.
(I’m sure it exists, but could they at least make it possible to find?)
Anyway the Goldman Sach’s prediction is from here. It’s quite interesting and I haven’t fully digested it yet.
Yeah, that would change the equation quite a bit. The equivalent of adding an extra 140 million people’s worth of electricity consumption is pretty substantial. Adding in the other risks associated with AI generated content seems less than optimal.
Computers are insanely energy efficient compared to the value added. Like, some random plastic toy/garbage whatever that traveled from the other side of the world will harm the Earth countless times for the couple of pennies “economic value”. LLMs fair much much better, the only IT thing that is similarly terrible than our general lifestyle from an added value/environmental cost ratio are cryptocurrencies.
Couldn’t the thousands of people now free from mundane work, work together now on solving climate issues or issues that require physical human labor?
This assumes that solving climate change is a technical problem.
I’m confused, I was suggesting that more humans will be available to fix the climate in some way…
In addition to @ksynwa’s comment, this assumes the problems can be solved. Are you willing to gamble everyone’s futures on AI fixing it?
It’s okay, people can just build endless stretches of wall. Problem solved. /s
But in all seriousness, LLMs/AI can also help automate so many processes that currently uses more energy. Food for office workers also costs energy. And if LLMs/AI can run byrocracies (and whole countries) more efficiently, there are potential energy savings there too.
How much more efficiently? Concrete numbers are important for comparisons. I estimate that ending capitalism would result in a 3-4x efficiency improvement across all resource usage, and I’m not sure LLMs come close.
I think the amount of wasted resources on Earth that could be handled more intelligently is pretty high. If new technology could help handle just some of this, it would help a lot. We are both only estimating here, but I would imagine it would be greater than the energy cost of LLMs, especially over time.
In addition to this, LLMs are in their infancy, and performance and energy optimizations have just started.
Cryptocurrency on the other hand…
I am also hoping the bubble bursts soon but because companies then will stop lying and overhyping. The tech has potential uses but people seriously suggesting getting diagnosis and therapy from LLMs does not seem healthy.
“stolen inputs” is exactly what is expected from a usual individual contributor by its manager (hence the popularity of slogans like “Uber for X” — steal the model and apply it everywhere where it may potentially stick). So I don’t think it will fail due to ineffectiveness — we’re “en mass” are ineffective too. BUT the leverage of the thing makes it quite dangerous — imagining some kind of fund that uses LLM for strategy and thus bancrupts let’s say some country. While it’s kinda OK when humane-led government does it, it may become quite a bust if it will be “AI-led” decision.
I don’t agree that reapplying a business model is the same as using huge swathes of writing that other people have done as an input to train a statistical model. In my view, business models are reapplied all the time, and that’s a working part of society (temporarily ignoring whether the business models themselves are good!). But what’s going on here is that the writing work needed to make the seemingly miraculous writing machine work has been taken without even any real recognition that it’s essential to making the writing machine work. It’s treated like writing on the Web is a given, not something people have spent hours toiling to make.
A quick search seems to indicate that this would be a drop in a very, very large bucket. Plus it could easily be offloaded to a green datacenter. Training seems to be about equivalent to ~60 cars per year. There are 284 million registered cars in the US. It seems quite absurd to compare the two, especially considering that cars burn gasoline whereas computers can hook up to the grid directly.
Seems a bit absurd. It’s obviously quite useful to many people. Reducing it to a “bullshit” machine is hyperbolic and nonsensical.
That’s true about LLMs, although I don’t know why it matters - lots of things don’t “understand”, can’t reason, etc and yet we consider them valuable. But it’s also specific to LLMs. ChatGPT can, for example, shell out to Python, allowing it to generate output that it has never seen before (for example, reversing a uuid).
Okay.
Sorry but this “bubble” isn’t bursting. Or at least, “AI” isn’t going anywhere. It’s use cases are so varied and so obvious that it feels like you have to be extremely biased to not see that.
I’m not sure monorepos are that common. Having worked at two and a half companies using monorepos, I’ve never seen any particular benefits.
The half was a half-assed attempt by a manager to unify all the codebases in his division using git and gitlab. Using tools not designed to support a monorepo for a monorepo is just terribly painful. Tbh traditional CI tooling (more even than the VC) really assumes a single project to build.
Amazon and Bloomberg use monorepos (or did in the 2000s and 2010s) and generally it was mostly painless, but not beneficial in any identifiable way. (Tbh Bloomberg was a bit bad but that was because it took 45 minutes to build their monolith and the source control was based on cvs).
Which is all to say, monorepos need special tooling but I can’t work out why anyone wants one.
Note I’ve worked at Meta on Source Control at the time of the merges and adoption of Mercurial. Monorepositories and polyrepositories enable very different workflows that solve very different problems. Both of them have tradeoffs and one is not better than the others. Which one to choose highly depends on a variety of factors such as the product you are developing and the engineering culture you are fostering.
How you benefit from a monorepositories, depends on the integration you have. A monorepository by itself, a single place to put code, solves few problems, and only allows for easy discoverability. Most importantly however, it’s a platform to build a strict vertical integration such as a common build system, a common ci integration, etc to it. If you build a full monorepo, including a single source build system such as buck or bazel, a common ci system integration, and release workflow you get the real value add:
One-Way-To-Do-Things: If you start a new project or jump into an existing project, it always looks the same. There is one way to format the code, there is one way to build the code, there is one way to do dependency management, one way to test, and one way to release. It’s true for all projects in the codebase. In an open engineering culture, this can be very desirable. You generally have a lot of velocity just from being able to pull in any part of the existing codebase you want, develop, push it and how to build, how to test, and how to deploy is all taken care for you.
Flexibility: You can evolve projects from binaries to libraries, from a library for a specific project to a common library for all of your engineering projects. Since you wrok in one big tree, moving, separating, recombining becomes easier. You dont’ have to adhere to the boundries of repositories. So you see more often than not that a library developed inside a project becomes a common library that people depend on. This is because its discoverable and easy to make common. It also means if you need changes upstream, it is very easy to just commit those yourself.
Single-Versioning: Most Monorepos go with a single version. This ensures your tree always builds against the most recent version. This makes it possible for upstream developers to understand their changes downstream easier, and make sure downstream is always able to build against the most recent version for security or performance reasons. (This also has drawbacks, putting tons of pressure of upstream, but it’s a choice). For instance if you must upgrade openssl in your tree, you can guarantee all projects build with the new version and will get released easily. You don’t have to worry about release schedules of different teams, figuring out how to get all downstreams to finally upgrade your package, etc.
Easy dependencies: A monorepo means, that all libraries are available to everyone. This means you have easy cross-organisational access to code without complicated package management, ownership, etc problems, or even worse, access problems. In a good monorepo, getting ANY dependency is just a one-liner in buck/bazel.
Debugging: If anything goes wrong, you can debug the whole stack. You have all the source code and can easily rebuild it.
Common CI System: You don’t have to add anything to get CI. All integrations, including for example static analysis, unit testing, integration testing, fuzzing, etc are coming out of the box. Your CI system can also make a lot of assumptions, such as keeping the monorepo hot and the build caches warm since 95% of it will serve the monorepo and no other repo.
There are more along those lines. It boils down: If you are large scale, have a centralised, open engineering culture and develop few, fairly related products that depend on a large set of common libraries, then a monorepository is a great approach. A lot of the problems that come from access problems, dependency management, ci systems, and so on dissapear. The cost is that you have to invest into source control and build systems instead. At certain scale, shifting that cost is useful.
To give you a bit more of an experience view: I want to develop a new backend service at FB: I checkout the monorepo, create a new directory, add a BUCK file. I now pull in service discovery, protocol buffer and the newest computer vision library into it and code away. When I am done, i commit it all, and click push. Now my new project instantly builds, gets code coverage, source code indexing, static analysis, formatting, unit tests, integration tests, and packaging for free, and I get access to the newest CV library by adding two lines to my config. I code away. 5-weeks later i notice a new commit in my project, changing the call of the CV library because they updated their underlying library. Now the CV team doesn’t have to wrry about maintaing the old version, since there is none, they just change the “world” instead.
Of course, and i’ll end my ramble here, there A LOT of drawbacks. Monorepo vs Polyrepo is not a “this one is better than the other”, they are deliberate workflow choice that come at costs both way, and you gotta choose the one that’s right for you.
One nice example of the usefulness of a monorepo: At some point the Hack team (FB’s PHP), decided to get rid of some of the more annoying footguns within PHP and change some syntax around: So instead of having to go to each team, build a plan, get commitments and chase them to the work and then hope it works out, instead they could literally just make the changes over a few days themselves. Change millions of lines in one go to remove certain constructs completely from both language and codebase in a big swoop, delete it a few weeks later from the Hack compiler itself and move on.
They would have been able to do the same thing with multiple repos, as long as they had access to them. The important difference is that they were empowered to make this change to other people’s code.
Yes and no. The problem here is not only human/political. It’s about tooling and visibility: are all the repos using the same tools? Can you automatically detect what CI system is being used? Can you be sure that it all works together? When do you know it’s all been deployed safely and the old code be deleted? Are you going to hit dependency versioning problems?
I think theoretically what you suggest works, but in practice it doesn’t.
How do you know that with a monorepo?
It’s easier to have a tool that can know what commits are in what running binaries if you don’t have to maintain knowledge of where binaries are built and what the right commit SHA was for each deployment.
Again, you can do all this. There’s nothing that stops writing the tools outside monorepos. It’s just much harder. Why spend time just to have multiple repos when you can just have a monorepo and be done with it?
Just have a monorepo? They also need specialized tools, for example a different VCS. Neither option “just” works.
Hah yes, this reminds me of a recent post from Rachel Kroll reminiscing about a huge Facebook outage, involving a bug that had been fixed but the fix was not deployed http://rachelbythebay.com/w/2024/03/05/outage/
This is super not true.
You don’t control what version of your compiler they are using. You don’t control the pace of upgrade. This is much easier in a single repository where everyone is on the latest compiler.
Plus the logistics of producing commits across a large number of repositories has complexity explosion: N pull requests, N commits, etc.
If you’ve changed many millions of lines of code in one repository and you’ve done it across a company that doesn’t have one repository you will understand the pain.
I think it’s an unfair comparison because there is a big tool investment in monorepos, but not similarly in handling many repos with coordinated changes.
Also the comment I replied to explicitly talked about having to get other people to do the work. That’s what I replied to.
Fair!
I dunno how generalizable that is; I’ve never heard of any other monorepo where the compiler for the language the application used was actually being developed in the same repository as the application.
Google does this. Running against the Google monorepo is the first step for Go to ensure a new version is backwards-compatible. I would guess, but do not know, the same is true for Starlark.
Their Angular team team does it too, kinda. When they make backwards incompatible changes to the framework, they update all 100+ Google Angular applications at the same time.
So you’re saying the private Google monorepo is the canonical repo for Google Go? It’s not developed in public?
From what I’ve read there is some special treatment to Go, Android and like 1 other product. Everything else is a monorepo. I think @cflewis is referring to the fact that changes to Go are tested against the monorepo - and while those aren’t open, if there’s a problem they share with the Go team.
This is correct. Go’s source of truth is the public GitHub repo. It is regularly imported into Google’s internal monorepo and tested to catch regressions.
$work doesn’t have the ruby/go/node sourcetrees unpacked into the repository, but it does have tooling to ensure you’re using the same version (down to the patchlevel) as production does.
I can create a single commit that upgrades the runtime and also changes the code to remove use of deprecated features / add use of new features. Even without being a compiler contributor, that’s a real benefit.
I understand and that sounds useful; however that is not the same thing as the parent was talking about.
codemods like this are a regular occurrence by the language teams (Hack, Python, etc) and (imo) key to continued success and PL improvement
The LLVM open source project uses a monorepo. The MLIR project was initiated by Google, and originally had its own repo. When it reached a sufficient level of maturity, ownership was transferred to the LLVM project, and that separate repo was migrated into the monorepo: https://discourse.llvm.org/t/mlir-landing-in-the-monorepo/53700
The migration must have been non-trivial. Why did LLVM do it?
I think it’s due to the difficulty of atomically commiting a change that updates both the LLVM core library and the MLIR code, if those are in two separate git repos. That’s a use case that the git tooling doesn’t have good support for. The coordination costs of managing tightly integrated code split across multiple repos slows down development.
Dealing with a similar pain in my dayjob. Even though I work on a “devops” team, no one knows how to use git effectively and only knows the basics of pushing and pulling commits and blowing up the tree with merge commits instead of squashing or rebasing anything.
This makes it especially painful when I have a project where I’m using submodules because it’s the best solution short of monorepos and no one knows how to properly clone or initialize that repo, so I get stuck working on it and helping whoever works on it every. time.
Years ago (OK, more like a decade ago), I had hoped that
git-subtreewould solve the issues that we faced with submodules, but… no.Submodules when described sounds like a solution to many problems. Submodules as implemented, either because the authors didn’t know what they were trying to achieve or that the Git model made it impossible to create effectively, are a nightmare.
In my experience a little scripting to fetch dependencies as part of the build system is much easier to work with than git submodules.
I think there’s a lot of back and forth about this because of at least two things:
With that said, I switched from a monorepo setup (the internal Neo4j cloud codebase) to a polyrepo setup (the current micro services codebase at my current employer Tibber), and I really miss the monorepo, on average. The main reason for that is: I found it a lot easier to stop accidental breaking changes at the Pull Request layer in a monorepo, because it was easier to write tests like “if you change this library rebuild all services that depend on it in the PR”. Now we often have mistakes where someone merges a change to a library, it passes PR tests, and then as soon as it’s merged in downstream CI builds in totally different teams and repos start breaking.
I agree, I notice the trend of “massive monorepos are fine because Google/Facebook do it” and it saddens me that people believe that nonsense; it is a huge pain to maintain such monstrous repos. What Google does not inform people about is the massive logistical and technological weight they must bear as a result of that decision, which they will most likely not reverse due to the sink cost fallacy and the desire to save face. Amazon and many other tech companies no longer use monorepos; instead, everything is deployed individually, allowing teams to deliver faster, which is a much superior way of doing software. Git is an incredible tool, and by far the greatest option; otherwise, everyone would use something else.
Many large FOSS projects are monorepos, e.g. rust-lang/rust
That doesn’t feel very monorepo to me: they have 215 repos listed on https://github.com/orgs/rust-lang/repositories
I’d class Rust as more of a monorepo if tools like Cargo and rustup were in there.
Python is probably a better example here, the full Python standard library is in the cPython repo: https://github.com/python/cpython/tree/main/Lib
UPDATE: Actually no, having found the tools/ folder Rust does feel like a monorepo https://github.com/rust-lang/rust/tree/master/src/tools (though it looks like Cargo is in there as a git submodule)
I wouldn’t call something that contains a single project/application, even a big one, a monorepo. The distinction of a monorepo is that everything in an organization is in one repo, notably both the clients of an API and the implementation of the API.
Otherwise you could argue that Git was always built for monorepos, since it was built for the Linux kernel, which is one large-ish repo.
That’s not a monorepo in the same sense. A real corporate monorepo would include Rust (including Cargo and all the related infra stuff), LLVM, all of the Mozilla/Firefox codebase (for historical reasons) and thousands of other small and large projects not directly related to Rust.
Rust is just a regular mid-size git repo.
It’s nice to do one PR instead of N PRs in N repos to ship a change. If you never worked across polyrepo boundaries then polyrepo is fine. I’ve found the toil from polyrepo workflows much more frustrating than the toil from monorepo workflows
The upside of monorepos is pretty huge–your entire system is essentially a single git commit, and all tests run so an upstream change that breaks a downstream system is much more likely to be caught before it is merged to master. The problem is that the monorepo build tooling almost universally sucks (yes, this includes Nix–not because Nix’s core philosophy is bad, but because the implementation misses the mark) such that it’s not really worth it for small organizations (you end up needing a dedicated team to operate the monorepo build tooling).
This is only true if every deploy replaces the entire system, and there are never rolling upgrades.
I desperately want a CI platform that ships some sort of reasonable monorepo support. Bazel and its prodigies are a scourge on the industry. They’re necessary but also just so hard to get people onboarded onto. There’s really gotta be a more user-friendly way to get incremental builds, and I think if a CI provider shipped with support for it many people would flock to “this is going to be faster for you and cheaper”
What would that look like?
Amazon moved away from Perforce to Git (IIRC, this was sometime around 2014) and generally do not use monorepos. (I’m a former Amazon employee who worked on some pretty old systems - e.g. one with almost 20 years of history - and occasionally had to spelunk into the depths of history to understand why something worked the way it did)
The build side of things has some similarities with monorepos though (I recall seeing others writing articles about the brazil build system which explains this in more detail)
I stand corrected. Another point against monorepos :)
At my last job, the team made a new repo with git flow for each data pipeline. This had horrible ergonomics. Some days I would write 30-40 pull requests to add a new tool to the dev branch and then another 30-40 pull requests to merge it to the main branch. If I was feeling saucy I would use the github CLI to automate things, but it still sucked. There ended up being over 100 repos for a 3 person team.
I eventually made a monorepo and moved most of the jobs over. Everyone was happy. The end.
That screenshot with the performance / acessibility / … score at the end, what tool is it from?
Lighthouse
This is not the right history. Google built Chrome because they became frustrated with having to cooperate with Mozilla; previously they had assigned engineers to work on Firefox. Chrome was almost certainly built so that Google would not have to worry about Mozilla changing their default search engine, or about Mozilla implementing/refusing to implement features that Google didn’t want/wanted.
The author was an early engineer on the Chrome project and the former TL of the Linux port. He says “in part”, fully understanding that Chrome was built for a number of reasons and that some of them are relevant to his post while others, frankly, aren’t.
https://hackerone.com/reports/293359#activity-2203160 via https://twitter.com/infosec_au/status/945048806290321408 seems to at least shed a bit more light on things. I don’t find this kind of behavior to be OK at all:
”Oh my God.
Are you seriously the Program Manager for Uber’s Security Division, with a 2013 psych degree and zero relevant industry experience other than technical recruiting?
LULZ”
Hrm. He really wants to be paid for this?
I mean, it’s a lot better than, say, promising a minimum of 500 for unlisted vulnerabilities and then repeatedly not paying it. Also, that’s not an unfair critique–if you’re a program manager in a field, I’d expect some relevant experience. Or, maybe, we should be more careful about handing out titles like program manager, project manager, product manager, etc. (a common issue outside of security!).
At the core of it, it seems like the fellow dutifully tried to get some low-hanging fruit and was rebuffed, multiple times. This was compounded when the issues were closed as duplicate or known or unimportant or whatever…it’s impossible to tell the difference from the outside between a good actor saying “okay this is not something we care about” and a bad actor just wanting to save 500 bucks/save face.
Like, the correct thing to have done would have been to say “Hey, thanks for reporting that, we’re not sure that that’s a priority concern right now but here’s some amount of money/free t-shirt/uber credits, please keep at it–trying looking .”
The fact that the company was happy to accept the work product but wouldn’t compensate the person for what sounded like hours and hours of work is a very bad showing.
No-one deserves to be talked to in that way, in any context, but especially not in a professional one.
There is no evidence that the title was “handed out”, especially since we don’t even know what the job description is.
I don’t presume to know what her job entails or whether or not she’s qualified, but titles should reflect reality or they lose their value. She certainly has a lot of endorsements on linkedin, which often carry more value than formal education.
It’s “Program Manager, Security” btw.
There’s no evidence that it wasn’t–the point I’m making is that, due to practices elsewhere in industry, that title doesn’t really mean anything concrete.
Yes it does have a dispatch table, and the linked post says it too:
It’s the case of gccgo maybe not go native compiler
If one wants run-time subtyping, one have no choice but to include a dispatch table.
You should point it out in the Medium post so that she can correct it. I doubt she’ll see it here :).
I saw it and made it clearer.
The post should have said Go doesn’t have a traditional dispatch table, the ivalue mechanism I’m explaining is also dispatching. I made it more obvious.
The point of the Go’s type system is the way it organizes the table, a method via an interface has its own entry that’s not associated with a parent type.
The design docs briefly mention Calvin (“This is another great paper.”), but without a close comparison. I’d be interested[1] in hearing the Cockroach team’s views on the tradeoffs they make differently etc. Are the replication and consistency guarantees the same? Throughput and latency?
[1] Since I’m speaking on Calvin next week: http://www.meetup.com/papers-we-love-too/events/171291972.
You are welcome to ask any clarifying questions on the list – cockroach-db@googlegroups.com
To be honest we’re at such an early stage that we may not have answers for you, but there’s no harm in asking.
That sounds like the data store a lot of people would want! I’m curious to know who is behind this project, because it cannot be just a hobby project.
See https://github.com/cockroachdb/cockroach/graphs/contributors.
FWIW one of my coworkers identified one of those as a former googler.
I’m under the impression this comes from Poptip, but it’s just my impression :)
andybons from Poptip, here. You may have seen me in the contributor list as the top committer, but if you look closely at the stats, Spencer has written the most code and wrote the original design document linked to in the README. He is the mad scientist behind Cockroach.
Poptip has been supportive, but it was not born within our walls and I do not work on it full-time.
From the list of authors, everyone works at Square, Inc. now and we are all ex-Google (except for one person). It is more than a side-project, but I, personally, don’t have the domain expertise that the other authors have within this space, so it has largely been a fun learning experience for me.
I hope this clarifies things a bit. I don’t want to speak for the other authors regarding some of the questions raised, but I can say they are brilliant people who I am lucky to work with.
Thanks Andrew: yes, it clarifies things a lot! This is an ambitious project: kudos for tackling this. Now that I know most authors work at Square, I understand better the emphasis on the ACID properties which, I guess, are quite useful to them.