Is this the first big Go project at Microsoft? I had no idea they could choose Go, I would assume that it just wouldn’t be acceptable, because they wouldn’t want to unnecessarily legitimize a Google project.
They had already lost the web platform by the time they switched to Chromium.
When I think of MS and ‘developers developers developers develop’, I was lead to believe that they’d have more pride in their own development platforms.
I wonder if the switch to Chromium, the embrace of Linux in Azure, the move from VS to VSCode, any of these, would have happened under Ballmer or Gates. I suspect Gates’ new book is like Merkel’s: retrospective but avoiding the most controversial questions: Russian gas and giving up dogfooding. Am I foolish to expect a more critical introspection?
I think corporate open source is now “proven”, and companies don’t worry so much about this kind of thing.
Google has used TypeScript for quite awhile now (even though they had their own Closure compiler, which was not staffed consistently, internally)
All of these companies are best thought of as “frenemies” :) They cooperate in some ways, and compete in others.
They have many common interests, and collaborate on say lobbying the US govt
When Google was small, and Microsoft was big, the story was different. They even had different employee bases.
But now they are basically peers, and employees go freely back and forth
I can see a blog post saying they made changes for FIPS compliance, which is usually understood to be for satisfying government requirements and not a security improvement. I can’t see a summary of any other changes.
MS is a pretty big company, so I wouldn’t be surprised if they have a ton of internal services in Go. Plus with Azure being a big part of their business, I doubt they care what language people use as long as you deploy it on their platforms.
Can confirm that some internal services were written in go as of a couple years ago. (I worked for Microsoft at the time.) I didn’t have a wide enough view to know if it was “a ton,” though.
I’m too lazy to look up a source but some of the error messages I’ve seen in azure make it clear Microsoft uses go to build their cloud.
Message like this one:
dial tcp 192.168.1.100:3000: connect: timeout
Keep in mind Microsoft also maintains their own go fork for fips compliance. I don’t know exactly their use case but I can tell you at Google we did the same thing for the same reason. I think the main fips compliant go binaries being shipped were probably gke/k8s related.
(Edit removed off topic ramble, sorry I have ADHD)
The TypeScript dev lead posted this response about the language choice on Reddit, for anyone who’s curious:
(dev lead of TypeScript here, hi!)
We definitely knew when choosing Go that there were going to be people questioning why we didn’t choose Rust. It’s a good question because Rust is an excellent language, and barring other constraints, is a strong first choice when writing new native code.
Portability (i.e. the ability to make a new codebase that is algorithmically similar to the current one) was always a key constraint here as we thought about how to do this. We tried tons of approaches to get to a representation that would have made that port approach tractable in Rust, but all of them either had unacceptable trade-offs (perf, ergonomics, etc.) or devolved in to “write your own GC”-style strategies. Some of them came close, but often required dropping into lots of unsafe code, and there just didn’t seem to be many combinations of primitives in Rust that allow for an ergonomic port of JavaScript code (which is pretty unsurprising when phrased that way - most languages don’t prioritize making it easy to port from JavaScript/TypeScript!).
In the end we had two options - do a complete from-scrach rewrite in Rust, which could take years and yield an incompatible version of TypeScript that no one could actually use, or just do a port in Go and get something usable in a year or so and have something that’s extremely compatible in terms of semantics and extremely competitive in terms of performance.
And it’s not even super clear what the upside of doing that would be (apart from not having to deal with so many “Why didn’t you choose Rust?” questions). We still want a highly-separated API surface to keep our implementation options open, so Go’s interop shortcomings aren’t particularly relevant. Go has excellent code generation and excellent data representation, just like Rust. Go has excellent concurrency primitives, just like Rust. Single-core performance is within the margin of error. And while there might be a few performance wins to be had by using unsafe code in Go, we have gotten excellent performance and memory usage without using any unsafe primitives.
In our opinion, Rust succeeds wildly at its design goals, but “is straightforward to port to Rust from this particular JavaScript codebase” is very rationally not one of its design goals. It’s not one of Go’s either, but in our case given the way we’ve written the code so far, it does turn out to be pretty good at it.
And it’s not even super clear what the upside of doing that would be (apart from not having to deal with so many “Why didn’t you choose Rust?” questions)
People really miss the forest for the trees.
I looked at the repo and the story seems clear to me: 12 people rewrote the TypeScript compiler in 5 months, getting a 10x speed improvement, with immediate portability to many different platforms, while not having written much Go before in their lives (although they are excellent programmers).
This is precisely the reason why Go was invented in the first place. “Why not Rust?” should not be the first thing that comes to mind.
I honestly do think the “Why not Rust?” question is a valid question to pop into someone’s head before reading the explanation for their choice.
First of all, if you’re the kind of nerd who happens to follow the JavaScript/TypeScript dev ecosystem, you will have seen a fair number of projects either written, or rewritten, in Rust recently. Granted, some tools are also being written/rewritten in other languages like Go and Zig. But, the point is that there’s enough mindshare around Rust in the JS/TS world that it’s fair to be curious why they didn’t choose Rust while other projects did. I don’t think we should assume the question is always antagonistic or from the “Rust Evangelism Strike Force”.
Also, it’s a popular opinion that languages with algebraic data types (among other things) are good candidates for parsers and compilers, so languages like OCaml and Rust might naturally rank highly in languages for consideration.
So, I honestly had the same question, initially. However, upon reading Anders’ explanation, I can absolutely see why Go was a good choice. And your analysis of the development metrics is also very relevant and solid support for their choice!
I guess I’m just saying, the Rust fanboys (myself, included) can be obnoxious, but I hope we don’t swing the pendulum too far the other way and assume that it’s never appropriate to bring Rust into a dev conversation (e.g., there really may be projects that should be rewritten in Rust, even if people might start cringing whenever they hear that now).
While tweaking a parser / interpreter a few years ago written in Go, I specifically replaced a struct with an ‘interface {}’ in order to exercise its pseudo-tagged-union mechanisms. Together with using type-switch form.
These day’s I’d actually make it a closed interface such that it is more akin to a tagged-union. Which I did for another project which was passing around instances of variant-structs (i.e. a tagged union), rather than building an AST.
So it is quite possible to use that pattern in Go as a form of sum-type, if for some reason one is inclined to use Go as the implementation language.
A relevant quote is that C# has “some ahead-of-time compilation options available, but they’re not on all platforms and don’t really have a decade or more of hardening.”
Yeah Hjelsberg also talks about value types being necessary, or at least useful, in making language implementations fast
If you want value types and automatically managed memory, I think your only choices are Go, D, Swift, and C# (and very recently OCaml, though I’m not sure if that is fully done).
I guess Hjelsberg is conceding that value types are a bit “second class” in C#? I think I was surprised by the “class” and “struct” split, which seemed limiting, but I’ve never used it. [1]
And that is a lesson learned from the Oils Python -> C++ translation. We don’t have value types, because statically typed Python doesn’t, and that puts a cap on speed. (But we’re faster than bash in many cases, though slower in some too)
Now that I’ve worked on a garbage collector, I see a sweet spot in languages like Go and C# – they have both value types deallocated on the stack and GC. Both Java and Python lack this semantic, so the GCs have to do more work, and the programmer has less control.
There was also a talk that hinted at some GC-like patterns in Zig, and I proposed that TinyGo get “compressed pointers” like Hotspot and v8, and then you would basically have that:
Yeah, I’m kind of curious about whether OCaml was considered at some point (I asked about this in the Reddit thread, haven’t gotten a reply yet).
OCaml seems much more similar to TS than Go, and has a proven track record when it comes to compilers. Maybe portability issues? (Good portability was mentioned as a must-have IIRC)
Not sure what more TypeScript would have needed. In fact, Flow’s JavaScript parser is available as a separate library, so they would have shaved off at least a month from the proof of concept…
Although thinking about it a bit more, I think Nim, Julia, (and maybe Crystal) are like C#, in that they are not as general as Go / D / Swift.
You don’t have a Foo* type as well as a Foo type, i.e. the layout is orthogonal to whether it’s a value or reference. Instead, Nim apparently has value objects and reference objects. I believe C# has “structs” for values and classes for references.
I think Hjelsberg was hinting at this category when saying Go wins a bit on expressiveness, and it’s also “as close to native as you can get with GC”.
I think the reason this Go’s model is uncommon is because it forces the GC to support interior pointers, which is a significant complication (e.g. it is not supported by WASM GC). Go basically has the C memory model, with garbage collection.
I think C#, Julia, and maybe Nim/Crystal do not support interior pointers (interested in corrections)
Someone should write a survey of how GC tracing works with each language :) (Nim’s default is reference counting without cycle collection.)
Yeah that’s interesting. Julia has a distinction between struct (value) and mutable struct (reference). You can use raw pointers but safe interior references (to an element of an array for example) include a normal reference to the (start of the) backing array, and the index.
I can understand how in Rust you can safely have an interior pointer as the borrow checker ensures a reference to an array element is valid for its lifetime (the array can’t be dropped or resized before the reference is dropped). I’m very curious - I would like to understand how Go’s tracing GC works with interior pointers now! (I would read such a survey).
Ok - Go’s GC seems to track a memory span for each object (struct or array), stored in kind of a span tree (interval tree) for easy lookup given some pointer to chase. Makes sense. I wonder if it smart enough to deallocate anything dangling from non-referenced elements of an array / fields of a struct, or just chooses to be conservative (and if so do users end up accidentally creating memory leaks very often)? What’s the performance impact of all of this compared to runtimes requiring non-interior references? The interior pointers themselves will be a performance win, at the expense of using an interval tree during the mark phase.
It’s been a few years since I’ve written any Go, but I have a vague recollection that the difference between something being heap or stack allocated was (sometimes? always?) implicit based on compiler analysis of how you use the value. Is that right? How easy it, generally, to accidentally make something heap-allocated and GC’d?
That’s the only thing that makes me nervous about that as a selling point for performance. I feel like if I’m worried about stack vs heap or scoped vs memory-managed or whatever, I’d probably prefer something like Swift, Rust, or C# (I’m not familiar at all with how D’s optional GC stuff works).
$ go build -gcflags "-m" main.go
.\main.go:8:14: *y escapes to heap
.\main.go:11:13: x does not escape
So the toolchain is pretty transparent. This is actually something I would like for the Oils Python->C++ compiler, since we have many things that are “obviously” locals that end up being heap allocated. And some not so obvious cases. But I think having some simple escape analysis would be great.
Hejlsberg said they got about 3x performance from native compilation and value types, which also halved the memory usage of the compiler. They got a further 3x from shared-memory multithreading. He talked a lot about how neither of those are possible with the JavaScript runtime, which is why it wasn’t possible to make tsc 10x faster while keeping it written in TypeScript.
Yeah but I can get bigger memory wins while staying inside JS by sharing the data structures between many tools that currently hold copies of the same data: the linter, the pretty-formatter, the syntax highlighter, and the type checker
I can do this because I make my syntax tree nodes immutable! TS cannot make their syntax tree nodes immutable (even in JS where it’s possible) because they rely on the node.parent reference. Because their nodes are mutable-but-typed-as-immutable, these nodes can never safely be passed as arguments outside the bounds of the TS ecosystem, a limitation that precludes the kind of cross-tool syntax tree reuse that I see as being the way forward
Hejlsberg said that the TypeScript syntax tree nodes are, in fact, immutable. This was crucial for parallelizing tsgo: it parses all the source files in parallel in the first phase, then typechecks in parallel in the second phase. The parse trees from the first phase are shared by all threads in the second phase. The two phases spread the work across threads differently. He talks about that kind of sharing and threading being impractical in JavaScript.
In fact he talks about tsc being designed around immutable and incrementally updatable data structures right from the start. It was one of the early non-batch compilers, hot on the heels of Roslyn, both being designed to support IDEs.
It’s true that I haven’t watched the interview yet, but I have confirmed with the team that the nodes are not immutable. My context is different than Hejlsberg’s context. For Hejlsberg if something is immutable within the boundaries of TS, it’s immutable. Since I work on JS APIs if something isn’t actually locked down with Object.freeze it isn’t immutable and can’t safely be treated as such. They can’t actually lock their objects down because they don’t actually completely follow the rules of immutability, and the biggest thing they do that you just can’t do with (real, proper) immutable structures is have a node.parent reference.
So they have this kinda-immutable tech, but those guarantees only hold if all the code that ever holds a reference to the node is TS code. That is why all this other infrastructure that could stand to benefit from a shared standard format for frozen nodes can’t: it’s outside the walls of the TS fiefdom, so the nodes are meant to be used as immutable but any JS code (or any-typed code) the trees are ever exposed to would have the potential to ruin them by mutating the supposedly-immutable data
To be more specific about the node.parent reference, if your tree is really truly immutable you need to replace a leaf node you must replace all the nodes on the direct path from the root to that leaf. TS does this, which is good.
The bad part is that then all the nodes you didn’t replace have chains of node.parent references that lead to the old root instead of the new one. Fixing this with immutable nodes would mean replacing every node in the tree, so the only alternative is to mutate node.parent, which means that 1) you can’t actually Object.freeze(node) and 2) you don’t get all the wins of immutability since the old data structure is corrupted by the creation of the new one.
See https://ericlippert.com/2012/06/08/red-green-trees/ for why Roslyn’s key innovation in incremental syntax trees was actually breaking the node.parent reference by splitting into the red and green trees, or as I call them paths and nodes. Nodes are deeply immutable trees and have no parents. Paths are like an address in a particular tree, tracking a node and its parents.
Hm yeah it was a very good talk. My summary of the type checking part is
The input to the type checker is immutable ASTs
That is, parsing is “embarassingly parallel”, and done per file
They currently divide the program into 4 parts (e.g. 100 files turns into 4 groups of 25 files), and they do what I’d call “soft sharding”.
That is, the translation units aren’t completely independent. Type checking isn’t embarassingly parallel. But you can still parallelize it and still get enough speedup – he says ~3x from parallelism, and ~3x from Go’s better single core perf, which gives you ~10x overall.
What wasn’t said:
I guess you have to de-duplicate the type errors? Because some type errors might come twice, since you are duplicating some work
Why the sharding is in 4 parts, and not # CPUs. Even dev machines have 8-16 cores these days, and servers can have 64-128 cores.
I guess this is just because, empirically, you don’t get more than 3x speedup.
That is interesting, but now I think it shows that TypeScript is not designed for parallel type checking. I’m not sure if other compilers do better though, like Rust (?) Apparently rustc uses the Rayon threading library. Though it’s hard to compare, since it also has to generate code
A separate thing I found kinda disappointing from the talk is that TypeScript is literally what the JavaScript code was. There was never a spec and will never be one. They have to do a line-for-line port.
There was somebody who made a lot of noise on the Github issue tracker about this, and it was basically closed “Won’t Fix” because “nobody who understands TypeScript well enough has enough time to work on a spec”. (Don’t have a link right now, but I saw it a few months ago)
Why the sharding is in 4 parts, and not # CPUs. Even dev machines have 8-16 cores these days, and servers can have 64-128 cores.
Pretty sure he said it was an arbitrary choice and they’d explore changing it. The ~10x optimization they’ve gotten so far is enough by itself to keep the project moving. Further optimization is bound to happen later.
I’m not sure if other compilers do better though, like Rust (?) Apparently rustc uses the Rayon threading library.
Work has been going on for years to parallelize rust’s frontend, but it apparently still has some issues, and so isn’t quite ready for prime time just yet, though it’s expected to be ready in the near term.
Under 8 cores and 8 threads, the parallel front end can reduce the clean build (cargo build with -Z threads=8 option) time by about 30% on average. (These crates are from compiler-benchmarks of rustc-perf)
I guess this is just because, empirically, you don’t get more than 3x speedup.
In my experience, once you start to do things “per core” and want to actually get performance out of it, you end up having to pay attention to caches, and get a bit into the weeds. Given just arbitrarily splitting up the work as part of the port has given a 10x speed increase, it’s likely they just didn’t feel like putting in the effort.
But check the chapters, they’re really split into good details. The video is interesting anyway, technically focused, no marketing spam. I can also highly recommend watching it.
Another point on “why Go and not C#” is that, he said, their current (typescript) compiler is highly functional, they use no classes at all. And Go is “just functions and data structures”, where C# has “a lot of classes”. Paraphrasing a little, but that’s roughly what he said.
Acknowledging some weak spots, Go’s in-proc JS interop story is not as good as some of its alternatives. We have upcoming plans to mitigate this, and are committed to offering a performant and ergonomic JS API.
This happens when the article author chooses to paywall it in Medium because it gives them a payout. They are basically using Lobsters as a free traffic source.
The pronunciation hint is no help, X followed by a vowel is always pronounced as in Xiaomi in my native language, so I can only read it as shit, it’s beyond automatic =P
I was looking in xit’s docs for a comparison to Jujutsu – specifically, why a whole new version control system is needed instead of a contribution to Jujutsu. The only direct mentions I found were in xit/docs/compat.md, which is about compatibility with Git:
[Jujutsu’s] approach to compatibility, however, is very different than xit’s. Jujutsu attains git compatibility by using the same on-disk repo format as git. This gives it a few advantages: […]
The repo format used by xit is completely different. It creates a .xit directory at the root of your project, and its internals have nothing in common with the .git directory you are used to.
I believe that a new on-disk repo format is critical to fixing many of git’s limitations. In particular, I think better merging and better large file support can’t be attained without moving to a new repo format. […] With this in mind, xit achieves git compatibility at the network layer instead of the storage layer.
This comparison fails to acknowledge that Jujutsu already supports the concept of multiple backends. Jujutsu’s Git backend is its better-supported one; “the native backend is used for testing purposes only”. Neither of Jujutsu’s current backends provide xit’s better large file support through using CDC, but Jujutsu’s roadmap already mentions the idea of implementing that technique. So Jujutsu will probably eventually also support usage with Git network compatibility but a different on-disk repo format.
xit’s goal of “better merging” seems to be a bigger difference. That same xit document mentions that Jujutsu uses the same merge algorithm as Git: three-way merge. The document says that xit, on the other hand, wants to “reduce the number of merge conflicts that occur in the first place”. It links to xit/docs/patch.md, which has a comparison of snapshot-based and patch-based version control.
In xit, there is a history of commits that closely mirrors git. Additionally, it computes patches for all changes to text files, which it uses when merging or cherry-picking. In this way, xit gets the primary benefit of patch-based systems (better merges), while using snapshots for everything else.
Jujutsu, like Git, is purely snapshot-based. I don’t see any mention of patch-based merging in Jujutsu’s docs or roadmap.
Merge conflicts depend a lot on your workflow. A good workflow can eliminate them.
Git is not purely snapshot-based. It very frequently calculates patches (like when you run git log) and uses them for patch-based shuffling of changes, as in cherry-pick, rebase, format-patch, am, ….
The workflow I like is to have a short feature branch for a merge request, that is based very close to the head of the main branch, and typically rebased onto the head before it gets merged. Then if there are any conflicts, they appear when applying the relevant patch, not when creating the merge commit.
Merges should always be conflict-free, so the three-way merge algorithm is basically irrelevant.
Jujutsu, like Git, is purely snapshot-based. I don’t see any mention of patch-based merging in Jujutsu’s docs or roadmap.
The jj copy-tracing proposal involves adopting certain patch-like semantics at the repo/file-level, although not the line/byte level. It’s interesting because it also requires keeping additional copy-tracing metadata alongside what you would expect from the Git object model.
When it comes to better merge algorithms, I’m more excited about things like mergiraf. Whatever small benefits an improved textual approach might bring, I expect that to pale in comparison to a merge based on an AST. Mergiraf is in early development as far as I can tell, but the idea is solid. I think it’s just a matter of time until a textual approach is basically irrelevant for day-to-day work.
Not sure I see this in such a rose-tinted way. The linked ‘bug report’ is full of people specifying very exact ways that they want to be warned beforehand: I want a blog post, the post has to be dedicated specifically to this one thing, I don’t read Reddit, the tool itself should loudly warn me several months in advance.
Notably, none of these comments actually volunteer any help with any of the above. Even the maintainer is talking about rustup as a ‘Rust project product’. I feel like everyone’s framing is wrong here…
Nothing is perfect for sure. There were some more spicy commentary on this on social media, for example. But, it’s all about the overall ratio, and the outcome.
Notably, none of these comments actually volunteer any help with any of the above.
I both agree and disagree. I think it’s normal in bug reports to report the bug, and wait for a decision to be made, before offering help. I’m sure that if they asked for help, they would have gotten it. There’s actually a pretty high degree of social cohesion here, which may not be obvious to outsiders, but I recognize virtually all of the names on that ticket, and so like, I think it’s sort of understood that if active help was desired, all you have to do is ask. The original reporter is on the language team, for example.
Even the maintainer is talking about rustup as a ‘Rust project product’.
The Rust Project has long described their output as a product. There’s pros and cons to this, but it’s been the norm for at least 12 years now. I do attribute that style of thinking to be one of the reasons why Rust has been successful.
Notably, none of these comments actually volunteer any help with any of the above. Even the maintainer is talking about rustup as a ‘Rust project product’. I feel like everyone’s framing is wrong here…
As an ex-lead of the Rust project: I don’t think the Rust project needs more volunteers all the time. It has capacity for fixes, so just stating problems is fine. A lot of people in the discussion are also people who contribute to Rust at other places, by writing libraries, etc.
I think treating a tool as a product of some kind is an important part of ensuring it delivers value to users. There are several things this mindset implies, such as having a product vision rather than just exposing your internals and calling it a day.
I wonder if it would work well to use paste bracketing for something like this. If you are pasting something that is mostly text but has a wildcard or two auto-escape it. I can also imagine similar features like if you type a quote, then paste, it will auto-escape any quotes in the pasted text.
It would probably do what you want most of the time but would probably have false-positives commonly enough that it would be negative overall. But maybe there are specific cases that are clear enough to be handled (like pasting something that starts with “https:” or immediately after a single quote).
Wouldn’t many URLs still contain the & character which would incorrectly break off the URL part-way and spawn background jobs that would almost certainly fail?
No, echo a&b works for me in fish, as does echo a&b://%20?q. I think fish might require a space before and/or after the & for it to create a background process.
I adopted hedy for an elementary school curriculum for one year, several years ago. I prefer Python syntax over JavaScript (code.org) and I like the ambition and concepts behind hedy. When I tried (several years ago) the implementation was lacking. We hit edge cases fairly often in a relatively small class size (negative). The implementers were gracious and welcoming to feedback (positive). But it was more a research project than a robust production teaching tool. I don’t know how it’s changed since.
IIRC at the time the backend was implemented in typescript. A more robust backend in something like Rust would probably have helped with edge cases.
At the time every program submitted was recorded, so we had to warn students to not put in any PII like their name or address into a program. Which was not great.
I would like to see more experimentation with how to slowly frog boil syntax knowledge. I would also like to see code.org expand their curriculum beyond block and javascript based coding to other languages. It’s really an amazing thing they’ve built.
I would like to see more experimentation with how to slowly frog boil syntax knowledge.
The decades-long research program that created the HtDP curriculum may be of interest. There’s a related teaching language and community, Pyret, that looks more like Python but shares many concepts with the Racket-based HtDP languages.
Thanks for the consideration. I clicked through. I think your expectations are off by an order of magnitude or two. When I start teaching kids they struggle with “what does the shift key do” and later “why do I need to put quote marks around both sides of my string” (not to mention “what is a string”).
Honestly, watching young 3rd grade minds smashed to bits by the minor amount of indirection provided by variables in a block based language is deeply humbling when I reflect on some of the complexity and abstraction I’m able to reason about relatively intuitively at this point.
Say you need to compute the sin of some angle
My students have never even heard of sin much less wanting to be able to compute something with it.
Hedy worked wonderfully, in gradually introducing syntax, but it missed (quality) gamification and polish (in the form of unreliable implementation). The thing I most want to preserve is joy and the ability to create. Blocks give that to kids. Text syntax is a huge leap already.
The move has been to use straight python rather than a dialect. An open question of mine is whether or not such frog-boil syntax rules helped in the long term or if throwing kids into the deep end was less confusing I.e. no starting with hate words and then gradually introducing quoting. The hardest thing with this age group is to keep them slightly challenged so they are learning but not so much that they are stuck. Joy and creation.
HtDP is a college curriculum! I think it’s reasonable for something like an AP high school course, but I wouldn’t try to teach third graders with it. Quite honestly, I wouldn’t try to teach kids “textual programming” until they’re already comfortable with a keyboard and with grammar and punctuation in their native language, as well as arithmetic. Seems like a recipe for frustration. What’s the rush?
I completely agree about joy and creation, though. I have a ten-year-old who’s taught himself quite a lot of programming basics or prerequisites just by creating custom items and command blocks in Minecraft. Sometimes he asks me for help, but mostly he’s learning by absorbing his environment, just like we all do.
AP high school course, but I wouldn’t try to teach third graders with it.
Why did you recommend it to the comment from an elementary school teacher?
Seems like a recipe for frustration. What’s the rush?
3rd is too young, but 5th is not. We want to teach them that there’s a bigger world out there, beyond blocks, before they get locked into a single paradigm of coding. Our curriculum also involves teaching typing.
I didn’t think of your comment as coming from an elementary school teacher. I was thinking about pedagogical language design, and pointing to the prior art that I’m aware of. If you’re not building a language, just trying to use something that already exists, and specifically for elementary school, then HtDP is probably not that helpful, and I’m sorry about that!
Let me try again… here’s an few-years-old lobsters story linking to a blog review of a much older book about how children relate to programming that I’ve personally found very useful in thinking about conceptual scaffolding: https://lobste.rs/s/r9thsc/mindstorms
For what it’s worth, if you’re using Python for teaching, you might check out the turtle graphics package in the standard library. “Batteries included!”
Isn’t third grade a bit too young? I’d say picking up some programming is OK for 16-year olds, as they get younger than that they wouldn’t really pick up anything very useful even as a foundation for the future.
I don’t think so. I have experimentally taught some Scratch to a bunch of second-graders during my brief stint as a school informatics teacher, and they were pretty responsive. (I quit the job for unrelated reason the next year.)
Some decade later, my own daughters have Scratch in their school curriculum, and my youngest one (will be 10 this year) additionally visits children’s programming courses by her own will, and as far as I can see, the children are extremely interested.
The goal, as far as I understand it, is not to prepare for a career in software development, but to introduce “constructing algorithms” as a tool of thought, as well as demystify computing a bit; like, “see, you can make things do this and that by your own, and it is all just ifs and cycles and responding to input/events,”
If, like me, you have a mix of iOS, Android (Graphene - primary), Windows, Mac OS and Linux (primary) but need to access your accounts on all of them, Passkeys are in an utterly terrible state. I don’t see that changing, as it seems to be all about vendor lock in, than anything benefitting a user.
Passkeys are in a nascent state as people are trying to figure out the optimal UX. I think within a couple of years we will see a clear trend to do initial set up/sign in to new devices with securely emailed magic links, then set up passkeys for easy subsequent logins. This will make lock-in a moot point–you can’t be locked in to a vendor if you can just log in with a magic link on any new device. And best of all, it will reduce to the point of nearly eliminating phishing attacks. That will be the key benefit for everyday regular users.
I still don’t understand why pledge(2)/unveil(2) have not been ported to other kernels. They have proven to work very well for a almost a decade now, as they were introduced in 2015.
We’re doing a capstone project with one of my students where we’re porting pledge to FreeBSD. We’re not doing unveil because I don’t know the internals as good as pledge, but hopefully something good comes out of this experiment.
You might want to reach out to some of the HardenedBSD community members. We have a port of pledge that is nearing completion and will likely be merged into HardenedBSD on the inside of a year, quicker if we have help. :-)
The Linux approach, for better or worse, is to provide more generic/complex kernel APIs, the two most similar to pledge/unveil being respectively seccomp-bpf and landlock. Given those, you can port pledge/unveil in userspace. But that obviously results in less uptake than an API blessed/mandated by the OS developers in the OpenBSD style.
For example, let’s say you want to do something on Linux like control whether or not some program you downloaded from the web is allowed to have telemetry.
This seems pretty easy on Linux with systemd, am I missing something? The program itself doesn’t even have to know about any control mechanisms like pledge or whatever, we can enforce it from outside.
pledge and unveil are specifically designed as tools for the programmer. They aren’t meant to be imposed from the outside as a sandbox. Theo has repeatedly rejected the idea of adding a utility to OpenBSD that runs an arbitrary command line under a particular pledge, for example. The intended use is for the developer (or knowledgeable porter) to pledge/unveil down to the resources that they know the program needs. It’s a tool for defensive programming: if your pledged process gets owned, it has as little ambient authority as possible.
I agree. The programmer knows better than the user what resources a program needs.
Besides, the external approach imposes a static set that stays the same during the entire runtime, which is sort of critical, mostly because many programs require more privileges during the initialization which can then be dropped later.
The programmer knows better than the user what resources a program needs.
I agree, but OTOH the user has the much greater incentive to care about sandboxing programs than their programmers (when the user and the programmer are not the same).
OK but my point is that the linked post says that this functionality is extremely difficult in Linux, but if we shift our thinking slightly to work with Linux’s execution resource controls, it seems to be quite easy.
It…depends. Linux sandboxing solutions are not particularly unified and depend on stitching a lot of pieces together, sometimes with confusing or bizarre edge cases. It’s not awful, but it’s still a fair bit of effort and a surprising amount of room for error.
(I will say that Landlock helps a lot here by being a relatively high-level set of access controls for things that are otherwise more difficult to sandbox.)
No, it really looks quite simple. Check the link I posted earlier. It’s as simple as adding a couple of lines to the systemd unit file or a couple of flags to the systemd-run command line.
There’s a significant structural difference between the two. I honestly don’t know why the Internet keeps lumping them together. Yes, both apply restrictions, but they’re completely different mechanisms. It’s like lumping email and keyloggers together because both ultimately boil down to delivering a series of keypresses to a third-party.
pledge and unveil are things that the program does, which has two major implications:
It’s generally expected that the program is written in such a manner that pledgeing a particular restriction is a supported mode of operation.
Breaching a pledge kills the program with SIGABRT and you get a stack trace pointing to wherever the promise was broken.
You can pledge or unveil whenever you want during a program’s execution (but there are restrictions to what you can “unpledge” or “un-unveil”). So you can e.g. start with full access rights to the user’s home path in order to read configuration, and you can drop filesystem access privileges entirely once you’ve read configuration data.
“External” sandboxing solutions allow you to apply the same restrictions, but:
They’re going to apply regardless of whether the program supports them or not. E.g. you can IPAddressDeny= an address, but if sending telemetry to that address is a blocking operation, you’ll just get a blocked program. That’s obviously not on the sandboxing method itself, but…
…breaching a restriction doesn’t necessarily give you any useful debugging context. The failure mechanism for breaching a restriction is usually operation-dependent. E.g. IPAddressDeny= packets just get dropped.
Those restrictions apply throughout a program’s execution lifecycle. So if you need higher privileges at start-up you’re out of luck, since external tools have no insight into what a program is doing.
The whole point of pledge and unveil is that you write your program so that you make certain promises about how it’s going to work, and sandboxing works as protection against accidentally breaking those promises, either straight as a bug, or through a bug getting exploited. They’re design-by-contract tools, more or less.
I’m a bit confused, I wouldn’t expect “Linux’s execution resource controls” to include systemd-specific APIs? Tbc these toggles do not cleanly map to OS primitives at all; you can’t just go “ah yes I want systemd’s IP address restrictions myself” without writing your own BPF filtering.
If you did just mean systemd specifically, note that it’s quite a bit trickier to, say, sandbox things in a flexible way with that. In particular, to sandbox something that’s not a service with systemd-run has the overhead of spawning the child under the service manager and piping the output, and you lose the ability to easily control and kill the child (because you’re just killing the proxying process, not the actual child).
I didn’t say it’s a ‘flat out replacement’, I said it achieves the same goals that were mentioned in the linked post ie preventing a piece of software I downloaded off the internet from ‘phoning home’ and uploading telemetry.
Okay I understand now.. The way I see it, the example in that post was a one-off example, and a pretty bad one, because I think from this you’ve misunderstood the purpose of pledge. While there may be some overlap in the functionaility of BPF filters they are far, far, from 1-1.
To help clear this up:
pledge is not for sandboxing existing/binary (i.e. already complied) software, nor limiting network connections in software (the former would preferably be done with network controls like firewalls).
pledge must be added to the source code of the software itself (either by a dev or maintainer).
on a violation of “promises” the program is killed
you can limit network connection but it’s simply “all or nothing”; your software gets all network, or no network.
So in this scenerio of telemetry:
if devs/maintainers have source access to add pledge then they also have source access to just rip out that telemetry code right?
There would be no point in using pledge here.
in using pledge, as soon as the software tries to do any networking, the program is killed! The connection is blocked – only because your software is now not running at all!
Yeah exactly, the overlap is what I’m talking about. This is what I was referring to earlier–if you shift your mindset a bit, it becomes pretty easy to externally impose similar security restrictions using systemd. If you don’t mind that the exact way it’s done is a bit different, you can achieve very similar goals.
Part of the pledge/unveil model is that you can progressively drop privileges during program execution. The observation is that programs may need a lot of privileges in order to start themselves up, and then hardly any for the remainder of runtime. For example, consider the recommendations for using pledge when opening handles to sndiod in OpenBSD:
If the sndio library is used in combination with pledge(2), then the sio_open() function needs the stdio, rpath, wpath, cpath, inet, unix, dns, and audio pledge(2) promises.
However:
Once no further calls to sio_open() will be made, all these pledge(2) promises may be dropped, except for the audio promise.
And in fact aucat(1) does exactly this, using a bunch of privileges at startup time (but dropping other very sensitive promises like proc and exec right away) and then dropping most of them when it’s time to start playback or recording from the sound device handle.
The purpose of pledge is for the programmer to mitigate the privileges available to exploits, not for the user to defend against code they chose to run. As far as I know, these goals, which are the main point of pledge and unveil, do not overlap at all with what you can do by filtering access via systemd.
They overlap with the goal that was stated in the linked post. That was what I was referring to. I agree that dropping complex combinations of privileges in Linux at different points within an application’s lifecycle doesn’t seem easy.
Yeah but the linked post misunderstands what pledge is actually for; “sandboxing untrusted binaries” is not an actual design goal of pledge and if it’s useful for that goal it’s only useful by accident.
Yeah so this is kinda weird: it makes it easy to manage the child as a single entity and then interactively control it. But systemd-run is, ofc, just a proxy, spawning the child as a transient unit and monitoring it until exit.
This effectively means that systemd-run does not itself control the spawned process’s lifetime. When you run it with –pty, Ctrl-C will kill the child normally… because it’s going through the tty. But if you’re running it detached from the tty (e.g. because you need to control stdout/stdin from your program), or you just signal the process directly, the actual service will remain running in the background. The only way of directly controlling that lifecycle is by grabbing the PID or (preferably) transient unit name and then targeting it directly or controlling it via the D-Bus interface.
But it’s a transient unit, it shows up in the output of systemctl list-units, right? That means you can operate on it with eg systemctl stop the-unit just like other systemd units?
It is, yeah, which is why I had mentioned the “it makes it easy to […] interactively control it”. It’s just a lot harder in turn to automatically do things like “oh the user did Ctrl-C, let me stop all my processes”.
From my admittedly passing acquaintance with both of them: Capsicum is much finer grained.
Pledge/unveil solves the 80% use case, Capsicum aims to solve the 100% use-case.
I.e. it would be very difficult, perhaps impossible to implement capsicum in pledge/unveil, but easy enough to do with capsicum.
That said, there is probably more veil/pledge actually implemented in binaries out in the wild. I think all of the base OpenBSD system is done now. Last I checked only some of FreeBSD has Capsicum implemented in base.
How do you like to run your bookmarklets? I have the bookmarks navbar hidden to save screen space and so bookmarklets are always hidden enough for me not to think about them.
I have a couple that I use all the time, like ‘scroll to top/bottom’, so I have them on my bookmarks toolbar itself. For the rest, I have them in a subfolder inside my bookmarks toolbar.
Not available on typical Mac laptop keyboards, and even if they were, often my hand is on the touchpad so it’s just more convenient to tap on a button instead of moving my head to a key.
While I think moving on from C to a memory safe systems programming language is a good idea, for various reasons, I don’t think Rust will end up being a good choice. Ultimately, I think the choice of Rust is going to hurt Linux.
Zig is nicer than C, but doesn’t bring any fundamental improvements. A big part of the reasoning for introducing Rust to the kernel despite the difficulty and complexity is that memory safety is a big deal, and Zig has no memory safety story at all.
The majority of bugs (quantity, not quality/severity) we have are due to the stupid little corner cases in C that are totally gone in Rust. Things like simple overwrites of memory (not that rust can catch all of these by far), error path cleanups, forgetting to check error values, and use-after-free mistakes. That’s why I’m wanting to see Rust get into the kernel, these types of issues just go away, allowing developers and maintainers more time to focus on the REAL
bugs that happen (i.e. logic issues, race conditions, etc.)
This is an extremely strong statement.
I think a few things are also interesting:
I think people are realizing how low quality the Linux kernel code is, how haphazard development is, how much burnout and misery is involved, etc.
I think people are realizing how insanely not in the open kernel dev is, how much is private conversations that a few are privy to, how much is politics, etc.
I think people are realizing how insanely not in the open kernel dev is, how much is private conversations that a few are privy to, how much is politics, etc.
The Hellwig/Ojeda part of the thread is just frustrating to read because it almost feels like pleading. “We went over this in private” “we discussed this already, why are you bringing it up again?” “Linus said (in private so there’s no record)”, etc., etc.
Dragging discussions out in front of an audience is a pretty decent tactic for dealing with obstinate maintainers. They don’t like to explain their shoddy reasoning in front of people, and would prefer it remain hidden. It isn’t the first tool in the toolbelt but at a certain point there is no convincing people directly.
Dragging discussions out in front of an audience is a pretty decent tactic for dealing with
With quite a few things actually. A friend of mine is contributing to a non-profit, which until recently had this very toxic member (they’ve even attempted felony). They were driven out of the non-profit very soon after members talked in a thread that was accessible to all members. Obscurity is often one key component of abuse, be it mere stubbornness or criminal behaviour. Shine light, and it often goes away.
IIRC Hintjens noted this quite explicitly as a tactic of bad actors in his works.
It’s amazing how quickly people are to recognize folks trying to subvert an org piecemeal via one-off private conversations once everybody can compare notes. It’s equally amazing to see how much the same people beforehand will swear up and down oh no that’s a conspiracy theory such things can’t happen here until they’ve been burned at least once.
This is an active, unpatched attack vector in most communities.
I’ve found the lowest example of this is even meetings minutes at work. I’ve observed that people tend to act more collaboratively and seek the common good if there are public minutes, as opposed to trying to “privately” win people over to their desires.
I think people are realizing how low quality the Linux kernel code is, how haphazard development is, how much burnout and misery is involved, etc.
Something I’ve noticed is true in virtually everything I’ve looked deeply at is the majority of work is poor to mediocre and most people are not especially great at their jobs. So it wouldn’t surprise me if Linux is the same. (…and also wouldn’t surprise me if the wonderful Rust rewrite also ends up poor to mediocre.)
yet at the same time, another thing that astonishes me is how much stuff actually does get done and how well things manage to work anyway. And Linux also does a lot and works pretty well. Mediocre over the years can end up pretty good.
After tangentially following the kernel news, I think a lot of churning and death spiraling is happening. I would much rather have a rust-first kernel that isn’t crippled by the old guard of C developers reluctant to adopt new tech.
Take all of this energy into RedoxOS and let Linux stay in antiquity.
I’ve seen some of the R4L people talk on Mastodon, and they all seem to hate this argument.
They want to contribute to Linux because they use it, want to use it, and want to improve the lives of everyone who uses it. The fact that it’s out there and deployed and not a toy is a huge part of the reason why they want to improve it.
Hopping off into their own little projects which may or may not be useful to someone in 5-10 years’ time is not interesting to them. If it was, they’d already be working on Redox.
The most effective thing that could happen is for the Linux foundation, and Linus himself, to formally endorse and run a Rust-based kernel. They can adopt an existing one or make a concerted effort to replace large chunks of Linux’s C with Rust.
IMO the Linux project needs to figure out something pretty quickly because it seems to be bleeding maintainers and Linus isn’t getting any younger.
They may be misunderstanding the idea that others are not necessarily incentivized to do things just because it’s interesting for them (the Mastodon posters).
Redox does have the chains of trying to do new OS things. An ABI-compatible Rust rewrite of the Linux kernel might get further along than expected (even if it only runs in virtual contexts, without hardware support (that would come later.))
Linux developers want to work on Linux, they don’t want to make a new OS. Linux is incredibly important, and companies already have Rust-only drivers for their hardware.
Basically, sure, a new OS project would be neat, but it’s really just completely off topic in the sense that it’s not a solution for Rust for Linux. Because the “Linux” part in that matters.
I read a 25+ year old article [1] from a former Netscape developer that I think applies in part
The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it. It doesn’t acquire bugs just by sitting around on your hard drive. Au contraire, baby! Is software supposed to be like an old Dodge Dart, that rusts just sitting in the garage? Is software like a teddy bear that’s kind of gross if it’s not made out of all new material?
Adopting a “rust-first” kernel is throwing the baby out with the bathwater. Linux has been beaten into submission for over 30 years for a reason. It’s the largest collaborative project in human history and over 30 million lines of code. Throwing it out and starting new would be an absolutely herculean effort that would likely take years, if it ever got off the ground.
The idea that old code is better than new code is patently absurd. Old code has stagnated. It was built using substandard, out of date methodologies. No one remembers what’s a bug and what’s a feature, and everyone is too scared to fix anything because of it. It doesn’t acquire new bugs because no one is willing to work on that weird ass bespoke shit you did with your C preprocessor. Au contraire, baby! Is software supposed to never learn? Are we never to adopt new tools? Can we never look at something we’ve built in an old way and wonder if new methodologies would produce something better?
This is what it looks like to say nothing, to beg the question. Numerous empirical claims, where is the justification?
It’s also self defeating on its face. I take an old codebase, I fix a bug, the codebase is now new. Which one is better?
Like most things in life the truth is somewhere in the middle. There is a reason there is the concept of a “mature node” in the semiconductor industry. They accept that new is needed for each node, but also that the new thing takes time to iron out the kinks and bugs. This is the primary reason why you see apple take new nodes on first before Nvidia for example, as Nvidia require much larger die sizes, and so less defects per square mm.
You can see this sometimes in software for example X11 vs Wayland, where adoption is slow, but most definetly progressing and now-days most people can see that Wayland is now, or is going to become the dominant tech in the space.
I don’t think this would qualify as dialectic, it lacks any internal debate and it leans heavily on appeals by analogy and intuition/ emotion. The post itself makes a ton of empirical claims without justification even beyond the quoted bit.
That means we can probably keep a lot of the old trusty Linux code around while making more of the new code safe by writing it in Rust in the first place.
I don’t think that’s a fair assessment of Spolsky’s argument or of CursedSilicon’s application of it to the Linux kernel.
Firstly, someone has already pointed out the research that suggests that existing code has fewer bugs in than new code (and that the older code is, the less likely it is to be buggy).
Secondly, this discussion is mainly around entire codebases, not just existing code. Codebases usually have an entire infrastructure around them for verifying that the behaviour of the codebase has not changed. This is often made up of tests, but it’s also made up of the users who try out a release of a codebase and determine whether it’s working for them. The difference between making a change to an existing codebase and releasing a new project largely comes down to whether this verification (both in terms of automated tests and in terms of users’ ability to use the new release) works for the new code.
Given this difference, if I want to (say) write a new OS completely in Rust, I need to choose: Do I want to make it completely compatible with Linux, and therefore take on the significant challenge of making sure everything behaves truly the same? Or do I make significant breaking changes, write my own OS, and therefore force potential adopters to rebuild their entire Linux workflows in my new OS?
The point is not that either of these options are bad, it is that they represent significant risks to a project. Added to the general risk that is writing new code, this produces a total level of risk that might be considered the baseline risk of doing a rewrite. Now risk is not bad per se! If the benefits of being able to write an OS in a language like Rust outweigh the potential risks, then it still makes sense to perform the rewrite. Or maybe the existing Linux kernel is so difficult to maintain that a new codebase really would be the better option. But the point that CursedSilicon was making by linking the Spolsky piece was, I believe, that the risks for a project like the Linux kernel are very high. There is a lot of existing, old code. And there is a very large ecosystem where either breaking or maintaining compatibility would each come with significant challenges.
Unfortunately, it’s very difficult to measure the risks and benefits here in a quantitative, comparable way, so I think where you fall on the “rewrite vs continuity” spectrum will depend mostly on what sort of examples you’ve seen, and how close you think this case is to those examples. I don’t think there’s any objective way to say whether it makes more sense to have something like R4L, or something like RedoxOS.
Firstly, someone has already pointed out the research that suggests that existing code has fewer bugs in than new code (and that the older code is, the less likely it is to be buggy).
I haven’t read it yet, but I haven’t made an argument about that, I just created a parody of the argument as presented. I’ll be candid, i doubt that the research is going to compel me to believe that newer code is inherently buggier, it may compel me to confirm my existing belief that testing software in the field is one good method to find some classes of bugs.
Secondly, this discussion is mainly around entire codebases, not just existing code.
I guess so, it’s a bit dependent on where we say the discussion starts - three things are relevant; RFL, which is not a wholesale rewrite, a wholesale rewrite of the Linux kernel, and Netscape. RFL is not about replacing the entire Linux kernel, although perhaps “codebase” here refers to some sort of unit, like a driver. Netscape wanted a wholesale rewrite, based on the linked post, so perhaps that’s what’s really “the single worst strategic mistake that any software company can make”, but I wonder what the boundary here is? Also, the article immediately mentions that Microsoft tried to do this with Word but it failed, but that Word didn’t suffer from this because it was still actively developed - I wonder if it really “failed” just because pyramid didn’t become the new Word? Did Microsoft have some lessons learned, or incorporate some of that code? Dunno.
I think I’m really entirely justified when I say that the post is entirely emotional/ intuitive appeals, rhetoric, and that it makes empirical claims without justification.
There’s a subtle reason that programmers always want to throw away the code and start over. The reason is that they think the old code is a mess. And here is the interesting observation: they are probably wrong. The reason that they think the old code is a mess is because of a cardinal, fundamental law of programming:
This is rhetoric. These are unsubstantiated empirical claims. The article is all of this. It’s fine as an interesting, thought provoking read that gets to the root of our intuitions, but I think anyone can dismiss it pretty easily since it doesn’t really provide much in the form of an argument.
It’s important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time.
Again, totally unsubstantiated. I have MANY reasons to believe that, it is simply question begging to say otherwise.
That’s all this post is. Over and over again making empirical claims with no evidence and question beggign.
We can discuss the risks and benefits, I’d advocate for that. This article posted doesn’t advocate for that. It’s rhetoric.
existing code has fewer bugs in than new code (and that the older code is, the less likely it is to be buggy).
This is a truism. It is survival bias. If the code was buggy, it would eventually be found and fixed. So all things being equal newer code is riskier than old code. But it’s also been impirically shown that using Rust for new code is not “all things being equal”. Google showed that new code in Rust is as reliable as old code in C. Which is good news: you can use old C code from new Rust projects without the risk that comes from new C code.
But it’s also been impirically shown that using Rust for new code is not “all things being equal”.
Yeah, this is what I’ve been saying (not sure if you’d meant to respond to me or the parent, since we agree) - the issue isn’t “new” vs “old” it’s things like “reviewed vs unreviewed” or “released vs unreleased” or “tested well vs not tested well” or “class of bugs is trivial to express vs class of bugs is difficult to express” etc.
I don’t disagree that the rewards can outweigh the risks, and in this case I think there’s a lot of evidence that suggests that memory safety as a default is really important for all sorts of reasons. Let alone the many other PL developments that make Rust a much more suitable language to develop in than C.
It’s a Ship of Theseus—at no point can you call it a “new” codebase, but after a period of time, it could be completely different code. I have a C program I’ve been using and modifying for 25 years. At any given point, it would have been hard to say “this is now a new codebase,
yet not one line of code in the project is the same as when I started (even though it does the same thing at it always has).
I don’t see the point in your question. It’s going to depend on the codebase, and on the nature of the changes; it’s going to be nuanced, and subjective at least to some degree. But the fact that it’s prone to subjectivity doesn’t mean that you get to call an old codebase with a single fixed bug a new codebase, without some heavy qualification which was lacking.
What’s old and new is poorly defined and yet there’s an argument being made that “old” and “new” are good indicators of something. If they’re so poorly defined that we have to bring in all sorts of additional context like the nature of the changes, not just when they happened or the number of lines changed, etc, then it seems to me that we would be just as well served to throw away the “old” and “new” and focus on that context.
I feel like enough people would agree more-or-less on what was an “old” or “new” codebase (i.e. they would agree given particular context) that they remain useful terms in a discussion. The general context used here is apparent (at least to me) given by the discussion so far: an older codebase has been around for a while, has been maintained, has had kinks ironed out.
There’s a really important distinction here though. The point is to argue that new projects will be less stable than old ones, but you’re intuitively (and correctly) bringing in far more important context - maintenance, testing, battle testing, etc. If a new implementation has a higher degree of those properties then it being “new” stops being relevant.
It’s also self defeating on its face. I take an old codebase, I fix a bug, the codebase is now new. Which one is better?
My point was that this statement requires a definition of “new codebase” that nobody would agree with, at least in the context of the discussion we’re in. Maybe you are attacking the base proposition without applying the surrounding context, which might be valid if this were a formal argument and not a free-for-all discussion.
If a new implementation has a higher degree of those properties
I think that it would be considered no longer new if it had had significant battle-testing, for example.
FWIW the important thing in my view is that every new codebase is a potential old codebase (given time and care), and a rewrite necessarily involves a step backwards. The question should probably not be, which is immediately better?, but, which is better in the longer term (and by how much)? However your point that “new codebase” is not automatically worse is certainly valid. There are other factors than age and “time in the field” that determine quality.
Methodologies don’t matter for quality of code. They could be useful for estimates, cost control, figuring out whom you shall fire etc. But not for the quality of code.
I’ve never observed a programmer become better or worse by switching methodology. Dijkstra would’ve not became better if you made him do daily standups or go through code reviews.
There are ways to improve your programming by choosing different approach but these are very individual. Methodology is mostly a beancounting tool.
When I say “methodology” I’m speaking very broadly - simply “the approach one takes”. This isn’t necessarily saying that any methodology is better than any other. The way I approach a task today is better, I think, then the way that I would have approached that task a decade ago - my methodology has changed, the way I think has changed. Perhaps that might mean I write more tests, or I test earlier, but it may mean exactly the opposite, and my methods may only work best for me.
I’m not advocating for “process” or ubiquity, only that the approach one tasks may improve over time, which I suspect we would agree on.
It’s the largest collaborative project in human history and over 30 million lines of code.
How many of those lines are part of the core? My understanding was that the overwhelming majority was driver code. There may not be that much core subsystem code to rewrite.
For a previous project, we included a minimal Linux build. It was around 300 KLoC, which included networking and the storage stack, along with virtio drivers.
That’s around the size a single person could manage and quite easy with a motivated team.
If you started with DPDK and SPDK then you’d already have filesystems and a copy of the FreeBSD network stack to run in isolated environments.
Once many drivers share common rust wrappers over core subsystems, you could flip it and write the subsystem in Rust. Then expose C interface for the rest.
I see that Drew proposes a new OS in that linked article, but I think a better proposal in the same vein is a fork. You get to keep Linux, but you can start porting logic to Rust unimpeded, and it’s a manageable amount of work to keep porting upstream changes.
Remember when libav forked from ffmpeg? Michael Niedermayer single-handedly ported every single libav commit back into ffmpeg, and eventually, ffmpeg won.
At first there will be extremely high C percentage, low Rust percentage, so porting is trivial, just git merge and there will be no conflicts. As the fork ports more and more C code to Rust, however, you start to have to do porting work by inspecting the C code and determining whether the fixes apply to the corresponding Rust code. However, at that point, it means you should start seeing productivity gains, community gains, and feature gains from using a better language than C. At this point the community growth should be able to keep up with the extra porting work required. And this is when distros will start sniffing around, at first offering variants of the distro that uses the forked kernel, and if they like what they taste, they might even drop the original.
I genuinely think it’s a strong idea, given the momentum and potential amount of labor Rust community has at its disposal.
I think the competition would be great, especially in the domain of making it more contributor friendly to improve the kernel(s) that we use daily.
I certainly don’t think this is impossible, for sure. But the point ultimately still stands: Linux kernel devs don’t want a fork. They want Linux. These folks aren’t interested in competing, they’re interested in making the project they work on better. We’ll see if some others choose the fork route, but it’s still ultimately not the point of this project.
Linux developers want to work on Linux, they don’t want to make a new OS.
While I don’t personally want to make a new OS, I’m not sure I actually want to work on Linux. Most of the time I strive for portability, and so abstract myself from the OS whenever I can get away with it. And when I can’t, I have to say Linux’s API isn’t always that great, compared to what the BSDs have to offer (epoll vs kqueue comes to mind). Most annoying though is the lack of documentation for the less used APIs: I’ve recently worked with Netlink sockets, and for the proc stuff so far the best documentation I found was the freaking source code of a third party monitoring program.
I was shocked. Complete documentation of the public API is the minimum bar for a project as serious of the Linux kernel. I can live with an API I don’t like, but lack of documentation is a deal breaker.
While I don’t personally want to make a new OS, I’m not sure I actually want to work on Linux.
I think they mean that Linux kernel devs want to work on the Linux kernel. Most (all?) R4L devs are long time Linux kernel devs. Though, maybe some of the people resigning over LKML toxicity will go work on Redox or something…
Re-Implementing the kernel ABI would be a ton of work for little gain if all they wanted was to upstream all the work on new hardware drivers that is already done - and then eventually start re-implementing bits that need to be revised anyway.
If the singular required Rust toolchain didn’t feel like such a ridiculous to bootstrap 500 ton LLVM clown car I would agree with this statement without reservation.
Zig is easier to implement (and I personally like it as a language) but doesn’t have the same safety guarantees and strong type system that Rust does. It’s a give and take. I actually really like Rust and would like to see a proliferation of toolchain options, such as what’s in progress in GCC land. Overall, it would just be really nice to have an easily bootstrapped toolchain that a normal person can compile from scratch locally, although I don’t think it necessarily needs to be the default, or that using LLVM generally is an issue. However, it might be possible that no matter how you architect it, Rust might just be complicated enough that any sufficiently useful toolchain for the language could just end up being a 500 ton clown car of some kind anyways.
Depends on which parts of GP’s statement you care about: LLVM or bootstrap. Zig is still depending on LLVM (for now), but it is no longer bootstrappable in a limited number of steps (because they switched from a bootstrap C++ implementation of the compiler to keeping a compressed WASM build of the compiler as a blob.
Yep, although I would also add it’s unfair to judge Zig in any case on this matter now given it’s such a young project that clearly is going to evolve a lot before the dust begins to settle (Rust is also young, but not nearly as young as Zig). In ten to twenty years, so long as we’re all still typing away on our keyboards, we might have a dozen Zig 1.0 and a half dozen Zig 2.0 implementations!
Yeah, the absurdly low code quality and toxic environment make me think that Linux is ripe for disruption. Not like anyone can produce a production kernel overnight, but maybe a few years of sustained work might see a functional, production-ready Rust kernel for some niche applications and from there it could be expanded gradually. While it would have a lot of catching up to do with respect to Linux, I would expect it to mature much faster because of Rust, because of a lack of cruft/backwards-compatibility promises, and most importantly because it could avoid the pointless drama and toxicity that burn people out and prevent people from contributing in the first place.
From the thread in OP, if you expand the messages, there is wide agreement among the maintainers that all sorts of really badly designed and almost impossible to use (safely) APIs ended up in the kernel over the years because the developers were inexperienced and kind of learning kernel development as they went. In retrospect they would have designed many of the APIs very differently.
It’s based on my forays into the Linux kernel source code. I don’t doubt there’s some quality code lurking around somewhere, but the stuff I’ve come across (largely filesystem and filesystem adjacent) is baffling.
Seeing how many people are confidently incorrect about Linux maintainers only caring about their job security and keeping code bad to make it a barrier to entry, if nothing else taught me how online discussions are a huge game of Chinese whispers where most participants don’t have a clue of what they are talking about.
I doubt that maintainers are “only caring about their job security and keeping back code” but with all due respect: You’re also just taking arguments out of thin air right now. What I do believe is what we have seen: Pretty toxic responses from some people and a whole lot of issues trying to move forward.
Seeing how many people are confidently incorrect about Linux maintainers only caring about their job security and keeping code bad to make it a barrier to entry
Huh, I’m not seeing any claim to this end from the GP, or did I not look hard enough? At face value, saying that something has an “absurdly low code quality” does not imply anything about nefarious motives.
Still, in GP’s case the Chinese whispers have reduced “the safety of this API is hard to formalize and you pretty much have to use it the way everybody does it” to “absurdly low quality”. To which I ask, what is more likely. 1) That 30-million lines of code contain various levels of technical debt of which maintainers are aware; and that said maintainers are worried even of code where the technical debt is real but not causing substantial issue in practice? Or 2) that a piece of software gets to run on literally billions of devices of all sizes and prices just because it’s free and in spite of its “absurdly low quality”?
Linux is not perfect, neither technically nor socially. But it sure takes a lot of entitlement and self-righteousness to declare it “of absurdly low quality” with a straight face.
GP here: I probably should have said “shockingly” rather than “absurdly”. I didn’t really expect to get lawyered over that one word, but yeah, the idea was that for a software that runs on billions of devices, the code quality is shockingly low.
Of course, this is plainly subjective. If your code quality standards are a lot lower than mine then you might disagree with my assessment.
That said, I suspect adoption is a poor proxy for code quality. Internet Explorer was widely adopted and yet it’s broadly understood to have been poorly written.
But it sure takes a lot of entitlement and self-righteousness to declare it “of absurdly low quality” with a straight face
I’m sure self-righteousness could get you to the same place, but in my case I arrived by way of experience. You can relax, I wasn’t attacking Linux—I like Linux—it just has a lot of opportunity for improvement.
I guess I’ve seen the internals of too much proprietary software now to be shocked by anything about Linux per se. I might even argue that the quality of Linux is surprisingly good, considering its origins and development model.
I think I’d lawyer you a tiny bit differently: some of the bugs in the kernel shock me when I consider how many devices run that code and fulfill their purposes despite those bugs.
FWIW, I was not making a dig at open source software, and yes plenty of corporate software is worse. I guess my expectations for Linux are higher because of how often it is touted as exemplary in some form or another. I don’t even dislike Linux, I think it’s the best thing out there for a huge swath of use cases—I just see some pretty big opportunities for improvement.
But it sure takes a lot of entitlement and self-righteousness to declare it “of absurdly low quality” with a straight face.
Or actual benchmarks: the performance the Linux kernel leaves on the table in some cases is absurd. And sure it’s just one example, but I wouldn’t be surprised if it was representative of a good portion of the kernel.
Well not quite but still “considered broken beyond repair by many people related to life time management” - which is definitely worse than “hard to formalize” when “the way ever[y]body does it” seems to vary between each user.
I love Rust but still, we’re talking of a language which (for good reasons!) considers doubly linked lists unsafe. Take an API that gets a 4 on Rusty Russell’s API design scale (“Follow common convention and you’ll get it right”), but which was designed for a completely different programming language if not paradigm, and it’s not surprising that it can’t easily be transformed into a 9 (“The compiler/linker won’t let you get it wrong”). But at the same time there are a dozen ways in which, according to the same scale, things could actually be worse!
What I dislike is that people are seeing “awareness of complexity” and the message they spread is “absurdly low quality”.
Note that doubly linked lists are not a special case at all in Rust. All the other common data structures like Vec, HashMap etc. also need unsafe code in their implementation.
Implementing these datastructures in Rust, and writing unsafe code in general, is indeed roughly a 4. But these are all already implemented in the standard library, with an API that actually is at a 9. And std::collections::LinkedList is constructive proof that you can have a safe Rust abstraction for doubly linked lists.
Yes, the implementation could have bugs, thus making the abstraction leaky. But that’s the case for literally everything, down to the hardware that your code runs on.
You’re absolutely right that you can build abstractions with enough effort.
My point is that if a doubly linked list is (again, for good reasons) hard to make into a 9, a 20-year-old API may very well be even harder. In fact, std::collections::LinkedList is safe but still not great (for example the cursor API is still unstable); and being in std, it was designed/reviewed by some of the most knowledgeable Rust developers, sort of by definition. That’s the conundrum that maintainers face and, if they realize that, it’s a good thing. I would be scared if maintainers handwaved that away.
Yes, the implementation could have bugs, thus making the abstraction leaky.
Bugs happen, but if the abstraction is downright wrong then that’s something I wouldn’t underestimate. A lot of the appeal of Rust in Linux lies exactly in documenting/formalizing these unwritten rules, and wrong documentation can be worse than no documentation (cue the negative parts of the API design scale!); even more so if your documentation is a formal model like a set of Rust types and functions.
That said, the same thing can happen in a Rust-first kernel, which will also have a lot of unsafe code. And it would be much harder to fix it in a Rust-first kernel, than in Linux at a time when it’s just feeling the waters.
In fact, std::collections::LinkedList is safe but still not great (for example the cursor API is still unstable); and being in std, it was designed/reviewed by some of the most knowledgeable Rust developers, sort of by definition.
At the same time, it was included almost as like, half a joke, and nobody uses it, so there’s not a lot of pressure to actually finish off the cursor API.
It’s also not the kind of linked list the kernel would use, as they’d want an intrusive one.
And yet, safe to use doubly linked lists written in Rust exist. That the implementation needs unsafe is not a real problem. That’s how we should look at wrapping C code in safe Rust abstractions.
The whole comment you replied to, after the one sentence about linked lists, is about abstractions. And abstractions are rarely going to be easy, and sometimes could be hardly possible.
That’s just a fact. Confusing this fact for something as hyperbolic as “absurdly low quality” is stunning example of the Dunning Kruger effect, and frankly insulting as well.
I personally would call Linux low quality because many parts of it are buggy as sin. My GPU stops working properly literally every other time I upgrade Linux.
No one is saying that Linux is low quality because it’s hard or impossible to abstract some subsystems in Rust, they’re saying it’s low quality because a lot of it barely works! I would say that your “Chinese whispers” misrepresents the situation and what people here are actually saying. “the safety of this API is hard to formalize and you pretty much have to use it the way everybody does it” doesn’t apply if no one can tell you how to use an API, and everyone does it differently.
Actually, the NT kernel of all things seems to have a pretty good reputation, and I wouldn’t dismiss the BSD kernels out of hand. I don’t know which kernel is better, but it seems you do. If you could explain how you came to this conclusion that would be most helpful.
*nod* I haven’t been a Windows person since shortly after the release of Windows XP (i.e. the first online activation DRM’d Windows) but, whenever I see glimpses of what’s going on inside the NT kernel in places like Project Zero: The Definitive Guide on Win32 to NT Path Conversion, it really makes me want to know more.
One thing that stood out to me is that systemd’s configuration approach is declarative while others are imperative, and this unlocks a lot of benefits, much like how a query planner can execute a declarative query better than most hand-written imperative queries could.
Yea, it’s unreasonbly effective, and in general all the components hang together really well.
In my experience the nicest systems to administer have kind of an “alternating layer” approach of imperative - declarative - imperative - declarative - … Papering over too much complexity with declarative causes trouble, and so does forcing (or encouraging, or allowing) too much imperative scripting at any given layer. Systemd (it seems to me, as a user) really nailed the layers to encapsulate declaratively - things you want to happen at boot, that depend on each other. Units are declarative but effectively “call into” imperative commands, which - in turn - ought to have their own declarative-ish configuration files, which - in turn - are best laid down by something programmable at the higher layer (something aware of larger state - what load balancers are up - what endpoints are up - what DNS servers are up, yadda yadda).
As an aside, my main beef with Kubernetes is that is has a bad design - or perhaps a missing design - for that next level up of configuration management. Way way way too much is squashed into a uniform morass of the generalized “object namespace”, and the only way to really get precise behavior out of that is to implement a custom controller, which is almost like taking on the responsibility of a mini-init system, or a mini-network config daemon.
What I want at this layer instead is a framework for writing a simple and declarative-ish DSL that lets me encode my own business domain. Not every daemon is a Deployment object. Not every service is a Service. I think we were just getting good at this in 2014 with systems like Salt and Chef, but K8s came in and sucked all the oxygen out of the room for a decade, not to mention that Salt and Chef did themselves no favors by simply being a metric boatload of not-always-easy-to-install scripting runtimes.
Years ago when I looked to contribute to Salt I was pretty astounded by the bugginess, but it worked for enough people and there was not and still is not a lot of competition in that space, so VMware acquired them.
We have yet to see the next generation of that sort of software. System Initiative may be the sole exception, and I am enthusiastic about the demos they put out. But I do think we’re missing something here.
Then I guess a story like this is also because it has nothing to do with computing? This is just a fact of life of using commercial technology in the current business environment.
gossip, shaming and outrage porn (which is not on topic)
a malicious act of surreptitiously introducing exfiltration into a somewhat popular product, so security (which is on topic)
a liability to company secrecy/copyright (so maybe law, which is on topic)
a badly implemented ai feature (which is on topic)
as some people mentioned, there could be privacy issues if it’s a personal repo (which is also on topic)
I do agree that the tag doesn’t really cover the issue (I mean, it is a nix-related tool, so that’s probably fine, but perhaps all of the aforementioned tags should also apply?) and the link is to the social media mob frenzy which is less than ideal.
There is no legal basis for sending all of your local files to a server. In particular there is no reasonable expectation that running a local command would result in a third party processing your data in this way or certainty that this is what the user intended. It is neither made obvious to the user and minimal in privacy impact, as required for a basis in Legitimate Interest, nor is any consent gathered.
I find that highly doubtful; we’re talking about code here. Maybe it would apply if you’re storing personal information in the specific repo, but even then it’s still a user-initiated action, and (at least the claim is) the service doesn’t keep any of the information.
At the very least there is a lot of grey area here, people can easily have repositories where they store personal information, eg someone else gave an example of data science repos where datasets are stored using LFS. Or another example, many code repositories have files where authors’ names and email addresses are mentioned, eg package metadata files.
The argument can definitely be made that this command was slurping up data without regard to whether it might be personal information or not.
Even if they were, you should not take legal advice from a lawyer you are not paying, and you should definitely check with your own lawyer if any doubt has been raised about legality.
So this is still kinda awful, but this seems to specifically be for the “devenv generate” command whose description explicitly states it’s to generate a new project using “AI”. For any current users, this wouldn’t affect the workflow (it’s a separate command from the usual new-project one), and the nature of the command would already imply it’s talking to an external server.
I think this misses what I perceive to be the main issue here. Yes, using the AI feature implies that your repo will be sent to an AI, but AI providers claim not to retain conversations sent via API and perhaps you’ve already chosen to trust them on this claim. But this generate feature seems to go out of its way to retain a copy on a server controlled by the devenv author, which is very surprising and shady.
this generate feature seems to go out of its way to retain a copy on a server controlled by the devenv author,
Do you have a pointer of this happening? Hope I don’t come across as “sealioning” here, but the boundary between privacy and productivity in AI-enhanced stuff is something we should all be very careful and factual about.
I don’t have a pointer to some place in some code that shows a copy being made, but the server receiving the code generation request is closed-source, as far as I know.
I’ve essentially derived this assesment from the context, especially this comment from Domen:
It’s going to make it really hard to improve the generation without telemetry, otherwise we wouldn’t have done it.
I can’t imagine how they could possibly “improve the generation” based on whatever it is that they are retaining when DO_NOT_TRACK is not set unless they retain the full input (which is a copy of the repo) so that they can feed it during model tuning (or maybe prompt tuning?)
I don’t think it’s really going “out of its way”: the LLM itself is hosting on devenv servers, and unfortunately the server side seems to be a black box…but by extension, there’s no particular indication of it doing that.
“you now need, at the very least, backups” This is true of every database; how is this a criticism of sqlite? “If you want to run the service across multiple machines you can use LiteFS” It seems like the core use case for sqlite is situations where your data and worker all fit on one machine. I’m unfamiliar with LiteFS besides this blog’s description, but its description argues in favor of sqlite because it describes LiteFS like it’s either a solution to that limitation or a bridging strategy while you migrate to another database.
“Migrations are not great in SQLite.” This sounds like a real concern but “not great” is vague and it suggests you “search online” to understand the scope of the problem. Maybe this is a compelling limitation but this is an empty two sentences.
“Decoupling storage from compute is the default architecture” This is point 1 again.
“Migrating from SQLite is not incredibly hard, but it’s not easy, and it’s still pointless work.” This is point 1 again. It’s also wrong: if any dependency or strategy stops being the correct decision, it’s not at all pointless to rework it. “SQLite, by default, is not very strict with your data, like enforcing types or foreign key constraints” This is a useful criticism; it seems like every time sqlite comes up I see a link to another list of settings and I’d say reviewing the options is mandatory where the defaults on mariadb/postgresql are fine for most apps. And “very different latency profiles” is useful warning about that migration path; every app ends up depending on a db-specific feature but 1 + n queries is a pretty big one to rework.
I’m not well-informed enough to agree or disagree with this post, just frustrated that it doesn’t make the best case it could.
Re: migrations: SQLite doesn’t support some commonplace ALTER TABLE behavior. Instead, you have to:
Start a transaction
Create a new table with your NOT NULL column without default, a constraint, whatever
Copy data from the old table to the new table
Disable foreign key enforcement
Drop the old table
Rename the new table to the old name
Re-enable foreign key enforcement
For large tables this will lock the database for a while as the data copies over.
Additionally, this is an error-prone process. Do this enough times and eventually you might get the steps wrong. Even if you don’t, I had an ORM delete data on a small project when it executed statements in an unintuitive, undocumented, and nondeterministic order, running the DROP before running the statement disabling foreign keys. The ORM’s built-in schema commands weren’t susceptible to this issue, but SQLite required me to use a lower-level approach.
To me this does not sound much less nutso than mysql online schema change machinery or vitess or whatever. Postgres is more better about this, sure. But sqlite is not bad, all the DDL is fully supported in transactions and it has a handy slot for storing your schema version number. If you’re using an ORM that is shredding your data, like, don’t use that software, but it’s not SQLite’s fault.
Yeah I agree this post does not make a compelling case. I have my own reasons I don’t love SQLite (really limited builtin methods and a too-flexible grammar) and I work for a Postgres company but from reading this article alone I can’t see a coherent reason not to use SQLite on the server.
I can’t see a coherent reason not to use SQLite on the server.
Did you miss the part about need to share a disk if you want more than one application server? Granted, not all applications need more than one copy running…
I guess the other option is that each application server has their own sqlite and you use RAFT or something to drive consensus between them… which maybe works if you can have at least 3 of these servers, at the expense of extreme complexity. :)
Granted, not all applications need more than one copy running…
Surely that’s the important point? SQLite is great when everything fits on a single machine, which is really the case for the vast majority of projects [citation needed].
When something is big enough to need multiple machines then it’s time to switch to a separate storage layer running PostgreSQL.
Isn’t this all just a case of picking the right tool for your situation?
This is the point of LiteFS - it replicates writes to other boxes - no NFS needed (and iirc locks on nfs were unreliable so you probably dont want to run the database file on nfs).
You DO need to take care to send writes to the leader but Fly (where the author works), makes this pretty easy. LiteStream (same author) is similar but is more of a continuous back up tool.
Right. A thing you can do is make your architecture more complicated by introducing LiteFS, or LiteStream, but it’s no longer “just sqlite” at that point.
I’ve been running Litestream for a while now and it’s VERY straight-forward. You start the process and point it at a SQLite database, it inexpensively streams WAL blocks to S3 or similar, you can use those for recovery.
It’s so simple, robust and cheap that I see it as a selling point for SQLite as a whole - much less complex/expensive than achieving point-in-time replica backups with PostgreSQL or MySQL.
Yeah but again, these work with only a single instance at a time. If that’s your setup it’s great, but many people need multi-instance setups and mounting a shared SQLite file on NFS in some cloud provider is not a great idea.
But it isn’t an NFS in the cloud, it replicates the WAL which is then made available, yes via a fuse file system but it’s not NFS, it’s not really any different that read replicas for Postgres or MySQL and it seems to be a lot easier to get going. You DO have HA on your DB don’t you?
If you don’t think it works for you, that’s fine, but I think you should understand how it works before making that decision.
Yeah but the point is that a cloud vendor DB gives me read replicas out of the box, I don’t need to set it up manually. Added to the other points, this makes it kinda hard to argue against PostgreSQL for a typical backend app.
The same is as true with sqlite as with other dbs. With Turso/libsql or litefs you can have a read replica materialized on the local nvme ssd of every application server. I’m not arguing SQLite is a better option for every app like everything in engineering the answer is “it depends”. But it feels unfair to dismiss SQLite based architecture because the sqlite3.c source file doesn’t directly automate some need and dismiss the sqlite vendors, and then compare that to Postgres + unlimited ancillary vendor support & automation.
Turso is one vendor. Cloudflare is another but its SQLite offering has a fairly low max database size. Meanwhile, most cloud providers have mature support for MySQL/Postgres. If the cloud provider situation changes for SQLite in the future and it is offered more widely, then the analysis changes, but until then I believe it’s fair. And so do the folks who actually make SQLite.
Sure… if you’re using AWS, Azure or Google… and yeah, even fly.io (though theirs is NOT managed) does too, but really, it’s run a single executable in the background - replay or routing writes to the master is more work of course, but it’s still pretty easy. And given the stance of the US govt lately towards my country, I am likely NOT to use any of those, preferring to host domestically (sorry to bring politics into this - but everything is a trade off for making technical decisions, including that) .
I once set up HA Postgres on Linode (floating ip,NBD and the watchdog) and it worked, but was cargo culted and seemed very fragile in it’s complexity. LiteFS in comparison seems clean and elegant and would be very robust against SPOF and as a benefit move the db reads to the edge, nearer to my hypothetical users.
But like I said, I am not running anything using it at the moment. The one project I AM considering doing would use Postgres for other reasons.
every app ends up depending on a db-specific feature but 1 + n queries is a pretty big one to rework
I think there might be a few cases in which SQLite forces one to use multiple queries where PostgreSQL wouldn’t, but it doesn’t usually, does it?
From that linked page:
So, SQLite is able to do one or two large and complex queries, or it can do many smaller and simpler queries. Both are efficient. An application can use either or both techniques, depending on what works best for the situation at hand.
“the situation at hand” could include “I want to leave open the option of replacing the RDBMS”.
“you now need, at the very least, backups” This is true of every database; how is this a criticism of sqlite?
Well, the article is pretty clear in what it’s trying to say, quote:
The value of SQLite is that it’s infrastructure-less. You don’t have to run anything additional to use it.
Which is saying, many choose to use sqlite because they don’t need to run anything extra - they just need to link their code with sqlite and it works. And trying to use sqlite on server-side negates that benefit. So, if that’s not why you want to use sqlite to begin with, this point doesn’t affect you at all.
Most of Asahi’s linux work is in Rust, such as their new GPU drivers for Apple Silicon. Part of that requires access to the DMA subsystem. An asahi maintainer other than Marcan wanted to upstream Rust bindings to DMA. These would have been part of the “Rust for Linux” subsystem, which explicitly is exempted from the usual kernel stability guarantees. The maintainer of DMA in Linux objected to them upstreaming those bindings, as he felt he would then be obligated to maintain the Rust code, and he did not want to maintain Rust. This was combined with some uncharitable comments on the Rust for Linux project’s goals. There has been a lot of debate on other internet forums about whether these comments were uncharitable about Rust, Rust for Linux, or some other distinction that personally I think doesn’t really matter.
Marcan and that maintainer then exchanged words on the LKML. Unsatisfied, Marcan appealed to his Mastodon for support. Linus then stepped in with a reply heavily condemning the appeal to social media and not really addressing the complaints that led to that point. Marcan later resigned from his kernel maintainerships, and now also from the Asahi project.
I think this understates the position of the maintainer. He sent a message to LKML that states it pretty clearly: he thinks a multi-language kernel is a terrible mistake. And I can understand his position — which to me demonstrates that there is a leadership failure happening, because this kind of fundamental conflict shouldn’t be allowed to go on this long. “Disagree and Commit”, as they say at Amazon.
Every additional bit that the another language creeps in drastically reduces the maintainability of the kernel as an integrated project. The only reason Linux managed to survive so long is by not having internal boundaries, and adding another language complely breaks this. You might not like my answer, but I will do everything I can do to stop this. This is NOT because I hate Rust.
…
The common ground is that I have absolutely no interest in helping to spread a multi-language code base. I absolutely support using Rust in new codebase, but I do not at all in Linux.
Since there hasn’t been a reply so far, I assume that we’re good with
maintaining the DMA Rust abstractions separately.
Hence, the next version of this patch series will have the corresponding
maintainer entry.
Danilo
I.e. after the explicit statement that the rust people, not Hellwig, would be maintaining it separately.
(And then we reach the post you quoted, where he makes it clear that his reasons are that he doesn’t want Rust in Linux at all, not that he doesn’t want to maintain it)
Yes, I understated the understatement. :) His stated goal is to use his influence and control over the DMA subsystem to prevent Rust (edit: or any other non-C language), from being in the kernel at all. As his NAK says:
If you want to make Linux
impossible to maintain due to a cross-language codebase do that in
your driver so that you have to do it instead of spreading this cancer
to core subsystems. (where this cancer explicitly is a cross-language
codebase and not rust itself […])
Classic LKML rhetorical style here, to use a loaded word like “cancer” to make what is actually a reasonable technical point.
It is not a reasonable technical point! It demonstrates a fundamental incuriosity about technical matters. Categorical technical statements are rarely reasonable.
I disagree in general, but not in specifics. I’ve spoken to Chris Hellman at lengths (literally 6 hours) about the topic of Rust and it was not combative. He was curious. It’s hard to sum up at length, but his disagreements are rooted in specifics and he’s a hardliner about the single source language thing. But not for the sake of being a hardliner. His hard-line stance comes out of different weighting of the issues at hand, not an outright dismissal.
Those people existing is to be expected and it doesn’t invalidate their opinions.
Obviously not a fan of LKML rhetoric, but yep, you get the tone you built over years.
You can disagree, but for the rest of us who haven’t had these super private, intimate exchanges, where we get to “really know” someone, what he puts into the public is our perception of him. To that end, he’s called the project a cancer, and declared he’s going to do whatever he can to stop it which is not what a curios person does. He comes across instead as a religious zealot or “hardliner” in your terms.
That’s fair, but it’s also a categorical mistake to read a (brusque and inappropriate) email and judge on a whole person for days and weeks on the internet. I’m not saying you “really need to know someone”, but you also need to be aware that hundreds of hours are spent on reading what those statements may mean.
I’ve personally had to make the decision whether to allow an additional language into a large codebase, and there are some major negatives to consider that are unrelated to the technical aspects of the language. The maintainer said three times in the above excerpts that he doesn’t have a negative opinion of Rust itself.
I’ve always heard that Google allows only four languages in their codebase, and one of those is only because they invented it. :)
By “categorical”, you mean saying “no second language could ever have upsides that overcome the downsides of having two languages”? I agree that’s a pretty conservative position. It’s hard to even imagine another candidate for second language, though (I dunno, Zig maybe?), so I didn’t take it as literally categorical.
And I also do not want another maintainer. If you want to make Linux
impossible to maintain due to a cross-language codebase do that in
your driver so that you have to do it instead of spreading this cancer
to core subsystems. (where this cancer explicitly is a cross-language
codebase and not rust itself, just to escape the flameware [sic] brigade).
I think Zig sees itself as a candidate second language for C projects, but that’s probably beside the point; nobody’s trying to bring it into the kernel.
Indeed, but that call was already decided when Rust support was first merged.
The goal of the experiment is to explore those tradeoffs. You can’t do that if you don’t merge things!
I’ve always heard that Google allows only four languages in their codebase
And yet they are one of the companies pushing Rust for Linux… They understand the tradeoff. The rust for linux people understand the tradeoff. They just think it is worth it.
Totally. But that does seem like a valid engineering question people could disagree about.
However, this is exactly the kind of huge impactful question that should get escalated to the chief architect. And if there’s a sanctioned scouting effort to gather experience to figure out the right answer, somebody found to be actively subverting that effort should be…corrected…by the chief architect.
It’s interesting that they won’t allow it in their internal codebase but they’re happy to let the Linux maintainers take on the burden of adding it there.
R4L is officially an “experiment” with goal to explore tradeoffs. I.E. satisfy curiosity.
This whole debate is so tired. The “including Rust” ship has, at least temporarily, sailed, and maintainers blocking it based on not wanting Rust/a second language doesn’t make sense when the whole point is to explore that option.
Maybe all the code will get deleted at some point, but the code needs to be merged first to then decide that based on how well it works in practice.
To have a thorough experiment, it must be merged into the official tree. One must be able to see how this interacts with a diverse set of hardware and software configurations, and also how this affects the development process inside the kernel community. Some maintainers are afraid of how the introduction of Rust would affect them in the future and the maintainability of the code base. Without it being in the official tree, there wouldn’t be any conclusion on those points.
It sounds like some of the kernel maintainers at least don’t want to have to pay the cost of this large-scale experiment? Like they don’t feel it’s worth the downsides of massively complicating the kernel into a multi-language codebase? Like they understand the potential benefits of memory safety but they feel they are outweighed by the risks?
These individuals believe that they know the result of the experimental before it’s been run. But they do not, because they cannot, by definition. Of course they don’t want to pay the cost, because change is hard, and deep down they know that there’s a chance that the experiment is successful, and thus their responsibilities will either change or they will have to cede responsibility to someone more capable.
Whether there is benefit to Rust in theory, there is little question. It provably solves problems that the kernel has. Whether it can solve those problems without causing larger ones is why the experiment is required. Blocking the experiment for essentially any reason is more or less indefenisble, because the entire point is that they don’t know.
That sounds way too confident and assured of a statement to me. You are discounting the opinion of subject matter experts as if they have no idea what they’re talking about. If someone came to you at your workplace and said that they want to run an experiment to convert your codebase to a new language, and they want you to help maintain and support that experiment while also delivering on all your existing obligations, would you be so quick to accept that? I don’t know about you, but personally I would and have pushed back on rewrite suggestions as a bad idea, despite the perceived benefits (for example, Management thinks that rewriting in JavaScript will make it easier to hire people for the project in the future).
Would rewriting in JavaScript have possibly made me redundant? Maybe. But it would also be massively expensive, cause huge churn, and have a real possibility of failing as a project. We can’t just ignore the very real possibility of risks because of the expected benefits.
There is no interpretation in which it’s too assured. My position is that the result of the experiment is unknown, and that the potential benefits are known, both of which are hard facts.
I have run such experiments in my workplace before. Some have succeeded and some have failed. The successes have massively outweighed the impact of the failures.
Assuming that you know the outcome before you begin is the pinnacle of hubris, which we see time and again with these entrenched maintainers. They may be domain experts, but they deserve every ounce of criticism that they receive. Closed mindedness is not how progress is achieved.
My point is that there must be some threshold that you use to decide whether some experiment is going to be useful to run or not even worth the effort because your experience and expertise tells you otherwise? Or would you accept every experiment that anyone proposed? Eg, suppose someone wanted to model your entire technical stack with TLA+ for high assurance purposes? On paper it sounds like a great idea–formally verify your concurrency properties–but you don’t see how a reasonable project lead might say ‘While this project could certainly bring great benefits, it’s not worth the tradeoff with our team size and capabilities right now. That may change in the future’?
Some threshold must exist, yes. Presumably, that threshold should be somewhere below “the progenitor of the system has decide that the experiment should be run”, which is currently the case of RFL.
Individual system maintainers should not and must not be able to usurp the direction of the project as a whole.
Your core misunderstanding is that we are coming from a tabula rasa. We are not, and if your position is that Linus’s previous endorsements shouldn’t stand, then we’re having an entirely different (and much less tenable) conversation.
“the progenitor of the system has decide that the experiment should be run”, which is currently the case of RFL.
Is it though? To my understanding the decision was only that they will try it and see if it works on both the technical and the social levels. There was never any guarantee given that Rust will be shipped in Linux at all costs. To my understanding Linus’s approach is to see if Rust is overall viable for the kernel, and that includes the opinions of the other subsystem maintainers. Because if they are overall against it, the experiment is dead in the water.
Again, that decision has already been made. There’s no point questioning it over and over. Doing so is distracting from the debates that are important now: how can leadership do better at enforcing these project wide decisions to not let the situation fester like R4L, and reviewing the Rust code without raising the same solved questions again and again (concern trolling).
If you’re genuinely curious about those questions, there’s plenty of text written about it. LWN is a good place to start.
I hear you, but it is a bit in the nature of project that are very distributed in decision making with high individual authority that indeed, decisions will be questioned again and again and again at every point. Those projects need a long breath and the willingness to talk about points again and again and again.
It comes with its strength, particularly, those projects are extremely resilient and fast in other regards. You can’t have both. Pushing top-down decisions in such an environment can have other unintended effects.
You’re right that it’s in rust/, tho unclear to me if it’s properly in the rust subsystem or not. You and others may be right that this patch may still go thru, that seems to depend on Greg KH and Linus.
so he has the authority to block it, but not to reject it? him being the guy referred to above as “the guy who blocked the patches that the Asahi Linux folks wanted.”
There has been a lot of debate on other internet forums about whether these comments were uncharitable about Rust, Rust for Linux, or some other distinction that personally I think doesn’t really matter.
Personally I think it does matter when a core maintainer calls a project that’s part of the kernel “cancer” and the guy who’s in charge doesn’t seem to care.
And I also do not want another maintainer. If you want to make Linux
impossible to maintain due to a cross-language codebase do that in
your driver so that you have to do it instead of spreading this cancer
to core subsystems. (where this cancer explicitly is a cross-language
codebase and not rust itself, just to escape the flameware brigade).
The cancer is not Marcan, Asahi, R4L, or Rust. The cancer is anything-but-C. The cancer is a two-language codebase.
Still a very discouraging comment from a maintainer, and it bucks Linus’ decision that trying out Rust in the kernel is OK, but the distortion of this comment has been bugging me for the last week.
I don’t have an opinion, but I think a fair and less charged way than “cancer” of stating the position is just this: using Rust would have positive benefits, but they are far outweighed by the negatives of allowing a second language into the kernel.
The pros and cons of RfL are somewhat subjective, and the funal balance is hotly debated.
But I find it quite telling that Hellwig’s core argument (that a multi-lamguage codebase requires more work) is held by people who didn’t try doing any of that work. Whereas the kernel devs who started using Rust are explicitly saying “the API compat churn isn’t that big a deal, we can do that work for you if you want”.
We mostly hear about the drama, but it seems that the overall feeling toward Rust among kernel devs (not just RfL devs) is generally positive.
I don’t think the metaphor conveys any technical meaning beyond what I said. I don’t know what “kills the organism” is supposed to relate to. Is the “organism” the kernel? The community of kernel developers? And what does “kill” equate to in reality? It would be better to have the discussion in concrete terms rather than metaphors.
It does convey the personal outrage of the maintainer better, certainly.
Well, notice that you aren’t absorbing my actual point too well, because you’re very focused on making your own.
yes, I’m ignoring the less absurd things you’ve said because I don’t know what to take seriously when you haven’t retracted the most absurd thing.
To give him the benefit of the doubt, he could have been trying to convey the technical meaning of “growth” in the sense of “if you let Rust in, we’ll just have more Rust”. However, that’s an utterly vacuous thing to say, because obviously the whole point of the exercise was to let Rust in and see what happens, and why would you do that unless you want to have more of it if the experiment succeeds?
one characteristic of cancerous growth is that it happens whether you want it to or not.
It conveys no useful information beyond “I don’t think Rust is a good idea at all, for reasons”; in other words, “the benefits don’t outweigh the negatives”. In fact it confuses matters because it implies he thinks Rust would be OK if only it didn’t grow.
it doesn’t imply that, but I do think he would be much more OK with a little bit of Rust if the scope were somehow guaranteed not to expand. like I don’t know, maybe there’s a carve-out for testing a particular class of rust drivers in linux to help find bugs, but eventually the drivers are ported to C for use in linux and the rust version is used in a separate kernel written from scratch in rust?
I’m actually trying to be polite to the guy here, because I truly believe it was an emotional outburst, not an attempt at a technical argument. His actual technical argument (stated much more clearly elsewhere) is that there should be no Rust at all in the kernel, ever, not that once there is some, there will be more. So if “cancer” was supposed to mean “growth” as a technical argument, he misstated his own argument.
so you wouldn’t say that you want to have no cancer at all in your body, ever? if it’s a non-cancerous tumor, I have much less of a problem with it being in my body.
I don’t want to dig into the minutiae of why this metaphor is inaccurate — my point is that using this metaphor at all is unprofessional and counterproductive because of the heavy emotional baggage it comes with. Just say “the introduction of Rust anywhere in the codebase will inevitably result in a growing amount of Rust in the codebase”.
okay, so do you retract your previous statement that “I don’t think the metaphor conveys any technical meaning beyond what I said”?
my intention was not to get into minutia either, but we’ve gone back and forth three times and you still haven’t retracted that statement so I don’t know where you stand.
It’s especially odd in this context, given the history of this metaphor regarding Linux.
so our enemies allow themselves to use the metaphor to efficiently communicate with each other, but we have to forswear it.
If you don’t want it there at all, it’s redundant to also complain that it’s growing. Cancer also has many other attributes besides “growing” that are irrelevant or contradictory to the position taken.
The reason he used that particular metaphor was to express disgust and horror, not to be technically accurate. If he just wanted to evoke “growth” he could have said it was like ivy. Disgust and horror are not an appropriate device to call on in this context, IMO.
Re Mr. Ballmer, the enemies were wrong that time, so don’t you think evoking that argument by using the same metaphor muddies the rhetorical water a bit?
if I can’t tell whether you stand by your statements, it feels futile to interpret them as if you believe them. that’s my issue with continuing in light of the fact that you are just leaving “I don’t think the metaphor conveys any technical meaning beyond what I said” without clarifying whether you believe it.
“complaining that it’s growing” is completely different from identifying the growth as a factor in the technical drawbacks.
Cancer has a multitude of characteristics, of which growth is only one. If anything, it has many characteristics contradictory to the point being made: it’s an undesirable outcome of natural processes, it is typically self-generated from within rather than deliberately introduced, it is hard to eliminate because it’s nourished by the same system that sustains healthy cells…
If you have to tell me exactly which part of the metaphor I’m supposed to pay attention to in order to get the technical meaning, then you aren’t conveying technical meaning, you’re muddying the water. Just say what you mean.
The additional meaning being conveyed here is not technical, it’s emotional, and I do agree emotional meaning was conveyed.
The additional meaning being conveyed here is not technical
I take this to mean that you still stand by the statement that “I don’t think the metaphor conveys any technical meaning beyond what I said.” if you treat all technical metaphors like this, I’m glad you’re not my coworker.
If you have to tell me exactly which part of the metaphor I’m supposed to pay attention to in order to get the technical meaning, then you aren’t conveying technical meaning, you’re muddying the water.
or it could mean I am conveying technical meaning but you’re being willfully obtuse. just maybe…
Well, notice that you aren’t absorbing my actual point too well, because you’re very focused on making your own.
To give him the benefit of the doubt, he could have been trying to convey the technical meaning of “growth” in the sense of “if you let Rust in, we’ll just have more Rust”. However, that’s an utterly vacuous thing to say, because obviously the whole point of the exercise was to let Rust in and see what happens, and why would you do that unless you want to have more of it if the experiment succeeds? It conveys no useful information beyond “I don’t think Rust is a good idea at all, for reasons”; in other words, “the benefits don’t outweigh the negatives”. In fact it confuses matters because it implies he thinks Rust would be OK if only it didn’t grow.
I’m actually trying to be polite to the guy here, because I truly believe it was an emotional outburst, not an attempt at a technical argument. His actual technical argument (stated much more clearly elsewhere) is that there should be no Rust at all in the kernel, ever, not that once there is some, there will be more. So if “cancer” was supposed to mean “growth” as a technical argument, he misstated his own argument.
I could be wrong, but I get the feeling that people didn’t quite get why Hellwig used the word “cancer”. I could be wrong about this in a lot of ways, maybe I too misunderstood what Hellwid meant, maybe I misjudged how others are interpreting what he said. But allow me to add my 2 cents.
People seem to be interpreting what Hellwig said (i.e. calling R4L “cancer”) as him saying R4L is bad, evil, a terrible thing, etc. And I totally understand why many would think that, and would agree this is a terrible choice of words. But I think that Hellwig’s focus is actually on the fact that cancer spreads. A better word to use would probably be like “infectious”, or “viral”. Although I disagree with him, I think what he was saying is that, despite what Rust developers promised, Rust will spread to more corners of the kernel and increase the maintenance burden and will have a negative impact on the kernel as a whole.
I think you’re right, but that doesn’t exculpate Hellwig in the least. One of the defining characteristics of cancer is that it is malignant. It seems pretty clear that he used this metaphor because he thought that Rust would spread and that it is dangerous and harmful.
“infectious” or “viral” have different connotations. viral would be if they thought the BSDs might catch it. infectious is not as specific as cancer; cancer specifically grows from one area and consumes more and more of the things next to it. infections can do that but not necessarily.
Unrelated to the rust issue, my understanding is that such toxic language actually isn’t all that strange in the Linux kernel maintainer community. It feels like they try somewhat not to overdo it, but they aren’t that much concerned about the toxicity. More like, “I’ll try but I don’t actually care”.
It’s useful in some cases, but when the effect of it is raising the temperature of the discussion, then I would say it isn’t useful in that instance. There are other ways to convey the same meaning that wouldn’t have attracted the same amount of strong emotions. We’ve seen examples of such alternate arguments in this thread that would have been more suitable.
sure. but then don’t fixate on the word, and acknowledge that their choice to use strong language can also be an effect of the raising temperature of the discussion, for which responsibility is shared among all participants.
There’s this idea that multi-language codebases are categorically a disaster, right? What really bothers me is how small-minded it is. There are many C and C++ codebases that have introduced Rust. It’s not easy but it is possible, and it often is worth it. Some attempts have been successful and others haven’t. You can learn so much from other people’s experiences here!
Simply calling multi-language codebases a cancer closes off discussion rather then opening it. That goes against the basic curiosity that is a hallmark of engineering excellence.
Honestly everything around R4L that has been going has just cemented in me the belief that the average long time Linux maintainer is fundamentally incompetent. They’re reasonably good at One Thing, but totally incurious and scared of anything outside of that One Thing they’re good at.
Is it possible that we could take an experienced kernel maintainer’s opinion seriously instead of dismissing it immediately as incurious and ‘against’ engineering excellence? If a co-worker who specializes in a different domain came to you and kept insisting that something you know is very difficult and not worth the tradeoff is actually quite easy, would you be incurious and ‘against’ engineering excellence if you dismissed their wild suggestions without a deep discussion of the tradeoffs?
Evaluating who to take seriously and who not to is a matter of judgment cultivated over a lifetime.
If a co-worker who specializes in a different domain came to you and kept insisting that something you know is very difficult and not worth the tradeoff is actually quite easy, would you be incurious and ‘against’ engineering excellence if you dismissed their wild suggestions without a deep discussion of the tradeoffs?
Depends on many things, but in general, yes, of course, especially if they were someone I took seriously on other matters. If nothing else, it’s a chance to sharpen my arguments. If I keep having the same arguments over and over I might even step back and write an essay or two explaining what I believe and why I believe it. Here’s one that’s public that was a result of the sort of back-and-forth you’re talking about.
Reading the discussions on this elsewhere, there’s no way to describe this that wasn’t going to get accused of bias by people holding strong opinions on either marcan or on the rust for linux project.
I’m not accusing of bias (which is more or less unavoidable), but of constructing a narrative. The facts are: he exchanged words on the LKML and then wrote Mastodon posts about the exchange. The narrative you’re constructing is: he exchanged words on LKML, then because he was unsatisfied with how that discussion was going, he wrote Mastodon posts about the exchange in an attempt to gather support.
I think it would be better to leave out the guesswork about intentions, or at least clearly separate it from the factual retelling of events.
Correction: Marcan never “exchanged words” with Christoph Hellwig on the patches, which had not even been posted by him. He just replied to the thread with a tirade.
It’s definitely a lot to download, but comparing WASM and JS bundle sizes isn’t entirely apples-to-apples. Byte for byte, JS is one of the slower assets you can put on your page due to its relatively complex parsing requirements.
I’d be interested to know where the balance point is between parsing speed and download speed.
the tricky thing is that the balancing point would depend on the connection speed of your user. some people still have really slow 3G as their primary internet connection.
But a JS bundle can be cached, no? And I’d be really surprised if browsers aren’t JIT-compiling the cached JS internally into some kind of executable bytecode representation.
Yeah, it’s a bit chunky for sure but not a dealbreaker. It’s deployed as a PWA, which helps; it’s heavily cached until the user clicks an “update” button.
Is this the first big Go project at Microsoft? I had no idea they could choose Go, I would assume that it just wouldn’t be acceptable, because they wouldn’t want to unnecessarily legitimize a Google project.
Their web browser runs on Chromium, I think it’s a little late to avoid legitimizing Google projects
They had already lost the web platform by the time they switched to Chromium.
When I think of MS and ‘developers developers developers develop’, I was lead to believe that they’d have more pride in their own development platforms.
I wonder if the switch to Chromium, the embrace of Linux in Azure, the move from VS to VSCode, any of these, would have happened under Ballmer or Gates. I suspect Gates’ new book is like Merkel’s: retrospective but avoiding the most controversial questions: Russian gas and giving up dogfooding. Am I foolish to expect a more critical introspection?
I think corporate open source is now “proven”, and companies don’t worry so much about this kind of thing.
Google has used TypeScript for quite awhile now (even though they had their own Closure compiler, which was not staffed consistently, internally)
All of these companies are best thought of as “frenemies” :) They cooperate in some ways, and compete in others.
They have many common interests, and collaborate on say lobbying the US govt
When Google was small, and Microsoft was big, the story was different. They even had different employee bases. But now they are basically peers, and employees go freely back and forth
Microsoft has actually maintained a security-hardened build of Go for a while now: https://devblogs.microsoft.com/go/
I can see a blog post saying they made changes for FIPS compliance, which is usually understood to be for satisfying government requirements and not a security improvement. I can’t see a summary of any other changes.
MS is a pretty big company, so I wouldn’t be surprised if they have a ton of internal services in Go. Plus with Azure being a big part of their business, I doubt they care what language people use as long as you deploy it on their platforms.
Can confirm that some internal services were written in go as of a couple years ago. (I worked for Microsoft at the time.) I didn’t have a wide enough view to know if it was “a ton,” though.
We embrace and we extend!
I’m too lazy to look up a source but some of the error messages I’ve seen in azure make it clear Microsoft uses go to build their cloud.
Message like this one:
Keep in mind Microsoft also maintains their own go fork for fips compliance. I don’t know exactly their use case but I can tell you at Google we did the same thing for the same reason. I think the main fips compliant go binaries being shipped were probably gke/k8s related.
(Edit removed off topic ramble, sorry I have ADHD)
Could this message just be coming from Kubernetes?
The TypeScript dev lead posted this response about the language choice on Reddit, for anyone who’s curious:
Source: https://www.reddit.com/r/typescript/comments/1j8s467/comment/mh7ni9g
People really miss the forest for the trees.
I looked at the repo and the story seems clear to me: 12 people rewrote the TypeScript compiler in 5 months, getting a 10x speed improvement, with immediate portability to many different platforms, while not having written much Go before in their lives (although they are excellent programmers).
This is precisely the reason why Go was invented in the first place. “Why not Rust?” should not be the first thing that comes to mind.
I honestly do think the “Why not Rust?” question is a valid question to pop into someone’s head before reading the explanation for their choice.
First of all, if you’re the kind of nerd who happens to follow the JavaScript/TypeScript dev ecosystem, you will have seen a fair number of projects either written, or rewritten, in Rust recently. Granted, some tools are also being written/rewritten in other languages like Go and Zig. But, the point is that there’s enough mindshare around Rust in the JS/TS world that it’s fair to be curious why they didn’t choose Rust while other projects did. I don’t think we should assume the question is always antagonistic or from the “Rust Evangelism Strike Force”.
Also, it’s a popular opinion that languages with algebraic data types (among other things) are good candidates for parsers and compilers, so languages like OCaml and Rust might naturally rank highly in languages for consideration.
So, I honestly had the same question, initially. However, upon reading Anders’ explanation, I can absolutely see why Go was a good choice. And your analysis of the development metrics is also very relevant and solid support for their choice!
I guess I’m just saying, the Rust fanboys (myself, included) can be obnoxious, but I hope we don’t swing the pendulum too far the other way and assume that it’s never appropriate to bring Rust into a dev conversation (e.g., there really may be projects that should be rewritten in Rust, even if people might start cringing whenever they hear that now).
While tweaking a parser / interpreter a few years ago written in Go, I specifically replaced a struct with an ‘interface {}’ in order to exercise its pseudo-tagged-union mechanisms. Together with using type-switch form.
https://github.com/danos/yang/commit/c98b220f6a1da7eaffbefe464fd9e734da553af0
These day’s I’d actually make it a closed interface such that it is more akin to a tagged-union. Which I did for another project which was passing around instances of variant-structs (i.e. a tagged union), rather than building an AST.
So it is quite possible to use that pattern in Go as a form of sum-type, if for some reason one is inclined to use Go as the implementation language.
That is great explanation of “Why Go and not Rust?”
If you’re looking for “Why Go and not AOT-compiled C#?” see here: https://youtu.be/10qowKUW82U?t=1154s
A relevant quote is that C# has “some ahead-of-time compilation options available, but they’re not on all platforms and don’t really have a decade or more of hardening.”
That interview is really interesting, worth watching the whole thing.
Yeah Hjelsberg also talks about value types being necessary, or at least useful, in making language implementations fast
If you want value types and automatically managed memory, I think your only choices are Go, D, Swift, and C# (and very recently OCaml, though I’m not sure if that is fully done).
I guess Hjelsberg is conceding that value types are a bit “second class” in C#? I think I was surprised by the “class” and “struct” split, which seemed limiting, but I’ve never used it. [1]
And that is a lesson learned from the Oils Python -> C++ translation. We don’t have value types, because statically typed Python doesn’t, and that puts a cap on speed. (But we’re faster than bash in many cases, though slower in some too)
Related comment about GC and systems languages (e.g. once you have a million lines of C++, you probably want GC): https://lobste.rs/s/gpb0qh/garbage_collection_for_systems#c_rrypks
There was also a talk that hinted at some GC-like patterns in Zig, and I proposed that TinyGo get “compressed pointers” like Hotspot and v8, and then you would basically have that:
https://lobste.rs/s/2ah6bi/programming_without_pointers#c_5g2nat
[1] BTW Guy Steele’s famous 1998 “growing a language” actually advocated value types in Java. AFAIK as of 2025, “Project Valhalla” has not landed yet
Compilers written in OCaml are famous for being super-fast. See eg OCaml itself, Flow, Haxe, BuckleScript (now ReScript).
Yeah, I’m kind of curious about whether OCaml was considered at some point (I asked about this in the Reddit thread, haven’t gotten a reply yet).
OCaml seems much more similar to TS than Go, and has a proven track record when it comes to compilers. Maybe portability issues? (Good portability was mentioned as a must-have IIRC)
Maybe, but given that Flow, its main competitor, distributes binaries for all major platforms: https://github.com/facebook/flow/releases/tag/v0.264.0
Not sure what more TypeScript would have needed. In fact, Flow’s JavaScript parser is available as a separate library, so they would have shaved off at least a month from the proof of concept…
Also Nim.
Also Julia.
There surely are others.
Yes good points, I left out Nim and Julia. And apparently Crystal - https://colinsblog.net/2023-03-09-values-and-references/
Although thinking about it a bit more, I think Nim, Julia, (and maybe Crystal) are like C#, in that they are not as general as Go / D / Swift.
You don’t have a
Foo*type as well as aFootype, i.e. the layout is orthogonal to whether it’s a value or reference. Instead, Nim apparently has value objects and reference objects. I believe C# has “structs” for values and classes for references.I think Hjelsberg was hinting at this category when saying Go wins a bit on expressiveness, and it’s also “as close to native as you can get with GC”.
I think the reason this Go’s model is uncommon is because it forces the GC to support interior pointers, which is a significant complication (e.g. it is not supported by WASM GC). Go basically has the C memory model, with garbage collection.
I think C#, Julia, and maybe Nim/Crystal do not support interior pointers (interested in corrections)
Someone should write a survey of how GC tracing works with each language :) (Nim’s default is reference counting without cycle collection.)
Yeah that’s interesting. Julia has a distinction between
struct(value) andmutable struct(reference). You can use raw pointers but safe interior references (to an element of an array for example) include a normal reference to the (start of the) backing array, and the index.I can understand how in Rust you can safely have an interior pointer as the borrow checker ensures a reference to an array element is valid for its lifetime (the array can’t be dropped or resized before the reference is dropped). I’m very curious - I would like to understand how Go’s tracing GC works with interior pointers now! (I would read such a survey).
Ok - Go’s GC seems to track a memory span for each object (struct or array), stored in kind of a span tree (interval tree) for easy lookup given some pointer to chase. Makes sense. I wonder if it smart enough to deallocate anything dangling from non-referenced elements of an array / fields of a struct, or just chooses to be conservative (and if so do users end up accidentally creating memory leaks very often)? What’s the performance impact of all of this compared to runtimes requiring non-interior references? The interior pointers themselves will be a performance win, at the expense of using an interval tree during the mark phase.
https://forum.golangbridge.org/t/how-gc-handles-interior-pointer/36195/5
It’s been a few years since I’ve written any Go, but I have a vague recollection that the difference between something being heap or stack allocated was (sometimes? always?) implicit based on compiler analysis of how you use the value. Is that right? How easy it, generally, to accidentally make something heap-allocated and GC’d?
That’s the only thing that makes me nervous about that as a selling point for performance. I feel like if I’m worried about stack vs heap or scoped vs memory-managed or whatever, I’d probably prefer something like Swift, Rust, or C# (I’m not familiar at all with how D’s optional GC stuff works).
Yes, that is a bit of control you give up with Go. Searching for “golang escape analysis”, this article is helpful:
https://medium.com/@trinad536/escape-analysis-in-golang-fc81b78f3550
So the toolchain is pretty transparent. This is actually something I would like for the Oils Python->C++ compiler, since we have many things that are “obviously” locals that end up being heap allocated. And some not so obvious cases. But I think having some simple escape analysis would be great.
Yes, the stack/heap distinction is made by the compiler, not the programmer, in Go.
Why did you leave JS/TS off the list? They seem to have left it off too and that confuses me deeply because it also has everything they need
Hejlsberg said they got about 3x performance from native compilation and value types, which also halved the memory usage of the compiler. They got a further 3x from shared-memory multithreading. He talked a lot about how neither of those are possible with the JavaScript runtime, which is why it wasn’t possible to make tsc 10x faster while keeping it written in TypeScript.
Yeah but I can get bigger memory wins while staying inside JS by sharing the data structures between many tools that currently hold copies of the same data: the linter, the pretty-formatter, the syntax highlighter, and the type checker
I can do this because I make my syntax tree nodes immutable! TS cannot make their syntax tree nodes immutable (even in JS where it’s possible) because they rely on the
node.parentreference. Because their nodes are mutable-but-typed-as-immutable, these nodes can never safely be passed as arguments outside the bounds of the TS ecosystem, a limitation that precludes the kind of cross-tool syntax tree reuse that I see as being the way forwardHejlsberg said that the TypeScript syntax tree nodes are, in fact, immutable. This was crucial for parallelizing tsgo: it parses all the source files in parallel in the first phase, then typechecks in parallel in the second phase. The parse trees from the first phase are shared by all threads in the second phase. The two phases spread the work across threads differently. He talks about that kind of sharing and threading being impractical in JavaScript.
In fact he talks about tsc being designed around immutable and incrementally updatable data structures right from the start. It was one of the early non-batch compilers, hot on the heels of Roslyn, both being designed to support IDEs.
Really, you should watch the interview https://youtu.be/10qowKUW82U
AIUI a typical LSP implementation integrates all the tools you listed so they are sharing a syntax tree already.
It’s true that I haven’t watched the interview yet, but I have confirmed with the team that the nodes are not immutable. My context is different than Hejlsberg’s context. For Hejlsberg if something is immutable within the boundaries of TS, it’s immutable. Since I work on JS APIs if something isn’t actually locked down with
Object.freezeit isn’t immutable and can’t safely be treated as such. They can’t actually lock their objects down because they don’t actually completely follow the rules of immutability, and the biggest thing they do that you just can’t do with (real, proper) immutable structures is have anode.parentreference.So they have this kinda-immutable tech, but those guarantees only hold if all the code that ever holds a reference to the node is TS code. That is why all this other infrastructure that could stand to benefit from a shared standard format for frozen nodes can’t: it’s outside the walls of the TS fiefdom, so the nodes are meant to be used as immutable but any JS code (or any-typed code) the trees are ever exposed to would have the potential to ruin them by mutating the supposedly-immutable data
To be more specific about the
node.parentreference, if your tree is really truly immutable you need to replace a leaf node you must replace all the nodes on the direct path from the root to that leaf. TS does this, which is good.The bad part is that then all the nodes you didn’t replace have chains of
node.parentreferences that lead to the old root instead of the new one. Fixing this with immutable nodes would mean replacing every node in the tree, so the only alternative is to mutatenode.parent, which means that 1) you can’t actuallyObject.freeze(node)and 2) you don’t get all the wins of immutability since the old data structure is corrupted by the creation of the new one.See https://ericlippert.com/2012/06/08/red-green-trees/ for why Roslyn’s key innovation in incremental syntax trees was actually breaking the
node.parentreference by splitting into the red and green trees, or as I call them paths and nodes. Nodes are deeply immutable trees and have no parents. Paths are like an address in a particular tree, tracking a node and its parents.You are not joking, just the hack to make type checking itself parallel is well worth an entire hour!
Hm yeah it was a very good talk. My summary of the type checking part is
That is, the translation units aren’t completely independent. Type checking isn’t embarassingly parallel. But you can still parallelize it and still get enough speedup – he says ~3x from parallelism, and ~3x from Go’s better single core perf, which gives you ~10x overall.
What wasn’t said:
I guess this is just because, empirically, you don’t get more than 3x speedup.
That is interesting, but now I think it shows that TypeScript is not designed for parallel type checking. I’m not sure if other compilers do better though, like Rust (?) Apparently rustc uses the Rayon threading library. Though it’s hard to compare, since it also has to generate code
A separate thing I found kinda disappointing from the talk is that TypeScript is literally what the JavaScript code was. There was never a spec and will never be one. They have to do a line-for-line port.
There was somebody who made a lot of noise on the Github issue tracker about this, and it was basically closed “Won’t Fix” because “nobody who understands TypeScript well enough has enough time to work on a spec”. (Don’t have a link right now, but I saw it a few months ago)
Pretty sure he said it was an arbitrary choice and they’d explore changing it. The ~10x optimization they’ve gotten so far is enough by itself to keep the project moving. Further optimization is bound to happen later.
Work has been going on for years to parallelize rust’s frontend, but it apparently still has some issues, and so isn’t quite ready for prime time just yet, though it’s expected to be ready in the near term.
In my experience, once you start to do things “per core” and want to actually get performance out of it, you end up having to pay attention to caches, and get a bit into the weeds. Given just arbitrarily splitting up the work as part of the port has given a 10x speed increase, it’s likely they just didn’t feel like putting in the effort.
Can you share the timestamp to the discussion of this hack, for those who don’t have one hour?
I think this one: https://www.youtube.com/watch?v=10qowKUW82U&t=2522s
But check the chapters, they’re really split into good details. The video is interesting anyway, technically focused, no marketing spam. I can also highly recommend watching it.
Another point on “why Go and not C#” is that, he said, their current (typescript) compiler is highly functional, they use no classes at all. And Go is “just functions and data structures”, where C# has “a lot of classes”. Paraphrasing a little, but that’s roughly what he said.
They also posted a (slightly?) different response on GitHub: https://github.com/microsoft/typescript-go/discussions/411
Yes please!
@cosmic-boi, is it in your power to enable the full article?
Everyone else, do we need a “paywall” tag? I’d rather avoid following links to them.
When this happens I flag it as “broken-link”. I cannot access the content so I think that is the correct flag.
This happens when the article author chooses to paywall it in Medium because it gives them a payout. They are basically using Lobsters as a free traffic source.
https://freedium.cfd/https://medium.com/shaffan-mustafas-blog/why-i-still-use-ruby-on-rails-7581464b86be
Does lobste.rs even allow posting links that require accounts?
Try this Friend Link: https://medium.com/shaffan-mustafas-blog/why-i-still-use-ruby-on-rails-7581464b86be?sk=02dbbd69852ab586a44064ccb3d8a2eb
That worked for me, at least this once, thank you. In future please consider an unencumbered link for submissions.
For sure. I’m working on building out my own blog (in Rails!) that won’t have a paywall :)
On the one hand, it’s neat to see people willing to experiment in this space!
On the other, it’s hard for me personally to imagine using anything other than
jjnow.The name seems unfortunate to me, as well. It’s pretty unappealing to me to use the word “zit” (where X often gets pronounced like a Z).
On the plus side the X is not meant to be pronounced like in Xiaiomi :^)
The pronunciation hint is no help, X followed by a vowel is always pronounced as in Xiaomi in my native language, so I can only read it as shit, it’s beyond automatic =P
Yeah, I’d have thought pronouncing it like ‘exit’ would have made more sense.
I’d be cautiously curious about the idea, but it even says:
I was looking in xit’s docs for a comparison to Jujutsu – specifically, why a whole new version control system is needed instead of a contribution to Jujutsu. The only direct mentions I found were in xit/docs/compat.md, which is about compatibility with Git:
This comparison fails to acknowledge that Jujutsu already supports the concept of multiple backends. Jujutsu’s Git backend is its better-supported one; “the native backend is used for testing purposes only”. Neither of Jujutsu’s current backends provide xit’s better large file support through using CDC, but Jujutsu’s roadmap already mentions the idea of implementing that technique. So Jujutsu will probably eventually also support usage with Git network compatibility but a different on-disk repo format.
xit’s goal of “better merging” seems to be a bigger difference. That same xit document mentions that Jujutsu uses the same merge algorithm as Git: three-way merge. The document says that xit, on the other hand, wants to “reduce the number of merge conflicts that occur in the first place”. It links to xit/docs/patch.md, which has a comparison of snapshot-based and patch-based version control.
Jujutsu, like Git, is purely snapshot-based. I don’t see any mention of patch-based merging in Jujutsu’s docs or roadmap.
Merge conflicts depend a lot on your workflow. A good workflow can eliminate them.
Git is not purely snapshot-based. It very frequently calculates patches (like when you run
git log) and uses them for patch-based shuffling of changes, as in cherry-pick, rebase, format-patch, am, ….The workflow I like is to have a short feature branch for a merge request, that is based very close to the head of the main branch, and typically rebased onto the head before it gets merged. Then if there are any conflicts, they appear when applying the relevant patch, not when creating the merge commit.
Merges should always be conflict-free, so the three-way merge algorithm is basically irrelevant.
The jj copy-tracing proposal involves adopting certain patch-like semantics at the repo/file-level, although not the line/byte level. It’s interesting because it also requires keeping additional copy-tracing metadata alongside what you would expect from the Git object model.
When it comes to better merge algorithms, I’m more excited about things like mergiraf. Whatever small benefits an improved textual approach might bring, I expect that to pale in comparison to a merge based on an AST. Mergiraf is in early development as far as I can tell, but the idea is solid. I think it’s just a matter of time until a textual approach is basically irrelevant for day-to-day work.
Nice summary, thanks!
Not sure I see this in such a rose-tinted way. The linked ‘bug report’ is full of people specifying very exact ways that they want to be warned beforehand: I want a blog post, the post has to be dedicated specifically to this one thing, I don’t read Reddit, the tool itself should loudly warn me several months in advance.
Notably, none of these comments actually volunteer any help with any of the above. Even the maintainer is talking about rustup as a ‘Rust project product’. I feel like everyone’s framing is wrong here…
Nothing is perfect for sure. There were some more spicy commentary on this on social media, for example. But, it’s all about the overall ratio, and the outcome.
I both agree and disagree. I think it’s normal in bug reports to report the bug, and wait for a decision to be made, before offering help. I’m sure that if they asked for help, they would have gotten it. There’s actually a pretty high degree of social cohesion here, which may not be obvious to outsiders, but I recognize virtually all of the names on that ticket, and so like, I think it’s sort of understood that if active help was desired, all you have to do is ask. The original reporter is on the language team, for example.
The Rust Project has long described their output as a product. There’s pros and cons to this, but it’s been the norm for at least 12 years now. I do attribute that style of thinking to be one of the reasons why Rust has been successful.
As an ex-lead of the Rust project: I don’t think the Rust project needs more volunteers all the time. It has capacity for fixes, so just stating problems is fine. A lot of people in the discussion are also people who contribute to Rust at other places, by writing libraries, etc.
I think treating a tool as a product of some kind is an important part of ensuring it delivers value to users. There are several things this mindset implies, such as having a product vision rather than just exposing your internals and calling it a day.
Nice blog, please timestamp the posts :-)
Hell yeah! This is gonna make pasting links into the shell so much easier ^^
I think I’ve used
?as a shell metacharacter on purpose about twice in my entire life. I strongly agree that dropping it is nice.I wonder if it would work well to use paste bracketing for something like this. If you are pasting something that is mostly text but has a wildcard or two auto-escape it. I can also imagine similar features like if you type a quote, then paste, it will auto-escape any quotes in the pasted text.
It would probably do what you want most of the time but would probably have false-positives commonly enough that it would be negative overall. But maybe there are specific cases that are clear enough to be handled (like pasting something that starts with “https:” or immediately after a single quote).
Fish 3.7.1 already does this.
Kitty just asks you every time.
But I think a control sequence which pastes raw would also work.
Wouldn’t many URLs still contain the
&character which would incorrectly break off the URL part-way and spawn background jobs that would almost certainly fail?No,
echo a&bworks for me infish, as doesecho a&b://%20?q. I thinkfishmight require a space before and/or after the&for it to create a background process.In bash,
echo a&bdoes not work.I adopted hedy for an elementary school curriculum for one year, several years ago. I prefer Python syntax over JavaScript (code.org) and I like the ambition and concepts behind hedy. When I tried (several years ago) the implementation was lacking. We hit edge cases fairly often in a relatively small class size (negative). The implementers were gracious and welcoming to feedback (positive). But it was more a research project than a robust production teaching tool. I don’t know how it’s changed since.
IIRC at the time the backend was implemented in typescript. A more robust backend in something like Rust would probably have helped with edge cases.
At the time every program submitted was recorded, so we had to warn students to not put in any PII like their name or address into a program. Which was not great.
I would like to see more experimentation with how to slowly frog boil syntax knowledge. I would also like to see code.org expand their curriculum beyond block and javascript based coding to other languages. It’s really an amazing thing they’ve built.
The decades-long research program that created the HtDP curriculum may be of interest. There’s a related teaching language and community, Pyret, that looks more like Python but shares many concepts with the Racket-based HtDP languages.
Thanks for the consideration. I clicked through. I think your expectations are off by an order of magnitude or two. When I start teaching kids they struggle with “what does the shift key do” and later “why do I need to put quote marks around both sides of my string” (not to mention “what is a string”).
Honestly, watching young 3rd grade minds smashed to bits by the minor amount of indirection provided by variables in a block based language is deeply humbling when I reflect on some of the complexity and abstraction I’m able to reason about relatively intuitively at this point.
My students have never even heard of sin much less wanting to be able to compute something with it.
Hedy worked wonderfully, in gradually introducing syntax, but it missed (quality) gamification and polish (in the form of unreliable implementation). The thing I most want to preserve is joy and the ability to create. Blocks give that to kids. Text syntax is a huge leap already.
The move has been to use straight python rather than a dialect. An open question of mine is whether or not such frog-boil syntax rules helped in the long term or if throwing kids into the deep end was less confusing I.e. no starting with hate words and then gradually introducing quoting. The hardest thing with this age group is to keep them slightly challenged so they are learning but not so much that they are stuck. Joy and creation.
HtDP is a college curriculum! I think it’s reasonable for something like an AP high school course, but I wouldn’t try to teach third graders with it. Quite honestly, I wouldn’t try to teach kids “textual programming” until they’re already comfortable with a keyboard and with grammar and punctuation in their native language, as well as arithmetic. Seems like a recipe for frustration. What’s the rush?
I completely agree about joy and creation, though. I have a ten-year-old who’s taught himself quite a lot of programming basics or prerequisites just by creating custom items and command blocks in Minecraft. Sometimes he asks me for help, but mostly he’s learning by absorbing his environment, just like we all do.
Why did you recommend it to the comment from an elementary school teacher?
3rd is too young, but 5th is not. We want to teach them that there’s a bigger world out there, beyond blocks, before they get locked into a single paradigm of coding. Our curriculum also involves teaching typing.
I didn’t think of your comment as coming from an elementary school teacher. I was thinking about pedagogical language design, and pointing to the prior art that I’m aware of. If you’re not building a language, just trying to use something that already exists, and specifically for elementary school, then HtDP is probably not that helpful, and I’m sorry about that!
Thanks for the apology. And genuinely appreciate the link, i just couldn’t connect the dots, which you just did.
Let me try again… here’s an few-years-old lobsters story linking to a blog review of a much older book about how children relate to programming that I’ve personally found very useful in thinking about conceptual scaffolding: https://lobste.rs/s/r9thsc/mindstorms
For what it’s worth, if you’re using Python for teaching, you might check out the turtle graphics package in the standard library. “Batteries included!”
Isn’t third grade a bit too young? I’d say picking up some programming is OK for 16-year olds, as they get younger than that they wouldn’t really pick up anything very useful even as a foundation for the future.
I don’t think so. I have experimentally taught some Scratch to a bunch of second-graders during my brief stint as a school informatics teacher, and they were pretty responsive. (I quit the job for unrelated reason the next year.)
Some decade later, my own daughters have Scratch in their school curriculum, and my youngest one (will be 10 this year) additionally visits children’s programming courses by her own will, and as far as I can see, the children are extremely interested.
The goal, as far as I understand it, is not to prepare for a career in software development, but to introduce “constructing algorithms” as a tool of thought, as well as demystify computing a bit; like, “see, you can make things do this and that by your own, and it is all just ifs and cycles and responding to input/events,”
Nope. They learn iteration (loops), variables, logic, and plenty more.
Great, but iOS’s Passwords apps can generate TOTPs now, and anyway OTPs are on their way to becoming obsolete tech thanks to passkeys. Hopefully soon.
If, like me, you have a mix of iOS, Android (Graphene - primary), Windows, Mac OS and Linux (primary) but need to access your accounts on all of them, Passkeys are in an utterly terrible state. I don’t see that changing, as it seems to be all about vendor lock in, than anything benefitting a user.
Passkeys are in a nascent state as people are trying to figure out the optimal UX. I think within a couple of years we will see a clear trend to do initial set up/sign in to new devices with securely emailed magic links, then set up passkeys for easy subsequent logins. This will make lock-in a moot point–you can’t be locked in to a vendor if you can just log in with a magic link on any new device. And best of all, it will reduce to the point of nearly eliminating phishing attacks. That will be the key benefit for everyday regular users.
I still don’t understand why pledge(2)/unveil(2) have not been ported to other kernels. They have proven to work very well for a almost a decade now, as they were introduced in 2015.
SerentityOS [1] [2] has ported pledge+unveil. Reply thread here mentions NanoVMS [3] too.
But i agree. It’s a really easy-to-use, simple protection that seems like a no-brainer for pretty much everything to have now.
1: https://man.serenityos.org/man2/pledge.html
2: https://awesomekling.github.io/pledge-and-unveil-in-SerenityOS/
3: https://nanovms.com/dev/tutorials/applying-sandbox-security-node-js-unikernels-openbsd-pledge-unveil
We’re doing a capstone project with one of my students where we’re porting pledge to FreeBSD. We’re not doing unveil because I don’t know the internals as good as pledge, but hopefully something good comes out of this experiment.
You might want to reach out to some of the HardenedBSD community members. We have a port of pledge that is nearing completion and will likely be merged into HardenedBSD on the inside of a year, quicker if we have help. :-)
The Linux approach, for better or worse, is to provide more generic/complex kernel APIs, the two most similar to pledge/unveil being respectively seccomp-bpf and landlock. Given those, you can port pledge/unveil in userspace. But that obviously results in less uptake than an API blessed/mandated by the OS developers in the OpenBSD style.
edit: Although see previous discussion for more caveats.
From the porting link:
This seems pretty easy on Linux with systemd, am I missing something? The program itself doesn’t even have to know about any control mechanisms like
pledgeor whatever, we can enforce it from outside.pledgeandunveilare specifically designed as tools for the programmer. They aren’t meant to be imposed from the outside as a sandbox. Theo has repeatedly rejected the idea of adding a utility to OpenBSD that runs an arbitrary command line under a particular pledge, for example. The intended use is for the developer (or knowledgeable porter) to pledge/unveil down to the resources that they know the program needs. It’s a tool for defensive programming: if your pledged process gets owned, it has as little ambient authority as possible.I agree. The programmer knows better than the user what resources a program needs.
Besides, the external approach imposes a static set that stays the same during the entire runtime, which is sort of critical, mostly because many programs require more privileges during the initialization which can then be dropped later.
I agree, but OTOH the user has the much greater incentive to care about sandboxing programs than their programmers (when the user and the programmer are not the same).
OK but my point is that the linked post says that this functionality is extremely difficult in Linux, but if we shift our thinking slightly to work with Linux’s execution resource controls, it seems to be quite easy.
It…depends. Linux sandboxing solutions are not particularly unified and depend on stitching a lot of pieces together, sometimes with confusing or bizarre edge cases. It’s not awful, but it’s still a fair bit of effort and a surprising amount of room for error.
(I will say that Landlock helps a lot here by being a relatively high-level set of access controls for things that are otherwise more difficult to sandbox.)
No, it really looks quite simple. Check the link I posted earlier. It’s as simple as adding a couple of lines to the systemd unit file or a couple of flags to the
systemd-runcommand line.There’s a significant structural difference between the two. I honestly don’t know why the Internet keeps lumping them together. Yes, both apply restrictions, but they’re completely different mechanisms. It’s like lumping email and keyloggers together because both ultimately boil down to delivering a series of keypresses to a third-party.
pledgeandunveilare things that the program does, which has two major implications:pledgeing a particular restriction is a supported mode of operation.pledgekills the program withSIGABRTand you get a stack trace pointing to wherever the promise was broken.pledgeorunveilwhenever you want during a program’s execution (but there are restrictions to what you can “unpledge” or “un-unveil”). So you can e.g. start with full access rights to the user’s home path in order to read configuration, and you can drop filesystem access privileges entirely once you’ve read configuration data.“External” sandboxing solutions allow you to apply the same restrictions, but:
IPAddressDeny=an address, but if sending telemetry to that address is a blocking operation, you’ll just get a blocked program. That’s obviously not on the sandboxing method itself, but…IPAddressDeny=packets just get dropped.The whole point of
pledgeandunveilis that you write your program so that you make certain promises about how it’s going to work, and sandboxing works as protection against accidentally breaking those promises, either straight as a bug, or through a bug getting exploited. They’re design-by-contract tools, more or less.I’m a bit confused, I wouldn’t expect “Linux’s execution resource controls” to include systemd-specific APIs? Tbc these toggles do not cleanly map to OS primitives at all; you can’t just go “ah yes I want systemd’s IP address restrictions myself” without writing your own BPF filtering.
If you did just mean systemd specifically, note that it’s quite a bit trickier to, say, sandbox things in a flexible way with that. In particular, to sandbox something that’s not a service with systemd-run has the overhead of spawning the child under the service manager and piping the output, and you lose the ability to easily control and kill the child (because you’re just killing the proxying process, not the actual child).
Yes, that’s what I meant.
But the whole point of systemd is that it makes it easy to control and kill the child processes that it runs…
Yes but these things are happening on different levels and mechanisms.
One is baked into the software by design, the other is not.
systemd is just flat out not a replacement for pledge+unveil, end of story. Completely different mechanisms.
I didn’t say it’s a ‘flat out replacement’, I said it achieves the same goals that were mentioned in the linked post ie preventing a piece of software I downloaded off the internet from ‘phoning home’ and uploading telemetry.
Okay I understand now.. The way I see it, the example in that post was a one-off example, and a pretty bad one, because I think from this you’ve misunderstood the purpose of pledge. While there may be some overlap in the functionaility of BPF filters they are far, far, from 1-1.
To help clear this up:
So in this scenerio of telemetry:
There would be no point in using pledge here.
Yeah exactly, the overlap is what I’m talking about. This is what I was referring to earlier–if you shift your mindset a bit, it becomes pretty easy to externally impose similar security restrictions using systemd. If you don’t mind that the exact way it’s done is a bit different, you can achieve very similar goals.
Part of the pledge/unveil model is that you can progressively drop privileges during program execution. The observation is that programs may need a lot of privileges in order to start themselves up, and then hardly any for the remainder of runtime. For example, consider the recommendations for using pledge when opening handles to sndiod in OpenBSD:
However:
And in fact
aucat(1)does exactly this, using a bunch of privileges at startup time (but dropping other very sensitive promises likeprocandexecright away) and then dropping most of them when it’s time to start playback or recording from the sound device handle.The purpose of pledge is for the programmer to mitigate the privileges available to exploits, not for the user to defend against code they chose to run. As far as I know, these goals, which are the main point of
pledgeandunveil, do not overlap at all with what you can do by filtering access via systemd.They overlap with the goal that was stated in the linked post. That was what I was referring to. I agree that dropping complex combinations of privileges in Linux at different points within an application’s lifecycle doesn’t seem easy.
I encourage you to read the rest of my messages in this thread to get the fuller context. It’s better than responding to one message at a time: https://lobste.rs/s/ny2s9f/openbsd_innovations#c_bds2yk
Yeah but the linked post misunderstands what pledge is actually for; “sandboxing untrusted binaries” is not an actual design goal of pledge and if it’s useful for that goal it’s only useful by accident.
Yeah so this is kinda weird: it makes it easy to manage the child as a single entity and then interactively control it. But systemd-run is, ofc, just a proxy, spawning the child as a transient unit and monitoring it until exit.
This effectively means that systemd-run does not itself control the spawned process’s lifetime. When you run it with –pty, Ctrl-C will kill the child normally… because it’s going through the tty. But if you’re running it detached from the tty (e.g. because you need to control stdout/stdin from your program), or you just signal the process directly, the actual service will remain running in the background. The only way of directly controlling that lifecycle is by grabbing the PID or (preferably) transient unit name and then targeting it directly or controlling it via the D-Bus interface.
But it’s a transient unit, it shows up in the output of
systemctl list-units, right? That means you can operate on it with egsystemctl stop the-unitjust like other systemd units?It is, yeah, which is why I had mentioned the “it makes it easy to […] interactively control it”. It’s just a lot harder in turn to automatically do things like “oh the user did Ctrl-C, let me stop all my processes”.
How does pledge/unveil compare to Capsicum?
From my admittedly passing acquaintance with both of them: Capsicum is much finer grained.
Pledge/unveil solves the 80% use case, Capsicum aims to solve the 100% use-case.
I.e. it would be very difficult, perhaps impossible to implement capsicum in pledge/unveil, but easy enough to do with capsicum.
That said, there is probably more veil/pledge actually implemented in binaries out in the wild. I think all of the base OpenBSD system is done now. Last I checked only some of FreeBSD has Capsicum implemented in base.
We ended up porting this to Nanos: https://nanovms.com/dev/tutorials/applying-sandbox-security-node-js-unikernels-openbsd-pledge-unveil .
How do you like to run your bookmarklets? I have the bookmarks navbar hidden to save screen space and so bookmarklets are always hidden enough for me not to think about them.
You can tag them and search the address bar by tag. I think defining them with a meaningful function name also helps the address bar search. Like so:
Thank you!
I have a couple that I use all the time, like ‘scroll to top/bottom’, so I have them on my bookmarks toolbar itself. For the rest, I have them in a subfolder inside my bookmarks toolbar.
What’s wrong with Home/End buttons?
Not available on typical Mac laptop keyboards, and even if they were, often my hand is on the touchpad so it’s just more convenient to tap on a button instead of moving my head to a key.
While I think moving on from C to a memory safe systems programming language is a good idea, for various reasons, I don’t think Rust will end up being a good choice. Ultimately, I think the choice of Rust is going to hurt Linux.
I hope to be proven wrong.
While I’m not exactly a Rust superfan, a bird in the hand is worth two in the bush.
Do you have anything in mind that would be a better choice than Rust?
I haven’t done low level programming for a long time, and I’ve never done kernel programming, so I’m probably the wrong person to ask.
That being said, I’ve heard good things about Zig.
Zig is nicer than C, but doesn’t bring any fundamental improvements. A big part of the reasoning for introducing Rust to the kernel despite the difficulty and complexity is that memory safety is a big deal, and Zig has no memory safety story at all.
I’d suggest that Zig has one memory safety story, namely that it has bounds checks (if not disabled) for arrays, so a spacial safety story.
That said, there are other reasons why I’ve gone off Zig.
This is an extremely strong statement.
I think a few things are also interesting:
I think people are realizing how low quality the Linux kernel code is, how haphazard development is, how much burnout and misery is involved, etc.
I think people are realizing how insanely not in the open kernel dev is, how much is private conversations that a few are privy to, how much is politics, etc.
The Hellwig/Ojeda part of the thread is just frustrating to read because it almost feels like pleading. “We went over this in private” “we discussed this already, why are you bringing it up again?” “Linus said (in private so there’s no record)”, etc., etc.
Dragging discussions out in front of an audience is a pretty decent tactic for dealing with obstinate maintainers. They don’t like to explain their shoddy reasoning in front of people, and would prefer it remain hidden. It isn’t the first tool in the toolbelt but at a certain point there is no convincing people directly.
With quite a few things actually. A friend of mine is contributing to a non-profit, which until recently had this very toxic member (they’ve even attempted felony). They were driven out of the non-profit very soon after members talked in a thread that was accessible to all members. Obscurity is often one key component of abuse, be it mere stubbornness or criminal behaviour. Shine light, and it often goes away.
IIRC Hintjens noted this quite explicitly as a tactic of bad actors in his works.
It’s amazing how quickly people are to recognize folks trying to subvert an org piecemeal via one-off private conversations once everybody can compare notes. It’s equally amazing to see how much the same people beforehand will swear up and down oh no that’s a conspiracy theory such things can’t happen here until they’ve been burned at least once.
This is an active, unpatched attack vector in most communities.
I’ve found the lowest example of this is even meetings minutes at work. I’ve observed that people tend to act more collaboratively and seek the common good if there are public minutes, as opposed to trying to “privately” win people over to their desires.
There is something to be said for keeping things between people with skin in the game.
It’s flipped over here, though, because more people want to contribute. The question is whether it’ll be stabe long-term.
Something I’ve noticed is true in virtually everything I’ve looked deeply at is the majority of work is poor to mediocre and most people are not especially great at their jobs. So it wouldn’t surprise me if Linux is the same. (…and also wouldn’t surprise me if the wonderful Rust rewrite also ends up poor to mediocre.)
yet at the same time, another thing that astonishes me is how much stuff actually does get done and how well things manage to work anyway. And Linux also does a lot and works pretty well. Mediocre over the years can end up pretty good.
After tangentially following the kernel news, I think a lot of churning and death spiraling is happening. I would much rather have a rust-first kernel that isn’t crippled by the old guard of C developers reluctant to adopt new tech.
Take all of this energy into RedoxOS and let Linux stay in antiquity.
I’ve seen some of the R4L people talk on Mastodon, and they all seem to hate this argument.
They want to contribute to Linux because they use it, want to use it, and want to improve the lives of everyone who uses it. The fact that it’s out there and deployed and not a toy is a huge part of the reason why they want to improve it.
Hopping off into their own little projects which may or may not be useful to someone in 5-10 years’ time is not interesting to them. If it was, they’d already be working on Redox.
The most effective thing that could happen is for the Linux foundation, and Linus himself, to formally endorse and run a Rust-based kernel. They can adopt an existing one or make a concerted effort to replace large chunks of Linux’s C with Rust.
IMO the Linux project needs to figure out something pretty quickly because it seems to be bleeding maintainers and Linus isn’t getting any younger.
They may be misunderstanding the idea that others are not necessarily incentivized to do things just because it’s interesting for them (the Mastodon posters).
Yep, I made a similar remark upthread. A Rust-first kernel would have a lot of benefits over Linux, assuming a competent group of maintainers.
along similar lines: https://drewdevault.com/2024/08/30/2024-08-30-Rust-in-Linux-revisited.html
Redox does have the chains of trying to do new OS things. An ABI-compatible Rust rewrite of the Linux kernel might get further along than expected (even if it only runs in virtual contexts, without hardware support (that would come later.))
Linux developers want to work on Linux, they don’t want to make a new OS. Linux is incredibly important, and companies already have Rust-only drivers for their hardware.
Basically, sure, a new OS project would be neat, but it’s really just completely off topic in the sense that it’s not a solution for Rust for Linux. Because the “Linux” part in that matters.
I read a 25+ year old article [1] from a former Netscape developer that I think applies in part
The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they’ve been fixed. There’s nothing wrong with it. It doesn’t acquire bugs just by sitting around on your hard drive. Au contraire, baby! Is software supposed to be like an old Dodge Dart, that rusts just sitting in the garage? Is software like a teddy bear that’s kind of gross if it’s not made out of all new material?Adopting a “rust-first” kernel is throwing the baby out with the bathwater. Linux has been beaten into submission for over 30 years for a reason. It’s the largest collaborative project in human history and over 30 million lines of code. Throwing it out and starting new would be an absolutely herculean effort that would likely take years, if it ever got off the ground.
[1] https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/
The idea that old code is better than new code is patently absurd. Old code has stagnated. It was built using substandard, out of date methodologies. No one remembers what’s a bug and what’s a feature, and everyone is too scared to fix anything because of it. It doesn’t acquire new bugs because no one is willing to work on that weird ass bespoke shit you did with your C preprocessor. Au contraire, baby! Is software supposed to never learn? Are we never to adopt new tools? Can we never look at something we’ve built in an old way and wonder if new methodologies would produce something better?
This is what it looks like to say nothing, to beg the question. Numerous empirical claims, where is the justification?
It’s also self defeating on its face. I take an old codebase, I fix a bug, the codebase is now new. Which one is better?
Like most things in life the truth is somewhere in the middle. There is a reason there is the concept of a “mature node” in the semiconductor industry. They accept that new is needed for each node, but also that the new thing takes time to iron out the kinks and bugs. This is the primary reason why you see apple take new nodes on first before Nvidia for example, as Nvidia require much larger die sizes, and so less defects per square mm.
You can see this sometimes in software for example X11 vs Wayland, where adoption is slow, but most definetly progressing and now-days most people can see that Wayland is now, or is going to become the dominant tech in the space.
The truth lies where it lies. Maybe the middle, maybe elsewhere. I just don’t think we’ll get to the truth with rhetoric.
Aren’t the arguments above more dialectic than rhetoric?
I don’t think this would qualify as dialectic, it lacks any internal debate and it leans heavily on appeals by analogy and intuition/ emotion. The post itself makes a ton of empirical claims without justification even beyond the quoted bit.
fair enough, I can see how one would make that argument.
“Good” is subjective, but there is real evidence that older code does contain fewer vulnerabilities: https://www.usenix.org/conference/usenixsecurity22/presentation/alexopoulos
That means we can probably keep a lot of the old trusty Linux code around while making more of the new code safe by writing it in Rust in the first place.
I don’t think that’s a fair assessment of Spolsky’s argument or of CursedSilicon’s application of it to the Linux kernel.
Firstly, someone has already pointed out the research that suggests that existing code has fewer bugs in than new code (and that the older code is, the less likely it is to be buggy).
Secondly, this discussion is mainly around entire codebases, not just existing code. Codebases usually have an entire infrastructure around them for verifying that the behaviour of the codebase has not changed. This is often made up of tests, but it’s also made up of the users who try out a release of a codebase and determine whether it’s working for them. The difference between making a change to an existing codebase and releasing a new project largely comes down to whether this verification (both in terms of automated tests and in terms of users’ ability to use the new release) works for the new code.
Given this difference, if I want to (say) write a new OS completely in Rust, I need to choose: Do I want to make it completely compatible with Linux, and therefore take on the significant challenge of making sure everything behaves truly the same? Or do I make significant breaking changes, write my own OS, and therefore force potential adopters to rebuild their entire Linux workflows in my new OS?
The point is not that either of these options are bad, it is that they represent significant risks to a project. Added to the general risk that is writing new code, this produces a total level of risk that might be considered the baseline risk of doing a rewrite. Now risk is not bad per se! If the benefits of being able to write an OS in a language like Rust outweigh the potential risks, then it still makes sense to perform the rewrite. Or maybe the existing Linux kernel is so difficult to maintain that a new codebase really would be the better option. But the point that CursedSilicon was making by linking the Spolsky piece was, I believe, that the risks for a project like the Linux kernel are very high. There is a lot of existing, old code. And there is a very large ecosystem where either breaking or maintaining compatibility would each come with significant challenges.
Unfortunately, it’s very difficult to measure the risks and benefits here in a quantitative, comparable way, so I think where you fall on the “rewrite vs continuity” spectrum will depend mostly on what sort of examples you’ve seen, and how close you think this case is to those examples. I don’t think there’s any objective way to say whether it makes more sense to have something like R4L, or something like RedoxOS.
I haven’t read it yet, but I haven’t made an argument about that, I just created a parody of the argument as presented. I’ll be candid, i doubt that the research is going to compel me to believe that newer code is inherently buggier, it may compel me to confirm my existing belief that testing software in the field is one good method to find some classes of bugs.
I guess so, it’s a bit dependent on where we say the discussion starts - three things are relevant; RFL, which is not a wholesale rewrite, a wholesale rewrite of the Linux kernel, and Netscape. RFL is not about replacing the entire Linux kernel, although perhaps “codebase” here refers to some sort of unit, like a driver. Netscape wanted a wholesale rewrite, based on the linked post, so perhaps that’s what’s really “the single worst strategic mistake that any software company can make”, but I wonder what the boundary here is? Also, the article immediately mentions that Microsoft tried to do this with Word but it failed, but that Word didn’t suffer from this because it was still actively developed - I wonder if it really “failed” just because pyramid didn’t become the new Word? Did Microsoft have some lessons learned, or incorporate some of that code? Dunno.
I think I’m really entirely justified when I say that the post is entirely emotional/ intuitive appeals, rhetoric, and that it makes empirical claims without justification.
This is rhetoric. These are unsubstantiated empirical claims. The article is all of this. It’s fine as an interesting, thought provoking read that gets to the root of our intuitions, but I think anyone can dismiss it pretty easily since it doesn’t really provide much in the form of an argument.
Again, totally unsubstantiated. I have MANY reasons to believe that, it is simply question begging to say otherwise.
That’s all this post is. Over and over again making empirical claims with no evidence and question beggign.
We can discuss the risks and benefits, I’d advocate for that. This article posted doesn’t advocate for that. It’s rhetoric.
This is a truism. It is survival bias. If the code was buggy, it would eventually be found and fixed. So all things being equal newer code is riskier than old code. But it’s also been impirically shown that using Rust for new code is not “all things being equal”. Google showed that new code in Rust is as reliable as old code in C. Which is good news: you can use old C code from new Rust projects without the risk that comes from new C code.
Yeah, this is what I’ve been saying (not sure if you’d meant to respond to me or the parent, since we agree) - the issue isn’t “new” vs “old” it’s things like “reviewed vs unreviewed” or “released vs unreleased” or “tested well vs not tested well” or “class of bugs is trivial to express vs class of bugs is difficult to express” etc.
Was restating your thesis in the hopes of making it clearer.
I don’t disagree that the rewards can outweigh the risks, and in this case I think there’s a lot of evidence that suggests that memory safety as a default is really important for all sorts of reasons. Let alone the many other PL developments that make Rust a much more suitable language to develop in than C.
That doesn’t mean the risks don’t exist, though.
Nobody would call an old codebase with a handful of fixes a new codebase, at least not in the contexts in which those terms have been used here.
How many lines then?
It’s a Ship of Theseus—at no point can you call it a “new” codebase, but after a period of time, it could be completely different code. I have a C program I’ve been using and modifying for 25 years. At any given point, it would have been hard to say “this is now a new codebase, yet not one line of code in the project is the same as when I started (even though it does the same thing at it always has).
I don’t see the point in your question. It’s going to depend on the codebase, and on the nature of the changes; it’s going to be nuanced, and subjective at least to some degree. But the fact that it’s prone to subjectivity doesn’t mean that you get to call an old codebase with a single fixed bug a new codebase, without some heavy qualification which was lacking.
If it requires all of that nuance and context maybe the issue isn’t what’s “old” and what’s “new”.
I don’t follow, to me that seems like a non-sequitur.
What’s old and new is poorly defined and yet there’s an argument being made that “old” and “new” are good indicators of something. If they’re so poorly defined that we have to bring in all sorts of additional context like the nature of the changes, not just when they happened or the number of lines changed, etc, then it seems to me that we would be just as well served to throw away the “old” and “new” and focus on that context.
I feel like enough people would agree more-or-less on what was an “old” or “new” codebase (i.e. they would agree given particular context) that they remain useful terms in a discussion. The general context used here is apparent (at least to me) given by the discussion so far: an older codebase has been around for a while, has been maintained, has had kinks ironed out.
There’s a really important distinction here though. The point is to argue that new projects will be less stable than old ones, but you’re intuitively (and correctly) bringing in far more important context - maintenance, testing, battle testing, etc. If a new implementation has a higher degree of those properties then it being “new” stops being relevant.
Ok, but:
My point was that this statement requires a definition of “new codebase” that nobody would agree with, at least in the context of the discussion we’re in. Maybe you are attacking the base proposition without applying the surrounding context, which might be valid if this were a formal argument and not a free-for-all discussion.
I think that it would be considered no longer new if it had had significant battle-testing, for example.
FWIW the important thing in my view is that every new codebase is a potential old codebase (given time and care), and a rewrite necessarily involves a step backwards. The question should probably not be, which is immediately better?, but, which is better in the longer term (and by how much)? However your point that “new codebase” is not automatically worse is certainly valid. There are other factors than age and “time in the field” that determine quality.
Methodologies don’t matter for quality of code. They could be useful for estimates, cost control, figuring out whom you shall fire etc. But not for the quality of code.
You’re suggesting that the way you approach programming has no bearing on the quality of the produced program?
I’ve never observed a programmer become better or worse by switching methodology. Dijkstra would’ve not became better if you made him do daily standups or go through code reviews.
There are ways to improve your programming by choosing different approach but these are very individual. Methodology is mostly a beancounting tool.
When I say “methodology” I’m speaking very broadly - simply “the approach one takes”. This isn’t necessarily saying that any methodology is better than any other. The way I approach a task today is better, I think, then the way that I would have approached that task a decade ago - my methodology has changed, the way I think has changed. Perhaps that might mean I write more tests, or I test earlier, but it may mean exactly the opposite, and my methods may only work best for me.
I’m not advocating for “process” or ubiquity, only that the approach one tasks may improve over time, which I suspect we would agree on.
If you take this logic to its end, you should never create new things.
At one point in time, Linux was also the new kid on the block.
The best time to plant a tree is 30 years ago. The second best time is now.
I don’t think Joel Spolsky was ever a Netscape developer. He was a Microsoft developer who worked on Excel.
My mistake! The article contained a bit about Netscape and I misremembered it
How many of those lines are part of the core? My understanding was that the overwhelming majority was driver code. There may not be that much core subsystem code to rewrite.
For a previous project, we included a minimal Linux build. It was around 300 KLoC, which included networking and the storage stack, along with virtio drivers.
That’s around the size a single person could manage and quite easy with a motivated team.
If you started with DPDK and SPDK then you’d already have filesystems and a copy of the FreeBSD network stack to run in isolated environments.
Once many drivers share common rust wrappers over core subsystems, you could flip it and write the subsystem in Rust. Then expose C interface for the rest.
Oh sure, that would be my plan as well. And I bet some subsystem maintainers see this coming, and resist it for reasons that aren’t entirely selfless.
That’s pretty far into the future, both from a maintainer acceptance PoV and from a rustc_codegen_gcc and/or gccrs maturity PoV.
Sure. But I doubt I’ll running a different kernel 10y from now.
And like us, those maintainers are not getting any younger and if they need a hand, I am confident I’ll get faster into it with a strict type checker.
I am also confident nobody in our office would be able to help out with C at all.
This cannot possibly be true.
It’s the largest collaborative open source os kernel project in human history
It’s been described as such based purely on the number of unique human contributions to it
I would expect Wikipedia should be bigger 🤔
I see that Drew proposes a new OS in that linked article, but I think a better proposal in the same vein is a fork. You get to keep Linux, but you can start porting logic to Rust unimpeded, and it’s a manageable amount of work to keep porting upstream changes.
Remember when libav forked from ffmpeg? Michael Niedermayer single-handedly ported every single libav commit back into ffmpeg, and eventually, ffmpeg won.
At first there will be extremely high C percentage, low Rust percentage, so porting is trivial, just git merge and there will be no conflicts. As the fork ports more and more C code to Rust, however, you start to have to do porting work by inspecting the C code and determining whether the fixes apply to the corresponding Rust code. However, at that point, it means you should start seeing productivity gains, community gains, and feature gains from using a better language than C. At this point the community growth should be able to keep up with the extra porting work required. And this is when distros will start sniffing around, at first offering variants of the distro that uses the forked kernel, and if they like what they taste, they might even drop the original.
I genuinely think it’s a strong idea, given the momentum and potential amount of labor Rust community has at its disposal.
I think the competition would be great, especially in the domain of making it more contributor friendly to improve the kernel(s) that we use daily.
I certainly don’t think this is impossible, for sure. But the point ultimately still stands: Linux kernel devs don’t want a fork. They want Linux. These folks aren’t interested in competing, they’re interested in making the project they work on better. We’ll see if some others choose the fork route, but it’s still ultimately not the point of this project.
While I don’t personally want to make a new OS, I’m not sure I actually want to work on Linux. Most of the time I strive for portability, and so abstract myself from the OS whenever I can get away with it. And when I can’t, I have to say Linux’s API isn’t always that great, compared to what the BSDs have to offer (epoll vs kqueue comes to mind). Most annoying though is the lack of documentation for the less used APIs: I’ve recently worked with Netlink sockets, and for the proc stuff so far the best documentation I found was the freaking source code of a third party monitoring program.
I was shocked. Complete documentation of the public API is the minimum bar for a project as serious of the Linux kernel. I can live with an API I don’t like, but lack of documentation is a deal breaker.
I think they mean that Linux kernel devs want to work on the Linux kernel. Most (all?) R4L devs are long time Linux kernel devs. Though, maybe some of the people resigning over LKML toxicity will go work on Redox or something…
That’s is what I was saying, yes.
I’m talking about the people who develop the Linux kernel, not people who write userland programs for Linux.
Re-Implementing the kernel ABI would be a ton of work for little gain if all they wanted was to upstream all the work on new hardware drivers that is already done - and then eventually start re-implementing bits that need to be revised anyway.
If the singular required Rust toolchain didn’t feel like such a ridiculous to bootstrap 500 ton LLVM clown car I would agree with this statement without reservation.
Would zig be a better starting place?
Zig is easier to implement (and I personally like it as a language) but doesn’t have the same safety guarantees and strong type system that Rust does. It’s a give and take. I actually really like Rust and would like to see a proliferation of toolchain options, such as what’s in progress in GCC land. Overall, it would just be really nice to have an easily bootstrapped toolchain that a normal person can compile from scratch locally, although I don’t think it necessarily needs to be the default, or that using LLVM generally is an issue. However, it might be possible that no matter how you architect it, Rust might just be complicated enough that any sufficiently useful toolchain for the language could just end up being a 500 ton clown car of some kind anyways.
Depends on which parts of GP’s statement you care about: LLVM or bootstrap. Zig is still depending on LLVM (for now), but it is no longer bootstrappable in a limited number of steps (because they switched from a bootstrap C++ implementation of the compiler to keeping a compressed WASM build of the compiler as a blob.
Yep, although I would also add it’s unfair to judge Zig in any case on this matter now given it’s such a young project that clearly is going to evolve a lot before the dust begins to settle (Rust is also young, but not nearly as young as Zig). In ten to twenty years, so long as we’re all still typing away on our keyboards, we might have a dozen Zig 1.0 and a half dozen Zig 2.0 implementations!
Yeah, the absurdly low code quality and toxic environment make me think that Linux is ripe for disruption. Not like anyone can produce a production kernel overnight, but maybe a few years of sustained work might see a functional, production-ready Rust kernel for some niche applications and from there it could be expanded gradually. While it would have a lot of catching up to do with respect to Linux, I would expect it to mature much faster because of Rust, because of a lack of cruft/backwards-compatibility promises, and most importantly because it could avoid the pointless drama and toxicity that burn people out and prevent people from contributing in the first place.
What is the, some kind of a new meme? Where did you hear it first?
From the thread in OP, if you expand the messages, there is wide agreement among the maintainers that all sorts of really badly designed and almost impossible to use (safely) APIs ended up in the kernel over the years because the developers were inexperienced and kind of learning kernel development as they went. In retrospect they would have designed many of the APIs very differently.
Someone should compile everything to help future OS developers avoid those traps! There are a lot of exieting non-posix experiments though.
It’s based on my forays into the Linux kernel source code. I don’t doubt there’s some quality code lurking around somewhere, but the stuff I’ve come across (largely filesystem and filesystem adjacent) is baffling.
Seeing how many people are confidently incorrect about Linux maintainers only caring about their job security and keeping code bad to make it a barrier to entry, if nothing else taught me how online discussions are a huge game of Chinese whispers where most participants don’t have a clue of what they are talking about.
I doubt that maintainers are “only caring about their job security and keeping back code” but with all due respect: You’re also just taking arguments out of thin air right now. What I do believe is what we have seen: Pretty toxic responses from some people and a whole lot of issues trying to move forward.
Huh, I’m not seeing any claim to this end from the GP, or did I not look hard enough? At face value, saying that something has an “absurdly low code quality” does not imply anything about nefarious motives.
I can personally attest to having never made that specific claim.
Indeed that remark wasn’t directly referring to GP’s comment, but rather to the range of confidently incorrect comments that I read in the previous episodes, and to the “gatekeeping greybeards” theme that can be seen elsewhere on this page. First occurrence, found just by searching for “old”: Linux is apparently “crippled by the old guard of C developers reluctant to adopt new tech”, to which GP replied in agreement in fact. Another one, maintainers don’t want to “do the hard work”.
Still, in GP’s case the Chinese whispers have reduced “the safety of this API is hard to formalize and you pretty much have to use it the way everybody does it” to “absurdly low quality”. To which I ask, what is more likely. 1) That 30-million lines of code contain various levels of technical debt of which maintainers are aware; and that said maintainers are worried even of code where the technical debt is real but not causing substantial issue in practice? Or 2) that a piece of software gets to run on literally billions of devices of all sizes and prices just because it’s free and in spite of its “absurdly low quality”?
Linux is not perfect, neither technically nor socially. But it sure takes a lot of entitlement and self-righteousness to declare it “of absurdly low quality” with a straight face.
GP here: I probably should have said “shockingly” rather than “absurdly”. I didn’t really expect to get lawyered over that one word, but yeah, the idea was that for a software that runs on billions of devices, the code quality is shockingly low.
Of course, this is plainly subjective. If your code quality standards are a lot lower than mine then you might disagree with my assessment.
That said, I suspect adoption is a poor proxy for code quality. Internet Explorer was widely adopted and yet it’s broadly understood to have been poorly written.
I’m sure self-righteousness could get you to the same place, but in my case I arrived by way of experience. You can relax, I wasn’t attacking Linux—I like Linux—it just has a lot of opportunity for improvement.
I guess I’ve seen the internals of too much proprietary software now to be shocked by anything about Linux per se. I might even argue that the quality of Linux is surprisingly good, considering its origins and development model.
I think I’d lawyer you a tiny bit differently: some of the bugs in the kernel shock me when I consider how many devices run that code and fulfill their purposes despite those bugs.
FWIW, I was not making a dig at open source software, and yes plenty of corporate software is worse. I guess my expectations for Linux are higher because of how often it is touted as exemplary in some form or another. I don’t even dislike Linux, I think it’s the best thing out there for a huge swath of use cases—I just see some pretty big opportunities for improvement.
Or actual benchmarks: the performance the Linux kernel leaves on the table in some cases is absurd. And sure it’s just one example, but I wouldn’t be surprised if it was representative of a good portion of the kernel.
Well not quite but still “considered broken beyond repair by many people related to life time management” - which is definitely worse than “hard to formalize” when “the way ever[y]body does it” seems to vary between each user.
I love Rust but still, we’re talking of a language which (for good reasons!) considers doubly linked lists unsafe. Take an API that gets a 4 on Rusty Russell’s API design scale (“Follow common convention and you’ll get it right”), but which was designed for a completely different programming language if not paradigm, and it’s not surprising that it can’t easily be transformed into a 9 (“The compiler/linker won’t let you get it wrong”). But at the same time there are a dozen ways in which, according to the same scale, things could actually be worse!
What I dislike is that people are seeing “awareness of complexity” and the message they spread is “absurdly low quality”.
Note that doubly linked lists are not a special case at all in Rust. All the other common data structures like
Vec,HashMapetc. also need unsafe code in their implementation.Implementing these datastructures in Rust, and writing unsafe code in general, is indeed roughly a 4. But these are all already implemented in the standard library, with an API that actually is at a 9. And
std::collections::LinkedListis constructive proof that you can have a safe Rust abstraction for doubly linked lists.Yes, the implementation could have bugs, thus making the abstraction leaky. But that’s the case for literally everything, down to the hardware that your code runs on.
You’re absolutely right that you can build abstractions with enough effort.
My point is that if a doubly linked list is (again, for good reasons) hard to make into a 9, a 20-year-old API may very well be even harder. In fact,
std::collections::LinkedListis safe but still not great (for example the cursor API is still unstable); and being in std, it was designed/reviewed by some of the most knowledgeable Rust developers, sort of by definition. That’s the conundrum that maintainers face and, if they realize that, it’s a good thing. I would be scared if maintainers handwaved that away.Bugs happen, but if the abstraction is downright wrong then that’s something I wouldn’t underestimate. A lot of the appeal of Rust in Linux lies exactly in documenting/formalizing these unwritten rules, and wrong documentation can be worse than no documentation (cue the negative parts of the API design scale!); even more so if your documentation is a formal model like a set of Rust types and functions.
That said, the same thing can happen in a Rust-first kernel, which will also have a lot of unsafe code. And it would be much harder to fix it in a Rust-first kernel, than in Linux at a time when it’s just feeling the waters.
At the same time, it was included almost as like, half a joke, and nobody uses it, so there’s not a lot of pressure to actually finish off the cursor API.
It’s also not the kind of linked list the kernel would use, as they’d want an intrusive one.
And yet, safe to use doubly linked lists written in Rust exist. That the implementation needs unsafe is not a real problem. That’s how we should look at wrapping C code in safe Rust abstractions.
The whole comment you replied to, after the one sentence about linked lists, is about abstractions. And abstractions are rarely going to be easy, and sometimes could be hardly possible.
That’s just a fact. Confusing this fact for something as hyperbolic as “absurdly low quality” is stunning example of the Dunning Kruger effect, and frankly insulting as well.
I personally would call Linux low quality because many parts of it are buggy as sin. My GPU stops working properly literally every other time I upgrade Linux.
No one is saying that Linux is low quality because it’s hard or impossible to abstract some subsystems in Rust, they’re saying it’s low quality because a lot of it barely works! I would say that your “Chinese whispers” misrepresents the situation and what people here are actually saying. “the safety of this API is hard to formalize and you pretty much have to use it the way everybody does it” doesn’t apply if no one can tell you how to use an API, and everyone does it differently.
I agree, Linux is the worst of all kernels.
Except for all the others.
Actually, the NT kernel of all things seems to have a pretty good reputation, and I wouldn’t dismiss the BSD kernels out of hand. I don’t know which kernel is better, but it seems you do. If you could explain how you came to this conclusion that would be most helpful.
NT gets a bad rap because of the OS on top of it, not because it’s actually bad. NT itself is a very well-designed kernel.
*nod* I haven’t been a Windows person since shortly after the release of Windows XP (i.e. the first online activation DRM’d Windows) but, whenever I see glimpses of what’s going on inside the NT kernel in places like Project Zero: The Definitive Guide on Win32 to NT Path Conversion, it really makes me want to know more.
More likely a fork that gets rusted from the inside out
Somewhere else it was mentioned that most developers in the kernel could just not be bothered with checking for basic things.
Nobody is forcing any of these people to do this.
One thing that stood out to me is that systemd’s configuration approach is declarative while others are imperative, and this unlocks a lot of benefits, much like how a query planner can execute a declarative query better than most hand-written imperative queries could.
It pairs very well with NixOS for this reason.
Yea, it’s unreasonbly effective, and in general all the components hang together really well.
In my experience the nicest systems to administer have kind of an “alternating layer” approach of imperative - declarative - imperative - declarative - … Papering over too much complexity with declarative causes trouble, and so does forcing (or encouraging, or allowing) too much imperative scripting at any given layer. Systemd (it seems to me, as a user) really nailed the layers to encapsulate declaratively - things you want to happen at boot, that depend on each other. Units are declarative but effectively “call into” imperative commands, which - in turn - ought to have their own declarative-ish configuration files, which - in turn - are best laid down by something programmable at the higher layer (something aware of larger state - what load balancers are up - what endpoints are up - what DNS servers are up, yadda yadda).
As an aside, my main beef with Kubernetes is that is has a bad design - or perhaps a missing design - for that next level up of configuration management. Way way way too much is squashed into a uniform morass of the generalized “object namespace”, and the only way to really get precise behavior out of that is to implement a custom controller, which is almost like taking on the responsibility of a mini-init system, or a mini-network config daemon.
What I want at this layer instead is a framework for writing a simple and declarative-ish DSL that lets me encode my own business domain. Not every daemon is a Deployment object. Not every service is a Service. I think we were just getting good at this in 2014 with systems like Salt and Chef, but K8s came in and sucked all the oxygen out of the room for a decade, not to mention that Salt and Chef did themselves no favors by simply being a metric boatload of not-always-easy-to-install scripting runtimes.
Years ago when I looked to contribute to Salt I was pretty astounded by the bugginess, but it worked for enough people and there was not and still is not a lot of competition in that space, so VMware acquired them.
We have yet to see the next generation of that sort of software. System Initiative may be the sole exception, and I am enthusiastic about the demos they put out. But I do think we’re missing something here.
I just saw that historical risk of technology is off topic: https://lobste.rs/s/hnlghh/what_punch_cards_teach_us_about_ai_risk
Then I guess a story like this is also because it has nothing to do with computing? This is just a fact of life of using commercial technology in the current business environment.
This is not a historical risk, this is an issue that can happen right now to any devenv user.
It’s an ethical/business risk and does not seem to have anything to do with computing per se.
Depending on how you look at it, it’s one of:
I do agree that the tag doesn’t really cover the issue (I mean, it is a nix-related tool, so that’s probably fine, but perhaps all of the aforementioned tags should also apply?) and the link is to the social media mob frenzy which is less than ideal.
So this is inaccurate, right?
It’s complicated. Someone commented that the
devenv generatefeature violates GDPR: https://github.com/cachix/devenv/pull/1700#issuecomment-2661070650I find that highly doubtful; we’re talking about code here. Maybe it would apply if you’re storing personal information in the specific repo, but even then it’s still a user-initiated action, and (at least the claim is) the service doesn’t keep any of the information.
Which part of it do you doubt? Are you saying that consent to upload the files is explicitly gathered? If so, how?
I’m doubting the GDPR applies as it’s specifically about personally identifiable information. Repositories aren’t typically covered by that.
At the very least there is a lot of grey area here, people can easily have repositories where they store personal information, eg someone else gave an example of data science repos where datasets are stored using LFS. Or another example, many code repositories have files where authors’ names and email addresses are mentioned, eg package metadata files.
The argument can definitely be made that this command was slurping up data without regard to whether it might be personal information or not.
Fair enough, but that’s not exactly the same as stating that it’s a GDPR violation per se.
They’re not a lawyer, right?
Even if they were, you should not take legal advice from a lawyer you are not paying, and you should definitely check with your own lawyer if any doubt has been raised about legality.
From the thread: https://chaos.social/@refi64@refi64.social/114009389091464531
I think this misses what I perceive to be the main issue here. Yes, using the AI feature implies that your repo will be sent to an AI, but AI providers claim not to retain conversations sent via API and perhaps you’ve already chosen to trust them on this claim. But this
generatefeature seems to go out of its way to retain a copy on a server controlled by thedevenvauthor, which is very surprising and shady.Do you have a pointer of this happening? Hope I don’t come across as “sealioning” here, but the boundary between privacy and productivity in AI-enhanced stuff is something we should all be very careful and factual about.
I don’t have a pointer to some place in some code that shows a copy being made, but the server receiving the code generation request is closed-source, as far as I know.
I’ve essentially derived this assesment from the context, especially this comment from Domen:
I can’t imagine how they could possibly “improve the generation” based on whatever it is that they are retaining when DO_NOT_TRACK is not set unless they retain the full input (which is a copy of the repo) so that they can feed it during model tuning (or maybe prompt tuning?)
I don’t think it’s really going “out of its way”: the LLM itself is hosting on devenv servers, and unfortunately the server side seems to be a black box…but by extension, there’s no particular indication of it doing that.
Lol I was just about to mention this clarification and then realized it was already posted.
This post caught my eye because we’re considering sqlite for lobsters, but this is not really a compelling set of points.
I’m not well-informed enough to agree or disagree with this post, just frustrated that it doesn’t make the best case it could.
Re: migrations: SQLite doesn’t support some commonplace
ALTER TABLEbehavior. Instead, you have to:NOT NULLcolumn without default, a constraint, whateverFor large tables this will lock the database for a while as the data copies over.
Additionally, this is an error-prone process. Do this enough times and eventually you might get the steps wrong. Even if you don’t, I had an ORM delete data on a small project when it executed statements in an unintuitive, undocumented, and nondeterministic order, running the
DROPbefore running the statement disabling foreign keys. The ORM’s built-in schema commands weren’t susceptible to this issue, but SQLite required me to use a lower-level approach.I’m very happy with the solution I built for this problem, which I’ve been using for several years: https://sqlite-utils.datasette.io/en/stable/cli.html#transforming-tables
Just so happens that there’s a “migrations” focused post also on Lobsters today.
To me this does not sound much less nutso than mysql online schema change machinery or vitess or whatever. Postgres is more better about this, sure. But sqlite is not bad, all the DDL is fully supported in transactions and it has a handy slot for storing your schema version number. If you’re using an ORM that is shredding your data, like, don’t use that software, but it’s not SQLite’s fault.
Yeah I agree this post does not make a compelling case. I have my own reasons I don’t love SQLite (really limited builtin methods and a too-flexible grammar) and I work for a Postgres company but from reading this article alone I can’t see a coherent reason not to use SQLite on the server.
Did you miss the part about need to share a disk if you want more than one application server? Granted, not all applications need more than one copy running…
I guess the other option is that each application server has their own sqlite and you use RAFT or something to drive consensus between them… which maybe works if you can have at least 3 of these servers, at the expense of extreme complexity. :)
Surely that’s the important point? SQLite is great when everything fits on a single machine, which is really the case for the vast majority of projects [citation needed].
When something is big enough to need multiple machines then it’s time to switch to a separate storage layer running PostgreSQL.
Isn’t this all just a case of picking the right tool for your situation?
This is the point of LiteFS - it replicates writes to other boxes - no NFS needed (and iirc locks on nfs were unreliable so you probably dont want to run the database file on nfs).
You DO need to take care to send writes to the leader but Fly (where the author works), makes this pretty easy. LiteStream (same author) is similar but is more of a continuous back up tool.
Right. A thing you can do is make your architecture more complicated by introducing LiteFS, or LiteStream, but it’s no longer “just sqlite” at that point.
I’ve been running Litestream for a while now and it’s VERY straight-forward. You start the process and point it at a SQLite database, it inexpensively streams WAL blocks to S3 or similar, you can use those for recovery.
It’s so simple, robust and cheap that I see it as a selling point for SQLite as a whole - much less complex/expensive than achieving point-in-time replica backups with PostgreSQL or MySQL.
Yeah but again, these work with only a single instance at a time. If that’s your setup it’s great, but many people need multi-instance setups and mounting a shared SQLite file on NFS in some cloud provider is not a great idea.
But it isn’t an NFS in the cloud, it replicates the WAL which is then made available, yes via a fuse file system but it’s not NFS, it’s not really any different that read replicas for Postgres or MySQL and it seems to be a lot easier to get going. You DO have HA on your DB don’t you?
If you don’t think it works for you, that’s fine, but I think you should understand how it works before making that decision.
Yeah but the point is that a cloud vendor DB gives me read replicas out of the box, I don’t need to set it up manually. Added to the other points, this makes it kinda hard to argue against PostgreSQL for a typical backend app.
The same is as true with sqlite as with other dbs. With Turso/libsql or litefs you can have a read replica materialized on the local nvme ssd of every application server. I’m not arguing SQLite is a better option for every app like everything in engineering the answer is “it depends”. But it feels unfair to dismiss SQLite based architecture because the sqlite3.c source file doesn’t directly automate some need and dismiss the sqlite vendors, and then compare that to Postgres + unlimited ancillary vendor support & automation.
Turso is one vendor. Cloudflare is another but its SQLite offering has a fairly low max database size. Meanwhile, most cloud providers have mature support for MySQL/Postgres. If the cloud provider situation changes for SQLite in the future and it is offered more widely, then the analysis changes, but until then I believe it’s fair. And so do the folks who actually make SQLite.
Sure… if you’re using AWS, Azure or Google… and yeah, even fly.io (though theirs is NOT managed) does too, but really, it’s run a single executable in the background - replay or routing writes to the master is more work of course, but it’s still pretty easy. And given the stance of the US govt lately towards my country, I am likely NOT to use any of those, preferring to host domestically (sorry to bring politics into this - but everything is a trade off for making technical decisions, including that) .
I once set up HA Postgres on Linode (floating ip,NBD and the watchdog) and it worked, but was cargo culted and seemed very fragile in it’s complexity. LiteFS in comparison seems clean and elegant and would be very robust against SPOF and as a benefit move the db reads to the edge, nearer to my hypothetical users.
But like I said, I am not running anything using it at the moment. The one project I AM considering doing would use Postgres for other reasons.
Right, if you want multi-instance setups you need LiteFS, not Litestream.
To be clear. I’ve not used it but from all appearances, it looks like it adds very little overhead or complexity.
I think there might be a few cases in which SQLite forces one to use multiple queries where PostgreSQL wouldn’t, but it doesn’t usually, does it?
From that linked page:
“the situation at hand” could include “I want to leave open the option of replacing the RDBMS”.
Well, the article is pretty clear in what it’s trying to say, quote:
Which is saying, many choose to use sqlite because they don’t need to run anything extra - they just need to link their code with sqlite and it works. And trying to use sqlite on server-side negates that benefit. So, if that’s not why you want to use sqlite to begin with, this point doesn’t affect you at all.
Can anyone explain what the kernel maintainer drama they’re talking about is?
Most of Asahi’s linux work is in Rust, such as their new GPU drivers for Apple Silicon. Part of that requires access to the DMA subsystem. An asahi maintainer other than Marcan wanted to upstream Rust bindings to DMA. These would have been part of the “Rust for Linux” subsystem, which explicitly is exempted from the usual kernel stability guarantees. The maintainer of DMA in Linux objected to them upstreaming those bindings, as he felt he would then be obligated to maintain the Rust code, and he did not want to maintain Rust. This was combined with some uncharitable comments on the Rust for Linux project’s goals. There has been a lot of debate on other internet forums about whether these comments were uncharitable about Rust, Rust for Linux, or some other distinction that personally I think doesn’t really matter.
Marcan and that maintainer then exchanged words on the LKML. Unsatisfied, Marcan appealed to his Mastodon for support. Linus then stepped in with a reply heavily condemning the appeal to social media and not really addressing the complaints that led to that point. Marcan later resigned from his kernel maintainerships, and now also from the Asahi project.
I think this understates the position of the maintainer. He sent a message to LKML that states it pretty clearly: he thinks a multi-language kernel is a terrible mistake. And I can understand his position — which to me demonstrates that there is a leadership failure happening, because this kind of fundamental conflict shouldn’t be allowed to go on this long. “Disagree and Commit”, as they say at Amazon.
https://lwn.net/ml/all/20250131075751.GA16720@lst.de/
I’ll go further, it misstates the maintainers position.
The maintainer never indicated “he felt he would then be obligated to maintain the Rust code”.
The maintainers “NAK” occurred in response to the following post
I.e. after the explicit statement that the rust people, not Hellwig, would be maintaining it separately.
(And then we reach the post you quoted, where he makes it clear that his reasons are that he doesn’t want Rust in Linux at all, not that he doesn’t want to maintain it)
Yes, I understated the understatement. :) His stated goal is to use his influence and control over the DMA subsystem to prevent Rust (edit: or any other non-C language), from being in the kernel at all. As his NAK says:
Classic LKML rhetorical style here, to use a loaded word like “cancer” to make what is actually a reasonable technical point.
It is not a reasonable technical point! It demonstrates a fundamental incuriosity about technical matters. Categorical technical statements are rarely reasonable.
I disagree in general, but not in specifics. I’ve spoken to Chris Hellman at lengths (literally 6 hours) about the topic of Rust and it was not combative. He was curious. It’s hard to sum up at length, but his disagreements are rooted in specifics and he’s a hardliner about the single source language thing. But not for the sake of being a hardliner. His hard-line stance comes out of different weighting of the issues at hand, not an outright dismissal.
Those people existing is to be expected and it doesn’t invalidate their opinions.
Obviously not a fan of LKML rhetoric, but yep, you get the tone you built over years.
You can disagree, but for the rest of us who haven’t had these super private, intimate exchanges, where we get to “really know” someone, what he puts into the public is our perception of him. To that end, he’s called the project a cancer, and declared he’s going to do whatever he can to stop it which is not what a curios person does. He comes across instead as a religious zealot or “hardliner” in your terms.
That’s fair, but it’s also a categorical mistake to read a (brusque and inappropriate) email and judge on a whole person for days and weeks on the internet. I’m not saying you “really need to know someone”, but you also need to be aware that hundreds of hours are spent on reading what those statements may mean.
Wanting to keep a large, complex, codebase in a single language is a very reasonable technical point.
Using the term “cancer” to describe it the adoption of another language, though, is inflammatory and probably didn’t help matters.
A general inclination towards this is reasonable (and one I agree with). A categorical statement is not.
Again, many other C and/or C++ projects have introduced Rust into them. Some have succeeded, while others have not. It is worth learning from them.
I’ve personally had to make the decision whether to allow an additional language into a large codebase, and there are some major negatives to consider that are unrelated to the technical aspects of the language. The maintainer said three times in the above excerpts that he doesn’t have a negative opinion of Rust itself.
I’ve always heard that Google allows only four languages in their codebase, and one of those is only because they invented it. :)
Yes, I was part of the first Rust team at Meta so I’m quite aware of the upsides and downsides! That’s why I said “categorical”.
By “categorical”, you mean saying “no second language could ever have upsides that overcome the downsides of having two languages”? I agree that’s a pretty conservative position. It’s hard to even imagine another candidate for second language, though (I dunno, Zig maybe?), so I didn’t take it as literally categorical.
This is the quote:
This is quite categorical!
I think Zig sees itself as a candidate second language for C projects, but that’s probably beside the point; nobody’s trying to bring it into the kernel.
Indeed, but that call was already decided when Rust support was first merged.
The goal of the experiment is to explore those tradeoffs. You can’t do that if you don’t merge things!
And yet they are one of the companies pushing Rust for Linux… They understand the tradeoff. The rust for linux people understand the tradeoff. They just think it is worth it.
Totally. But that does seem like a valid engineering question people could disagree about.
However, this is exactly the kind of huge impactful question that should get escalated to the chief architect. And if there’s a sanctioned scouting effort to gather experience to figure out the right answer, somebody found to be actively subverting that effort should be…corrected…by the chief architect.
It’s interesting that they won’t allow it in their internal codebase but they’re happy to let the Linux maintainers take on the burden of adding it there.
What’s your source for Google not allowing Rust in their internal code base? Google publishes crate audits for google3: https://github.com/google/rust-crate-audits/blob/main/manual-sources/google3-audits.toml
I’m not sure where you get that from - I worked at Google and literally all the code I wrote was in rust
In the google3 main repo?
Is the Linux kernel a place for curiosity or for consistency?
R4L is officially an “experiment” with goal to explore tradeoffs. I.E. satisfy curiosity.
This whole debate is so tired. The “including Rust” ship has, at least temporarily, sailed, and maintainers blocking it based on not wanting Rust/a second language doesn’t make sense when the whole point is to explore that option.
Maybe all the code will get deleted at some point, but the code needs to be merged first to then decide that based on how well it works in practice.
Why does it need to be merged though? If it’s officially an experiment, maintain it in a separate R4L tree, and everybody’s happy?
To have a thorough experiment, it must be merged into the official tree. One must be able to see how this interacts with a diverse set of hardware and software configurations, and also how this affects the development process inside the kernel community. Some maintainers are afraid of how the introduction of Rust would affect them in the future and the maintainability of the code base. Without it being in the official tree, there wouldn’t be any conclusion on those points.
It sounds like some of the kernel maintainers at least don’t want to have to pay the cost of this large-scale experiment? Like they don’t feel it’s worth the downsides of massively complicating the kernel into a multi-language codebase? Like they understand the potential benefits of memory safety but they feel they are outweighed by the risks?
These individuals believe that they know the result of the experimental before it’s been run. But they do not, because they cannot, by definition. Of course they don’t want to pay the cost, because change is hard, and deep down they know that there’s a chance that the experiment is successful, and thus their responsibilities will either change or they will have to cede responsibility to someone more capable.
Whether there is benefit to Rust in theory, there is little question. It provably solves problems that the kernel has. Whether it can solve those problems without causing larger ones is why the experiment is required. Blocking the experiment for essentially any reason is more or less indefenisble, because the entire point is that they don’t know.
That sounds way too confident and assured of a statement to me. You are discounting the opinion of subject matter experts as if they have no idea what they’re talking about. If someone came to you at your workplace and said that they want to run an experiment to convert your codebase to a new language, and they want you to help maintain and support that experiment while also delivering on all your existing obligations, would you be so quick to accept that? I don’t know about you, but personally I would and have pushed back on rewrite suggestions as a bad idea, despite the perceived benefits (for example, Management thinks that rewriting in JavaScript will make it easier to hire people for the project in the future).
Would rewriting in JavaScript have possibly made me redundant? Maybe. But it would also be massively expensive, cause huge churn, and have a real possibility of failing as a project. We can’t just ignore the very real possibility of risks because of the expected benefits.
There is no interpretation in which it’s too assured. My position is that the result of the experiment is unknown, and that the potential benefits are known, both of which are hard facts.
I have run such experiments in my workplace before. Some have succeeded and some have failed. The successes have massively outweighed the impact of the failures.
Assuming that you know the outcome before you begin is the pinnacle of hubris, which we see time and again with these entrenched maintainers. They may be domain experts, but they deserve every ounce of criticism that they receive. Closed mindedness is not how progress is achieved.
My point is that there must be some threshold that you use to decide whether some experiment is going to be useful to run or not even worth the effort because your experience and expertise tells you otherwise? Or would you accept every experiment that anyone proposed? Eg, suppose someone wanted to model your entire technical stack with TLA+ for high assurance purposes? On paper it sounds like a great idea–formally verify your concurrency properties–but you don’t see how a reasonable project lead might say ‘While this project could certainly bring great benefits, it’s not worth the tradeoff with our team size and capabilities right now. That may change in the future’?
Some threshold must exist, yes. Presumably, that threshold should be somewhere below “the progenitor of the system has decide that the experiment should be run”, which is currently the case of RFL.
Individual system maintainers should not and must not be able to usurp the direction of the project as a whole.
Your core misunderstanding is that we are coming from a tabula rasa. We are not, and if your position is that Linus’s previous endorsements shouldn’t stand, then we’re having an entirely different (and much less tenable) conversation.
Is it though? To my understanding the decision was only that they will try it and see if it works on both the technical and the social levels. There was never any guarantee given that Rust will be shipped in Linux at all costs. To my understanding Linus’s approach is to see if Rust is overall viable for the kernel, and that includes the opinions of the other subsystem maintainers. Because if they are overall against it, the experiment is dead in the water.
where do the interests of actual linux users factor into this?
I like it when drivers work correctly and don’t break every other time I update the kernel due to weird concurrency issues.
then you should update your kernel twice as often, then it will be every fourth time instead of every other time.
Again, that decision has already been made. There’s no point questioning it over and over. Doing so is distracting from the debates that are important now: how can leadership do better at enforcing these project wide decisions to not let the situation fester like R4L, and reviewing the Rust code without raising the same solved questions again and again (concern trolling).
If you’re genuinely curious about those questions, there’s plenty of text written about it. LWN is a good place to start.
I hear you, but it is a bit in the nature of project that are very distributed in decision making with high individual authority that indeed, decisions will be questioned again and again and again at every point. Those projects need a long breath and the willingness to talk about points again and again and again.
It comes with its strength, particularly, those projects are extremely resilient and fast in other regards. You can’t have both. Pushing top-down decisions in such an environment can have other unintended effects.
It’s not something to take up (period, but even more so) with the R4L devs.
Doing that is like berating a cashier for the price of an item.
It sounds like the decision has actually not been made, just dictated by ‘top management’ and now getting pushback from the rank-and-file?
The guy who blocked it is ‘top management’, inasmuch as that exists for Linux.
sorry, the guy who blocked what? has the decision been made?
The guy who blocked the patches that the Asahi Linux folks wanted, which I think were to allow writing some rust drivers.
The guy who blocked has the authority to reject patches in their subsystem.
It wasn’t a patch in their subsystem though, it was in
rust/and used the DMA subsystem.You’re right that it’s in
rust/, tho unclear to me if it’s properly in the rust subsystem or not. You and others may be right that this patch may still go thru, that seems to depend on Greg KH and Linus.but did they have the authority to reject the patch?
I don’t believe so.
so he has the authority to block it, but not to reject it? him being the guy referred to above as “the guy who blocked the patches that the Asahi Linux folks wanted.”
So the original statement, “that decision has already been made,” is supposed to mean that the decision not to allow that patch has already been made?
ThinkChaos was talking about the decision to do the Rust for Linux (R4L) experiment.
I was talking about the decision to block the patch.
Sorry for the confusion.
how could the decision have been made, if the guy who blocked the patches is ‘top management’, inasmuch as that exists for Linux?
how was the decision made?
I sure hope it’s for both! If we cannot find a balance between the two, then we have failed as engineers.
Personally I think it does matter when a core maintainer calls a project that’s part of the kernel “cancer” and the guy who’s in charge doesn’t seem to care.
Marcan got a lot of mileage out of theatrically misinterpreting the “cancer” comment, and you’re carrying it here.
The actual comment:
The cancer is not Marcan, Asahi, R4L, or Rust. The cancer is anything-but-C. The cancer is a two-language codebase.
Still a very discouraging comment from a maintainer, and it bucks Linus’ decision that trying out Rust in the kernel is OK, but the distortion of this comment has been bugging me for the last week.
R4L is a project to make Linux a two-language code base. So he quite explicitly called the goal of the project cancer.
Yep, that’s what the commenter above is referring to. He thinks that project goal is unworthy. Or, well, in his words, cancer.
I don’t have an opinion, but I think a fair and less charged way than “cancer” of stating the position is just this: using Rust would have positive benefits, but they are far outweighed by the negatives of allowing a second language into the kernel.
The pros and cons of RfL are somewhat subjective, and the funal balance is hotly debated.
But I find it quite telling that Hellwig’s core argument (that a multi-lamguage codebase requires more work) is held by people who didn’t try doing any of that work. Whereas the kernel devs who started using Rust are explicitly saying “the API compat churn isn’t that big a deal, we can do that work for you if you want”.
We mostly hear about the drama, but it seems that the overall feeling toward Rust among kernel devs (not just RfL devs) is generally positive.
but then you lose the meaning conveyed by the metaphor, of something that grows beyond its intended scope until it kills the organism.
I don’t think the metaphor conveys any technical meaning beyond what I said. I don’t know what “kills the organism” is supposed to relate to. Is the “organism” the kernel? The community of kernel developers? And what does “kill” equate to in reality? It would be better to have the discussion in concrete terms rather than metaphors.
It does convey the personal outrage of the maintainer better, certainly.
thread depth limit reached; continuing here.
yes, I’m ignoring the less absurd things you’ve said because I don’t know what to take seriously when you haven’t retracted the most absurd thing.
one characteristic of cancerous growth is that it happens whether you want it to or not.
it doesn’t imply that, but I do think he would be much more OK with a little bit of Rust if the scope were somehow guaranteed not to expand. like I don’t know, maybe there’s a carve-out for testing a particular class of rust drivers in linux to help find bugs, but eventually the drivers are ported to C for use in linux and the rust version is used in a separate kernel written from scratch in rust?
so you wouldn’t say that you want to have no cancer at all in your body, ever? if it’s a non-cancerous tumor, I have much less of a problem with it being in my body.
I agree that “kill” could have multiple meanings.
I think “growing beyond its intended scope” is technical meaning that is not included in “the benefits outweigh the negatives.” you disagree?
Cancer has no intended scope, right? “Allowing into the kernel” conveys that there shouldn’t be any in the kernel.
sure. so do you think “growing” is a technically meaningful word that is not implied in “the benefits outweigh the negatives”?
I don’t want to dig into the minutiae of why this metaphor is inaccurate — my point is that using this metaphor at all is unprofessional and counterproductive because of the heavy emotional baggage it comes with. Just say “the introduction of Rust anywhere in the codebase will inevitably result in a growing amount of Rust in the codebase”.
It’s especially odd in this context, given the history of this metaphor regarding Linux.
okay, so do you retract your previous statement that “I don’t think the metaphor conveys any technical meaning beyond what I said”?
my intention was not to get into minutia either, but we’ve gone back and forth three times and you still haven’t retracted that statement so I don’t know where you stand.
so our enemies allow themselves to use the metaphor to efficiently communicate with each other, but we have to forswear it.
If you don’t want it there at all, it’s redundant to also complain that it’s growing. Cancer also has many other attributes besides “growing” that are irrelevant or contradictory to the position taken.
The reason he used that particular metaphor was to express disgust and horror, not to be technically accurate. If he just wanted to evoke “growth” he could have said it was like ivy. Disgust and horror are not an appropriate device to call on in this context, IMO.
Re Mr. Ballmer, the enemies were wrong that time, so don’t you think evoking that argument by using the same metaphor muddies the rhetorical water a bit?
if I can’t tell whether you stand by your statements, it feels futile to interpret them as if you believe them. that’s my issue with continuing in light of the fact that you are just leaving “I don’t think the metaphor conveys any technical meaning beyond what I said” without clarifying whether you believe it.
“complaining that it’s growing” is completely different from identifying the growth as a factor in the technical drawbacks.
Cancer has a multitude of characteristics, of which growth is only one. If anything, it has many characteristics contradictory to the point being made: it’s an undesirable outcome of natural processes, it is typically self-generated from within rather than deliberately introduced, it is hard to eliminate because it’s nourished by the same system that sustains healthy cells…
If you have to tell me exactly which part of the metaphor I’m supposed to pay attention to in order to get the technical meaning, then you aren’t conveying technical meaning, you’re muddying the water. Just say what you mean.
The additional meaning being conveyed here is not technical, it’s emotional, and I do agree emotional meaning was conveyed.
I take this to mean that you still stand by the statement that “I don’t think the metaphor conveys any technical meaning beyond what I said.” if you treat all technical metaphors like this, I’m glad you’re not my coworker.
or it could mean I am conveying technical meaning but you’re being willfully obtuse. just maybe…
Well, notice that you aren’t absorbing my actual point too well, because you’re very focused on making your own.
To give him the benefit of the doubt, he could have been trying to convey the technical meaning of “growth” in the sense of “if you let Rust in, we’ll just have more Rust”. However, that’s an utterly vacuous thing to say, because obviously the whole point of the exercise was to let Rust in and see what happens, and why would you do that unless you want to have more of it if the experiment succeeds? It conveys no useful information beyond “I don’t think Rust is a good idea at all, for reasons”; in other words, “the benefits don’t outweigh the negatives”. In fact it confuses matters because it implies he thinks Rust would be OK if only it didn’t grow.
I’m actually trying to be polite to the guy here, because I truly believe it was an emotional outburst, not an attempt at a technical argument. His actual technical argument (stated much more clearly elsewhere) is that there should be no Rust at all in the kernel, ever, not that once there is some, there will be more. So if “cancer” was supposed to mean “growth” as a technical argument, he misstated his own argument.
I don’t see the misinterpretation, lonjil said
Not
Or whatever phrase you would need to make your objection stand.
I could be wrong, but I get the feeling that people didn’t quite get why Hellwig used the word “cancer”. I could be wrong about this in a lot of ways, maybe I too misunderstood what Hellwid meant, maybe I misjudged how others are interpreting what he said. But allow me to add my 2 cents.
People seem to be interpreting what Hellwig said (i.e. calling R4L “cancer”) as him saying R4L is bad, evil, a terrible thing, etc. And I totally understand why many would think that, and would agree this is a terrible choice of words. But I think that Hellwig’s focus is actually on the fact that cancer spreads. A better word to use would probably be like “infectious”, or “viral”. Although I disagree with him, I think what he was saying is that, despite what Rust developers promised, Rust will spread to more corners of the kernel and increase the maintenance burden and will have a negative impact on the kernel as a whole.
I think technical leaders should focus on clarity rather than everyone else trying to do exegesis on what they write.
That’s very fair, I was not trying to defend Hellwig’s communication skills in the least here.
I think you’re right, but that doesn’t exculpate Hellwig in the least. One of the defining characteristics of cancer is that it is malignant. It seems pretty clear that he used this metaphor because he thought that Rust would spread and that it is dangerous and harmful.
yes, which is a perfectly valid technical position and to disallow a maintainer from expressing that position is completely insane.
“infectious” or “viral” have different connotations. viral would be if they thought the BSDs might catch it. infectious is not as specific as cancer; cancer specifically grows from one area and consumes more and more of the things next to it. infections can do that but not necessarily.
actively expelling the cancer probably would have drawn even more ire.
Unrelated to the rust issue, my understanding is that such toxic language actually isn’t all that strange in the Linux kernel maintainer community. It feels like they try somewhat not to overdo it, but they aren’t that much concerned about the toxicity. More like, “I’ll try but I don’t actually care”.
The only reason they’ve toned it down is due to financial pressure from major funders of LF.
The fact that many of those LF funders are also the employers of major Linux maintainers is also relevant, I feel.
I think it’s kind of crazy to label a vivid and extremely useful metaphor as “toxic.”
It’s useful in some cases, but when the effect of it is raising the temperature of the discussion, then I would say it isn’t useful in that instance. There are other ways to convey the same meaning that wouldn’t have attracted the same amount of strong emotions. We’ve seen examples of such alternate arguments in this thread that would have been more suitable.
sure. but then don’t fixate on the word, and acknowledge that their choice to use strong language can also be an effect of the raising temperature of the discussion, for which responsibility is shared among all participants.
There’s this idea that multi-language codebases are categorically a disaster, right? What really bothers me is how small-minded it is. There are many C and C++ codebases that have introduced Rust. It’s not easy but it is possible, and it often is worth it. Some attempts have been successful and others haven’t. You can learn so much from other people’s experiences here!
Simply calling multi-language codebases a cancer closes off discussion rather then opening it. That goes against the basic curiosity that is a hallmark of engineering excellence.
Honestly everything around R4L that has been going has just cemented in me the belief that the average long time Linux maintainer is fundamentally incompetent. They’re reasonably good at One Thing, but totally incurious and scared of anything outside of that One Thing they’re good at.
Do you suppose there might be some kind of incentive structure behind this pattern?
That seems likely, though I don’t think I can usefully speculate on the details.
Is it possible that we could take an experienced kernel maintainer’s opinion seriously instead of dismissing it immediately as incurious and ‘against’ engineering excellence? If a co-worker who specializes in a different domain came to you and kept insisting that something you know is very difficult and not worth the tradeoff is actually quite easy, would you be incurious and ‘against’ engineering excellence if you dismissed their wild suggestions without a deep discussion of the tradeoffs?
Evaluating who to take seriously and who not to is a matter of judgment cultivated over a lifetime.
Depends on many things, but in general, yes, of course, especially if they were someone I took seriously on other matters. If nothing else, it’s a chance to sharpen my arguments. If I keep having the same arguments over and over I might even step back and write an essay or two explaining what I believe and why I believe it. Here’s one that’s public that was a result of the sort of back-and-forth you’re talking about.
Your post strayed away from a factual re-telling of facts and towards narrativization here.
Reading the discussions on this elsewhere, there’s no way to describe this that wasn’t going to get accused of bias by people holding strong opinions on either marcan or on the rust for linux project.
I’m not accusing of bias (which is more or less unavoidable), but of constructing a narrative. The facts are: he exchanged words on the LKML and then wrote Mastodon posts about the exchange. The narrative you’re constructing is: he exchanged words on LKML, then because he was unsatisfied with how that discussion was going, he wrote Mastodon posts about the exchange in an attempt to gather support.
I think it would be better to leave out the guesswork about intentions, or at least clearly separate it from the factual retelling of events.
marcan has posted about this, so it’s hardly guesswork: https://lore.kernel.org/rust-for-linux/208e1fc3-cfc3-4a26-98c3-a48ab35bb9db@marcan.st/
Correction: Marcan never “exchanged words” with Christoph Hellwig on the patches, which had not even been posted by him. He just replied to the thread with a tirade.
Isn’t that still a lot for frontend?
I think modern SPA frameworks have been getting a lot better at lazy loading things, right?
go-applooks sick though. It’s gona be on my to-try list now.It’s definitely a lot to download, but comparing WASM and JS bundle sizes isn’t entirely apples-to-apples. Byte for byte, JS is one of the slower assets you can put on your page due to its relatively complex parsing requirements.
I’d be interested to know where the balance point is between parsing speed and download speed.
the tricky thing is that the balancing point would depend on the connection speed of your user. some people still have really slow 3G as their primary internet connection.
And some may still prefer less data usage to a minor speed up
But a JS bundle can be cached, no? And I’d be really surprised if browsers aren’t JIT-compiling the cached JS internally into some kind of executable bytecode representation.
Spoke a bit too soon. It seems like https://go-app.dev/reference crashed for me on iOS + Safari. One big downside for using wasm.
Yeah, it’s a bit chunky for sure but not a dealbreaker. It’s deployed as a PWA, which helps; it’s heavily cached until the user clicks an “update” button.