I’ve said it before and I’ll say it again: as a child of OCaml and C++, Rust currently is the best language for production compiler-shaped things.
Though it is curious that in this case it seems that the major motivation wasn’t any inherent compiler-friendly language properties (combination of ADTs and unboxed data structures), but rather more mundane software engineering considerations, in particular, the ease of gluing together a parser generator, a JavaScript runtime, an HTTP client, and a work-stealing threadpool.
Wasn’t it simply due to tree-sitter not having a Java library implementation with a significant uptake/user?
Don’t get me wrong, rust is a pretty good language for several niches and is a breath of fresh air in the low-level domain, I just don’t think that it can glue together some other combination of features better than many other languages, say Java (which would be pretty good with even this combination besides this single library that was developed in rust).
Wasn’t it simply due to tree-sitter not having a Java library implementation with a significant uptake/user?
I think this is what I am saying? That combining libraries is just easier in Rust typically.
I just don’t think that it can glue together some other combination of features better than many other languages
I absolutely do think this though! I like to say that Rust delivers on Scala’s promise to be the language for library authors. There are many specific things that make Rust better at this
Build system. Comparing with Java, gradle is hugely complex. gradle book is longer than rust book! I found it much easier to learn Rust than to learn gradle.
Package manager: Rust is very explicit about semver and rules allowing the collection of interdependent libraries to evolve together. My understanding is that most over ecosystems didn’t quite caught up yet. That’s the super trick of Rust : it uses a restrictive build system and a restrictive versioning scheme which severely limit what you can express, but the flip side of this is automated large scale composition of software.
No single global namespace. Traditionally, all libraries just add themselves to some sort of global search path, which hard-breaks if there’s a naming conflict, and there will be a naming conflict when you end up depending on two versions of the same library. Rust is very meticulous about not creating any kind of a shared namespace. Names are a property of an edge in the dependency graph, the same node might have different names written along its different edges.
Tying to semver, this give a nice property that 1.2.0 and 1.3.0 are unified, while 1.3.0 and 2.0.0 are duplicated, given the library authors flexible API evolution strategy.
These days, C++ and Java also try to get something like this through their modules, but that’s a recent development.
Traits allow for a flexible composition, because the trait and its impl need not to be developed by the same team. That’s quiet a bit more flexible than OOP (but quiet a bit less flexible than full blown ML-style modules)
Type system sometimes helps with composition. I think there’s only one big notable example, but this example is huge: Rust tracks thread safety and would double check that you are not using single-threaded data structures concurrently. You basically can through rayon at your problem, and rustc would point out the places where you need the mutex
Rust has an option to be low-level, so there’s a higher ROI for libraries. Tree Sitter runtime is C, and not Java, because you can put C anywhere. That’s the same deal with Rust. This is outright cheating! Before Rust, a library author could pick Java, and eat slowness, or they could pick C++, and restrict the usability of their library only to the C++ experts, or eat slowness and pain on bridging C++ and some high level level language. These days, you can safely pick Rust and use that to pull all of the C++ tricks, but the end result is still directly usable by high-level programmers. Liberally allocating Rust and GC Java are not that far from each other, all things considered.
Well, this is the more fundamental problem of specialist vs generalist. Of course a specialist (cargo) can be better optimized, be more ergonomic, while being simpler, but it comes with a caveat - it can only handle single-language projects. Many projects grow to incorporate multiple languages, code generators, etc, and while I’m sure you can put some escape hatch into cargo to make it possible, you then lose all the advantages a build system natively brings (parallelism and caching). Gradle is definitely not liked by many, but it is a generalist build tool, and its ability to make android builds possible speaks for it.
Namespacing
Which language are you thinking of? Java (well before its module system) is well-known for its reverse domain namespacing at every level, which actually helped in preventing certain kinds of supply chain attacks, which npm and even cargo are susceptible to.
I mostly agree with your other points, and as I mentioned, I really like this language. What I was primarily pointing at in my comment is the size of its economy - which is growing at a very healthy rate, but it is not yet anywhere close to the vastness of Java’s.
One additional note for the ‘no data races due to the type system’ point - this is one thing which is not as big of a problem when you have a higher level language, like java. Data races there are absolutely safe and well-defined, the execution of the program won’t go down a UB path as it does in case of low-level languages like Rust.
Well, this is the more fundamental problem of specialist vs generalist.
Yes! This is basically one ring problem: https://www.tedinski.com/2018/01/30/the-one-ring-problem-abstraction-and-power.html. Staying on Pareto frontier, the more restricted your build system is, the easier is to compose libraries. Rust made a correct in the current context decision that a composable library ecosystem is more important than convenience of a build for any particular project. If cargo doesn’t cut it for you for your particular application project, use buck2! But when you upload your library code to crates.io to be reused by others, it has to be buildable by Cargo, so that it can be easily re-used by others.
well-known for its reverse domain namespacing at every level
Pre-modules, all these reverse domain names lived in the same global namespace. You couldn’t have two differentcom.mycompany.Foo classes on the class path (outside of classloader shenanigans). But, in large projects composed of many small libraries, you’ll need two have two different Foos, because at some point you’ll have the same library in two different versions,
I agree that cargo made the correct choice here, especially that it indirectly biases people towards pure rust libraries, which is an under appreciated property of an ecosystem. [1] I haven’t heard of buck2 before, will check it out!
Ah, this is what you meant. If we are being pedantic, in the JVM’s case a class loader - class with canonical name tuple is the unique basis, so you can actually load two versions of the same package with some class loader magic (this is how application servers, some hot reload work). Of course this is not exposed in the Java language itself natively.
[1] Java happened to achieve this through different “means” (by java’s ffi not being too ergonomic, plus it being faster than interpreted managed languages, so not needing it all that much), and now can profit off of it in case of, say, virtual threads.
in the JVM’s case a class loader - class with canonical name tuple is the unique basis, so you can actually load two versions of the same package with some class loader magic (this is how application servers, some hot reload work).
Yep, worked with this in Kafka Connect’s plugin loading - each plugin has its own classloader to separate out what could be incompatible versions of the same class.
I’ve said it before and I’ll say it again: as a child of OCaml and C++, Rust currently is the best language for production compiler-shaped things.
Though it is curious that in this case it seems that the major motivation wasn’t any inherent compiler-friendly language properties (combination of ADTs and unboxed data structures), but rather more mundane software engineering considerations, in particular, the ease of gluing together a parser generator, a JavaScript runtime, an HTTP client, and a work-stealing threadpool.
Wasn’t it simply due to tree-sitter not having a Java library implementation with a significant uptake/user?
Don’t get me wrong, rust is a pretty good language for several niches and is a breath of fresh air in the low-level domain, I just don’t think that it can glue together some other combination of features better than many other languages, say Java (which would be pretty good with even this combination besides this single library that was developed in rust).
I think this is what I am saying? That combining libraries is just easier in Rust typically.
I absolutely do think this though! I like to say that Rust delivers on Scala’s promise to be the language for library authors. There are many specific things that make Rust better at this
Well, this is the more fundamental problem of specialist vs generalist. Of course a specialist (cargo) can be better optimized, be more ergonomic, while being simpler, but it comes with a caveat - it can only handle single-language projects. Many projects grow to incorporate multiple languages, code generators, etc, and while I’m sure you can put some escape hatch into cargo to make it possible, you then lose all the advantages a build system natively brings (parallelism and caching). Gradle is definitely not liked by many, but it is a generalist build tool, and its ability to make android builds possible speaks for it.
Which language are you thinking of? Java (well before its module system) is well-known for its reverse domain namespacing at every level, which actually helped in preventing certain kinds of supply chain attacks, which npm and even cargo are susceptible to.
I mostly agree with your other points, and as I mentioned, I really like this language. What I was primarily pointing at in my comment is the size of its economy - which is growing at a very healthy rate, but it is not yet anywhere close to the vastness of Java’s.
One additional note for the ‘no data races due to the type system’ point - this is one thing which is not as big of a problem when you have a higher level language, like java. Data races there are absolutely safe and well-defined, the execution of the program won’t go down a UB path as it does in case of low-level languages like Rust.
Yes! This is basically one ring problem: https://www.tedinski.com/2018/01/30/the-one-ring-problem-abstraction-and-power.html. Staying on Pareto frontier, the more restricted your build system is, the easier is to compose libraries. Rust made a correct in the current context decision that a composable library ecosystem is more important than convenience of a build for any particular project. If cargo doesn’t cut it for you for your particular application project, use buck2! But when you upload your library code to crates.io to be reused by others, it has to be buildable by Cargo, so that it can be easily re-used by others.
Pre-modules, all these reverse domain names lived in the same global namespace. You couldn’t have two different
com.mycompany.Fooclasses on the class path (outside of classloader shenanigans). But, in large projects composed of many small libraries, you’ll need two have two different Foos, because at some point you’ll have the same library in two different versions,I agree that cargo made the correct choice here, especially that it indirectly biases people towards pure rust libraries, which is an under appreciated property of an ecosystem. [1] I haven’t heard of buck2 before, will check it out!
Ah, this is what you meant. If we are being pedantic, in the JVM’s case a class loader - class with canonical name tuple is the unique basis, so you can actually load two versions of the same package with some class loader magic (this is how application servers, some hot reload work). Of course this is not exposed in the Java language itself natively.
[1] Java happened to achieve this through different “means” (by java’s ffi not being too ergonomic, plus it being faster than interpreted managed languages, so not needing it all that much), and now can profit off of it in case of, say, virtual threads.
Yep, worked with this in Kafka Connect’s plugin loading - each plugin has its own classloader to separate out what could be incompatible versions of the same class.
You don’t have to use Gradle. I vastly prefer Maven still.