On the other side, the std lib itself is woefully lacking: no RNG, cryptography or serialization.
You’d prefer we either be frozen with the 0.1 version of the RNG API and rustc_serialize instead of Serde or have a Python/Java-esque situation where everyone tells you to ignore the standard library implementation and use a de facto-standard successor instead. (eg. Requests for Python and its internal never-will-be-in-stdlib urllib3, with basically any other networking code in the standard library being “use Twisted instead” for ages.)
Rust’s development philosophy was informed by the saying in the Python world that the standard library is where modules go to die, because the API stability requirements strangle further development. That’s why crates like rand are effectively “pieces of the standard library distributed through cargo”… because distributing through Cargo allows you to use older libraries with newer toolchains and vice versa.
Every time I see this argument, it comes across as people seeming to believe that you can force a simple solution to a complex problem.
Even some things that should have been language features since day one - like async traits and yield generators - are supplied as 3rd party macros.
Rust follows a “release a minimum viable product and extend it later” model and they already regret the things they were over-eager to implement and have to support in deprecated form in perpetuity, like the description and cause methods on the Error trait.
Discussion is ongoing on ways to improve this. For example, one idea is keyword generics, which could allow functions to be generic over whether they’re being called in an async context. (The idea of abstracting over & vs. &mut hasn’t seen as much progress because it tends to get into the weeds over whether it’s actually a good idea, given the underlying semantic difference between them and how likely it is to actually need different code to achieve correctness.)
However Rust also decided to turn Box/Arc into fat pointers (similar to Go, and opposite to Java/.NET).
Your first two points sound like “I believe the “dyn Trait wouldn’t be a fat pointer”-ness of C++-style classes with internal vtables has benefits so valuable to things like C FFI and lock-free compare-and-swap that they should be offered as an option in the language itself”, which I can see reasonable discussion around. However, your third point is talking about generics being limited to Sized things… which is hard to argue as anything other than faulting Rust for not universally using C++-style classical inheritance for polymorphism since, if it’s anything but the only option, you’re back to some things satisfying the constraint and some not.
Rust provides a workaround in form of dedicated crates that offer thin dynamic pointers as well, but since they are not part of standard lib, it’s unlikely that you’ll be able to use them across different libraries in the ecosystem without extra work.
The general answer for situations like this is that, as with what happened with lazy_static → once_cell → std::sync::Once*, the Rust upstream are waiting to see if anything emerges in the ecosystem which is used ubiquitously enough to justify bringing it in.
(Rust has a philosophy that, if it can be prototyped, iterated on, and prove its desirability as a macro first, it should. Heck, ? began as try! even though it was part of the standard library in both forms.)
For example, delegate is used in two to three orders of magnitude fewer crates than once_cell and lazy_static, depending on how much overlap there is between them, and there’s the added barrier to getting implementation inheritance in the standard library that there’s no single way to do it that’s unarguably best. Each one has trade-offs and none has been excessively more desirable than the others so far.
I get why it’s there, but forcing it blindly and everywhere as a default behaviour is fucking bullshit: which apparently is acknowledged by the authors themselves, since the common way of getting immutable/mutable reference from an array is to split it into two objects using method that operates using unsafe pointers under the hood. Shutout to all haters saying that unsafe Rust is not idiomatic: it’s not only idiomatic, it’s necessary.
Honestly, this section’s take on the borrow checker feels not unlike calling math bullshit because Gödel’s incompleteness theorems exist.
I’m reminded Monty from xiph.org explaining that Gibbs Effect ripples are “just part of what a band-limited signal is”.
Even if Rust didn’t have to deal with being suitable for niches where you can’t bring your own tracing garbage collector, those Share-XOR-Mutate semantics are necessary to make ensuring correctness of multi-threaded code at compile time tractable.
Another thing about borrow checker is that it has very shallow understanding of your code. It also explicitly makes a conservative assumption that if you call method over some reference, this method will try to access ALL fields of that references, forcing any other field accessed outside of it to be invalidated.
Partial borrows are another topic of ongoing discussion. (Basically, given limited developer bandwidth, they’ve been put low on the priority list compared to things like the work to get async as close to the level of ergonomics as sync enjoys as possible.)
In fact, one of the things that’s been prioritized higher than partial borrows is the Polonius project to rewrite the borrow checker to finally be able to address Non-Lexical Lifetimes Problem Case #3. (i.e. making the borrow checker smarter)
Performance ceiling vs time to performance
Fair… though, for me, what matters is that Rust is much more likely to throw up a compiler error if I blindly try to do something which would have me fall off the fast path, while GCed languages will just happily pessimize performance to avoid distracting me from thinking about my business logic.
I’ve always seen Rust as less about performance and more about predictability.
…though…
Time to performance which basically means: “how long it takes to solve a problem with acceptable performance”. And personally: Rust falls behind many managed languages on that metric, mainly because of things like borrow checker, and multi-threading issues etc. which I cover later.
Given that I see Rust in the context of multi-threading to be more about correctness, I’m reminded of this quote:
“Why are you in such a hurry for your wrong answers anyway?” – Attributed to Edgser Dijkstra
So once you finally get your PhD from lock algorithms in Rust, you finally are at the level where you can do the job as efficiently as Go dev after learning the whole language in 2 hours.
You are aware, I hope, that Go isn’t memory-safe in the presence of data races. IIRC, it has to do with passing around certain kinds of multi-word values without synchronization in ways that can invoke undefined behaviour.
Panics
I fully agree. The lack of a maintained tool in the vein of Rustig or findpanics is embarrassing and, along with LLVM not providing the information necessary to turn “this optimization used to get applied and now it doesn’t” into a compile-time error, is one of the biggest things I dislike about Rust.
Async traits are in the process of being implemented.
There’s one thing about async traits that I’m of two minds of: I think that we could have stabilized async fn in traits by de sugaring to Box<dyn Future> ages ago, with the object safety restrictions and (tiny) perf hit that comes from that, to provide value to people earlier while we still worked on impl trait in trait, but fear that if we had done that the pressure to ship impl trait in trait would have deflated and that project would never have seen the light of day, causing the “imperfect and temporary” solution to stick around.
Generators exist in a “not yet safe for unconstrained use” form as the underlayer of async/await and are in the process of being implemented more generally.
The issue with generators where that there are 3 related things (in order of blocking stabilization more to less):
the traits involved, including Stream/AsyncIterator that have been under discussion for years
the decision on how desugaring of async generators should be (and how to deal with pinning rules specifically, if you have to always pin the whole stream manually which is more flexible but more annoying or not which is easier to start doing but more restrictive)
design around whether to provide special syntax to iterate on streams (think for await f in stream). You don’t need this because you can use while let always, adding await to patterns is a can of worms (it can be made to work, also amusingly you could make it so let await x = y; and let x = y.await; were equivalent, but what you also want is some way of selecting parallel batch size, like rayon’s par_iter as well)
Fair… though, for me, what matters is that Rust is much more likely to throw up a compiler error if I blindly try to do something which would have me fall off the fast path, while GCed languages will just happily pessimize performance to avoid distracting me from thinking about my business logic.
I would like it if we could supplement this gap more with Rudy’s lint’s: make the easy code fast, make the code do what the user expects (“I’m returning a boxed a trait here, so method access are through a v-table”) but detect the cases where an optimizing compiler would switch strategies and tell the user how to write it differently (“this function is used in a hot loop, and only ever returns one of two types, use impl Trait instead”). This keeps the developer in control, choices are explicit, and mild changes in the code don’t have sudden hidden performance cliffs because of changes from the assumption. Basically, every rule where Swift would change its code gen strategy should be a lint.
Yes, but changing the underlying mechanism to impl Trait was a backwards compatible change. I called out that the trade off would have been potentially delaying benefits for a few but enable a lot of people earlier.
Yes, but changing the underlying mechanism to impl Trait was a backwards compatible change.
I don’t see how that could be true? If the feature is implemented by desugaring to Box then trying to remove that desugaring is also a backwards-incompatible change. For example, assuming an implementation similar to https://github.com/dtolnay/async-trait, you might have:
let foo = some_async_trait_obj.some_method();
let foo_ptr = Box::leak(foo.into_inner());
Hiding memory allocations in normal-looking code also goes completely against the Rust use case, how are you going to verify that a particular code path has no allocations in it if any trait object’s method call might potentially be a hidden Box::new() ?
That’s why crates like rand are effectively “pieces of the standard library distributed through cargo”…
Interesting you use rand as an example, because it seems like the opposite situation in play – the crates.io rand library isn’t maintained by the Rust developers, it’s just under some random GitHub org at https://github.com/rust-random/rand.
There are some crates.io packages that are like extended parts of the stdlib (flate2, git2, getopts) but they’re not necessarily the most popular solution in their space. It’s also difficult to discover which packages are under rust-lang/ and which aren’t without digging through the crates.io index metadata.
In contrast, Go has a larger built-in stdlib than Rust, but also has an extended stdlib distributed as regular packages. golang.org/x/crypto, golang.org/x/text, and golang.org/x/net/http2 are all maintained by the Go developers – if you want a SHA2-256 checksum then the stdlib has you covered, the more esoteric BLAKE2b is in golang.org/x/crypto.
In Rust if you use the crates.io sha256 package then you’re depending on https://github.com/baoyachi/sha256-rs, which as of the most recent version (v1.6.0, released 3 days ago) doesn’t even have no_std support.
I certainly agree that we need better tooling to indicate which things are officially “external pieces of the standard library”. However, by Python 2.7, half of Python’s standard library was under de facto deprecation in favour of libraries with no connection to the stdlib dev team, with the identity of those packages being communicated through tribal knowledge.
Putting stuff in Rust’s actual standard library wouldn’t prevent the “Batteries Included, But They’re Leaking” effect and there’s only so much that “this is officially maintained” can do to put a thumb on the “who’s competing more effectively?” scale.
Again, this comes back to “Go’s standard library is what it is because Google is paying for the manpower, not because declaring things std magically summons more contributors”.
Having seen the state Python was in, the philosophy behind Rust’s standard library is that it’s for two kinds of things:
Interface types that need to be in core or std for language constructs to depend on. (Which is why Result and Future are in std for ? and async to depend on, but the http crate is the de facto standard source of HTTP request and response interface definitions and is external.)
Things that are so universal that they wind up in every non-trivial project and which have stabilized on an agreed-upon solution. (eg. Every non-trivial project includes once_cell, lazy_static, or both, with it being agreed that the younger once_cell is superior for use in new code. Hence the work to incorporate a version of once_cell into std as std::sync::Once*.)
That’s also why, once they finish getting async traits implemented, they intend to work on adding traits that’ll finally allow people to stop specifically writing support for each async executor they want to be compatible with.
A lot of people are also unaware that std has replaced the innards of the standard library HashMap, Mutex, and std::sync::mpsc with more performant third-party crates since v1.0 (hashbrown, parking_lot, and crossbeam_channel, respectively) but you may still want to use them directly to access features precluded by needing to preserve the std APIs externally, such as hashbrown’s no_std support, upstream parking_lot’s non-poisoning mutex API, or crossbeam_channel’s support for MPMC queueing. (I’m not 100% certain, but I believe hashbrown is being depended on while parking_lot and crossbeam_channel are being vendored and possibly patched.)
My opinion on the Python standard library is that its problems are not caused by things being in the standard library, they’re caused by Python’s habit of shipping half-baked functionality in its stable releases. Nobody complains about the Python stdlib packages that through design or luck were working from the beginning.
Rust has the stable/beta/nightly system, which means things like sha256 could hang out in nightly for a while and get its API tweaked according to real-world usage before being stabilized. Python just … ships things, in whatever form they’re in on release day, and relies on its dynamic type system to let fixes get shimmed in later.
Again, this comes back to “Go’s standard library is what it is because Google is paying for the manpower, not because declaring things std magically summons more contributors”.
I disagree that Rust’s small standard library and/or small extended standard library is due to a lack of contributors. IMO it’s due to the libs-team putting themselves in the blocking path of every new library feature, but refusing to expand libs-team large enough to handle the review load.
Any reasonably capable Rust developer could write and submit a competent Rust equivalent to Go’s encoding/base64 or hash/crc32, but if the ACP review and RFC review take multiple years to proceed (with uncertain chance of success) then what’s the point?
If money is the concern, you could imagine the libs-team having a trusted set of “available for hire” folks who are available to do a preliminary review. Someone who wants a feature could pay $5000 or whatever to make sure the ACP and/or RFC are good enough for the libs-team, then take that proposal through a fast-track review process.
But there’s no such mechanism, it’s all just slow-grinding bureaucracy by people who simultanously complain about the workload but also restrict their own team capacity to the point that it can barely keep up with maintenance.
We already have deprecated methods on things like the Error trait which took half a decade to be realized to be the wrong implementation, and that’s with most of the churn being farmed out to error-chain → failure → anyhow/this-error (with some people using a form of anyhow named eyre) while fehler, written by one of the Rust team, languished and got retired because I’m apparently not alone in not wanting more exception-like syntax in Rust error-handling, and other error-handling things like snafu still manage to carve out very non-trivial niches despite “anyhow+this-error, oh, actually, eyre is better than anyhow” being what everyone gets pushed toward by the common wisdom.
It’s not a function of them being half-baked. It’s a function of the reality that it can take 5 or 10 years (or more) to realize that an idea isn’t ideal. That’s why C++ has classical inheritance baked into its core while Rust doesn’t, or why C++ has three different APIs for things like retrieving members from a vector with the nicest, most concise one being the most hazardous to use because it came first. It’s why APIs that remain available from 90s-era Windows NT contain a frozen snapshot of every piece of 90s common wisdom about good software design except for the part where they realized early on that they couldn’t make a microkernel performant enough. (eg. the deprecated fibers APIs and the “APIs, not escape sequences” design of the Win32 console that they finally turned into a GNU Screen-esque compatibility proxy with the advent of Windows Terminal since SSH has become the dominant way to use consoles.)
The Rust devs are optimizing for the reality that all languages will accumulate legacy baggage by trying to strike a balance which allows as much of the language as possible to be swapped out without turning it into Forth.
We already have deprecated methods on things like the Error trait which took half a decade to be realized to be the wrong implementation,
Were they the wrong implementation, though? Sure, they’re deprecated now, but they still work for both old and new code. It’s natural that not all new functionality is completely disjoint from old functionality – sometimes the old stuff gets replaced by a better solution. Also by my calendar it only took three years (Rust v1.0 in 2015, Error::source() added in v1.30 in 2018), but that’s a minor quibble.
Regarding the specific example of Error, my only gripe about its evolution is how long it took to become available from core. I actually ended up commenting on the stabilization issue[0], and maybe had some small influence on core::error::Error getting unwedged from the whole backtrace / provider thing.
and that’s with most of the churn being farmed out to error-chain → failure → anyhow/this-error […]
I might be missing context here, but what do those packages have to do with the standard library? As far as I know none of the Rust stdlib’s error handling logic was imported from a third-party package.
It’s natural that a design space as wide and varied as “generic handling with arbitrary nesting of dynamically-allocated error values” will have a lot of ideas on how to implement it, but the Rust stdlib can’t use those designs anyway, so from the standpoint of trying to expand the stdlib all that churn … kinda doesn’t matter? Like, who actually cares about anyhow vs this-error or whatever? Just be careful not to expose them in your library’s public API and pretty much anything will work well enough.
It’s not a function of them being half-baked. It’s a function of the reality that it can take 5 or 10 years (or more) to realize that an idea isn’t ideal.
This may be a difference of opinion, but I don’t think it’s important that the standard library contain only things that are “ideal”. 5-10 years is an extraordinarily long time for a stdlib feature to be unstable – by that time all of the potential users have already implemented it themselves and moved on.
That’s why C++ has classical inheritance baked into its core while Rust doesn’t, or why C++ has three different APIs for things like retrieving members from a vector with the nicest, most concise one being the most hazardous to use because it came first.
But it’s not the lack of design time that caused C++ to be like that! C++ is designed the way it is because that’s what the designers of C++ want it to be! That’s like saying that Rust shouldn’t have stabilized until it had found a way to eliminate the borrow checker and &str / &[u8] distinction.
The Rust devs are optimizing for the reality that all languages will accumulate legacy baggage by trying to strike a balance which allows as much of the language as possible to be swapped out without turning it into Forth.
I understand that’s what they’re trying to do, my position is that they should stop trying to do that.
There is clearly a middle ground between Python and Go – which allow silly nonsense like XML or HTTP stdlib modules, despite XML being way too complex for a stdlib and HTTP being an evolving standard – and the abject minimalism of Lua or Scheme.
I might be missing context here, but what do those packages have to do with the standard library? As far as I know none of the Rust stdlib’s error handling logic was imported from a third-party package.
The “native” Error handling experience is something people have been complaining about often since day one, but the Rust standard library is what it is because all the churn was kept external to it. That’s the relevance.
This may be a difference of opinion, but I don’t think it’s important that the standard library contain only things that are “ideal”.
Then you’re at odds with the core precept that the Rust devs abide by. Since the Rust v1.0 Stability Promise, they want to minimize how much sub-optimal “use this third-party thing instead” cruft piles up in the standard library.
…given how much they hew to that tenet, discussing the difference of opinion is only relevant if you are trying to decide whether to start a fork.
5-10 years is an extraordinarily long time for a stdlib feature to be unstable – by that time all of the potential users have already implemented it themselves and moved on.
…and they’re fine with that, as demonstrated with their approach to lazy_static, once_cell, and std::sync::Once* and their unconcernedness that it’s taking so long for a clear and popular winner to emerge on the question of implementation inheritance. (i.e. “Your language doesn’t need to have everything. If it’s that hard to find a solution that meets the standard, maybe the best answer is to do nothing”.)
But it’s not the lack of design time that caused C++ to be like that! C++ is designed the way it is because that’s what the designers of C++ want it to be!
True. As is every language. I’m saying that what the Rust designers want is for Cargo to be the complement to the editions system, given that editions can’t remove stuff from the standard library.
That’s like saying that Rust shouldn’t have stabilized until it had found a way to eliminate the borrow checker and &str / &[u8] distinction.
I disagree. In fact, I think that the ignorance of the theoretical underpinnings of those constructs that you’re demonstrating is so high that it becomes “not sure if arguing in good faith”.
First, because the borrow checker is just as much about ensuring the correctness of multi-threaded code in the presence of mutable state as it is about managing memory, and the difficulty of preserving that property while eliminating it is “cutting edge of theoretical research”-level difficult, not “let’s just find a sufficiently comfortable design for this API” difficult. (i.e. That comparison, to me, comes across as the same class of misunderstanding as “Mathematicians could give division by zero a defined behaviour tomorrow if they wanted to.” …if you give division by zero a defined behaviour, you can use it to write proofs for things like 1 = 2.)
Second, because the &str / &[u8] distinction isn’t a technical limitation. It’s an intentional decision to enforce invariants at the level of the type system which would have been more clear if they’d been named ValidUtf8String and ByteString instead. Basically, implying that it’s a problem to be solved, rather than a solution to a problem, is akin to implying that C’s refusal to compile a goto into the middle of some other function is a problem to be solved, rather than a solution to a problem. (specifically, an explicit decision to enforce structured programming principles at the language level.)
There is clearly a middle ground between Python and Go – which allow silly nonsense like XML or HTTP stdlib modules, despite XML being way too complex for a stdlib and HTTP being an evolving standard – and the abject minimalism of Lua or Scheme.
Rust is hardly Lua or Scheme. Personally, I think that they’re striking a fairly good balance, aside from the lack of an official analogue to https://blessed.rs/
The “native” Error handling experience is something people have been complaining about often since day one,
Who and why? As far as I can tell the native error handling is pretty much perfect, since it allows rich error information to be propagated up the stack with very little overhead. A struct mypkg::Error { code: u32 } fits in a single register, and even a pretty complex enum mypkg::RichError<'a> { Something(&'a SomethingDetails), ... } will get a nice compact representation that’s equivalent to a manually-written union.
Conversely, libraries like anyhow require a heap allocation for each error value, and it’s pretty difficult to figure out what types an anyhow::Error actually contains. Instead of returning Result<Something, SomeError> the code returns Result<..., anyhow::Error> and the actual error type can vary at runtime.
I disagree. In fact, I think that the ignorance of the theoretical underpinnings of those constructs that you’re demonstrating is so high that it becomes “not sure if arguing in good faith”. […]
You misunderstand my point.
The distinction between &str and &[u8] is important to the design or Rust for all the reasons you mention. My point is that the current C++ design (including operator[] being the most ergonomic and also most unsafe way to access a std::vector) is what the designers of C++ want it to be. I don’t think the C++ committee considers operator[] to be a failure of the design process, and if they’d had an extra 5 years to ponder it I think they would have produced the same outcome.
When you say “It’s a function of the reality that it can take 5 or 10 years (or more) to realize that an idea isn’t ideal. That’s why C++ has classical inheritance baked into its core while Rust doesn’t,” it reads to me as if you think the designers of C++ would have implemented a different design if they’d spent longer thinking about the problem, which I don’t agree with.
Rust is hardly Lua or Scheme. Personally, I think that they’re striking a fairly good balance, aside from the lack of an official analogue to https://blessed.rs/
Rust’s approach to the standard library is much closer to that of Lua or Scheme than it is to Python or Go, in ways that I think cause adoption of Rust to be slower than it could otherwise be given a richer stdlib.
Regarding sites like https://blessed.rs, I don’t think they’re useful. The crates.io download statistics do well enough for people who just want a catalog of popular libraries, and that page has no analysis or benchmarks or code audit reports or anything that would help guide someone who’s trying to figure out which Rust libraries are good enough to use as-is.
Generally anyone who hasn’t resolved to use either anyhow and/or thiserror-like libraries or Box<dyn Error> and sees it as a flaw in the language that you need a third-party crate to avoid writing “all that boilerplate”… so basically every newcomer and anyone who is opinionated enough to stick to their guns that it’s a flaw to need third-party crates to define custom error types less concisely than class MyError(Exception): pass
…especially when, from what I remember, Box<dyn Error> only became viable a while into Rust’s lifespan. (And I’m not just talking about the syntax change for how you name a trait object.)
(These days, you tend to see those people faulting Rust for not having structural types so you could do something like fn foo(bar: Bax) -> Result<Quux, ErrA | ErrB>)
My point is that the current C++ design (including operator[] being the most ergonomic and also most unsafe way to access a std::vector) is what the designers of C++ want it to be. I don’t think the C++ committee considers operator[] to be a failure of the design process, and if they’d had an extra 5 years to ponder it I think they would have produced the same outcome.
I dunno. On the one hand, from what I’ve read about them, the C++ committee do seem to have a stubborn old man “In my day, we didn’t have no seatbelts or crumple zones. That’s a skill issue.” mindset toward what a programming language should be, even to this day but, on the other hand, C++ was trying to be state of the art for “C with Classes” so I could see them deciding differently before they became so emotionally invested in their current design.
Rust’s approach to the standard library is much closer to that of Lua or Scheme than it is to Python or Go, in ways that I think cause adoption of Rust to be slower than it could otherwise be given a richer stdlib.
I see it as a “the candle that burns twice as hot burns half as long” situation. Yes, it could help Rust to get uptake faster, but it’ll sabotage its longevity in the face of inevitable changing surroundings.
every newcomer and anyone who is opinionated enough to stick to their guns that it’s a flaw to need third-party crates to define custom error types less concisely than class MyError(Exception): pass
Well, the whole challenge with going from a super high-level scripting language like Python to a low-level systems language like Rust is that things are going to get more verbose. That’s just a natural consequence of systems programming requiring more precise control over compiler output and runtime behavior.
Sometimes I get the feeling that people coming from Python/Ruby/JavaScript to Rust would have been happier if they instead went to Java or C# (or Kotlin / Scala / F#), but they avoided those languages for non-technical reasons. There’s nothing wrong with requiring an allocation per error value if your language also has a garbage collector and doesn’t distinguish the stack from the heap in type signatures.
(These days, you tend to see those people faulting Rust for not having structural types so you could do something like fn foo(bar: Bax) -> Result<Quux, ErrA | ErrB>)
Yeah, that’s definitely a high-level way of thinking about it. Structural type systems are interesting for interfaces because they allow looser coupling between libraries (I like Go’s interface, and IIRC Swift does something similar) but it’s difficult to say what role they’d serve as concrete types in Rust.
What kind of ABI would Result<_, ErrA | ErrB> even have? Presumably an implicit enum, but then matching on it becomes tricky (would Rust need Go-style type switches?) and the API evolution into an actual named enum is unclear.
Nobody should be using that package! Nobody with any common sense would include that package in a contemporary list of high-quality Django-related packages!
It reminds me of those autogenerated blogspam articles that pop up in Google results when one searches “best oven degreaser” or whatever.
Sometimes I get the feeling that people coming from Python/Ruby/JavaScript to Rust would have been happier if they instead went to Java or C# (or Kotlin / Scala / F#), but they avoided those languages for non-technical reasons.
“You can’t feasibly write in-process loadable modules for CPython using Java or C# (or Kotlin / Scala / F#)” is a non-technical reason?
(Seriously. My big reason for using Rust is that I can write modules that can then be shared between my PyQt, Django, WebAssembly, etc. projects as well as used in fast-starting, statically-linked CLI tools.)
Yeah, that’s definitely a high-level way of thinking about it. Structural type systems are interesting for interfaces because they allow looser coupling between libraries (I like Go’s interface, and IIRC Swift does something similar) but it’s difficult to say what role they’d serve as concrete types in Rust.
What kind of ABI would Result<_, ErrA | ErrB> even have? Presumably an implicit enum, but then matching on it becomes tricky (would Rust need Go-style type switches?) and the API evolution into an actual named enum is unclear.
nod One of the big oppositions listed in RFC discussions is that the risk of a pile of type aliases containing a semantically significant A | A after a semver-compatible update to a transitive dependency is about as high as allowing type inference in function signatures.
Nobody should be using that package! Nobody with any common sense would include that package in a contemporary list of high-quality Django-related packages!
…and that’s surfaced in rows like “Last Updated”. I’m assuming it’s included in case nothing else meets your needs and you want to take up maintaining a fork for your project as has happened with things like tikv-jemallocator in Rust. (Now the successor, with the original crate being updated as a duplicate of it for compatibility.)
“You can’t feasibly write in-process loadable modules for CPython using Java or C# (or Kotlin / Scala / F#)” is a non-technical reason?
“I want to write business logic in a statically-typed compiled language with automatic memory management and a rich library ecosystem but don’t want to use JVM or .NET languages” would be an example of a non-technical reason.
If you want to be able to declare new error types with Python syntax, write your library in Python. If you want to write a CPython extension module, use PyErr_NewExceptionWithDoc and accept that CPython extension modules written in C (or Rust) have different ergononomics than Python.
(Seriously. My big reason for using Rust is that I can write modules that can then be shared between my PyQt, Django, WebAssembly, etc. projects as well as used in fast-starting, statically-linked CLI tools.)
If you want to write code in a language that is designed for systems programming then you’ll have to accept the tradeoffs that come with that. One of those tradeoffs is that the needs of people using it for microcontroller firmware or OS kernels will have a strong influence on the language design, including in the idiomatic way to represent errors.
In languages like Python or Java, errors contain a great deal of information about their underlying cause – formatted messages, stack traces, wrapped errors from further down the call stack. All of that extra detail requires dynamic memory allocation, which means it can’t be the standard error mechanism in a systems language like Rust. And the size of error values affects how quickly those functions can get called: an error code that fits into a register can be returned in %rbx, a rich error struct that takes up 64 bytes requires extra stack manipulation overhead on every call.
The Cambrian explosion you describe in third-party exception-ish libraries for Rust is a symptom of people using a language that is inappropriate for the level of abstraction they want to write code in, not a sign that the Rust standard library needs a new approach to error handling.
I never said the standard library needs a new approach to error handling. I said that the standard library is as clean as it is because they don’t rush to be batteries included in the face of “no single solution currently in use is head and shoulders above the others” situations.
Rust has the stable/beta/nightly system, which means things like sha256 could hang out in nightly for a while and get its API tweaked according to real-world usage before being stabilized.
How is that better than distributing that same code as crates until they are deemed stable enough for inclusion in the std? Doing what you propose would mean that people using stable Rust (a majority of developers) wouldn’t be able to use those APIs anyways.
*nod* I prioritize stability (I came from Python for the compile-time correctness and the “fearless upgrades”-focused ecosystem) and apparently enough other people do too that they’re worried about not getting enough people to hammer on nightly features before they get stabilized.
(Seeing the “nightly” badge on lib.rs is an automatic Ctrl+W for me. I’d sooner NIH something than make my projects incompatible with the stable-channel compiler.)
“nightly” crates aside, stuff in crates is available for beta-test by all Rust users, which is why they do it that way when it’s an option.
What’s the path to that crate getting into core? I’m having a hard time seeing one.
On the other hand, an unstable core::encoding:base64 could start off as a Rust adaptation of the design principles in https://github.com/simdutf/simdutf, so it’s not starting from scratch. The API of a base64 codec is not large, so it would have a pretty fast path to stabilization (maybe 1-2 years at most). And once it’s stabilized, Rust would have a clear solution to the question of “how do I do base64 encoding in a no_std project without having to run c2rust on simdutf?”.
Doing what you propose would mean that people using stable Rust (a majority of developers) wouldn’t be able to use those APIs anyways.
That’s fine. They can continue to use whatever other solution they were already using, and once the std / core version stabilizes they can switch over to it (or not) on their own schedule. Or they can do something like the futures package and extract snapshots of the unstable stdlib code into a crates.io package.
This is an incredibly strange article. It has a few technical inaccuracies (Box IS a Sized type, the size of a pointer to an object doesn’t depend on the size of the object itself), but more fundamentally, it doesn’t actually answer the question it poses in the title.
For example, it complains that fat pointers are bad because they’re not supported by FFI. Nobody* writing a business app is going to care about FFI.
The rest of the article is full of “standard” complaints about Rust that I think have been fairly widely debunked, or just represent a not-well-informed view of things (e.g., the borrow checker is too hard, async sucks, etc) but even if true none of these criticisms are specific to business apps, it’s just a critique of the language itself.
I also just had to laugh at this bit:
While languages such as Go enable you to write pretty much entire HTTP service from standard lib alone, this bazaar-style package management comes with the burden: whenever you need to solve any mundane problem, you land in a space where everything has at least 7 different crates available, but half of them are actually a toy projects and most of them were not maintained for the last 5 years. And don’t get me started about audits to check if one of 600 dependencies of your hello world app won’t be used for supply chain attacks.
Yes, dependency management is a concern, but comparing to Go which famously refuses to implement basic features in the language, and then expects you to import xXx69roflcopterxXx’s github repo which is widely accepted as the best library in the ecosystem is a bit hilarious to me.
Yes, yes, I’m sure that somebody somewhere has tried to write a business app with FFI included, but it’s certainly not the norm.
Yes, dependency management is a concern, but comparing to Go which famously refuses to implement basic features in the language, and then expects you to import xXx69roflcopterxXx’s github repo which is widely accepted as the best library in the ecosystem is a bit hilarious to me.
Rust is the exact same, it just hides the GitHub repo names better. The Cargo package json would in Go be spelled "github.com/maciejhirsz/json-rust".
Well, not the exact same, because if you also decide to write your own json package for Rust you can’t register it in crates.io due to its no-namespaces naming policy. Go doesn’t restrict how many packages are allowed to match the pattern "github.com/*/json".
Rust requires more of implementation details to be visible, like the locks, dynamic dispatch, or mutability of references. Compared to Java or Go, this is a pedantic level.
However, I still like strictness of Rust’s type system. Especially exclusive ownership makes APIs clearer and more robust – I never need to worry if an object my function is working on could be mutated from somewhere else.
The advantage in multi-threading isn’t about async at all, or mere ability to use many cores. It’s about whole-program guarantee there’s no accidental sharing and data races that could lead to data corruption, even in complex programs. This is a hard problem, and in Go it takes a lot of vigilance where you have both first-class multithreading and non-thread-safe built-in collections.
you land in a space where everything has at least 7 different crates available, but half of them are actually a toy projects and most of them were not maintained for the last 5 years
The other side of the dependency problem I’m curious about is how teams efficiently get crates through security and legal review and how they handle updates to those crates.
I’m biased, but I think crates.io search is bad (does not have active maintenance as a ranking signal), and contributes to the perception. When looking through the lens of https://lib.rs it’s much better.
As for legal and security: with tooling. cargo-deny for licensing. cargo-vet to ensure that deps are either from trusted authors, or had their code reviewed.
I don’t know, maintenance matters for a HTTP library, it doesn’t matter for a strictyaml parser. One of those is constantly evolving, and the other is a standard which is immutable and won’t change. There’s an argument to be made that Rust is itself evolving and so dependencies should evolve with it, but a lot of the time it’s fine if some things were written and worked on over the course of a few months and then left. There’s plenty of perfectly serviceable crates out there like that. I encounter them all the time when doing embedded stuff. The ADS1015 is a chip with a know interface and once you write a working driver for it, you’re done. Now embedded HAL might come out with a 2.0, but until then, you don’t need to touch it.
That’s optimizing the general case for an edge case.
There are some old “done” crates, but literally 99.5% of old crates are junk. With such a terrible false positive rate it’s hard to even find the timeless ones.
These are real numbers — I’ve actually checked. I’ve searched for crates that are old but good, using multiple methods, and I could not find more than about 100 that may be useful. 150 is scraping the bottom of the barrel. That may sound like a significant number, but that was among 31000 old crates. That’s 200 junk crates for every non-junk one. The timeless ones aren’t even that good — they’re mostly unoptimized classic hash functions, simple macros, and opinionated wrappers for std functions.
Sturgeon’s law holds true for the Rust ecosystem too. In my estimates, 11% of crates are good, 25% maybe okay, and the quality goes sharply down from there. It’s full of “hello world”s, abandoned forks, unfinished prototypes, namesquatting and various grades of spammy behavior.
Rust has more “done” crates, but most of them still release some minor patch every few years. It’s really not a big ask to make a release once every 3-5 years just to mark a crate as still usable. At least there’s an excuse to bump Rust edition.
Avoiding hard decisions about ranking to give a chance to every edge-case crate doesn’t work. You just get search results full of crap, and ironically the crap crowds-out the good crates, making them harder to find.
Is it really the edge case? I am not entirely convinced.
The fact that 99.5% of crates are junk doesn’t really matter in this discussion since that’s true for most code written and the only difference is that here the barrier to entry (to publishing) is very low.
I think if crates got marked “WIP” by default and non-WIP crates got prioritised over WIP ones in search results, you could probably solve most of these problems. Kindly ask people to consider whether they consider their crate ready for general use or not, and I think most of these unfinished crates would just never drop their WIP marker.
Pre 1.0 when using semver isn’t quite the same as WIP, it’s a useful region where “nothing is compatible with anything else” which means you can just keep bumping the minor version. The alternative is to always bump the major version but this would now imply “not WIP” which might also not be correct.
The tooling is nice, but my curiosity is more along the lines of “this is how our programmers work with the legal and security departments to get and keep our package list approved.” Poking around, it looks like what I really want is an article about operationalizing something like OpenSSF’s Securing Software Repositories in a corporate environment.
I’d say this needs to
s/No/Not yet/its answer.You’d prefer we either be frozen with the 0.1 version of the RNG API and
rustc_serializeinstead of Serde or have a Python/Java-esque situation where everyone tells you to ignore the standard library implementation and use a de facto-standard successor instead. (eg. Requests for Python and its internal never-will-be-in-stdliburllib3, with basically any other networking code in the standard library being “use Twisted instead” for ages.)Rust’s development philosophy was informed by the saying in the Python world that the standard library is where modules go to die, because the API stability requirements strangle further development. That’s why crates like
randare effectively “pieces of the standard library distributed through cargo”… because distributing through Cargo allows you to use older libraries with newer toolchains and vice versa.Every time I see this argument, it comes across as people seeming to believe that you can force a simple solution to a complex problem.
Except that, if they chose to do that, it wouldn’t mean they arrived sooner… it’d mean Day One is still in the future.
Rust follows a “release a minimum viable product and extend it later” model and they already regret the things they were over-eager to implement and have to support in deprecated form in perpetuity, like the
descriptionandcausemethods on theErrortrait.Async traits are in the process of being implemented. Generators exist in a “not yet safe for unconstrained use” form as the underlayer of
async/awaitand are in the process of being implemented more generally.Discussion is ongoing on ways to improve this. For example, one idea is keyword generics, which could allow functions to be generic over whether they’re being called in an async context. (The idea of abstracting over
&vs.&muthasn’t seen as much progress because it tends to get into the weeds over whether it’s actually a good idea, given the underlying semantic difference between them and how likely it is to actually need different code to achieve correctness.)Your first two points sound like “I believe the “
dyn Traitwouldn’t be a fat pointer”-ness of C++-style classes with internal vtables has benefits so valuable to things like C FFI and lock-free compare-and-swap that they should be offered as an option in the language itself”, which I can see reasonable discussion around. However, your third point is talking about generics being limited toSizedthings… which is hard to argue as anything other than faulting Rust for not universally using C++-style classical inheritance for polymorphism since, if it’s anything but the only option, you’re back to some things satisfying the constraint and some not.The general answer for situations like this is that, as with what happened with lazy_static → once_cell →
std::sync::Once*, the Rust upstream are waiting to see if anything emerges in the ecosystem which is used ubiquitously enough to justify bringing it in.(Rust has a philosophy that, if it can be prototyped, iterated on, and prove its desirability as a macro first, it should. Heck,
?began astry!even though it was part of the standard library in both forms.)For example, delegate is used in two to three orders of magnitude fewer crates than
once_cellandlazy_static, depending on how much overlap there is between them, and there’s the added barrier to getting implementation inheritance in the standard library that there’s no single way to do it that’s unarguably best. Each one has trade-offs and none has been excessively more desirable than the others so far.Honestly, this section’s take on the borrow checker feels not unlike calling math bullshit because Gödel’s incompleteness theorems exist.
I’m reminded Monty from xiph.org explaining that Gibbs Effect ripples are “just part of what a band-limited signal is”.
Even if Rust didn’t have to deal with being suitable for niches where you can’t bring your own tracing garbage collector, those Share-XOR-Mutate semantics are necessary to make ensuring correctness of multi-threaded code at compile time tractable.
Partial borrows are another topic of ongoing discussion. (Basically, given limited developer bandwidth, they’ve been put low on the priority list compared to things like the work to get async as close to the level of ergonomics as sync enjoys as possible.)
In fact, one of the things that’s been prioritized higher than partial borrows is the Polonius project to rewrite the borrow checker to finally be able to address Non-Lexical Lifetimes Problem Case #3. (i.e. making the borrow checker smarter)
Fair… though, for me, what matters is that Rust is much more likely to throw up a compiler error if I blindly try to do something which would have me fall off the fast path, while GCed languages will just happily pessimize performance to avoid distracting me from thinking about my business logic.
I’ve always seen Rust as less about performance and more about predictability.
…though…
Given that I see Rust in the context of multi-threading to be more about correctness, I’m reminded of this quote:
You are aware, I hope, that Go isn’t memory-safe in the presence of data races. IIRC, it has to do with passing around certain kinds of multi-word values without synchronization in ways that can invoke undefined behaviour.
I fully agree. The lack of a maintained tool in the vein of Rustig or findpanics is embarrassing and, along with LLVM not providing the information necessary to turn “this optimization used to get applied and now it doesn’t” into a compile-time error, is one of the biggest things I dislike about Rust.
There’s one thing about async traits that I’m of two minds of: I think that we could have stabilized async fn in traits by de sugaring to
Box<dyn Future>ages ago, with the object safety restrictions and (tiny) perf hit that comes from that, to provide value to people earlier while we still worked on impl trait in trait, but fear that if we had done that the pressure to ship impl trait in trait would have deflated and that project would never have seen the light of day, causing the “imperfect and temporary” solution to stick around.The issue with generators where that there are 3 related things (in order of blocking stabilization more to less):
for await f in stream). You don’t need this because you can usewhile letalways, addingawaitto patterns is a can of worms (it can be made to work, also amusingly you could make it solet await x = y;andlet x = y.await;were equivalent, but what you also want is some way of selecting parallel batch size, like rayon’spar_iteras well)I would like it if we could supplement this gap more with Rudy’s lint’s: make the easy code fast, make the code do what the user expects (“I’m returning a boxed a trait here, so method access are through a v-table”) but detect the cases where an optimizing compiler would switch strategies and tell the user how to write it differently (“this function is used in a hot loop, and only ever returns one of two types, use impl Trait instead”). This keeps the developer in control, choices are explicit, and mild changes in the code don’t have sudden hidden performance cliffs because of changes from the assumption. Basically, every rule where Swift would change its code gen strategy should be a lint.
This would have made
asynctraits depend on a memory allocator, which would rule out its use from many interesting use cases (e.g. embedded systems).Yes, but changing the underlying mechanism to impl Trait was a backwards compatible change. I called out that the trade off would have been potentially delaying benefits for a few but enable a lot of people earlier.
I don’t see how that could be true? If the feature is implemented by desugaring to
Boxthen trying to remove that desugaring is also a backwards-incompatible change. For example, assuming an implementation similar to https://github.com/dtolnay/async-trait, you might have:Hiding memory allocations in normal-looking code also goes completely against the Rust use case, how are you going to verify that a particular code path has no allocations in it if any trait object’s method call might potentially be a hidden
Box::new()?Interesting you use
randas an example, because it seems like the opposite situation in play – the crates.iorandlibrary isn’t maintained by the Rust developers, it’s just under some random GitHub org at https://github.com/rust-random/rand.There are some crates.io packages that are like extended parts of the stdlib (
flate2,git2,getopts) but they’re not necessarily the most popular solution in their space. It’s also difficult to discover which packages are underrust-lang/and which aren’t without digging through the crates.io index metadata.In contrast, Go has a larger built-in stdlib than Rust, but also has an extended stdlib distributed as regular packages.
golang.org/x/crypto,golang.org/x/text, andgolang.org/x/net/http2are all maintained by the Go developers – if you want a SHA2-256 checksum then the stdlib has you covered, the more esoteric BLAKE2b is ingolang.org/x/crypto.In Rust if you use the crates.io
sha256package then you’re depending on https://github.com/baoyachi/sha256-rs, which as of the most recent version (v1.6.0, released 3 days ago) doesn’t even haveno_stdsupport.I certainly agree that we need better tooling to indicate which things are officially “external pieces of the standard library”. However, by Python 2.7, half of Python’s standard library was under de facto deprecation in favour of libraries with no connection to the stdlib dev team, with the identity of those packages being communicated through tribal knowledge.
Putting stuff in Rust’s actual standard library wouldn’t prevent the “Batteries Included, But They’re Leaking” effect and there’s only so much that “this is officially maintained” can do to put a thumb on the “who’s competing more effectively?” scale.
Again, this comes back to “Go’s standard library is what it is because Google is paying for the manpower, not because declaring things
stdmagically summons more contributors”.Having seen the state Python was in, the philosophy behind Rust’s standard library is that it’s for two kinds of things:
coreorstdfor language constructs to depend on. (Which is whyResultandFutureare instdfor?andasyncto depend on, but the http crate is the de facto standard source of HTTP request and response interface definitions and is external.)once_cell,lazy_static, or both, with it being agreed that the youngeronce_cellis superior for use in new code. Hence the work to incorporate a version ofonce_cellintostdasstd::sync::Once*.)That’s also why, once they finish getting async traits implemented, they intend to work on adding traits that’ll finally allow people to stop specifically writing support for each async executor they want to be compatible with.
A lot of people are also unaware that
stdhas replaced the innards of the standard libraryHashMap,Mutex, andstd::sync::mpscwith more performant third-party crates since v1.0 (hashbrown, parking_lot, and crossbeam_channel, respectively) but you may still want to use them directly to access features precluded by needing to preserve thestdAPIs externally, such as hashbrown’sno_stdsupport, upstream parking_lot’s non-poisoning mutex API, or crossbeam_channel’s support for MPMC queueing. (I’m not 100% certain, but I believe hashbrown is being depended on while parking_lot and crossbeam_channel are being vendored and possibly patched.)My opinion on the Python standard library is that its problems are not caused by things being in the standard library, they’re caused by Python’s habit of shipping half-baked functionality in its stable releases. Nobody complains about the Python stdlib packages that through design or luck were working from the beginning.
Rust has the stable/beta/nightly system, which means things like
sha256could hang out in nightly for a while and get its API tweaked according to real-world usage before being stabilized. Python just … ships things, in whatever form they’re in on release day, and relies on its dynamic type system to let fixes get shimmed in later.I disagree that Rust’s small standard library and/or small extended standard library is due to a lack of contributors. IMO it’s due to the libs-team putting themselves in the blocking path of every new library feature, but refusing to expand libs-team large enough to handle the review load.
Any reasonably capable Rust developer could write and submit a competent Rust equivalent to Go’s
encoding/base64orhash/crc32, but if the ACP review and RFC review take multiple years to proceed (with uncertain chance of success) then what’s the point?If money is the concern, you could imagine the libs-team having a trusted set of “available for hire” folks who are available to do a preliminary review. Someone who wants a feature could pay $5000 or whatever to make sure the ACP and/or RFC are good enough for the libs-team, then take that proposal through a fast-track review process.
But there’s no such mechanism, it’s all just slow-grinding bureaucracy by people who simultanously complain about the workload but also restrict their own team capacity to the point that it can barely keep up with maintenance.
We already have deprecated methods on things like the
Errortrait which took half a decade to be realized to be the wrong implementation, and that’s with most of the churn being farmed out to error-chain → failure → anyhow/this-error (with some people using a form of anyhow named eyre) while fehler, written by one of the Rust team, languished and got retired because I’m apparently not alone in not wanting more exception-like syntax in Rust error-handling, and other error-handling things like snafu still manage to carve out very non-trivial niches despite “anyhow+this-error, oh, actually, eyre is better than anyhow” being what everyone gets pushed toward by the common wisdom.It’s not a function of them being half-baked. It’s a function of the reality that it can take 5 or 10 years (or more) to realize that an idea isn’t ideal. That’s why C++ has classical inheritance baked into its core while Rust doesn’t, or why C++ has three different APIs for things like retrieving members from a vector with the nicest, most concise one being the most hazardous to use because it came first. It’s why APIs that remain available from 90s-era Windows NT contain a frozen snapshot of every piece of 90s common wisdom about good software design except for the part where they realized early on that they couldn’t make a microkernel performant enough. (eg. the deprecated fibers APIs and the “APIs, not escape sequences” design of the Win32 console that they finally turned into a GNU Screen-esque compatibility proxy with the advent of Windows Terminal since SSH has become the dominant way to use consoles.)
The Rust devs are optimizing for the reality that all languages will accumulate legacy baggage by trying to strike a balance which allows as much of the language as possible to be swapped out without turning it into Forth.
Were they the wrong implementation, though? Sure, they’re deprecated now, but they still work for both old and new code. It’s natural that not all new functionality is completely disjoint from old functionality – sometimes the old stuff gets replaced by a better solution. Also by my calendar it only took three years (Rust v1.0 in 2015,
Error::source()added in v1.30 in 2018), but that’s a minor quibble.Regarding the specific example of
Error, my only gripe about its evolution is how long it took to become available fromcore. I actually ended up commenting on the stabilization issue[0], and maybe had some small influence oncore::error::Errorgetting unwedged from the whole backtrace / provider thing.[0] https://github.com/rust-lang/rust/issues/103765#issuecomment-1801053112
I might be missing context here, but what do those packages have to do with the standard library? As far as I know none of the Rust stdlib’s error handling logic was imported from a third-party package.
It’s natural that a design space as wide and varied as “generic handling with arbitrary nesting of dynamically-allocated error values” will have a lot of ideas on how to implement it, but the Rust stdlib can’t use those designs anyway, so from the standpoint of trying to expand the stdlib all that churn … kinda doesn’t matter? Like, who actually cares about
anyhowvsthis-erroror whatever? Just be careful not to expose them in your library’s public API and pretty much anything will work well enough.This may be a difference of opinion, but I don’t think it’s important that the standard library contain only things that are “ideal”. 5-10 years is an extraordinarily long time for a stdlib feature to be unstable – by that time all of the potential users have already implemented it themselves and moved on.
But it’s not the lack of design time that caused C++ to be like that! C++ is designed the way it is because that’s what the designers of C++ want it to be! That’s like saying that Rust shouldn’t have stabilized until it had found a way to eliminate the borrow checker and
&str/&[u8]distinction.I understand that’s what they’re trying to do, my position is that they should stop trying to do that.
There is clearly a middle ground between Python and Go – which allow silly nonsense like XML or HTTP stdlib modules, despite XML being way too complex for a stdlib and HTTP being an evolving standard – and the abject minimalism of Lua or Scheme.
The “native” Error handling experience is something people have been complaining about often since day one, but the Rust standard library is what it is because all the churn was kept external to it. That’s the relevance.
Then you’re at odds with the core precept that the Rust devs abide by. Since the Rust v1.0 Stability Promise, they want to minimize how much sub-optimal “use this third-party thing instead” cruft piles up in the standard library.
…given how much they hew to that tenet, discussing the difference of opinion is only relevant if you are trying to decide whether to start a fork.
…and they’re fine with that, as demonstrated with their approach to lazy_static, once_cell, and
std::sync::Once*and their unconcernedness that it’s taking so long for a clear and popular winner to emerge on the question of implementation inheritance. (i.e. “Your language doesn’t need to have everything. If it’s that hard to find a solution that meets the standard, maybe the best answer is to do nothing”.)True. As is every language. I’m saying that what the Rust designers want is for Cargo to be the complement to the editions system, given that editions can’t remove stuff from the standard library.
I disagree. In fact, I think that the ignorance of the theoretical underpinnings of those constructs that you’re demonstrating is so high that it becomes “not sure if arguing in good faith”.
First, because the borrow checker is just as much about ensuring the correctness of multi-threaded code in the presence of mutable state as it is about managing memory, and the difficulty of preserving that property while eliminating it is “cutting edge of theoretical research”-level difficult, not “let’s just find a sufficiently comfortable design for this API” difficult. (i.e. That comparison, to me, comes across as the same class of misunderstanding as “Mathematicians could give division by zero a defined behaviour tomorrow if they wanted to.” …if you give division by zero a defined behaviour, you can use it to write proofs for things like
1 = 2.)Second, because the &str / &[u8] distinction isn’t a technical limitation. It’s an intentional decision to enforce invariants at the level of the type system which would have been more clear if they’d been named
ValidUtf8StringandByteStringinstead. Basically, implying that it’s a problem to be solved, rather than a solution to a problem, is akin to implying that C’s refusal to compile agotointo the middle of some other function is a problem to be solved, rather than a solution to a problem. (specifically, an explicit decision to enforce structured programming principles at the language level.)Rust is hardly Lua or Scheme. Personally, I think that they’re striking a fairly good balance, aside from the lack of an official analogue to https://blessed.rs/
Who and why? As far as I can tell the native error handling is pretty much perfect, since it allows rich error information to be propagated up the stack with very little overhead. A
struct mypkg::Error { code: u32 }fits in a single register, and even a pretty complexenum mypkg::RichError<'a> { Something(&'a SomethingDetails), ... }will get a nice compact representation that’s equivalent to a manually-writtenunion.Conversely, libraries like
anyhowrequire a heap allocation for each error value, and it’s pretty difficult to figure out what types ananyhow::Erroractually contains. Instead of returningResult<Something, SomeError>the code returnsResult<..., anyhow::Error>and the actual error type can vary at runtime.You misunderstand my point.
The distinction between
&strand&[u8]is important to the design or Rust for all the reasons you mention. My point is that the current C++ design (includingoperator[]being the most ergonomic and also most unsafe way to access astd::vector) is what the designers of C++ want it to be. I don’t think the C++ committee considersoperator[]to be a failure of the design process, and if they’d had an extra 5 years to ponder it I think they would have produced the same outcome.When you say “It’s a function of the reality that it can take 5 or 10 years (or more) to realize that an idea isn’t ideal. That’s why C++ has classical inheritance baked into its core while Rust doesn’t,” it reads to me as if you think the designers of C++ would have implemented a different design if they’d spent longer thinking about the problem, which I don’t agree with.
Rust’s approach to the standard library is much closer to that of Lua or Scheme than it is to Python or Go, in ways that I think cause adoption of Rust to be slower than it could otherwise be given a richer stdlib.
Regarding sites like https://blessed.rs, I don’t think they’re useful. The crates.io download statistics do well enough for people who just want a catalog of popular libraries, and that page has no analysis or benchmarks or code audit reports or anything that would help guide someone who’s trying to figure out which Rust libraries are good enough to use as-is.
A page like https://asomers.github.io/mock_shootout/ (comparison of mocking libs for Rust) or https://github.com/MartinThoma/algorithms/tree/master/Python/json-benchmark (JSON libraries for Python) would be much more useful, since it provides at least some basis for comparison.
Generally anyone who hasn’t resolved to use either anyhow and/or thiserror-like libraries or
Box<dyn Error>and sees it as a flaw in the language that you need a third-party crate to avoid writing “all that boilerplate”… so basically every newcomer and anyone who is opinionated enough to stick to their guns that it’s a flaw to need third-party crates to define custom error types less concisely thanclass MyError(Exception): pass…especially when, from what I remember,
Box<dyn Error>only became viable a while into Rust’s lifespan. (And I’m not just talking about the syntax change for how you name a trait object.)(These days, you tend to see those people faulting Rust for not having structural types so you could do something like
fn foo(bar: Bax) -> Result<Quux, ErrA | ErrB>)I dunno. On the one hand, from what I’ve read about them, the C++ committee do seem to have a stubborn old man “In my day, we didn’t have no seatbelts or crumple zones. That’s a skill issue.” mindset toward what a programming language should be, even to this day but, on the other hand, C++ was trying to be state of the art for “C with Classes” so I could see them deciding differently before they became so emotionally invested in their current design.
I see it as a “the candle that burns twice as hot burns half as long” situation. Yes, it could help Rust to get uptake faster, but it’ll sabotage its longevity in the face of inevitable changing surroundings.
Fair. Let’s say a site like https://djangopackages.org/ and its grids instead.
Well, the whole challenge with going from a super high-level scripting language like Python to a low-level systems language like Rust is that things are going to get more verbose. That’s just a natural consequence of systems programming requiring more precise control over compiler output and runtime behavior.
Sometimes I get the feeling that people coming from Python/Ruby/JavaScript to Rust would have been happier if they instead went to Java or C# (or Kotlin / Scala / F#), but they avoided those languages for non-technical reasons. There’s nothing wrong with requiring an allocation per error value if your language also has a garbage collector and doesn’t distinguish the stack from the heap in type signatures.
Yeah, that’s definitely a high-level way of thinking about it. Structural type systems are interesting for interfaces because they allow looser coupling between libraries (I like Go’s
interface, and IIRC Swift does something similar) but it’s difficult to say what role they’d serve as concrete types in Rust.What kind of ABI would
Result<_, ErrA | ErrB>even have? Presumably an implicitenum, but then matching on it becomes tricky (would Rust need Go-style type switches?) and the API evolution into an actual namedenumis unclear.That seems to have a related problem, it’s got a comparison but the package selection is way too un-curated.
I used Django ~15 years ago in my first job out of university, and one of the packages I uploaded to PyPI at that time was
django-genshi. It’s not been updated since, and yet when I go to https://djangopackages.org/grids/g/template-engine-adapters/ it’s listed!Nobody should be using that package! Nobody with any common sense would include that package in a contemporary list of high-quality Django-related packages!
It reminds me of those autogenerated blogspam articles that pop up in Google results when one searches “best oven degreaser” or whatever.
“You can’t feasibly write in-process loadable modules for CPython using Java or C# (or Kotlin / Scala / F#)” is a non-technical reason?
(Seriously. My big reason for using Rust is that I can write modules that can then be shared between my PyQt, Django, WebAssembly, etc. projects as well as used in fast-starting, statically-linked CLI tools.)
nod One of the big oppositions listed in RFC discussions is that the risk of a pile of type aliases containing a semantically significant
A | Aafter a semver-compatible update to a transitive dependency is about as high as allowing type inference in function signatures.…and that’s surfaced in rows like “Last Updated”. I’m assuming it’s included in case nothing else meets your needs and you want to take up maintaining a fork for your project as has happened with things like tikv-jemallocator in Rust. (Now the successor, with the original crate being updated as a duplicate of it for compatibility.)
“I want to write business logic in a statically-typed compiled language with automatic memory management and a rich library ecosystem but don’t want to use JVM or .NET languages” would be an example of a non-technical reason.
If you want to be able to declare new error types with Python syntax, write your library in Python. If you want to write a CPython extension module, use
PyErr_NewExceptionWithDocand accept that CPython extension modules written in C (or Rust) have different ergononomics than Python.If you want to write code in a language that is designed for systems programming then you’ll have to accept the tradeoffs that come with that. One of those tradeoffs is that the needs of people using it for microcontroller firmware or OS kernels will have a strong influence on the language design, including in the idiomatic way to represent errors.
In languages like Python or Java, errors contain a great deal of information about their underlying cause – formatted messages, stack traces, wrapped errors from further down the call stack. All of that extra detail requires dynamic memory allocation, which means it can’t be the standard error mechanism in a systems language like Rust. And the size of error values affects how quickly those functions can get called: an error code that fits into a register can be returned in
%rbx, a rich error struct that takes up 64 bytes requires extra stack manipulation overhead on every call.The Cambrian explosion you describe in third-party exception-ish libraries for Rust is a symptom of people using a language that is inappropriate for the level of abstraction they want to write code in, not a sign that the Rust standard library needs a new approach to error handling.
I never said the standard library needs a new approach to error handling. I said that the standard library is as clean as it is because they don’t rush to be batteries included in the face of “no single solution currently in use is head and shoulders above the others” situations.
How is that better than distributing that same code as crates until they are deemed stable enough for inclusion in the std? Doing what you propose would mean that people using stable Rust (a majority of developers) wouldn’t be able to use those APIs anyways.
*nod* I prioritize stability (I came from Python for the compile-time correctness and the “fearless upgrades”-focused ecosystem) and apparently enough other people do too that they’re worried about not getting enough people to hammer on nightly features before they get stabilized.
(Seeing the “nightly” badge on lib.rs is an automatic Ctrl+W for me. I’d sooner NIH something than make my projects incompatible with the stable-channel compiler.)
“nightly” crates aside, stuff in crates is available for beta-test by all Rust users, which is why they do it that way when it’s an option.
Because “they are deemed stable enough for inclusion in the std” doesn’t happen.
Look at
base64(https://github.com/marshallpierce/rust-base64) as an example. Its still on a v0.x version after 7 years of development, the API requires allocation (Engine::encodereturns aString), and it doesn’t seem to be using SIMD at all.What’s the path to that crate getting into
core? I’m having a hard time seeing one.On the other hand, an unstable
core::encoding:base64could start off as a Rust adaptation of the design principles in https://github.com/simdutf/simdutf, so it’s not starting from scratch. The API of a base64 codec is not large, so it would have a pretty fast path to stabilization (maybe 1-2 years at most). And once it’s stabilized, Rust would have a clear solution to the question of “how do I do base64 encoding in ano_stdproject without having to run c2rust on simdutf?”.That’s fine. They can continue to use whatever other solution they were already using, and once the
std/coreversion stabilizes they can switch over to it (or not) on their own schedule. Or they can do something like thefuturespackage and extract snapshots of the unstable stdlib code into a crates.io package.This is an incredibly strange article. It has a few technical inaccuracies (Box IS a Sized type, the size of a pointer to an object doesn’t depend on the size of the object itself), but more fundamentally, it doesn’t actually answer the question it poses in the title.
For example, it complains that fat pointers are bad because they’re not supported by FFI. Nobody* writing a business app is going to care about FFI.
The rest of the article is full of “standard” complaints about Rust that I think have been fairly widely debunked, or just represent a not-well-informed view of things (e.g., the borrow checker is too hard, async sucks, etc) but even if true none of these criticisms are specific to business apps, it’s just a critique of the language itself.
I also just had to laugh at this bit:
Yes, dependency management is a concern, but comparing to Go which famously refuses to implement basic features in the language, and then expects you to import xXx69roflcopterxXx’s github repo which is widely accepted as the best library in the ecosystem is a bit hilarious to me.
Rust is the exact same, it just hides the GitHub repo names better. The Cargo package
jsonwould in Go be spelled"github.com/maciejhirsz/json-rust".Well, not the exact same, because if you also decide to write your own
jsonpackage for Rust you can’t register it in crates.io due to its no-namespaces naming policy. Go doesn’t restrict how many packages are allowed to match the pattern"github.com/*/json".I’m not sure what you call a business app. All the ones I know always have some dll-based runtime plugin / extension mechanism.
First, beware the word “should”.
Second, history is not as simple as “what seems to clear to me with hindsight”.
I think it’s mostly fair assessment.
Rust requires more of implementation details to be visible, like the locks, dynamic dispatch, or mutability of references. Compared to Java or Go, this is a pedantic level.
However, I still like strictness of Rust’s type system. Especially exclusive ownership makes APIs clearer and more robust – I never need to worry if an object my function is working on could be mutated from somewhere else.
The advantage in multi-threading isn’t about async at all, or mere ability to use many cores. It’s about whole-program guarantee there’s no accidental sharing and data races that could lead to data corruption, even in complex programs. This is a hard problem, and in Go it takes a lot of vigilance where you have both first-class multithreading and non-thread-safe built-in collections.
The other side of the dependency problem I’m curious about is how teams efficiently get crates through security and legal review and how they handle updates to those crates.
I’m biased, but I think crates.io search is bad (does not have active maintenance as a ranking signal), and contributes to the perception. When looking through the lens of https://lib.rs it’s much better.
As for legal and security: with tooling.
cargo-denyfor licensing.cargo-vetto ensure that deps are either from trusted authors, or had their code reviewed.I don’t know, maintenance matters for a HTTP library, it doesn’t matter for a strictyaml parser. One of those is constantly evolving, and the other is a standard which is immutable and won’t change. There’s an argument to be made that Rust is itself evolving and so dependencies should evolve with it, but a lot of the time it’s fine if some things were written and worked on over the course of a few months and then left. There’s plenty of perfectly serviceable crates out there like that. I encounter them all the time when doing embedded stuff. The ADS1015 is a chip with a know interface and once you write a working driver for it, you’re done. Now embedded HAL might come out with a 2.0, but until then, you don’t need to touch it.
…which reminds me of this quote from a video on the advantage of Rust being so focused on correctness and “fearless upgrades”.
That’s optimizing the general case for an edge case.
There are some old “done” crates, but literally 99.5% of old crates are junk. With such a terrible false positive rate it’s hard to even find the timeless ones.
These are real numbers — I’ve actually checked. I’ve searched for crates that are old but good, using multiple methods, and I could not find more than about 100 that may be useful. 150 is scraping the bottom of the barrel. That may sound like a significant number, but that was among 31000 old crates. That’s 200 junk crates for every non-junk one. The timeless ones aren’t even that good — they’re mostly unoptimized classic hash functions, simple macros, and opinionated wrappers for std functions.
Sturgeon’s law holds true for the Rust ecosystem too. In my estimates, 11% of crates are good, 25% maybe okay, and the quality goes sharply down from there. It’s full of “hello world”s, abandoned forks, unfinished prototypes, namesquatting and various grades of spammy behavior.
Rust has more “done” crates, but most of them still release some minor patch every few years. It’s really not a big ask to make a release once every 3-5 years just to mark a crate as still usable. At least there’s an excuse to bump Rust edition.
Avoiding hard decisions about ranking to give a chance to every edge-case crate doesn’t work. You just get search results full of crap, and ironically the crap crowds-out the good crates, making them harder to find.
Is it really the edge case? I am not entirely convinced.
The fact that 99.5% of crates are junk doesn’t really matter in this discussion since that’s true for most code written and the only difference is that here the barrier to entry (to publishing) is very low.
I think if crates got marked “WIP” by default and non-WIP crates got prioritised over WIP ones in search results, you could probably solve most of these problems. Kindly ask people to consider whether they consider their crate ready for general use or not, and I think most of these unfinished crates would just never drop their WIP marker.
If only there were a version numbering scheme that came with a WIP marker built in!
Pre 1.0 when using semver isn’t quite the same as WIP, it’s a useful region where “nothing is compatible with anything else” which means you can just keep bumping the minor version. The alternative is to always bump the major version but this would now imply “not WIP” which might also not be correct.
The tooling is nice, but my curiosity is more along the lines of “this is how our programmers work with the legal and security departments to get and keep our package list approved.” Poking around, it looks like what I really want is an article about operationalizing something like OpenSSF’s Securing Software Repositories in a corporate environment.