1. 5

    This one feels boring to me compared to 2018, which is exactly what I was hoping. After the 2018 edition, a few prominent companies were very clear to the Rust team that if the same thing happened every 3 years, that Rust would not be a sustainable choice for their long-term projects.

    1. 10

      Fun fact, this edition is just as big as the previous one!

      The Rust team has IMHO made a mistake of making one big marketing push under “2018” umbrella for two very different things:

      1. Breaking changes in the 2018 edition, which were actually quite small

      2. Advertising all 2015-compatible features added in previous couple years to highlight the progress Rust has made since 1.0

      And this has created a wrong impression as if everything was suddenly added at once, breaking, and hidden behind the edition flag. When in fact that was multiple years of small backwards-compatible changes, almost all of which are available in all editions.

      The 2018 edition compilation flag only changed:

      • Reserved async, dyn and try as keywords
      • Changed meaning of use paths slightly
      • Elevated a few warnings to errors

      All other changes were not related to the edition, and were added backwards-compatibly to the 2015 syntax (except async syntax sugar, because of the new keyword, but all the underlying std::future machinery works in 2015 edition too). Most of them were released even before 2018 was announced and re-advertised them as new again.

      1. 1

        Excuse the flippant reply, but so what? The value of Rust is not determined by mega corps.

        1. -1

          Sorry for my rather flippant reply, but – so what? How is this relevant, and why would people care?

          Is this some US thing where corporations are people, so everyone has to take care to not hurt their feelings or something?

        1. 4

          I am frightened by the behavior and development style of (some) principal gnupg/libbcrypt developers:

          1. The fix release 1.9.1 apparently did not build on x86: https://twitter.com/FiloSottile/status/1355153432081031171
          2. Feature requests to use ASAN are closed: https://twitter.com/FiloSottile/status/1355194247482384392
          3. It appears that even a basic CI is missing, leave alone CI runs with sanitizers and static code analysis (e.g. Facebooks infer)

          I really like to switch away from gpg as fast as possible. Luckily there are already some alternatives, like https://sequoia-pgp.org/, emerging.

          1. 1

            I would also look at age: https://github.com/FiloSottile/age

            1. 2

              This does not seem to be OpenPGP based, which rules it out as gpg replacement.

          1. 3

            Yes for my own site the RSS feed is the most downloaded thing by far. I don’t include the page content so bandwidth is not a problem but it is weird that many readers are clearly ignoring the TTL and just requesting it over and over.

            1. 4

              Misconfigured readers/central aggregators have been a bane of RSS almost since the beginning. Developers don’t always read specs, or test.

              1. 9

                Even worse, commonly used readers like ttrss adamantly refuse to do the bare minimum like using etags. ttrss makes requests every 15 minutes to a feed that only really updates once per day at most. I’d have to convince all of my readers that use ttrss to fix their settings instead of them being a good net citizen and following recommendations to make the protocol work better.

                1. 6

                  That’s horrific. I would even say antisocial. Shit like this degrades the entire ecosystem — RSS is already kludgy enough as it is without developers petulantly refusing to adopt the few scalability measures available.

                  Plus, the developer’s response is childish:

                  … get a better hosting provider or something. your sperging out over literal bytes wasted on http redirects on your blog and stuff is incredibly pathetic.

                  Pathetic, indeed.

                    1. 2

                      I kind of liked ttrss until I understood how the developer acts in general. I’ve moved to miniflux since.

                      I’ve also understood that ttrss had some issues regarding security that the developer just refused to address or fix, due to reasons.

                    2. 1

                      Tempting to vary on a response header so one could prepend a bad netizen warning post to the top of the list for those readers that are being problematic.

                      1. 1

                        Having HTTP features (cache-control and last-modified) duplicated in the RSS spec is really annoying. Developers don’t want to write a custom cache for RSS feeds. I don’t know why supporting redundant caching measures encoded in XML would make a piece of software better. Why wouldn’t a HTTP cache be sufficient?

                        1. 3

                          AFAIK we are talking about HTTP caching in this thread. There are sites that don’t include headers like Etag or even Last-Modified, and there are clients that ignore them and just send unconditional requests every time.

                          There are one or two RSS properties related to this, but the only one I remember specifies a suggested poll interval. That does overlap to some degree with Cache-Control, but I don’t remember what the reasoning was for it. I’m certainly not going to defend RSS as a format, it’s pretty crude and the “official” specs are awful.

                        2. 1

                          haha! tiny tiny rss happens to be the #1 useragent for my site!

                    1. 3

                      This is fine, but it does mean you cannot use the browser for GET requests. This may be a good thing as it pushes people to use curl, postman, etc.

                      1. 1

                        It strikes me that writing these distributed storage doohickeys without too many interesting bugs is incredibly difficult.

                        One thing I wonder about. If you ran the same kind of fault injection tests with ASAN or UBSAN compiled Scylla binaries, would that unearth potential crash/corruption bugs in rarely exercised code paths?

                        Great writeup as always. Thank you @aphyr. ♥️

                        1. 7

                          Simulation makes many of these distributed race conditions pop out in just a few milliseconds after hitting save. If you begin your distributed system codebase with the knowledge that race conditions need to be avoided, it’s actually pretty straightforward to tease them out before you ever open a pull request that introduces them. This technique, also known as discrete event simulation, is a more common technique in places where safety is prioritized, like automotive software. FoundationDB is a prominent user of this technique, and their track record speaks to the results.

                          But most distributed systems creators build their systems in a fundamentally bug-prone way that obscures their race conditions from them, and they tend to put-off race condition testing, leaving them with no alternative other than these black-box techniques that are far, far less efficient both in CPU and human terms, yet nevertheless almost always yield paydirt because of the poor engineering practices that the codebase started with. Kyle will continue to have success for a long time :)

                          1. 2

                            I am not an expert in this space. It reads to me like you are suggesting that people establish a light interface that abstracts over the networking details. This interface (for example: receive and tick functions) allow people to write unit, property, fuzz, etc tests that leverage this interface to exercise the system. This sounds very reasonable.

                            On a somewhat related note, it frustrates me to no end that very few web frameworks allow me to inject a request and get back out a response without going over the network.

                            1. 3

                              It reads to me like you are suggesting that people establish a light interface that abstracts over the networking details.

                              Kind of — though I’m not sure “abstracts over” is the best way to describe it, or even that “simulation” or “simulator” is the best way to describe what’s actually being suggested. I think the lede is buried here:

                              write your code in a way that can be deterministically tested on top of a simulator

                              The important thing is to implement each component of your system as an isolated and fully deterministic state machine. Nothing should be autonomous, everything should be driven by external input. This requires treating the network as a totally separate layer — something that provides input to, and receives output from, the system. It also requires treating time itself as a dependency, which can have significant impact on design choices. For example, you can’t have a component that does something every N seconds, you have to model that as e.g. a function that should be called every N seconds. Or, you can’t use an HTTP client that has an N second request timeout, you have to e.g. use requests that are explicitly canceled by the caller after they determine a timeout period has elapsed.

                              This doesn’t require your system to follow the actor model, where components interact by sending and receive messages. I think the specific framing of the article kind of obscures that.

                              If every component in the system is a side-effect-free state machine, then everything is also perfectly isolated. That means you can construct arbitrarily complex topologies, drive them with whatever sequence of operations you can construct, and verify system behavior, all in-memory and for cheap. This is I guess what’s meant by simulation. But, at least to me, that’s the obvious consequence of the design, it’s the design that’s worth focusing on.

                              1. 3

                                I need to rewrite my simulation post to make a few aspects more clear, I think. Determinism is orthogonal to simulability. You get the race condition detection with some function that “scrambles” events that are concurrent. This can be accomplished with a simpler network abstraction and on the whole is far cheaper to implement than determinism, and it’s far far cheaper to retrofit into a legacy system than full determinism, yet it’s what yields bug discovery paydirt.

                                Determinism is what lets you get a non-flaky regression test in some cases “for free” which counts as a mechanically generated test. For this property to hold, your deterministic business logic must interact with your deterministic testing in such a way that when you modify your business logic or tests, it does not change the “history” that the test induces. A common example of a type of deterministic test that nevertheless fails to hold this nice property is to use a simple random number seed as the deterministic replay key. When business logic or test logic changes, the seed causes different behavior. The seed will still be nice for deterministic debugging before code changes, but it’s not useful for verifying that the bug has possibly been addressed if the seed causes no failure after a code change.

                                For something that preserves history a bit better, I tend to opt for tests that generate a sequence of actions, rather than a simple replay seed. This is still not enough to guarantee that your deterministic test is actually a regression test, but it’s a step in the right direction.

                                So, simulation shuffles concurrent events during test, causing many interesting bus to jump out very quickly. Determinism gives you replay for helping with understanding the issue. Sequence-of-events tests is nice for preserving history and improving the regression test nature of a mechanically generated test. Each of these combined has been known in engineering for decades as “discrete event simulation”.

                                1. 2

                                  Determinism is orthogonal to simulability.

                                  True. But reading your post makes me think neither determinism or simulability (woof, what a word) is actually the thing to focus on. They both feel like obvious emergent properties of a system that models all of its dependencies explicitly. That feels like the important thing — externalizing and injecting dependencies.

                        1. 5

                          “ We are diving deep into machine learning in PHP and…”

                          I’m sorry, what? Why PHP? Why not use Python and the bevy of ML tools already available? I mean, I get adding another language to the stack is not ideal but libraries like numpy are invaluable.

                          1. 14

                            Ted made a classic mistake in unsafe code (one I’ve made myself). This is why the Rust community normatively discourages writing unsafe code if it can be avoided. If I were a code reviewer, I’d be paying specific attention to any module containing unsafe code.

                            It’s worth saying that the as_ptr documentation explicitly notes this issue, and another option instead of the one shown would be to use CString::into_raw which you can safely cast from *mut c_char to *const c_char.

                            1. 7

                              I’m not a Rust developer, but it seems to me Ted’s point here is that it seems this scenario would be harder to catch by the code reviewer paying special attention, while any reviewer of Go code would see the opening/accessing of a resource (even if they were themselves on shaky ground with unsafe code) and think “that next line better be a defer statement to clean up”.

                              1. 13

                                First, I want to say I appreciate Ted for voicing his experience with Rust in this area, and I do think he hit this speed bump honestly, did his best to resolve it, and is understandably displeased with the experience. Nothing I say here is intended as a judgment of Ted’s efforts or skill. This is a hard thing to do right, and he did his best.

                                To your comment: I agree that is part of his point. I don’t agree in his assessment that the Go code is “clearer” [1]. In Rust, any unsafety is likely to receive substantial scrutiny, and passing clippy is table-stakes (in this case, Clippy flags the bad code, which Ted notes). I view the unsafe presence in the module as a red-flag that the entire module must be assessed, as the module is the safety boundary due to Rust’s privacy rules (see, “Working with Unsafe” from the Rustonomicon).

                                That said, Rust can improve in this area, and work is ongoing to better specify unsafe code, and develop norms, documentation, and tooling for how to do unsafe code correctly. In this case, I’d love to see the Clippy lint and other safety lints like it incorporated into the compiler in the future.

                                [1] Assessing “clarity” between two languages is really tough because clarity is evaluated in the context of socialization around a language’s norms, so a person can only effectively make the comparison if they are at least sufficiently knowledgeable in both languages to understand the norms around code smells, which I don’t think Ted is.

                                1. 5

                                  Is clippy used that pervasively? I’d rather have that particular warning in the compiler itself, in this case. Clippy seems to be super verbose and I don’t care too much for a tool that has strong opinions on code style.

                                  1. 8

                                    Feel free to file an issue to uplift the warning into rustc: Clippy warnings, especially deny-by-default clippy warnings – are totally fair game for uplifting into rustc if they’re seen as being useful enough.

                                    Clippy has a lower bar for lints, but for high value correctness lints like this one the only reason a lint is in clippy is typically “rustc is slower to accept new lints”, and if a lint has great success from baking in clippy, it can be upstreamed.

                                    People use clippy quite pervasively. Note that in this case the clippy lint that is triggered is deny-by-default, and all deny-by-default clippy lints are “this is almost definitely a bug” as opposed to “your code looks ugly” or whatever. You can easily turn off the different classes of clippy lints wholesale, and many people who want correctness or perf lints but not complexity or style lints do just that.

                                    1. 2

                                      In my experience, yes Clippy is used pervasively (but maybe I’m wrong).

                                      1. 1

                                        We have a few Rust projects at $work and we use cargo clippy -- -D warnings as part of our builds. It’s a very popular tool.

                                  2. 3

                                    This is not about unsafe code as this code works in safe rust but it is still not correct. Proof

                                    I think using as_ptr (and similar functions) in this way should 100% be an error.

                                    1. 1

                                      However, there’s no bug in the example you’ve written as the pointer is never dereferenced.

                                    2. 1

                                      Be careful (to other readers) that into_raw used in the same way here would leak the memory. Really as_ptr is very suspicious in any code, in exactly the same way that c_str is in C++.

                                      1. 1

                                        Unable to edit the post anymore, but others are right to point out that I missed part of the point here, and it’s important to note that CString::into_raw will leak the memory unless you reclaim it with CString::from_raw, and that CString::as_ptr is a challenging API because it relies on the lifetime of the original CString in a way that isn’t unsafe per Rust’s formal definition, but is still sharp-edged and may easily and silently go wrong. It’s caught by a Clippy lint, but there are some for whom Clippy is too aggressive.

                                        Thanks @hjr3 and @porges.

                                      1. 2

                                        I feel like this writer just picked a tool vendor whose goals didn’t align with his own. It seems like a matter of pros and cons, not right and wrong. Hasn’t Google always been less inclined to backward compatibility than Amazon and Microsoft? They didn’t even keep the lights on at Google Reader when it was the world’s leading RSS product.

                                        1. 11

                                          And this article is arguing that Google’s attitude towards backwards compatibility is why GCP is trailing behind AWS and Azure. The examples may be cherry picked but the point stands.

                                          1. 4

                                            It’s also of note that the author used to work for Google, so trying out GCP would have been a natural thing for him.

                                            Also, who out there would want cloud products where there’s a random chance of one of them getting a deprecation notice every 3-6 months (where deprecation means that a service is just gone), and generating that much churn? Like, that seems extreme. I don’t think even the Rails community, which does do more deprecations and that has more version to version churn, would want to deal with that.

                                          1. 1

                                            Amazing work. I thought PHP was going the way of Perl. The resurgence in the community and the speed of development has been astonishing.

                                            1. 22

                                              And no, you don’t have to learn assembly and start writing your web application in that. A good way to start would be to split up libraries. Instead of creating one big library that does everything you could ever possibly need, just create many libraries. Your god-like library could still exist, but solely as a wrapper.

                                              That’s exactly what the linked “dependency hell” example did. They imported create-react-app, which is a thin wrapper combining 15 other dependencies, which each wrap a few more, which wrap a few more… Also, I have no idea how he got 13,000 dependencies when the package-lock only has 7,400 lines.

                                              Also, node’s style of “many one-line dependencies” is specific to node, as it was encouraged by the npm company as best practice. Saying “Developers … have also become lazy” is unwarranted.

                                              1. 4

                                                The output of npm is a bit misleading with regards to dependencies installed. People looking to make a point about too many dependencies often quote that number instead of looking at the real dependency count.

                                              1. 3

                                                I encourage anyone writing an API to also write a client for that API. Experiencing how the client uses the API is invaluable. You can also use it in your functional API tests!

                                                1. 1

                                                  How come this wasn’t the default in a language with a type system as soft as ice cream in the summer?

                                                  1. 6

                                                    Early PHP was essentially gluing functoins from C into an HTML template. Back then, the convention was to make the PHP function definition the same as the C function definition. This made sense at the time because the people who were using PHP were already familiar with C.

                                                    There were no named arguments in C, thus there were no named arguments in PHP.

                                                    1. 1

                                                      Not sure if the first version of PHP had functions in the first place. The announcement does not mention any kind of programming syntax:

                                                      https://groups.google.com/forum/#!msg/comp.infosystems.www.authoring.cgi/PyJ25gZ6z7A/M9FkTUVDfcwJ

                                                      1. 1

                                                        True, you have a point. Still, if you are going to add functions, why add such a poor implementation?

                                                    1. 1

                                                      Why not just use an atomic for the second example?

                                                      1. 7

                                                        It’s rather “why integer counter, instead some more real world data structure?”, as I wanted to show mutex specifically.

                                                        Similar point can be made about atomics: Rust allows non-atomic load at the end, but requires atomic loads from other threads. But I felt that a mutex is a better illustration because it is more boring, and because it shows nice RAII guard API.

                                                        1. 3

                                                          The second example extends to any data type.

                                                        1. 6

                                                          Most linters or IDEs will catch that you’re ignoring an error

                                                          I’m not always using linters or IDEs. Does that mean that I should be able to just ignore the error? I don’t think so

                                                          Plan for failure, not success

                                                          The default behavior in Go assumes success. You have to explicitly plan for failure

                                                          has sufficient context as to what layers of the application went wrong. Instead of blowing up with an unreadable, cryptic stack trace

                                                          I don’t know how about you, but math: square root of negative number -15 doesn’t exactly tell my why the error occurred. I’d rather look into that “cryptic” stack trace to find out where that negative number got used.

                                                          1. 0

                                                            math: square root of negative number -15

                                                            math.Sqrt(-15) = NaN. I have no idea where you’ve seen that particular error message. Cases like that are exceptional and are a programmer’s mistake (e.g. division by zero), therefore Go would panic with a full stack trace.

                                                            But let’s assume that there is some square root function that returns an error. We usually add annotations to error messages, e.g. in Go ≥1.13 fmt.Errorf("compute something: %w", err) and later check that errors.Is(err, math.ErrNegativeSqrt), so instead of

                                                            math: square root of negative number -15
                                                            

                                                            you’d actually see

                                                            solve equation: compute discriminant: square root of negative number -15
                                                            

                                                            which is way more descriptive than a stack trace if there are multiple similar error paths along the call stack (from my experience, that’s almost always true). With exceptions you’d have to carefully follow the stack trace line-by-line to find the cause of an error.

                                                            Also, in many languages discovering that a particular function throws an exception without code analyzers is pretty hard.

                                                            1. 5

                                                              I have no idea where you’ve seen that particular error message.

                                                              I just took it as an example.

                                                              We usually add annotations to error messages

                                                              Usually. But the default for go is just return err. Which leaves you with the problematic one.

                                                              which is way more descriptive

                                                              I don’t know how about you, but in a project with >10k LOC I’d like to see, in which exact file do you have to open for an error. I won’t always know the project by heart, and to find it normally would take a good couple of minutes.

                                                              if there are multiple similar error paths along the call stack

                                                              Uhhh, you just look into the first function along the call stack and go along in such case. It’s not that difficult. I know it can look scary, but stack traces usually have all the information that you need, and then some. Just learning their syntax will allow you to efficiently find causes of bugs. That cannot be said for Go, as different projects can use different styles of error annotations.

                                                              With exceptions you’d have to carefully follow the stack trace line-by-line to find the cause of an error.

                                                              I usually read less than half of it before realizing where the error lies. There is a ton of information in a call stack, and you don’t always need all of it.

                                                              Also, in many languages discovering that a particular function throws an exception without code analyzers is pretty hard.

                                                              In a simple case, looking for raise/throw/etc. is enough. But yes, this is a problem. Some languages do have a solution, e.g. Java, but those aren’t that common yet. I’d like improvement here.

                                                              1. 3

                                                                What happens if “solve equation” calls “compute discriminant” multiple times? There is no line number telling me where it was called.

                                                                My preference is a good error message with a stack trace. I can choose to ignore the stack trace when looking at logs/output. Also, (in some languages) a stack trace can be generated without an exception.

                                                                1. 1

                                                                  line numbers assume a few things:

                                                                  • you have the same version of the code everywhere. in a production environment, you can easily have multiple versions of the same thing during deployments, upgrades, etc. Those situations are the most common situations in which you would encounter an error for the first time since, as we just said, you’re deploying potentially new code.
                                                                  • you’re only talking about comprehending the error data as it happens, and not historically, in a log file somewhere. What if you’re looking at log data from a few days ago? Now you have to cross-reference your stack trace with what version of the app was running at the time that stack trace was generated. Ok, now check out the code at that time. Now step through the line numbers and run the program in your head to try to understand the story of how the error happened. -OR- just … read the error’s story of how it happened in a Go program.
                                                                  • a stack trace is generally not going to capture local variables. Even if you know what line you’re on, you might not know what parameters that function was called with. With explicit wrapping of error values, you append to the error value the necessary contextual information, as decided by a human author, for a human to understand the cause of the error.

                                                                  I don’t miss exceptions and stack traces after using Go for a few years. Both are, in my experience, actively worse than handling errors at the call site and wrapping errors with descriptive contextual information.

                                                                  1. 2

                                                                    I use both error context and stack traces; you can add stack traces to errors quite easily (about 20/30 lines of code) and it never hurts to have more information. In some cases there are performance problems with stack traces (an issue in any language), but those are rare enough that I’m not overly worried about it.

                                                                    Cases where error context fails for me is when I’ve accidentally added identical context to two errors. This is usually a mistake on my part (copy/paste, or just laziness) and having the stack trace is helpful. I also find it easier in development because I can just go to “foo.go line 20”.

                                                                    I replaced all pkg/errors calls with some sed-fu to Go 1.13 errors, but I added back stack traces after a few weeks with a small library (about 100 lines in total).

                                                                    Either way, I don’t think context and stack traces need to be exclusive, and there is value in both in different contexts.

                                                            1. 14

                                                              TL;DR: the article asks “how often does Rust change?” And the answer is “every six weeks”. Regardless of how big the changes are, you have to check every six weeks to see if something changed. Like someone said the other day on Lobsters, it’s like getting your pizza sliced into 42 slices versus 12. Sure it’s the same amount of pizza but the cognitive load is higher.

                                                              The endless release posts with tons of stuff in them makes it feel like a lot is happening.

                                                              TFA seems to argue that yes, Rust changes a lot, but they aren’t “big” changes.

                                                              The problem is, big or not, I’m trying to learn the language and it feels like a moving target. As a neophyte I don’t know what’s important about the new version and what isn’t.

                                                              It’s also interesting to note the number of language changes in each release. Standard library changes are one thing, but language changes make things more difficult to learn.

                                                              For example, from the release of Python 3.5 (which introduced explicit async/await syntax, the last major syntax revision) in late 2015, there’s been around 40 Python releases across both Python 2.x and Python 3.x. If you only track Python 3, there were around 30 releases. The majority of these releases were bug fixes, with no or minimal standard library changes. There were of course some big standard library changes, but they were the exception.

                                                              In the same time frame, there were around 65 releases of Rust. These changes were not just bug fixes but all over the place.

                                                              My point is, with Rust I’d have to go through twice as many release notes and while the list of changes is generally small, it’s not always clear what’s a big change and what isn’t. With Python, it’s obvious that “fixed bug in how whatever works” is a bug fix. In Rust “this macro can now be applied to structures” doesn’t mean anything to a neophyte. Is that a big change? A small one? I don’t know.

                                                              Of course you can find counter examples in Python releases and bug fixes in Rust releases, but it just feels different. It feels impossible to keep up.

                                                              Compare to languages like C, where there were literally no changes between the last two standard revisions (C11 and C18), or Go which has had around 10 significant releases in five years and an explicit language standard that allows alternative implementations. (Go has had more than ten minor releases in that time, but anything where only the minor version changes are bug fixes).

                                                              I really like Rust. The type system is beautiful. The ownership model is gorgeous and innovative. The toolchain is fantastic. I just feel like using it would be on a treadmill, every six weeks poring over release notes to see what changed. Sure I can stick to Rust 2018, but by doing that I’ve run into third party code that used newer features that I didn’t know about. I’ve also run into trouble with distro-packaged Rust being “too old” to build something I find online. Sure I can use rustup, but I like having the OS packaged and supported tools.

                                                              (Again, of course, other languages have problems with the packaged version being too old but I’ve used Python for many years and not had that many problems and Rust for a couple of months and run into it twice).

                                                              I really, truly, think that Rust’s rapid release schedule is my and perhaps others’ primary barrier to adoption.

                                                              (Also, and this is a minor thing, but one of the major Rust contributors tweeted (or retweeted) a thing about how “COBOL was designed by women and just works and C was designed by men and it’s impossible to use securely.” Despite being inaccurate it also struck me as really unwelcoming. I was following that person’s Twitter feed specifically to start participating in the Rust community and that did not encourage me.)

                                                              1. 14

                                                                Regardless of how big the changes are, you have to check every six weeks to see if something you rely on changed.

                                                                Or else what? If you only check every 12 weeks and read a pair of posts at a time, or only check once a year, what goes wrong? Your code will keep on working.

                                                                You miss out on anything new and nice, but only as much as if Rust was releasing less often.

                                                                1. 7

                                                                  Well partially because it’s not just my code.

                                                                  If I want to use a third-party crate (which is almost a given because of the way Rust’s ecosystem is designed*), I need to audit that crate to make sure it’s secure and correct for my purposes. A rapidly-changing language makes it harder for me to read other people’s code regardless of how static my code is.

                                                                  * this is not necessarily a negative thing

                                                                  1. 10

                                                                    Among the changes Steve enumerated here, are there any that you feel would impact your ability to audit code without having known about the feature ahead of time?

                                                                    I would think if you are auditing and encounter something unfamiliar like const fn or dyn Trait or some new standard library method, it’s easy enough to then learn about it via search and the Rust documentation.

                                                                    For standard library methods, by far the main way people learn about them is not from release posts. They find them in documentation when searching for that functionality, or look it up when they see it in someone’s code. But in a safe language this works for the things classed as “language changes” as well. The feeling of being compelled to read the release notes isn’t well founded in my experience.

                                                                    1. 4

                                                                      The ?, for example, was a problem for me getting started. I’d never seen it before and while I understand its purpose and it makes great sense, I then had to go say “okay this way of doing things that I’d already read about and started getting a handle on has two ways of expressing it.”

                                                                      It’s not the end of the world, of course, or that difficult to understand, it just seems like there are a lot of little changes like that that increase cognitive load for me.

                                                                      Again, this is just me and my opinion. Obviously Rust is very popular and this release strategy works for them. I’m just pointing out what has been a difficulty for me and, given the articles posted recently here, for some others too.

                                                                      1. 6

                                                                        You now shifted from “changes every 6 weeks” is a problem to “this syntax change was a problem”. I believe you that ? Was confusing at first. However, the post shows that syntax changes are rare. If and when you do see it, you go look up what it is. No way that is happening every 6 weeks.

                                                                        1. 13

                                                                          You now shifted from “changes every 6 weeks” is a problem to “this syntax change was a problem”.

                                                                          You asked for an example from a limited set of possibilities, and I gave one. I didn’t imply my list was exhaustive. You then accuse me of changing my argument…because I gave you an example from one of the possibilities you asked and explained how I felt about it as a new user of the language.

                                                                          I’ve said several times that this release schedule seems to work for Rust and a lot of people like it. I’ve also said that it seems to be a barrier to adoption for some people, myself included. I’m choosing to try to overcome because I think Rust is a fascinating and worthwhile language.

                                                                          I don’t appreciate the implication that I’m somehow lying in my impression of a programming language or trying to “win” some debate here with gotcha tactics. That Rust changes often is a concern that a lot of people seem to have. As someone who has decided to learn the language, my impression is that I think this argument holds some water. I’ve presented my feelings in response to a topical post regarding the same issue.

                                                                          1. 2

                                                                            I’m not denying that try/? change was an issue for you, but how could Rust avoid causing you that problem?

                                                                            You’re saying that it should release less frequently, but I don’t see how that avoids the fact that it changed at all. If Rust didn’t change ? in 2016, it would change it in 2018. The result in both cases is the same: you — a Rust 2015 user — wouldn’t know the new syntax.

                                                                            1. 2

                                                                              Nobody expects a language to be static. It’s like I said above, it’s easier to understand larger release notes twice a year than smaller release notes eight times a year, at least for me and obviously some other people.

                                                                              There’s also the issue that, okay, it’s not Rust 2015….but it’s not Rust 2018 either. It’s something in between. I can’t say “oh yeah, that’s from Rust 2018,” I have to say “that’s from Rust 1.27.0” or whatever. There are many more versions of Rust to keep track of than just 2015 and 2018. I learn the ? syntax and that’s great…but that doesn’t teach me Rust 2018, it just teaches me part of one of eight or nine steps between the editions.

                                                                              (And maybe I’m wrong but there doesn’t seem to be any concrete sum documentation for the Rust editions. The documentation at rust-lang seems to track stable and there doesn’t appear to be a “Rust 2015” snapshot of the documentation except in dead-tree form. Please correct me if I’m wrong. That means that as I’m studying the language, the documentation changes out from under me, not a lot but some.)

                                                                              As I said above, a lot of people seem to like this sort of schedule with smaller changes more often. That’s great.

                                                                              What I feel like some Rust advocates aren’t getting is that some people don’t like this schedule and the Edition mechanism doesn’t seem to totally fix the problems they have with it…and that’s okay. People are allowed to say “I like Rust but I’m having trouble with this part and it’s a major pain point.” Rather than try to “prove them wrong” about how they feel about the release schedule maybe acknowledge that it can be a problem for some people.

                                                                              1. 2

                                                                                Why do you have to know from which version a thing is? I get that in C or C++ you have to know when each feature has been added, because there are different camps that settle on different past decades, but in Rust there’s no such thing. If you know a feature has been released, you can use it.

                                                                                there doesn’t seem to be any concrete sum documentation for the Rust editions

                                                                                There has been only one so far, and it’s here: https://doc.rust-lang.org/edition-guide/

                                                                                The documentation at rust-lang seems to track stable and there doesn’t appear to be a “Rust 2015” snapshot

                                                                                Old docs are archived here: https://doc.rust-lang.org/1.30.0/

                                                                                But “Rust 2015” isn’t a version of Rust. It’s not a thing that can be targetted or snapshotted. It’s a parsing mode that changes interpretation of a few bits of syntax. It’s like "use strict" in JavaScript or Quirks Mode in HTML. “2015” parsing mode in latest Rust has all the new features of the latest Rust, except a couple that would conflict with some keywords/syntax in 2015.

                                                                                The Rust team created a lot of confusion around this by marketing “cool new features we’ve released in the past year you may have missed” together with the “new parsing mode” switch, creating a false impression that “2015” and “2018” are different languages. They’re the same language with the same standard library, but “2015” mode can use async as a variable name.

                                                                2. 2

                                                                  Thanks for chiming in with this view. It makes a lot of sense.

                                                                  However, I also feel that the Internet encourages this sort of firehose-style of releases. To many developers, a slow release cadence indicates project stagnation. And, to them, stagnation does not imply stability, but rather, a dead-end. I’m pretty sure the rust team puts releases frequently for other reasons that are more important, but there are virtues to signaling frequent project activity. IOW, I don’t think the Rust core team caters to the “always be committing” crowd, but they do try to stay on the collective radar.

                                                                  FWIW I have been on both sides of the module change and I lost a few hours to trying to understand it. But that’s been about it so far. Haven’t ventured into asynchronous stuff yet.

                                                                  1. 1

                                                                    Worth pointing out that Python has been around since the early 90s. How much did Python change at an equivalent point in its lifecycle? How would Python look if it had been locked in amber at that point?

                                                                    1. 10

                                                                      People get hung up a lot on Python 2/3, but forget how much Python changed prior to that. During the Python 2.x series, the language gained:

                                                                      • The bool type
                                                                      • Opt-in unified type hierarchy and “new-style” classes with full operator overloading
                                                                      • Context managers/the with statement
                                                                      • Unified integral type
                                                                      • Integral versus “true” division
                                                                      • The set type
                                                                      • The logging module
                                                                      • The unittest and doctest modules
                                                                      • The modern import-hook system
                                                                      • Lazy iterables and the itertools module
                                                                      • Generator expressions
                                                                      • The functools module
                                                                      • Ternary conditional expressions
                                                                      • Absolute versus relative imports
                                                                      • Decorators
                                                                      • The format() method and formatting mini-language
                                                                      • The collections module
                                                                      • Abstract classes and methods

                                                                      And on and on… sure, you could say that code written for Python 2.0 still worked on Python 2.7 (which wasn’t actually true – there were deprecations and removals in the 2.x series!), but idiomatic Python 2.7 contains so many things that would be unrecognizable to someone who knew only idiomatic 2.0 that it might as well be a completely different language.

                                                                      1. 2

                                                                        No doubt, but early Rust was very different from modern Rust (there was a garbage collector at one point, major differences in the type system, etc).

                                                                        I’m not saying Rust doesn’t deserve its youthful vigor and changes because of course it does. I also don’t think Python was as popular that early in its life as Rust is, relatively speaking (at least from my fuzzy memories of those ancient days where I worked mostly in Perl and C). Python didn’t really explode to be everywhere until after the 2.0 release, IIRC. Python 2.0 and Python 3.8 are of course pretty different, but there’s something like 20 years there versus ten for Rust with I would argue even greater changes.

                                                                        Regardless of the relative age of the languages, I do think Rust’s change schedule is a barrier to adoption for a lot of people (with the anecdotal evidence of the couple of articles posted recently here talking about just that).

                                                                        Again, that’s just me. I’m sure a lot of people like Rust’s release schedule and it seems to be working well given Rust’s rising popularity.

                                                                        1. 0

                                                                          Why would you compare that to Rust’s changes now and not to Rust’s pre-1.0 changes?

                                                                      1. 2

                                                                        I prefer https://github.com/ulid/spec instead. It is sortable and does not wreck indices.

                                                                        1. 4

                                                                          Lots of good things were originally unintended or semi-intended results of technical limitations. The /usr split is still a good idea today even if those technical limitations no longer exist. It’s not a matter of people not understanding history, or of people not realising the origins of things, but that things outgrow their history.

                                                                          Rob’s email is, in my opinion, quite condescending. Everyone else is just ignorantly cargo-culting their filesystem hierarchy. Or perhaps not? Perhaps people kept the split because it was useful? That seems a bit more likely to me.

                                                                          1. 19

                                                                            I’m not sure it is still useful.
                                                                            In fact, some linux distributions have moved to a “unified usr/bin” structure, where /bin, /sbin/, and /usr/sbin all are simply symlinks (for compatibility) to /usr/bin. Background on the archlinux change.

                                                                            1. 2

                                                                              I’m not sure it is still useful.

                                                                              I think there’s a meaningful distinction there, but it’s a reasonable decision to say ‘there are tradeoffs for doing this but we’re happy with them’. What I’m not happy with is the condescending ‘there was never any good reason for doing this and anyone that supports it is just a cargo culting idiot’ which is the message I felt I was getting while reading that email.

                                                                              In fact, some linux distributions have moved to a “unified usr/bin” structure, where /bin, /sbin/, and /usr/sbin all are simply symlinks (for compatibility) to /usr/bin. Background on the archlinux change.

                                                                              I’m not quite sure why they chose to settle on /usr/bin as the one unified location instead of /bin.

                                                                              1. 14

                                                                                That wasn’t the argument though. There was a good reason for the split (they filled up their hard drive). But that became a non-issue as hardware quickly advanced. Unless you were privy to these details in the development history of this OS, of course you would copy this filesystem hierarchy in your unix clone. Cargo culting doesn’t make you an idiot, especially when you lack design rationale documentation and source code.

                                                                                1. 2

                                                                                  … it’s a reasonable decision to say ‘there are tradeoffs for doing this but we’re happy with them’. What I’m not happy with is the condescending ‘there was never any good reason for doing this and anyone that supports it is just a cargo culting idiot’ which is the message I felt I was getting while reading that email.

                                                                                  Ah. Gotcha. That seems like a much more nuanced position, and I would tend to agree with that.

                                                                                  I’m not quite sure why they chose to settle on /usr/bin as the one unified location instead of /bin

                                                                                  I’m not sure either. My guess is since “other stuff” was sticking around in /usr, might as well put everything in there. /usr being able to be a single distinct mount point that could ostensibly be set as read-only, may have had some bearing too, but I’m not sure.
                                                                                  Personally, I think I would have used it as an opportunity to redo hier entirely into something that makes more sense, but I assume that would have devolved into endless bikeshedding, so maybe that is why they chose a simpler path.

                                                                                  1. 3

                                                                                    My guess is since “other stuff” was sticking around in /usr, might as well put everything in there. /usr being able to be a single distinct mount point that could ostensibly be set as read-only, may have had some bearing too, but I’m not sure.

                                                                                    That was a point further into the discussion. I can’t find the archived devwiki entry for usrmerge, but I pulled up the important parts from Allan.

                                                                                    Personally, I think I would have used it as an opportunity to redo hier entirely into something that makes more sense, but I assume that would have devolved into endless bikeshedding, so maybe that is why they chose a simpler path.

                                                                                    Seems like we did contemplate /kernel and /linker at one point in the discussion.

                                                                                    What convinced me of putting all this in /usr rather than on / is that I can have a separate /usr partition that is mounted read only (unless I want to do an update). If everything from /usr gets moved to the root (a.k.a hurd style) this would require many partitions. (There is apparently also benefits in allowing /usr to be shared across multiple systems, but I do not care about such a setup and I am really not sure this would work at all with Arch.)

                                                                                    https://lists.archlinux.org/pipermail/arch-dev-public/2012-March/022629.html

                                                                                    Evidently, we also had an request to symlink /bin/awk to /usr/bin/awk for distro compatability.

                                                                                    This actually will result in more cross-distro compatibility as there will not longer be differences about where files are located. To pick an example, /bin/awk will exist and /usr/bin/awk will exist, so either hardcoded path will work. Note this currently happens for our gawk package with symlinks, but only after a bug report asking for us to put both paths sat in our bug tracker for years…

                                                                                    https://lists.archlinux.org/pipermail/arch-dev-public/2012-March/022632.html

                                                                                    And bug; https://bugs.archlinux.org/task/17312

                                                                              2. 18

                                                                                Sorry, I can’t tell from your post - why is it still useful today? This is a serious question, I don’t recall it ever being useful to me, and I can’t think of a reason it’d be useful.

                                                                                1. 2

                                                                                  My understanding is that on macOS, an OS upgrade can result in the contents of /bin being overwritten, while the /usr/local directory is left untouched. For that reason, the most popular package manager for macOS (Homebrew) installs packages to /usr/local.

                                                                                  1. 1

                                                                                    I think there are cases where people want / and /usr split, but I don’t know why. There are probably also arguments that the initramfs/initrd is enough of a separate system/layer for unusual setups. Don’t know.

                                                                                    1. 2

                                                                                      It’s nice having /usr mounted nodev, whereas I can’t have / mounted nodev for obvious reasons. However, if an OS implements their /dev via something like devfs in FreeBSD, this becomes a non-issue.

                                                                                      1. 2

                                                                                        Isn’t /dev an own mountpoint anyways?

                                                                                        1. 1

                                                                                          It is on FreeBSD, which is why I mentioned devfs, but idk what the situation is on Linux, Solaris and AIX these days off the top of my head. On OpenBSD it isn’t.

                                                                                          1. 2

                                                                                            Linux has devtmpfs per kernel default.

                                                                                  2. 14

                                                                                    The complexity this introduced has far outweighed any perceived benefit.

                                                                                    1. 13

                                                                                      I dunno, hasn’t been useful to me in the last 20 years or so. Any problem that it solves has a better solution in 2020, and probably had a better solution in 1990.

                                                                                      1. 6

                                                                                        Perhaps people kept the split because it was useful? That seems a bit more likely to me.

                                                                                        Do you have a counter-example where the split is still useful?

                                                                                        1. 3

                                                                                          The BSDs do have the related /usr/local split which allows you to distinguish between the base system and ports/packages, which is useful since you may want to install different versions of things included in the base system (clang and OpenSSL for example). This is not really applicable to Linux of course, since there is no ‘base system’ to make distinct from installed software.

                                                                                          1. 3

                                                                                            Doesn’t Linux have the same /usr/local split? It’s mentioned in the article.

                                                                                            1. 5

                                                                                              I tend to rush for /opt/my-own-prefix-here (or per-package), myself, mainly to make it clear what it is, and avoid risk of clobbering anything else in /usr/local (like if it’s a BSD). It’s also in the FHS, so pedants can’t tell you you’re doing it wrong.

                                                                                              1. 4

                                                                                                It does - this is generally used for installing software outside the remit of the package manager (global npm packages, for example), and it’s designated so by the FHS which most distributions follow (as other users have noted in this thread), but it’s less prominent since most users on Linux install very little software not managed by the package manager. It’s definitely a lot more integral in BSD-land.

                                                                                                1. 3

                                                                                                  […] since most users on Linux install very little software not managed by the package manager

                                                                                                  The Linux users around me still do heaps of ./configure && make install; but, I see your point when contrasted against the rise of PPAs, Docker and nodenv/rbenv/pyenv/…

                                                                                                  1. 3

                                                                                                    Yeah, I do tons of configure make install stuff, sometimes of things that are also in the distro - and this split of /usr/local is sometimes useful because it means if I attempt a system update my custom stuff isn’t necessarily blasted.

                                                                                                    But the split between /bin and /usr/bin is meh.

                                                                                              2. 1

                                                                                                That sounds sensible. Seems like there could be a command that tells you the difference. Then, a versioning scheme that handles the rest. For example, OpenVMS had file versioning.

                                                                                          1. 2

                                                                                            Many of programs that were upgraded to be 64 bit compatible are now buggy. This makes Catalina feel more unstable.

                                                                                            1. 2

                                                                                              Implicits are one of the things I did not like about Scala. I prefer Python’s be explicit philosophy.

                                                                                                1. 2

                                                                                                  SMH, of course. Forgot they started blogging about this recently.

                                                                                                    1. 1

                                                                                                      Oh that’s nice.