1. 48
  1. 69

    I read this and felt weird about it initially but couldn’t quite put my finger on it. From my experience, using Rust has lead to faster developer velocity (even for CRUD apps) simply because the strong type system allows you to encode invariants to be checked at compile time, and the tooling and libraries are amazing. This article seems to be about a project that was being developed right around async/await release, which was a rough time for using libraries. Most of the sharp edges are gone now, but I don’t doubt that the state of the ecosystem at that point affected this person’s experience.

    However, I do think there is a situation where this advice holds true (even for someone as heavily biased as I am), which is for a very specific kind of startup: a hyper-growth startup with 100%+ YoY engineer hiring. The issue with Rust is not that its slower to develop in, I don’t think that is true, its that in order to develop quickly in Rust you have to program in Rust. And frankly, most new developers to Rust have no idea how to program in Rust because so many languages do not feature strong type systems. And the problem is that if your influx of new developers who need to learn Rust is too large, you won’t be able to properly onboard them. Trying to write Java using Rust is horrible (I’ve worked with a number of colleagues who I’ve had to gently steer away from OO design patterns that they were used to, simply because they make for really difficult Rust code, and are largely obsoleted by the type system).

    It isn’t even lifetimes or borrowing that are necessarily tricky, in my experience issues with lifetimes are fairly rare for people and they almost always immediately seek out an experienced Rust dev for guidance (you only need a handful to deal with all questions on Lifetimes; my current team only has me and its been a non-issue). The bigger problems are around how to structure code. Type-driven development is not something most people have experience with, so they tend to stick to really simple Structs and Enums, to their detriment.

    For instance, I commonly see new Rust developers doing something like this:

    fn double_or_multiply(x: i32, y: Option<i32>, double: bool) -> Result<i32> {
        if double {
            if y.is_some() {
                return Err("Y should not be set");
            }
            x * 2
        } else {
            if y.is_none() {
                 return Err("Y should be set");
            }
            x * y.unwrap()
        }
    }
    

    Yes, I know its a completely contrived example, but I’m sure you’re familiar with that kind of pattern in code. The issue is that this is using the shallow aspects of Rust’s type system – you end up paying for all of Rust but only reaping the benefits of 10% of it. Compare that to what you could do by leveraging the type system fully:

    enum OpKind {
       Double(x),
       Multiply(x, y),
    }
    
    fn double_or_multiply(input: OpKind) -> i32 {
        match input {
             Double(x) => x * 2,
             Multiply(x, y) => x * y,
        }
    }
    

    Note how the error has disappeared as there is no way to call this function improperly. That means fewer tests to write, less code to maintain, and APIs that can’t be used improperly. One of the most interesting questions I get commonly when I promote this style of code is “how do I test that it fails then?”; its always fun to explain that it can’t fail[1] and there is no need to write a test for the failure. The developer efficiency benefits from this style of thinking everywhere is massive and more than pays for the cost of using Rust.

    But developers from other languages take time to get to this point, and it does take time and effort from experienced Rust developers to get everyone on the same page. Too many new people and I can see more and more code leaking in as the first example, which means you get minimal benefits of Rust with all the cost.

    I can’t argue with this person’s experience, as much as I love Rust and think it has features that make it an incredibly productive language, but I think the core issue is that the majority of developers do not have experience with strong type-system thinking. As more languages start adding types I’m hopeful this problem becomes less prevalent, because the productivity differences between developers who understand type-driven development vs. those who don’t is large (in languages with strong type-systems).

    [1] Technically it can panic, which I do explain, but for the majority of cases that is a non-issue. Generally if there is a panic-situation that you think you might have to handle you use a different method/function which returns a Result and bubble that up. Panics are largely unhandled (and for good reason; they aren’t exceptions and should be considered major bugs).

    1. 11

      FWIW Matt Welsh is a very experienced programmer and former computer science professor at Harvard:

      https://en.wikipedia.org/wiki/Matt_Welsh_(computer_scientist)

      (I know of him from his research on SEDA, an influential web server architecture, as well as having a good blog.)

      So this comments strikes me as a bit out in left field … I don’t doubt it’s your experience, but it doesn’t seem relevant to the article

      1. 9

        I’m not familiar with who Matt Welsh is, but I found his post to be well written and accurate to his experience. My comment was simply a reflection of my own experience, which I think differs.

        I don’t see how my comment isn’t relevant to the article, but I am open to feedback if there is something specific you felt made my comment out of left field!

        1. 7

          Not GP, and I hope this doesn’t come off as too negative, but your comment is pretty dismissive of the idea that Matt Welsh could have substantive issues with the design of Rust. You seem to imply that the problems he ran into stem from a lack of experience:

          I think the core issue is that the majority of developers do not have experience with strong type-system thinking.

          Your example about double_or_multiply is great, but IMO it’s a pretty elementary example for most readers here, as well as Matt Welsh.

          The general vibe is like this: someone complains that a particular proof in graduate-level mathematics is hard to read, and then you respond with a tutorial on pre-calculus. I like your comment, and it is relevant, but it doesn’t feel like it responds to the article, or takes the author seriously.

          1. 5

            Thanks for the explanation, I do appreciate it. I hoped to not make my comment be dismissive of the article, and more as a refinement of which aspect it was talking about. I think the fundamental issue is that I completely disagree with Matt Welsh that programming in Rust lowers developer efficiency, and so my comment was exploring a reason for why he may feel that way.

            The example was simple, and more just for having something to ground my argument in (ie. Rust makes you faster because you write less code overall), as well as having something for developers who are unfamiliar with type-driven development to see. I didn’t mean to imply it as Matt Welsh doesn’t know that, but more that type-driven development is a completely different style of programming, and onboarding devs to a new language is easy but to a new style is hard.

            Clearly my point wasn’t made as clear as I had hoped, and thank you for pointing out where it felt like I was off-base. I do think it’s important to disagree, and I’ve never been one to defer to another just because of their accomplishments or prestige.

            I’m thinking it might make sense for me to write my own post on my thoughts on developing quickly in Rust, and hopefully I can take these ideas and your feedback and make something that pushes the conversation forward on what makes a productive language without coming across as dismissive of others experience :)

            1. 1

              You:

              …a hyper-growth startup with 100%+ YoY engineer hiring… in order to develop quickly in Rust you have to program in Rust. And frankly, most new developers to Rust have no idea how to program in Rust because so many languages do not feature strong type systems. And the problem is that if your influx of new developers who need to learn Rust is too large, you won’t be able to properly onboard them.

              Matt:

              We hired a ton of people during my time at this company, but only about two or three of the 60+ people that joined the engineering team had previous experience with Rust.

              This was in two years, which he claims was 10x growth in headcount, so from ~6 people to 60 in two years, with only 3 people who knew Rust. Basically, well above your 100% YoY hiring threshold for being able to onboard Rust engineers.

              You:

              I completely disagree with Matt Welsh that programming in Rust lowers developer efficiency

              I don’t think you do disagree with him :)

              My interpretation of his claim is that in a rapidly growing organisation that needs to ship product rapidly, and where you don’t have a lot of Rust expertise already, Rust may not be a good fit. What you are claiming is that given enough experience with Rust, it can accelerate your day to day development. But that to do so you need to give developers sufficient time to onboard to Rust such that they can become familiar and efficient with it (which of course is true for any language, but Rust onboarding likely takes a lot longer than, say, Go, or Python, or C#).

              Both of these claims can be true because they are discussing different aspects of software engineering at different scales.

              Related, I’d be interested in hearing what your experience has been on how long it takes to onboard a complete Rust novice, to the point they are at close to 100% productivity.

              1. 3

                My interpretation of his claim is that in a rapidly growing organisation that needs to ship product rapidly, and where you don’t have a lot of Rust expertise already, Rust may not be a good fit.

                You’re right, I do agree with this claim. My understanding of Matt’s article is that he argues this is a function of Rust as a language, whereas I argue this is a function of a general lack of understanding of type-driven programming.

                It is effectively semantics, but important semantics, because I don’t think the solution is to avoid using Rust/Haskell/OCaml/etc except in niche situations, it should be to educate developers in type-driven development. But of course that is a much bigger problem, and not easy, so I can see where one might think I’m just arguing a point entirely separate from what the article is talking about.

                I think you’re right, the point I am making is mostly tangential, and I could have made that more clear. Thanks for explaining your view!

                Related, I’d be interested in hearing what your experience has been on how long it takes to onboard a complete Rust novice, to the point they are at close to 100% productivity.

                I’ll take the easy way out and say it depends :P but it is hard to put an exact number on it simply because I don’t have a ton of data to really say. Rust is still fairly niche, even at companies using it (ie. Rust for core services, but most devs work on services in other langs talking to the Rust core), and usually devs wanting to work on the Rust stuff already know Rust to an extent. My current team is getting ~6 or 7 devs without Rust experience next quarter (who have no idea they’ll be working in Rust) so hopefully I’ll have much better data soon! (Absurd bureaucratic policies maybe have good side effects)

                From my limited experience onboarding people with no Rust experience, I’ve seen devs without type-driven design experience pick up a productive level in two to three months (including new-grads), but take a lot longer to get to being comfortable with using the type system to its full power (maybe a year to get mostly comfortable, but still have gaps). However, they’ve largely been passionate and qualified devs I’ve worked with (even the ones that were new-grads), so I think my opinion is biased here.

                I’d guess that a developer with strong type system experience can pick up a near 100% productivity in a few weeks, but I have yet to onboard a dev with this background who doesn’t already know Rust, so this is largely from my own experience learning Rust and talking to others.

                I really wish I had experience with a hyper growth startup using Rust, and seeing first-hand the failure modes onboarding that many people to Rust. But I agree with Matt’s assessment that it’s the wrong language for that situation given the average software developer’s experience, and I have my doubts about the efficacy of hyper growth startups in the first place.

      2. 8

        On a related note, I’m very curious to see what happens with the recent Twitter situation. If Twitter manages to thrive, I think many companies are going to take notice and cut back on developers. The easy pay-day for software engineers could be at an end, and fewer developers will have to handle larger and larger systems. In that case, I’d imagine building things which are robust will outweigh building things quickly. If you have 10% the number of engineers you want to minimize the amount of incident response you are doing (1 out of 100 devs dealing with oncall every week is very different from 1 out of 10 devs dealing with oncall every week in terms of productivity; now the buggy systems have a 10% hit on productivity rather than 1%).

        I’m both worried (large scale cutbacks to mimic Twitter would not be fun) but also somewhat optimistic that it would lead to more reliable systems overall. Counter-intuitively, I think Rust/Haskell/Ocaml and friends would thrive in that world, as choosing a language for quick onboarding of hundreds of devs is no longer a constraint.

        1. 19

          I draw the exact opposite conclusion:

          Tighter pursestrings mean less resources allocated to engineers screwing around playing with new languages and overengineering solutions to stave off boredom.

          There will probably be cases where a business truly uses Rust or something else to differentiate itself technically in performance or reliability, but the majority of folks will go with cheaper tooling with easier-to-replace developers.

          1. 11

            I agree. People will go where the libraries are. If you have 1/10 the number of people you aren’t going to spend your time reimplementing the AWS SDK. You are going to stick to the beaten path.

            1. 2

              I’m sure you meant that as more of a general example than a specific one, but: https://aws.amazon.com/about-aws/whats-new/2021/12/aws-sdk-rust-developer-preview/

              1. 1

                Yeah, I meant it generically. More niche languages are missing a lot of libraries. If you have fewer people you probably want to spend less time reinventing the wheel.

                I know for any one language people will probably come out of the wood work and say “I don’t run into any issues.” but it’s more about perception in the large.

            2. 2

              You make a really good point, and I’ve been mulling on it. My logic was based on the idea that if the software industry suddenly shrank to 10% of its size, the engineers maintaining buggy systems would burn out, while those maintaining robust systems would not. Sort of a forced evolution-by-burnout.

              But I think you’re right, tighter purse strings means less experimentation, so the tried-and-true would benefit. So who knows! Hopefully it’s not something we will ever learn the answer to :)

              1. 2

                The department I run, after over 50% casualty rate this year, has made it a major focus to consolidate, simplify, and emphasize better-documented and smaller systems specifically to handle this. :)

                I hope it works out, but these are going to be interesting times whatever happens. I just personally wish engineers in tech as a culture hadn’t overplayed their hand.

            3. 5

              I can probably set your mind at ease about Twitter (but not the other tech companies having layoffs, nor the new management there who is utterly clueless). Since at least 2009, Twitter’s implicit/unspoken policy was that engineers are cheaper than machines. In other words, it’s more cost-effective to hire a team to optimize the bejeezus out of some service or another, if they can end up cutting server load by 10%. If their policy was based on any real financial data (I have no idea), good dev and devops people will continue to be in high demand, if only to reduce overall machine costs.

            4. 3

              Any recommend way to learn about that from your experience (except than being lucky enough to have an experienced Rust programmer to help you out)?

              Maybe something like exercise.org?

              1. 4

                I’m a fan of trial-by-fire, and if you really want to understand type-driven development then learning and using Haskell is what I’d recommend. Rust is a weird language because it seems really novel, but it really only has the ownership model as unique (and even then, Ada + spark had it first). Everything else is just the best bits borrowed from other languages.

                Haskell’s type system is more powerful, and the docs for libraries heavily lean into the types-as-documentation. I’m not good enough at Haskell to write production software in it, but getting to that “aha!” moment with the language has paid dividends in using Rust effectively.

                1. 3

                  Rust is a weird language because it seems really novel, but it really only has the ownership model as unique (and even then, Ada + spark had it first

                  Ada/SPARK did not have Rust’s affine types ownership model.

                2. 4

                  Just wanted to mention that it is actually https://exercism.org for others’ sake. Didn’t want to let that go unnoticed as it is a wonderful platform for learning!

                  1. 2

                    Elm! If you want a beginner-friendly way to epiphany, work through the Elm tutorial. That was my first, visceral, experience of the joy of sum types / zero runtime errors / a compiler that always has my back.

                    Why via Elm? Because it’s a small and simple language that is user-friendly in every fibre of its being. This leaves you with lots of brain cycles to play with sum types.

                    • Friendly error messages that always give you hints on what to do next. (Elms error messages were so spectacularly good, and that goodness was so novel, that for a while there was a whole buzz in all sorts of language communities saying “our error messages should be more like Elm’s”. Rust may be the most prominent success.)
                    • You’re building a web page, something you can see and interact with.
                    • Reliable refactoring, if your refactor is incomplete the compiler will tell you.
                  2. 2

                    fn double_or_multiply(x: i32, y: Option, double: bool)

                    my 2c: I know it’s a contrived example, but even outside of rust it’s generally (not always) a bad idea (e.g. it’s a code smell) to have a function that does different thing based on a boolean.

                    Also, a good linter/code review should help with the kind of issue you’re pointing to.

                    1. 2

                      In hopes it’s instructive, your code samples are an instance of parse don’t validate where you push all your error checking logic to one place in the code.

                      1. 3

                        Yes it is :) I’m a huge fan of that article, though I’ve found it can sometimes be difficult for someone who isn’t familiar with strong types already. Thank you for sharing the link, I think it’s a great resource for anyone interested in reading more!

                      2. 1

                        I’m new to Rust, could you provide an example of calling your second function? I’ve only just passed the enum chapter of the book and that is the exact chapter that made me excited about working with Rust.

                        1. 7

                          Of course! You would call it like so:

                          double_or_multiply(OpKind::Double(33));
                          
                          double_or_multiply(OpKind::Multiply(33, 22));
                          

                          Its good to hear your excitement from enums in Rust, as I think they are an under-appreciated aspect of the language. Combining structs + enums is super powerful for removing invalid inputs, especially nesting them into each other. The way I think about designing any API is: how can I structure the input the user supplies such that they can’t pass in something incorrect?

                          I wish I could find a source for how to design APIs, as there is some place out there which lists the different levels of quality of an API:

                          • low: the obvious way to use the API is incorrect, hard to use correctly
                          • medium: can be used incorrectly or correctly
                          • high: the obvious way to use the API is correct, hard to use incorrectly
                          • best: no way to incorrectly use the API, easy to use correctly
                          1. 6

                            You may be thinking of Rusty Russell’s API design levels.

                            1. 1

                              Yes! That was exactly what I was looking for, thank you!

                          2. 3
                            double_or_multiply(Double(2)) // = 2*x = 4
                            // Or
                            double_or_multiply(Multiply(3,7)) // = 3 * 7 = 21
                            
                          3. 1

                            Can you explain how it could panic?

                            1. 2

                              Multiplication overflow, which actually would only happen in debug mode (or release with overflow checks enabled). So in practice it likely couldn’t panic (usually nobody turns on overflow checks) (see below)

                              1. 6

                                It’s not that uncommon. Overflow checks are generally off because of perceived bad performance, but companies interested in correctness favor a crash over wrapping. Example: Google Android…

                                https://source.android.com/docs/setup/build/rust/building-rust-modules/overview

                                Overflow checking is on by default in Android for Rust, which requires overflow operations to be explicit.

                                1. 2

                                  I stand corrected! I’m curious what the performance impact is, especially in hot loops. Though I imagine LLVM trickery eliminates a lot of overflow checks even with them enabled

                                  1. 2

                                    I remember numbers flying around on Twitter, most of what I hear is that it is in neglectible ranges. Particularly that if it becomes a problem, there‘s API for actually doing wrapping ops.

                                    Sadly, as often, I can‘t find a structured document that outlines this, even after a bit of searching. Sorry, I‘d love if I had more.

                                2. 1

                                  So, it’s specific for this example, if the enum was over structs with different types and the function did something else, it wouldn’t necessarily panic, right?

                                  Is there a way to make this design panic-proof?

                                  1. 5

                                    Yes, the panicking is specific to the example. And you can make it panic-proof if none of the function calls within can panic. IIRC its still an open design problem of how to mark functions as “no panic” in rust so the compiler checks it [1][2]. There are some libraries to do some amount of panic-proofing at compile-time[3] but I haven’t used them. I thought there was a larger RFC ticket for the no-panic attribute but I can’t find it right now.

                                    [1] https://github.com/rust-lang/project-error-handling/issues/49 [2] https://github.com/rust-embedded/wg/issues/551 [3] https://crates.io/crates/no-panic

                            2. 30

                              Basically, the problems that Rust is designed to avoid can be solved in other ways — by good testing, good linting, good code review, and good monitoring.

                              This sounds like a false economy to me. In my experience catching problems already at compilation stage is pretty much always a saving over catching bugs later in the life cycle – especially if it goes into production and is caught via monitoring. It might initially feel like higher velocity, but as the code complexity grows, it really isn’t.

                              1. 16

                                I’m reminded of the facetious adage from science that “a year in the library can save a day in the lab.”

                                1. 11

                                  You have reminded me of one of my favorites, which is, “weeks of programming can save you hours of planning.”

                              2. 28

                                This is an important lesson. Rust is aimed at being a safer systems language. There are a lot of safe applications languages with more mature ecosystems, better tooling, and so on. If you are in a situation where you don’t need fine-grained control over memory layout, where you can tolerate the memory and jitter of a global garbage collector, then Rust is as much the wrong choice as C would be. Rust is a good choice for situations where, if Rust didn’t exist, C would be a good choice. If you wouldn’t consider using C for a project, don’t use Rust. Use Erlang/Elixir, C#/F#, JavaScript/TypeScript, Swift, Pony, or anything else that targets application programming problems.

                                1. 20

                                  There are a lot of safe applications languages with more mature ecosystems, better tooling, and so on

                                  I think this is sadly not entirely true. On balance, I would say Rust’s “quality of implementation” story is ahead of mature languages, and that’s exactly what creates this “let’s write CRUD in Rust” pressure.

                                  Let’s say I write something where I don’t need Rust, what are my choices?

                                  • Erlang/Elexir – dynamic types, the same, if not bigger, level of weirdness in Rust.
                                  • F# last time I’ve checked, the build system was pretty horrible (specifying files in a specific order in an XML file)
                                  • C# – until recently, it had nullability problems, and was pretty windows-specific. maybe it is a viable option now. My two questions would be: a) how horrible is the build system? b) how hard is it to produce a static binary which I can copy from one linux PC to another without installing any runtime libraries?
                                  • JavaScript/TypeScript – npm is a huge time sink. Once deno matures (or Rome conquers the world) it might become a reasonable alternative
                                  • Swift – doesn’t really exist outside of Mac? A lot of core things were in mac-specific libraries last time I’ve checkd
                                  • Pony – super niche at this point?
                                  • Java – nullability, problems with static binaries, horrible build system
                                  • Kotlin – Java sans nullability
                                  • Go – if you value utmost simplicity more than sum types, that is a splendid choice. I think Go is the single language which is in the same class as Rust when it comes to “quality of implementation”, and, arguably, it’s even better than Rust in this respect. If only it had enums and null-checking…
                                  • OCaml – this one is pretty ideal when it comes to the language machinery, but two+ build systems, two standard libraries, etc, make it not a reasonable choice in comparison.

                                  Really, it seems that there’s “Go with enums / OCaml with channels and Cargo”-shaped hole in our language-space these days, which is pretty imperfectly covered by existing options. I really wish that such language existed, this would relieve a lot of design pressure from Rust and allow it to focus on systems use-case more.

                                  1. 9

                                    JavaScript/TypeScript – npm is a huge time sink. Once deno matures (or Rome conquers the world) it might become a reasonable alternative

                                    I’d have a hard time finding a productive lead or senior engineer for Node.js complaining that npm is a timesink and the number one thing that makes the language not productive. The teams I work with are incredibly productive with Node (and would have a very hard time onboarding with rust), and most of the time we only have to touch package.json once every few weeks.

                                    1. 6

                                      Ocaml 5 has channels along with the new multi core support. And algebraic effects.

                                      1. 4

                                        [… ] this would relieve a lot of design pressure from Rust and allow it to focus on systems use-case more.

                                        Considering that a lot of more serious “system use-cases” (the Linux kernel, Firefox) tend to either fight or ditch Cargo, could it be that its current design is optimized for the wrong use-cases?

                                        1. 3

                                          That’s a great question.

                                          The primary answer is indeed that’s not the use-case Cargo is optimized for, and that was the right thing to do. With the exception of the kernel, all folks who ditch Cargo still make use of crates.io ecosystem. And producing re-usable libraries is the use-case Cargo is very heavily skewed towards. The main shtick of Cargo is the (informal, implementation-defined) specification of Cargo.toml file and .crate archive which provide a very specific interface for re-usable units of code. Because every library on crates.io is build in exactly same inflexible way, re-using the libraries is easy.

                                          The secondary answer is that, while Cargo could be much better (defaults steer you towards poor compile times) for the case of developing a leaf artifact, rather than a library, it’s pretty decent. Stuff like ripgrep, cargo, rust-analyzer, wasmtime are pretty happy with Cargo. There’s a lot of “systems-enough” software which doesn’t need a lot of flexibility in the build process.

                                          But yeah, ultimately, if you build something BIG, you probably have a generic build-system building the thing, and the way to plug Rust there is by plugging rustc, not cargo. Though, yeah, Cargo could’ve been somewhat more transparent to make this easier.

                                          1. 1

                                            The main shtick of Cargo is the (informal, implementation-defined) specification of Cargo.toml file […]

                                            An even bigger shtick is build.rs, a rudimentary build system built with environment variables. For example, in build2 we could map all the concepts of Cargo.toml (even features), but dealing with build.rs is a bleak prospect.

                                            1. 2

                                              Yeah, build.rs is super-problematic. It’s a necessary evil escape hatch to be able to use C code, if you have too.

                                              I think the most promising solution in this space is the metabuild proposal: rather than compiling C “by hand” in build.rs, you specify declarative dependencies on C libraries in Cargo.toml, and then a generic build.rs (eg, the same binary shared between all crates) reads that meta and builds C code. This would provide a generic hook for generic build systems to supply Cargo with C deps.

                                              Sadly, that didn’t go anywhere:

                                              • the design we arrived at required first-class support in Cargo, which didn’t materialize. I feel that an xtask-shaped polyfil would’ve worked better.
                                              • we lacked a brave person to actually submit tonnes of PRs to actually move ecosystem towards declarative native deps.
                                              1. 1

                                                Yeah, in a parallel universe there is a standard build system and package manager for C/C++ with Rust (and other languages) that are built “on top” simply reusing that and getting all the C/C++ libraries for free.

                                        2. 3

                                          I think this overstates the importance of the output being a single static binary. Sometimes that’s a desirable goal, no question. But if you are writing code that will be deployed to a containerized environment, which is the case for a lot of CRUD web apps, installing runtime dependencies (libraries, interpreter, etc.) is a no-op from an operational point of view because the libraries are bundled in the container image just like they’d be bundled in a static binary.

                                          1. 3

                                            But then you’re paying the cost of building container images - which is a huge part of deployment time costs if I understand people’s experience reports correctly.

                                            1. 5

                                              Even with static binaries you’re probably still looking at containerisation for a lot of intermediate things, or runtime requirements, if only because of the state of infrastructure right now. There isn’t really “k8s but without containers”.

                                              1. 2

                                                Not really. Building Docker images with proper caching is pretty quick, within about a couple of minutes in my experience. Far more time will be spent running tests (or in bad CI systems, waiting for an agent to become ready).

                                                1. 1

                                                  👍 It only adds seconds on top of iterated builds if done well. Could take an extra minute or two on a fresh machine

                                                2. 1

                                                  It’s pretty quick if you choose not to include a whole operating system

                                                3. 2

                                                  Yeah, for specific “let’s build a CRUD app” use-case static linking is not that important.

                                                  But it often happens that you need to write many different smaller things for different use-cases. If you pick the best tool for the job for every small use-case, you’ll end up with a bloated toolbox. Picking a single language for everything has huge systemic savings.

                                                  This is the angle which makes me think that “small artifacts” is an important part of QoI.

                                                4. 3

                                                  the same, if not bigger, level of weirdness in Rust.

                                                  In the case of Elixir, I’d argue that it actually writes a great deal like Python or Ruby, plus some minor practical guardrails (immutability) and an idiom around message-passing.

                                                  The really weird stuff you don’t hit until Advanced/Advanced+ BEAM wizard status.

                                                  1. 1

                                                    This is your regularly scheduled reminder that D exists. It recently added sumtypes to the standard library!

                                                    1. 6
                                                      $ cat main.d
                                                      import std.stdio;
                                                      
                                                      void main(){
                                                          auto r = f();
                                                          writeln("hello, ub");
                                                          writeln(*r);
                                                      }
                                                      
                                                      auto f() {
                                                          int x = 92;
                                                          return g(&x);
                                                      }
                                                      
                                                      auto g(int* x) {
                                                          return x;
                                                      }
                                                      
                                                      $ dmd main.d && ./main
                                                      
                                                      hello, ub
                                                      4722720
                                                      

                                                      If, by default, I can trivially trigger UB, the language loses a lot of points on QoI. I think there’s some safe pragma, or compilation flag, or something to fix this, but it really should be default if we think about language being applicable to CRUD domain.

                                                      1. 2

                                                        This is fixed by enabling the experimental DIP1000 feature via -dip1000:

                                                         11: Error: returning `g(& x)` escapes a reference to local variable `x`
                                                        

                                                        DIP1000 is in testing and will hopefully become standard soon.

                                                        That said, just don’t use pointers. You practically never need them. We have large backend services in production use that never use pointers at all.

                                                    2. 1

                                                      Scala. The horsepower and ecosystem of the JVM, plus the type system inspired by much more academic languages. In reality though, all of the above mainstream languages are plenty good enough for almost all line of business work. And where they have shortcomings, certainly Rust does have its own shortcomings as outlined in the OP. So it’s all about tradeoffs, as always.

                                                      1. 1

                                                        tbh, I think the only reason to use Scala over Kotlin is having an already existing large Scala code-base which went all-in on type-level programming (which I think is surprisingly the case for a lot of bank’s internal departments).

                                                        Scala deserves a lot of credit for normalizing functional programming in the industry, it was the harbinger of programming language renaissance, but, as a practical tool, I don’t think it is there.

                                                        1. 1

                                                          As far as I know Kotlin still doesn’t have a general pattern-matching implementation. This is so important for business logic correctness that I find it difficult to imagine managing a serious production codebase without it. I use it on an almost daily basis in my projects.

                                                    3. 6

                                                      Use Erlang/Elixir, C#/F#, JavaScript/TypeScript, Swift, Pony, or anything else that targets application programming problems.

                                                      Didn’t Wallaroo move from Pony to Rust around this time last year?

                                                      I remember the corecursive interview with Sean Allen in 2020 talking about his developer experience with Pony “All of us who worked on Wallaroo in the early days have a bit of scar tissue where even though none of us had hit a compiler bug in forever, we were still like, ‘Is that a bug in my code or is that a bug in the compiler?’”

                                                      Having worked at a startup where we used Haskell for an elaborate CRUD app, this sounds about as painful. They sacked the engineering director after I left and I believe they’ve moved to Typescript for everything now.

                                                      I wouldn’t put Pony on that list just like I wouldn’t put Haskell on that list.

                                                      I can also say that my buddy who’s a manager at a big-data company has had everyone under him switch over to Java from Scala.

                                                      So I’d probably put Java and Golang at the top of the list for a CRUD app.

                                                      1. 2

                                                        Great interview, enjoyed reading it.

                                                        Tbh I think the trade off here is how “business logicky” your app is, rather than crud or whatever. At a certain point something like scala really helps you find cascading implications of changes. If you have that kind of app, all else being equal, a rich static type system is going to improve productivity. The more you’re mostly just doing io with code that’s fairly self contained, the easier it is to crank out code with a golang or mypy like type system.

                                                        Similarly the more you need to spend time wrangling for throughput the better your life will be with systems that focus on safety and control of execution.

                                                        Twitter started out on rails, and that makes sense when you don’t need your application to actually be bug free or fast but you do need to just get stuff out there. We continue to have so many different programming systems because there are so many different points in the space that it’s worth optimizing for.

                                                        (For anyone who thinks I’m being dismissive- I spend my days writing golang with only api tests mostly to run sql, and it’s a solid choice for that kind of thing).

                                                    4. 17

                                                      the problems that Rust is designed to avoid can be solved in other ways — by good testing, good linting, good code review, and good monitoring

                                                      This stuck out to me, as the four things the author mentions here are the exact things that have always been in short supply in my current line of work, on under-resourced teams at a public sector employer. Obviously Rust is not the only productive language out there offering good compile-time guarantees and the firm guiding hand of a type system, but in general I wonder if “just let me crush code and we’ll fix it in post catch the issues in the test suite/CR” isn’t an abundance mindset. I frequently have to fix things (in Python or Perl or JavaScript) where nobody including me has looked at the code in 3 or more years and there are no tests; I’d give my kingdom for mypy --strict on those occasions.

                                                      1. 17

                                                        Coming from a Haskell background, many of the complaints here are things that strike me as strange. For example,

                                                        What really bites is when you need to change the type signature of a load-bearing interface and find yourself spending hours changing every place where the type is used

                                                        Or, you can make changes to load bearing interfaces and the compiler tells you everywhere you need to update something, which makes it mechanical work and removes the possibility of human error.

                                                        when I’m building a new feature I usually don’t have all the data types, APIs, and other fine details worked out up front. I’m often just farting out code

                                                        If you have a strong type system, you tend to fart around in the type system instead.

                                                        The expected load on this service was going to be on the order no more than a few queries per second, max, through the lifetime of this particular system

                                                        So why was this a microservice at all? Why not just a nice monolith where you can take advantage of the type system strongly?

                                                        1. 16

                                                          Rust has made the decision that safety is more important than developer productivity.

                                                          I think this forgets that safety improves developer productivity in the long run. But what Rust actually tries to achieve is safety without loosing performance, and that comes at a cost of developer productivity. Plenty of other languages are safe as well, but they usually sacrifice performance to achieve that.

                                                          I think the main question in choosing to use Rust is “do you value performance more than productivity?”. And the answer to that depends on the system.

                                                          1. 14

                                                            You will have a hard time hiring Rust developers.

                                                            It’s kinda funny - we’re a small startup (6 people) and I keep getting great CVs to my email, even while we don’t search. When we do we get around 50~ per week, mostly high end candidates. I think there are many Rust developers who just wait for the opportunity to use that as their main language.

                                                            To give some perspective, our main project had to be low level, so we did Rust. Then, we needed to do a backend service and the designated engineer decided to go with Rust (though it wasn’t our first choice because of prejudice as seen above). He shipped it very fast, high performance, well documented, many battries included and edge cases covered just because the eco system makes you be robust. Adding new features is easy because it’s hard to break things real bad.

                                                            1. 11

                                                              I feel like a lot of what is said here can also apply to other “fancy” but rigorous languages like Haskell. It solves a lot of issues well, but the deployment velocity might suffer a bit and it can be hard to hire for.

                                                              1. 8

                                                                I commented on this on twitter, and I think it’s accurate, but there’s one point missed in the comments here that I think is worth diving into.

                                                                Any choice of rust web framework made now is probably wrong long-term.

                                                                One could say that axum is maybe the least bad option today, but given the state of affairs the major version dance of the couple frameworks on major versions make them less easy to work with than the non-major-versioned libraries. Because the pace of change requires them to bump major versions. I think the rust web frameworks are nice, but they need to shake out like all the other major languages have, with a few good ones and many semi-abandoned ones that simply made wrong design decisions.

                                                                I hope this comment doesn’t apply a year from now. A statistical view of some of the frameworks, but I would need a crystal ball to pick the one or more that’s going to become the de facto standard. https://github.com/flosse/rust-web-framework-comparison

                                                                1. 6

                                                                  If you’re really worried about that, you could just use hyper. It’s the basis of all the higher-level frameworks I’m aware of.

                                                                  Frankly, I feel like most high level web frameworks are really unnecessary for most services. They reduce the boilerplate, sure, but boilerplate is easy. And you can always do routing with a simple match statement. Just use query params instead of url params. URL params are a scam, anyways.

                                                                  1. 3

                                                                    I think the issue is most jobs are focused on day 1 productivity (commit to master on day 1), so spinning up someone on Rust has a lot of levels to it. why .to_string() what’s Option, what’s Result, what’s an enum (oh crap you learned it after Option and Result so your concept map is muddled), what’s a crate, no really what’s a crate, etc etc. Which in a safe space for learning, can be learned really fast, but perhaps not at hyperscale. I personally think hyperscale is a side-effect of hyper-greed, a lack of cash discipline and a surfeit of hopefulness, but you can be the judge of past hyperscaler corp’s maintenance of scale. Some have, some haven’t.

                                                                2. 8

                                                                  Couple of things I want to highlight from this article that bear repeating (indeed, perhaps, stamping onto the arms of developers tempted to forget):

                                                                  The only reason we were using Rust was because the original authors of the system were Rust experts, not because it was an especially good fit for building this kind of service.

                                                                  There can be tremendous advantages your team is missing out on due to what is effectively an accident of history. In the case of a CRUD app with a couple of routes making a database call, Express (JS) or Sinatra (Ruby) or Flask (Python) can just bury the competition in productivity. It took longer to write this comment than a simple index API route in those frameworks/languages. Just because your founding team likes X doesn’t mean that decision shouldn’t be reviewed.

                                                                  We started having weekly “learn Rust” sessions for the team to help share knowledge and expertise. This was all a significant drain on the team’s productivity and morale as everyone felt the slow rate of development.

                                                                  Developers need to learn the domain, learn the process, and learn the tooling during the course of doing their work. Choosing non-standard or difficult tooling means that their efficiency will suffer. It sounds like the author’s team navigated this as best they could with professional development, but the real solution of using an easier and better-fit tool would have been preferable.

                                                                  I’m often just farting out code trying to get some basic idea working and checking whether my assumptions about how things should work are more-or-less correct.

                                                                  My favorite point that the author makes. In my experience, especially on high-velocity teams trying out lots of stuff, the exact things that make languages like Rust, Haskell, and others so good at their domains make them liabilities. If your language requires a strong and carefully-expressed model of the business rules and business logic in order to do anything, until you have that model you will do nothing.

                                                                  ~

                                                                  Overall, a neat article, and I’m glad to hear some skepticism.

                                                                  1. 6

                                                                    I work at a startup that uses Rust and this rings very true to me.

                                                                    I like programming Rust. It’s fun and its interesting and it makes me feel smart when I use it.

                                                                    hiring, onboarding, code reviews, estimates, sprint planning, library maturity, being on-call, maintaining existing projects? yeah. That’s a different thing.

                                                                    1. 4

                                                                      A lot of people talk up the reliability and ease of maintenance in rust. What do you find challenging about it? Are there edge cases you wish you had known about before?

                                                                      1. 1

                                                                        well, I can only comment on my experience. When I say maintaining existing projects, I don’t think “edge cases” is the right framing, because the problem is not really something about the language itself; it’s not a problem with Rust’s capabilities as a language. I also don’t think “reliability” is the right framing because it’s assuming that maintenance just means keeping the thing online.

                                                                        In reality, maintenance really means that engineer A wrote something that fit requirements alpha, and now the requirements have changed to requirements beta, and engineer B has to modify the project that was built by someone else for different requirements to fit their new requirements. Maybe engineer A is gone, maybe engineer A is busy; you can’t just require that all changes go through the same person, because if you can’t exchange work fluidly between engineers, you’re never going to keep the velocity you need in a startup environment. In a startup, requirements tend to change very rapidly because you’re still finding product-market fit.

                                                                        Hiring people that already know how to program Rust is hard enough. Hiring people that know how to operate Rust in production is even harder. Applicants that do known Rust very often have not used Rust in a team or production setting.

                                                                        The obvious problem with hiring a bunch of people that don’t really know Rust is that you wind up with a lot of projects that were written by people who were learning Rust when they wrote it. Most people think the consequence of this is that you have two styles of code floating around your org: “Rust by experienced Rust programmers” and “Rust by inexperienced Rust programmers”. Reality is actually much worse. In reality, people who come to Rust from Go, Python, JavaScript, Java, etc… each has a specific and idiosyncratic way of learning Rust. Instead of having 2 styles of code in your org, you really wind up with N+1 styles of code in your org: 1 style for each of the N contexts that you hired from and then 1 style for people that are writing code in your org’s native style. If you have a stable team that’s been working together and they’re all coming from $language and they all start programming Rust together, you wind up with two kinds of code: “Rust by people thinking like $language” and “Rust by people who have been writing Rust for a while”. When you have a team that’s new to working with one another and comes from a large diversity of contexts and programming backgrounds, what you really wind up with is “Rust by people thinking like $language” for each $language in the set of all languages that describe the prior experience of all of your engineers.

                                                                        Operators have a pretty hard time with it. Most people say that learning to write Rust comfortably and productively takes about three months of programming Rust regularly, but what about the people on your team that need to interact with your projects on an infrequent basis? It’s very common for someone to have a role where they do a lot of operations-focused work, but might want to contribute to code on an infrequent basis. Maybe they want to update a library because a vulnerability was found in one of our dependencies, do they have the toolset? Can they update the dependency? What if they find a small bug, will they be able to fix a small bug they noticed in production, or will they have to send it to another engineer to fix it? Virtually all of these engineers could fix a small bug in a Go, Python, or Java program, but fixing even a small bug in a Rust program is often very difficult for people who don’t use the language regularly. Often times they can’t even understand what the error messages are talking about.

                                                                        The testing ecosystem of Rust is also fairly immature. Support for benches in tests is still in nightly, and none of us want to be depending on nightly in production. Custom test runners is also a nightly feature. Sure, these things exist, but there’s not a lot of stable tooling build on top of them, because their foundations are not stable.

                                                                        For context, it’s a game company, we’re making an MMO, and the client uses Unreal Engine. Our engineers come from places like Epic, Google, Amazon, Riot, Blizzard, etc. These are smart, experienced engineers, people with a lot of production experience on large scale server deployments and large scale multiplayer games.

                                                                        I think Rust is a very capable language and I think my teammates are very capable engineers; I don’t think the problem is either the capabilities of the language or the capabilities of the people. I enjoy writing Rust, and I think most people I work with would say the thing, but thinking it’s cool and thinking it’s effective are different things. I’m convinced it’s cool, but I’m not convinced that it’s particularly well-suited for rapidly-growing teams.

                                                                        1. 1

                                                                          First off, thank you for taking the time to leave a detailed comment!

                                                                          I misunderstood what you meant by maintenance. What you are describing I might think of as brown field development or legacy code bases. E.g., continued development and evolution of a codebase. I was thinking about operational maintenance. I assumed that might imply rough tooling around maintenance tasks and therefore be a bit of an edge case in program life cycle. I think I have a better idea what you mean now.

                                                                          I see how it could be challenging to get everyone to write the same code style in Rust. It’s a bit of a kitchen sink language and it’s fairly young so there may not be enough cultural norms (like the Zen of Python) or people to enforce said norms. By contrast, Go, Python, and Java have normalized conventions, which makes it easier to assimilate into the community.

                                                                          It makes me wonder how long it took other languages to “find their voice”. C++ is famous for shops having their own subset so it seems like not every language necessarily comes to normalize a set of conventions. I’m not deep in the Rust ecosystem so I don’t feel qualified to say how that is developing. Hopefully enough people will develop the cultural knowledge to provide a good base new shops to adopt Rust.

                                                                          Thanks again for your thoughts!

                                                                    2. 5

                                                                      The high-level point that if you quickly hire a lot of people without Rust experience, they need more time to get up to speed compared to Go, is likely correct.

                                                                      It’s incorrect and beside the point, though, to bring up C++ and complaints about Rust you hear from C++ developers. First, ownership and lifetimes shouldn’t be conceptually new for C++ programmer. Second, as pcwalton pointed out on Twitter, relative to Go, Rust favors performance—Go is safe, too. The complaint that Rust favors safety is a C++ PoV complaint that seems off the mark here.

                                                                      1. 2

                                                                        Go is memory-safe, but it is also error prone by striving to finish a program with ill-defined failure cases or invariants.

                                                                        1. 3

                                                                          Go is memory safe-ish. It’s not threadsafe and data races can lead to memory errors.

                                                                      2. 4

                                                                        This is a large reason I’ve been drawn more to Zig and Python than Rust in my personal graphics/audio/gamedev related projects. I like to prototype numerical algorithms quickly, and numpy is killer for that. When I want control over memory (which absolutely matters in certain prototypes), I’ll drop to Zig. It provides better guardrails than C or C++, gives me access to all my favorite C libs through @cImport, but is still much faster to prototype with than Rust. [1]

                                                                        Like the article mentions if you’re writing an OS kernel (or filesystem, or database, or a cryptography library, or a mission-critical scheduler, or any number of other multithreaded + safety + performance sensitive code [2]), Rust is a great choice. In established organizations, Rust also has an edge since larger companies are more focused on risk-mitigation, have larger teams, and benefit from stronger SDLC constraints to keep code consistent when dozens of programmers get their grubby paws on everything. But I can definitely see how a small startup that needs a bunch of short-term wins to survive would struggle with Rust. I’m sure there are startups building products where safety and performance are main features, and I bet Rust would provide an edge in some of those cases as well. But for a SaaS CRUD app with a handful of developers, it probably wouldn’t rank high on my list of languages to use.

                                                                        [1]: I’d pick Rust even for my prototypes if worked on my problems that were multithreaded, but the safety guarantees of Zig are strong enough for the small, single-threaded programs I usually write. I’ve really enjoyed contributing a few small Rust patches to open source projects though, so I might also pick it if I wanted to make collaboration easy.

                                                                        [2]: Although, it might even be tricky with some of these examples since they could require a decent chunk of unsafe code which is another sticking point for Rust.

                                                                        1. 1

                                                                          I just don’t understand this need to be first out the door with features. Microsoft, a behemoth of a company, has never been first to market anything in its lifetime, except maybe BASIC interpreters (maybe). Operating system for the PC? Nope (there were three available up on the release of the IBM PC, and Microsoft wasn’t the first choice). GUI? Nope. Spreadsheet? Nope. Word processor? Nope. Slide presentations? Nope. Cloud offerings? Nope. Linux support? Nope. It hasn’t seemed to hurt them being second, or third, or fourth, to market one bit.

                                                                          1. 5

                                                                            I think it’s more about getting your product out there before you run out of money.