1. 40
  1.  

  2. 33

    There’s a huge cultural problem around dependencies and a total lack of discipline for what features get used in most Rust projects.

    sled compiles in 6 seconds flat on my laptop, despite being a fairly complex Rust embedded database. Most database compile times are measured in minutes, even if they are written in C, C++ or Go.

    I feel that slow compilation times for a library are totally disrespectful to any users, who probably just want to solve a relatively simple issue by bringing in your library as a dependency anyway. But in the Rust ecosystem it’s not uncommon at all for a simple dependency that could have probably been written in 50 lines of simple code to pull in 75 dependencies to get the job done that you need it for. Pretty much all of my friends working in the Rust blockchain space have it especially bad since they tend to have to pull in something like 700 dependencies and spend 3 minutes just for linking for one commonly used dependency.

    Things I avoid to make compilation fast:

    • proc macros - these are horrible for compile times
    • build.rs same as above, also causes friction with tooling
    • deriving traits that don’t get used anywhere (side note, forcing users to learn your non-std trait is just as bad as forcing them to learn your non-std macro. It introduces a huge amount of friction into the developer experience)
    • generics for things I only use one concrete version of internally. conditional compilation is very easy to shoot yourself in the foot with but sometimes it’s better than generics for testing-only functionality.
    • dependencies for things I could write in a few dozen lines myself - the time saved for a project that I sometimes compile hundreds of times per day and have been building for over 4 years is a huge win. Everybody has bugs, and my database testing tends to make a lot of them pop out, but I can fix mine almost instantly, whereas it takes a lot of coordination to get other people to fix their stuff.

    Also, CI on every PR tends to finish in around 6 minutes despite torturing thousands of database instances with fault injection and a variety of other tests that most people only run once before a big release.

    Developer-facing latency is by far one of the most effective metrics to optimize for. It keeps the project feeling fun to hack on. I don’t feel dread before trying out my changes due to the impending waiting time. Keeping a project nice to hack on is what keeps engineers hacking on it, which means it’s also the most important metric for any other metrics like reliability and performance for any project that you hope to keep using over years. But most publicly published Rust seems to be written with an expiration date of a few weeks, and it shows.

    1. 11

      My take-away from the article is that open source allows different people to play different roles: the original dev got it working to their own satisfaction. Another user polished off some cruft. Everybody wins.

      I feel that slow compilation times for a library are totally disrespectful to any users …

      If someone write software to solve a problem and shares it as open source, I don’t consider it disrespectful regardless of the code quality. Only if someone else is compelled to use it or is paying for it would the developer have any obligation, IMO.

      1. 10

        Same experience here. When I started writing durduff, I picked clap for parsing CLI arguments. After a while. I used cargo-deps to generate a graph of dependencies and it turned out that the graph was dominated by the dependencies pulled in by clap. So I switched to getopts. It cut the build times by something like 90%.

        Another example: atty term_size depends on libc, but when you look at its source code, it has a lot of code duplicated with libc. The constants which need to be passed to the ioctl, with conditional compilation, because they have different values on different platforms. It seems to be a common theme: wrapping libraries, while still duplicating work. I replaced atty term_size with a single call to libc (in the end I stopped doing even that). One less dependency to care about.

        That said, I still think that waiting a couple seconds for a project as small as durduff to compile is too much. It also slows down syntastic in vim: it’s really irritating to wait several seconds for text to appear every time I open a rust file in vim. It’s even worse with bigger projects like rustlib.

        As for avoiding generics: I use them a lot for testing things in isolation. Sort of like what people use interfaces for in Go. With the difference that I don’t pay the runtime cost for it. I’m not giving this one up.

        BTW Thank you for flamegraph-rs! Last weekend it helped me find a performance bottleneck in durduff and speed the whole thing up three-fold.

        EDIT: I got crates mixed up. It was term_size, not atty that duplicated code from libc.

        1. 11

          it turned out that the graph was dominated by the dependencies pulled in by clap.

          Did you try disabling some or all of Clap’s default features?

          With respect to the OP, it’s not clear whether they tried this or whether they tried disabling any of regex’s features. In the latter case, those features are specifically intended to reduce compilation times and binary size.

          Another example: atty depends on libc, but when you look at its source code, it has a lot of code duplicated with libc. The constants which need to be passed to the ioctl, with conditional compilation, because they have different values on different platforms. It seems to be a common theme: wrapping libraries, while still duplicating work

          Huh? “A lot”? Looking at the source code, it defines one single type: Stream. It then provides a platform independent API using that type to check whether there’s a tty or not.

          I replaced atty with a single call to libc. One less dependency to care about.

          I normally applaud removing dependencies, but it’s likely that atty is not a good one to remove. Unless you explicitly don’t care about Windows users. Because isatty in libc doesn’t work on Windows. The vast majority of the atty crate is specifically about handling Windows correctly, which is non-trivial. That’s exactly the kind of logic that should be wrapped up inside a dependency.

          Now, if you don’t care about Windows, then sure, you might have made a good trade off. It doesn’t really look like one to me, but I suppose it’s defensible.

          That said, I still think that waiting a couple seconds for a project as small as durduff to compile is too much. It also slows down syntastic in vim: it’s really irritating to wait several seconds for text to appear every time I open a rust file in vim.

          It takes about 0.5 seconds for cargo check to run on my i5-7600 after making a change in your project. Do you have syntastic configured to use cargo check?

          1. 4

            Did you try disabling some or all of Clap’s default features?

            I disabled some of them. It wasn’t enough. And with the more fireworky features disabled, I no longer saw the benefit of clap over getopts, when getopts has less dependencies.

            Huh? “A lot”? Looking at the source code, it defines one single type: Stream. It then provides a platform independent API using that type to check whether there’s a tty or not.

            Ok, I got crates mixed up. It was term_size (related) that did that, when it could just rely on what’s already in libc (for unix-specific code). Sorry for the confusion.

            I normally applaud removing dependencies, but it’s likely that atty is not a good one to remove. Unless you explicitly don’t care about Windows users. Because isatty in libc doesn’t work on Windows.

            Yes, I don’t care about Windows, because reading about how to properly handle output to the windows terminal and output that is piped somewhere else at the same time left me with the impression that it’s just too much of a pain.

            It takes about 0.5 seconds for cargo check to run on my i5-7600 after making a change in your project. Do you have syntastic configured to use cargo check?

            I’ll check when I get back home.

            1. 21

              I no longer saw the benefit of clap over getopts, when getopts has less dependencies.

              Well, with getopts you start out of the gate with a bug: it can only accept flags and arguments that are UTF-8 encoded. clap has OS string APIs, which permit all possible arguments that the underlying operating system supports.

              You might not care about this. But I’ve had command line tools with similar bugs, and once they got popular enough, end users invariably ran into them.

              Now, I don’t know whether this bug alone justifies that extra weight of Clap. Although I do know that Clap has to go out of its way (with additional code) to handle this correctly, because dealing with OS strings is hard to do in a zero cost way.

              Yes, I don’t care about Windows, because reading about how to properly handle output to the windows terminal and output that is piped somewhere else at the same time left me with the impression that it’s just too much of a pain.

              I think a lot of users probably expect programs written in Rust to work well on Windows. This is largely because of the work done in std to provide good platform independent APIs, and also because of the work done in the ecosystem (including myself) to build crates that work well on Windows.

              My argument here isn’t necessarily “you should support Windows.” My argument here is, “it’s important to scrutinize all costs when dropping dependencies.” Particularly in a conversation that started with commentary such as “lack of discipline.” Discipline cuts both ways. It takes discipline to scrutinize all benefits and costs for any given technical decision.

              1. 15

                Just wanted to say thanks for putting in the effort to support Windows. ripgrep is one of my favorite tools, and I use it on Windows as well as Linux.

                1. 1

                  I checked and I have a line like this in my .vimrc:

                  let g:rust_cargo_check_tests = 1
                  

                  That’s because I was annoyed that I didn’t see any issues in test code only to be later greeted with a wall of errors when compiling. Now I made a small change and cargo check --tests took 5 seconds on my AMD Ryzen 7 2700X Eight-Core Processor.

                  Well, with getopts you start out of the gate with a bug: it can only accept flags and arguments that are UTF-8 encoded. clap has OS string APIs, which permit all possible arguments that the underlying operating system supports.

                  I’ll reconsider that choice.

          2. 4

            But in the Rust ecosystem it’s not uncommon at all for a simple dependency that could have probably been written in 50 lines of simple code to pull in 75 dependencies to get the job done that you need it for.

            I don’t think I have experienced this. Do you have an example of this on crates.io?

            1. 1

              Not quite to that degree, but I’ve seen it happen. Though I’ve also seen the ecosystem get actively better about this – or maybe I just now have preferred crates that don’t do this very much.

              rand does this to an extent by making ten million little sub-crates for different algorithms and traditionally including them all by default, and rand is everywhere, so I wrote my own version. num also is structured that way, though seems to leave less things on by default, and deals with a harder problem domain than rand.

              The main example of gratuitous transitive dependencies I recall in recent memory was a logging library – I thought it was pretty_env_logger but can’t seem to find it right now. It used winconsole for colored console output, which pulls in the 10k lines of cgmath, which pulls in rand and num both… so that it can have a single function that takes a single Vector2D.

              …sorry, this is something I find bizarrely fun. I should probably make more time for it again someday.

            2. 1

              Minimizing your dependencies has further advantages like making the overall system easier to understand or avoiding library update problems.

              Part of this is simply insufficient tooling.

              Rebuilding all your dependencies should be rare. In practice, it happens way too often, e. g. frequently on every CI run without better build systems. That is madness. You can avoid it by e.g. using Nix or bazel.

              I terms of timing, I’d also love to understand why linking is quite slow - for me often the slowest part.

              But all in all, bloated compile times in dependencies would not be a major decision factor for me in choosing a library. Bloated link times or bloated compile times of my own crates are, since they affect my iteration speed.

              That said, I think if you are optimizing the compile time of your crate, you are respecting your own time and that of your contributors. Time well spent!

              1. 1

                deriving traits that don’t get used anywhere

                I take it you mean custom traits, and not things like Default, Eq/Ord, etc?

                1. 9

                  check this out:

                  #[derive()]
                  pub struct S { inner: String }
                  

                  (do this in your own text editor)

                  1. 2dd (yank lines), 400p (paste 400 times)
                  2. %s/struct\ S/\=printf("struct S%d", line('.')) name all of those S’s to S + line number
                  3. time cargo build - 0.10s on my laptop
                  4. %s/derive()/derive(Debug)
                  5. time cargo build - 0.58s on my laptop
                  6. %s/derive(Debug)/derive(Debug, PartialOrd, PartialEq)
                  7. time cargo build - 2.46s

                  So, yeah, that deriving actually slows things down a lot, especially in a larger codebase.

                  1. 3

                    This is particularly annoying for Debug because either you eat the compile time or you have to go derive Debug on various types every time you want to actually debug something. Also if you don’t derive Debug on public types for a library then users of the library can’t do it themselves.

                    In languages like Julia and Zig that allow reflection at specialization time this tradeoff doesn’t exist. Eg in zig:

                    pub fn debug(thing: var) !void {
                        const T = @TypeOf(thing);
                        if (std.meta.trait.hasFn("debug")(T)) {
                            // use custom impl if it exists
                            thing.debug();
                        } else {
                            // otherwise reflect on type
                            switch (@typeInfo(T)) {
                                ...
                            }
                        }
                    }
                    

                    This function will work on any type but will only get compiled for types to which it is actually applied in live code so there’s no compile time overhead for having it available. But the reflection is compiled away at specialization time so there is no runtime overhead vs something like derive.

                    1. 2

                      Some numbers from a fairly large project:

                      $ cargo vendor
                      $ rg --files | grep -E '*.rs' | xargs wc -l | sort -n | tail -n 1
                       1234130 total
                      $ rg --files | grep -E '*.rs' | xargs grep -F '#[derive' | grep -o -E '\(|,' | wc -l
                      22612
                      

                      If we extrapolate from your example a minimum of 2ms extra compile time per derive, this is adding >45s to the compile time for a debug build. But:

                      $ cargo clean && time cargo build
                      Finished dev [unoptimized + debuginfo] target(s) in 20m 58s
                      
                      real	20m58.636s
                      user	107m34.211s
                      sys	10m57.734s
                      
                      $ cargo clean && time cargo build --release
                      Finished release [optimized + debuginfo] target(s) in 61m 25s
                      real	61m25.930s
                      user	406m27.001s
                      sys	11m30.052s
                      

                      So number of dependencies and amount of specialization are probably the low hanging fruit in this case.

                      1. 1

                        Doh, bash fail.

                        $ rg --files -0 | grep -zE '\.rs$' | wc -l --files0-from=- | tail -n 1
                        2123768 total
                        $ rg --files -0 | grep -zE '\.rs$' | xargs -0 cat | grep '\[derive' | grep -oE '\(|,' | wc -l
                        22597
                        

                        Same conclusion though.

                      2. 2

                        Experiment independently reproduced, very nice results. I never realized this was significantly expensive!

                        1. 1

                          Thank you for this reply. It’s absolutely beautiful. You made an assertion, and this backs it up in a concise, understandable, and trivially reproducible way.

                      3. 0

                        Without build.rs how are create authors going to mine Bitcoin on your computer?

                        1. 2

                          With a make install target, just like the olden days.

                      4. 13

                        Here’s a bit of a counter-point: https://wiki.alopex.li/LetsBeRealAboutDependencies

                        Cargo and npm make all dependencies (including transitive dependencies) very visible. Languages that mainly use system libraries make only direct dependencies visible, and OS/distro/package managers take care of the rest. There’s still a lot of dependencies involved, but obtained and built in a less in-your-face way.

                        For example, if you add Rust’s reqwest it will pull in lots of stuff - async I/O, TLS stack, HTTP/2 implementation, gzip, brotli, punycode and Unicode tables for it. Looks very heavy! But libcurl.so has all the same things. You just don’t see them built in front of you, so it looks like 1 dependency, and not 150.

                        So it’s not necessarily more code that is pulled in. Where Rust is failing short is distributing dependencies in source form that takes time to compile, rather than some precompiled form.

                        1. 3

                          Where Rust is failing short is distributing dependencies in source form that takes time to compile, rather than some precompiled form.

                          This analysis is spot on. Rust should really fix this.

                          1. 1

                            Agreed, here’s the best background on this decision I’ve found:: https://gankra.github.io/blah/swift-abi/

                        2. 4

                          I dunno, the C++ application I’m working on at work takes about 10 seconds to build and link 1000 lines of code, and it has precisely two dependencies worth talking about. Rust is WAY faster to build. :-P

                          1. 3

                            Just to give some sort of an upper bound, I tried to build a Haskell program the other day from source. Stack launched builds successfully and worked for 27 minutes, after which part the process failed, having completed about 60% of all the builds I would’ve needed for that application.

                            Rust builds tend to be quite a lot speedier – not to mention that they tend to not fail.

                            1. 2

                              Rust builds tend to be quite a lot speedier – not to mention that they tend to not fail.

                              They sometimes fail for me. It’s always because of a language feature supported in a newer compiler. Sadly, there is no line in Cargo.toml saying how recent a compiler you need to compile a project. And to be honest, it’s a bit weird that such things are tied to the compiler and not a “language version” (“editions” in Rust). It makes it hard to be hopeful about a future with multiple Rust implementations.

                            2. 1

                              Reading this I have to wonder, are crates simply too big? The trouble with Rust dependencies is that if you want one function you have to compile the whole crate*. I like the thrust of this article, insofar as we should look more closely at we’re actually asking rustc to do rather than blaming it for all our problems. I think it gives slightly too much responsibility to the developer of the final application though—there is no guarantee that a lean alternative crate exists for a given task. Making crates smaller in scope or using feature flags more often(*) is a community-wide effort that would help substantially.

                              1. 5

                                Unfortunately, people also freak out when they see many dependencies. “OMG, this project has 274 dependencies! <insert left-pad joke>!”. This creates an incentive to make fewer, larger crates to bring down the scary number of dependencies, even if that actually increases the amount of unused code.

                              2. 1

                                I haven’t dug into it much yet, but wouldn’t something akin to ccache help with dependency compilation times?

                                1. 5

                                  It does to a degree; for example, Mozilla’s sccache is a ccache like tool with optional support for sharing build artefacts on cloud storage services which ships with support for the Rust toolchain (it works as a wrapper around rustc, so it should work even if you don’t use Cargo to build Rust code). Linking times can still be slow though, and obviously the cache isn’t useful if you’re running beta or nightly and frequently updating your compiler.

                                  1. 3

                                    Or using a build system like bazel or Nix.

                                  2. 1

                                    26s for whole-program compilation and 6s for sled compilation are pretty good. Rust compile times aren’t as bad as I thought.

                                    However, 26s is still very long when you’re working interactively. Does Rust support incremental compilation of a whole file? That is, when you recompile a file where only one function has changed, do you pay for recompiling the whole file, or just the changed function? How long does this take for moderately-sized files, say 500-1k lines? SBCL takes 34s to recompile my whole research project along with its two dozen dependencies, 0.13s to recompile the 400-line main file, and 0.03s to recompile one of the largest functions in that file (which can be done independently of the rest of the file) - perceptually instantaneous.