1. 46
  1.  

  2. 11

    That bit about “Rust itself can’t even properly clean up its memory” is not that it leaks (it doesn’t), but that the standard library calls abort() when it fails to allocate memory.

    The reasoning behind that design was that programs literally can’t run out of memory due to Linux’s overcommit and OOM killer. Fortunately, this is changing, and there will be an option to have it panic.

    1. 4

      As I’ve mentioned on lobsters before, abort-on-OOM isn’t good enough for implementing commercial databases either. So I’m pretty stoked for panic-on-OOM to land.

      Rust would be the perfect language to build a database. And in the 3 years since I wrote those comments Rust has only gotten better. I eagerly await the day Rust supplants C++ in every domain—I love writing Rust!

      1. 5

        I think you will find C itself does not ship with alternative allocators, you must build those yourself. C programmers build these core abstractions themselves a lot of the time, but nothing in rust stops you from doing the same with a slightly different mindset. In rust it is entirely possible to write your own collections and smart pointers that use custom allocators.

        I feel if you had the resources to write a commercial database, writing an alternative String or Vec would not be a challenge for you. You just have to avoid using ‘std’ for everything, which I feel is not that hard.

        1. 5

          Yes. However in C/C++, such things have been done for decades. My concern is less about what is technically possible, and more about what is a defensible business decision.

          1. 3

            You just have to avoid using ‘std’ for everything, which I feel is not that hard.

            Not totally necessary either, because even your custom-written types which use your special allocator will be able to yield &str and &[u8], and impl iterators.

            1. 2

              Even when using custom types, other functions in std allocate, and thus could potentially abort your entire database process. Using no_std would be the best option at this time.

              Of course there are useful databases without such levels of crash resiliency, e.g. Cassandra written in Java. But people don’t pay money to license Cassandra, do they?

              1. 2

                If you’re trying to avoid allocations at all, just don’t use libstd. Libcore has no allocator and has everything you need language wise. You can always copy non-allocating parts of libstd that you want into your code (or a supporting llibrary) directly, or copy allocating parts and modify them to handle failure.

          2. 3

            Rust would be the perfect language to build a database. And in the 3 years since I wrote those comments Rust has only gotten better. I eagerly await the day Rust supplants C++ in every domain—I love writing Rust!

            It is already. Rust already powers banking systems through TiKV by Pingcap. Just not as much in the US as rather in Asia.

            1. 2

              Indeed! Do you happen to know whether TiDB gets used as well as TiKV?

              1. 2

                Oh, sorry. My mistake. Actually, they all use TiDB, TiKV just being the storage layer.

        2. 8

          The pull request on hyper is pretty interesting to follow: https://github.com/hyperium/hyper/pull/2278

          1. 3

            But hyper depends on httparse which uses unsafe CPU-specific SIMD that people like me can’t understand from the first glance. There are a few tests but no mentions of fuzzing and no comments and explanations about the SIMD part. How safe is that?

              1. 2

                Curl has been getting more buggy since the introduction of HTTP/2. This is understandable, HTTP/2 is an order of magnitude more complex than HTTP/1.1. Hopefully, the Rust backend will allow alleviating those bugs.

                1. 2

                  I really like this approach over rewriting the entire thing in rust and changing input/output (like ripgrep, etc). Why change the interface if nothing is terribly wrong.

                  1. 7

                    ripgrep wasn’t a “RIIR” initiative. The grep interface has been changing in various other tools for a long time, starting with at least ack (written in Perl) in the mid 00s. The Silver Searcher also did it, and is written in C. There are various other grep alternatives written in C++, Python, Ruby and Go.

                    That is to say, plenty of people do take issue with grep’s interface. Or at least, find an alternative interface to work better for them in some very common usage scenarios.