Threads for kornel

freddyb 8 hours ago | on: Traversal-resistant file APIs

I wonder… does something like that exist in rust? I wouldn’t mind starting a crate like that as a side project :)

kornel 4 hours ago | on: Traversal-resistant file APIs

There is safe-path that I think is technically it, but it gives impression of being container-specific, and the crate has only a single 0.1.0 release.

There’s also fs_at building block.

kornel 27 hours ago | on: Here’s how I use LLMs to help me write code

I’ve tried leaning on LLMs on a couple of test projects, and ended up disappointed (I’ve tried Zed with Claude 3.7, ChatGPT 4o & o3-mini, and GitHub copilot).

It’s okay for 100-line programs, but gets cumbersome and counterproductive around 1000 lines. It’s creates a promising start, but it can’t finish what it started.

There’s no good way to get an LLM to refactor code other than by re-generating it. When a change needs to span more than a small function, this becomes slow and tedious. I can’t even leave it to rewrite a long program for as long as it needs to, because it may randomly stop half way. It requires babysitting. It never does things perfectly on the first try, so either I have to fix it myself manually, or wait again and check again and wait again.

In non-toy projects, the context window size becomes a problem too. Even if I just want to add a new function, the LLM needs to know the structure of my program – the functions available, the database schema, data structures (otherwise it’s forced to hallucinate them or reinvent duplicates). For programs that are more than a couple of files, providing the right context is tedious, and maxing out the context window gets expensive.

I’ve also tried Copilot for navigating large projects like LLVM and the Rust compiler, and it had no clue about them. It clearly just searched the codebase and tried to guess the answer from the results. That’s not smart, that’s just a second-hand retelling of grep.

legoktm 14 hours ago | on: Here’s how I use LLMs to help me write code

I find I get the most value out of the Codeium VS Code extension for larger projects. It primarily provides a smarter autocomplete, and you can also select a block of code, and prompt it to refactor it in a specific way.

Absolutely no idea how it would scale to projects the size of LLVM/rustc though. It definitely gets sluggish/worse results on larger files, so is definitely some context limitations.
1. kornel 13 hours ago | on: Here’s how I use LLMs to help me write code
  
  I’ve tried that as well, but I don’t get much value out of autocomplete. In the autocomplete mode there isn’t room to give precise instructions, so it doesn’t guess that well. The latency is high, so even when it guesses what I want, it doesn’t feel like it’s saving me time.

24

riki 2 days ago | on: Why Go's Error Handling is Awesome

I was hoping this article would compare if err != nil to more modern approaches (Rust’s ?) and not just Java-style exceptions, but unfortunately it doesn’t.

I’d be more interested to read an article that weighs the approaches against each other.

One point the article misses is how value-based error handling works really nicely when you don’t have constructors (either in your language, or at least your codebase, in case you’re using C++.)

8

weberc2 2 days ago | on: Why Go's Error Handling is Awesome

I’ve been pretty disappointed by Rust’s approach to error handling. It improves upon two “problems” in Go which IMHO are not actually problems in practice: if err != nil boilerplate and unhandled return values, but making good error types is fairly hard–either you manually maintain a bunch of implementations of the Error trait (which is a truly crushing amount of boilerplate) or you use something like anyhow to punt on errors (which is generally considered to be poor practice for library code) or you use some crate that generates the boilerplate for you via macros. The latter seems idyllic, but in practice I spend about as much time debugging macro errors as I would spend just maintaining the implementations manually.

In Go, the error implementation is just a single method called Error() that returns a string. Annotating that error, whether in a library or a main package, is just fmt.Errorf("calling the endpoint: %w", err). I don’t think either of them do a particularly good job of automating stack trace stuff, and I’m not sure about Rust, but at least Go does not have a particularly good solution for getting more context out of the error beyond the error message–specifically parameter values (if I’m passing a bunch of identifiers, filepaths, etc down the call stack that would be relevant for debugging, you have to pack them into the error message and they often show up several times in the error message or not at all because few people have a good system for attaching that metadata exactly once).

A lot of people have a smug sense of superiority about their language’s approach to error handling, which (beyond the silliness of basing one’s self-esteem on some programming language feature) always strikes me as silly because even the best programming languages are not particularly good at it, or at least not as good as I imagine it ought to be.
1. 9
  
  mikedorf edited 2 days ago | on: Why Go's Error Handling is Awesome
  
  a bunch of implementations of the Error trait (which is a truly crushing amount of boilerplate)
  
  you usually just need to impl Display, which I wouldn’t call a “crushing” amount of boilerplate
  
  or you use some crate that generates the boilerplate for you via macros.
  
  thiserror is pretty good, although tbqh just having an enum Error and implementing display for it is good enough. I’ve done some heavy lifting with error handling before but that’s usually to deal with larger issues, like making sure errors are Clone + Serialize + Deserialize and can keep stacktraces across FFI boundaries.
  1. 3
    
    weberc2 38 hours ago | on: Why Go's Error Handling is Awesome
    
    It’s pretty rarely “just” impl Display though, right? If you want automatic conversions from some upstream types you need to implement From, for example. You could not do it, but then you’re shifting the boilerplate to every call site. Depending on other factors, you likely also need Debug and Error. There are likely others as well that I’m not thinking about.
    1. 3
      
      mikedorf 38 hours ago | on: Why Go's Error Handling is Awesome
      
      #[derive(Debug)] and impl Display makes the impl of Error trivial (impl Error for E {}). If you’re wrapping errors then you probably want to implement source(). thiserror is a nice crate for doing everything with macros, and it’s not too heavy so the debugging potential is pretty low.
      
      One advantage of map_err(...) everywhere instead of implementing From is that it gives you access to file!() and line!() macros so you can get stack traces out of your normal error handling.
      1. weberc2 edited 34 hours ago | on: Why Go's Error Handling is Awesome
        
        thiserror is a nice crate for doing everything with macros, and it’s not too heavy so the debugging potential is pretty low.
        
        I’ve used thiserror and a few other crates, and I still spend a lot more time than I’d like debugging macro expansions. To the point where I waffle between using it and maintaining the trait implementations by hand. I’m not sure which of the two is less work on balance, but I know that I spend wayyy more time trying to make good error types in Rust than I do with Go (and I’d like to reiterate that I think there’s plenty of room for improvement on Go’s side).
        
        One advantage of map_err(…) everywhere instead of implementing From is that it gives you access to file!() and line!() macros so you can get stack traces out of your normal error handling.
        
        Maybe I should try this more. I guess I wish there was clear, agreed-upon guidance for how to do error handling in Rust. It seems like lots of people have subtly different ideas about how to do it–you mentioned just implementing Display while others encourage thiserror and someone else in this thread suggested Box<dyn Error> while others suggest anyhow.
        
        mikedorf 32 hours ago | on: Why Go's Error Handling is Awesome
        
        The rule of thumb I’ve seen is anyhow for applications and thiserror or your own custom error type for libraries, and if thiserror doesn’t fit your needs (for example, needing clone-able or serializable errors, stack traces, etc). Most libraries I’ve seen either use thiserror if they’re wrapping a bunch of other errors, or just have their own error type which is usually not too complex.
2. 8
  
  veqq edited 2 days ago | on: Why Go's Error Handling is Awesome
  
  a smug sense of superiority about their language’s approach to error handling
  
  Surprisingly, you don’t see people mention Common Lisp’s condition system in these debates
  1. 3
    
    weberc2 2 days ago | on: Why Go's Error Handling is Awesome
    
    That’s too bad, I genuinely enjoy learning about new (to me) ways of solving these problems, I just dislike the derisive fervor with which these conversations take place.
3. 6
  
  kornel 41 hours ago | on: Why Go's Error Handling is Awesome
  
  You discount anyhow as punting on errors, but Go’s Error() with a string is the same strategy.
  
  If you want that, you don’t even need anyhow. Rust’s stdlib has Box<dyn Error>. It supports From<String>, so you can use .map_err(|err| format!("calling the endpoint: {err}")). There’s downcast() and .source() for chaining errors and getting errors with data, if there’s more than a string (but anyhow does that better with .context()).
  1. 2
    
    weberc2 39 hours ago | on: Why Go's Error Handling is Awesome
    
    Ah, I didn’t know about downcast(). Thanks for the correction.
4. wrs edited 35 hours ago | on: Why Go's Error Handling is Awesome
  
  One source of differences in different languages’ error handling complexity is whether you think errors are just generic failures with some human-readable context for logging/debugging (Go makes this easy), or you think errors have meaning that should be distinguishable in code and handled by code (Rust assumes this). The latter is inherently more complicated, because it’s doing more. You can do it either way in either language, of course, it’s just a question of what seems more idiomatic.
  1. weberc2 34 hours ago | on: Why Go's Error Handling is Awesome
    
    I don’t think I agree. It’s perfectly idiomatic in Go to define your own error types and then to handle them distinctly in code up the stack. The main difference is that Rust typically uses enums (closed set) rather than Go’s canonical error interface (open set). I kind of think an open set is more appropriate because it gives upstream functions more flexibility to add error cases in the future without breaking the API, and of course Rust users can elect into open set semantics–they just have to do it a little more thoughtfully. The default in Go seems a little more safe in this regard, and Go users can opt into closed set semantics when appropriate (although I’m genuinely not sure off the top of my head when you need closed set semantics for errors?). I’m sure there are other considerations I’m not thinking of as well–it’s interesting stuff to think about!
    1. wrs edited 32 hours ago | on: Why Go's Error Handling is Awesome
      
      Maybe “idiomatic” isn’t quite the right word and I just mean “more common”. As I say, you can do both ways in both languages. But I see a lot of Go code that propagates errors by just adding a string to the trace, rather than translating them into a locally meaningful error type. (E.g.,
      
      return fmt.Errorf("Couldn't do that: %w", err)
      
      so the caller can’t distinguish the errors without reading the strings, as opposed to
      
      return &ErrCouldntDoThat{err} // or equivalent
      
      AFAIK the %w feature was specifically designed to let you add strings to a human-readable trace without having to distinguish errors.
      
      Whereas I see a lot of Rust code defining a local error type and an impl From to wrap errors in local types. (Whether that’s done manually or via a macro.)
      
      Maybe it’s just what code I’m looking at. And of course, one could claim people would prefer the first way in Rust, if it had a stdlib way to make a tree of untyped error strings.
      1. weberc2 32 hours ago | on: Why Go's Error Handling is Awesome
        
        But I see a lot of Go code that propagates errors by just adding a string to the trace, rather than translating them into a locally meaningful error type
        
        Right, we usually add a string when we’re just passing it up the call stack, so we can attach contextual information to the error message as necessary (I don’t know why you would benefit from a distinct error type in this case?). We create a dedicated error type when there’s something interesting that a caller might want to switch on (e.g., resource not found versus resource exists).
        
        AFAIK the %w feature was specifically designed to let you add strings to a human-readable trace without having to distinguish errors.
        
        It returns a type that wraps some other error, but you can still check the underlying error type with errors.Is() and errors.As(). So I might have an API that returns *FooNotFoundErr and its caller might wrap it in fmt.Errorf("fetching foo: %w", err), and the toplevel caller might do if errors.As(err, &fooNotFoundErr) { return http.StatusNotFound }.
        
        Whereas I see a lot of Rust code defining a local error type and an impl From to wrap errors in local types. (Whether that’s done manually or via a macro.)
        
        I think this is just the open-vs-closed set thing? I’m curious where we disagree: In Go, fallible functions return an error which is an open set of error types, sort of like Box<dyn Error>, and so we don’t need a distinct type for each function that represents the unique set of errors it could return. And since we’re not creating a distinct error type for each fallible function, we may still want to annotate it as we pass it up the call stack, so we have fmt.Errorf() much like Rust has anyhow! (but we can use fmt.Errorf() inside libraries as well as applications precisely because concrete error types aren’t part of the API). If you have to make an error type for each function’s return, then you don’t need fmt.Errorf() because you just add the annotation on your custom type, but when you don’t need to create custom types, you realize that you still want to annotate your errors.
      2. denz 32 hours ago | on: Why Go's Error Handling is Awesome
        
        This is true, usually you create a specific error type on the fly when you understand that the caller needs to distinguish it.
5. 3
  
  aconbere 2 days ago | on: Why Go's Error Handling is Awesome
  
  I tend to agree that rusts error handling is both better and worse. In day to day use I can typically get away with anyhow or dyn Error but it’s honestly a mess, and one that I really dread when it starts barking at me.
  
  On the other hand… I think being able to chain ‘?’ blocks is a god send for legibility, I think Result is far superior to err.
  
  I certainly bias towards Rusts overall but it’s got real issues.
  1. 5
    
    masklinn 42 hours ago | on: Why Go's Error Handling is Awesome
    
    There is one thing to be said against ?: it does not encourage the addition of contextual information, which can make diagnosing an error more difficult when e.g. it gets expect-ed (or logged out) half a dozen frames above with no indication of the path it took.
    
    However I that is hardly unsolvable. You could have e.g. ?("text") which wraps with text and returns, and ?(unwrapped) which direct returns (the keyword being there to encourage wrapping, one could even imagine extending this to more keywords e.g. ?(panic)` would be your unwrap).
    1. 1
      
      aconbere 37 hours ago | on: Why Go's Error Handling is Awesome
      
      In a chain i’ll just map_err which as soon as the chain is multiline looks and works well. Inline it’s not excellent ha.
      1. 1
        
        masklinn 37 hours ago | on: Why Go's Error Handling is Awesome
        
        Oh yeah I’m not saying it’s not possible to decorate things (it very much is), just pointing out that the incentives are not necessarily in that direction.
        
        If I was a big applications writer / user of type-erased errors, I’d probably add a wrapping method or two to Result (if I was to use “raw” boxed error, as IIRC anyhow has something like that already).
4

markwatson 2 days ago | on: Why Go's Error Handling is Awesome

I’ve often wondered if people would like Java exceptions more if it only supported checked exceptions. You still have the issue of exceptions being a parallel execution flow / go-to, but you lose the issue of random exceptions crashing programs. In my opinion it would make the language easier to write, because the compiler would force you to think about all the ways your program could fail at each level of abstraction. Programs would be more verbose, but maybe it would force us to think more about exception classes.

Tl;Dr Java would be fine if we removed RuntimeException?
1. 12
  
  masklinn 2 days ago | on: Why Go's Error Handling is Awesome
  
  You’d need to make checked exceptions not horrendous to use to start with e.g. genericity exception transparency, etc…
  
  It would also necessarily yield a completely different language, consider what would happen if NPEs were checked.
  1. 1
    
    glesica 2 days ago | on: Why Go's Error Handling is Awesome
    
    consider what would happen if NPEs were checked.
    
    Basically, Kotlin, so yeah, totally agree with you.
2. 6
  
  carlana edited 2 days ago | on: Why Go's Error Handling is Awesome
  
  No, Go has unchecked exceptions. They’re called “panics”.
  
  What makes Go better than Java is that you return the error interface instead of a concrete error type, which means you can add a new error to an existing method without breaking all your callers and forcing them to update their own throws declarations.
  
  The creator of C# explains the issue well here: https://www.artima.com/articles/the-trouble-with-checked-exceptions#part2
  1. 3
    
    gf0 46 hours ago | on: Why Go's Error Handling is Awesome
    
    You can just throw Exception (or even a generic) in Java just fine, though if all you want is an “error interface”.
    
    Java’s problem with checked exceptions is simply that checked exceptions would probably require effect types to be ergonomic.
4

ketchupfriend 2 days ago | on: Why Go's Error Handling is Awesome

Looks like it’s been updated:

Rust, for example, has a good compromise of using option types and pattern matching to find error conditions, leveraging some nice syntactic sugar to achieve similar results.

I’m also personally quite fond of error handling in Swift.
1. 2
  
  masklinn 2 days ago | on: Why Go's Error Handling is Awesome
  
  Rust, Zig, and Swift all have interesting value-oriented results. Swift more so since it added, well, Result and the ability to convert errors to that.
  1. 5
    
    matklad edited 2 days ago | on: Why Go's Error Handling is Awesome
    
    Zig’s not really value oriented. It’s more like statically typed error codes.

1

kornel 3 days ago | on: Medium data and small data

Not having access to >4GB at all in a process would be inconvenient, but maybe we can bring back the NEAR and FAR pointers from the 16-bit era, but this time for 32/64-bit pointers?

SSDs are getting so ridiculously fast now that it’s absolutely feasible to just naively load (or mmap) any large file into memory.

23

kornel 5 days ago | on: Why do I find Rust inadequate for text compression codecs?

The article looks more like a list of reasons to prefer Rust?

The function that C couldn’t vectorize looks okay to me in Rust. Rust’s stricter pointer aliasing helps autovectorize loops like this. In this case rse32(u8 data[restrict K], u8 out[restrict N]) improves C too, but Rust can apply it to any mutable slice, and has true immutability too.

Panicking bounds checks are annoying, but they’re optional. When optimizing hot code like that you’re going to spot them anyway (cargo asm makes it convenient). They can be eliminated in many ways — using iterators, or adding a length check outside of the loop, and there’s always unsafe that isn’t any less safe than indexing in C.

1

dist1ll 5 days ago | on: Why do I find Rust inadequate for text compression codecs?

It’s unfortunate that get_unchecked is so verbose. I wish there was an expression block syntax for disabling these checks, similar to unsafe.
1. 2
  
  hauleth 4 days ago | on: Why do I find Rust inadequate for text compression codecs?
  
  In theory that could be achieved with some macro magic.
2. 2
  
  Halkcyon 4 days ago | on: Why do I find Rust inadequate for text compression codecs?
  
  It’s “verbose” because it’s explicit.
  1. 4
    dist1ll 4 days ago | on: Why do I find Rust inadequate for text compression codecs?
    
    There are ways to make it less verbose without sacrificing explicitness. Compare this hypothetical syntax
    
    bounds_unchecked { zdotc.val[a] += Cd(&x[i]).val[a] * Cd(&y[i]).val[a]; zdotc.val[b] += Cd(&x[i]).val[b] * Cd(&y[i]).val[b]; }
    
    with this
    
    unsafe { *zdotc.val.get_unchecked_mut(a) += Cd(&x.get_unchecked(i)).val.get_unchecked(a) * Cd(&y.get_unchecked(i)).val.get_unchecked(a); *zdotc.val.get_unchecked_mut(b) += Cd(&x.get_unchecked(i)).val.get_unchecked(b) * Cd(&y.get_unchecked(i)).val.get_unchecked(b); }
    1. 3
      
      kylewlacy 4 days ago | on: Why do I find Rust inadequate for text compression codecs?
      
      You could also get more-or-less the same ergonomics without special syntax today. All you’d need is a wrapper type like UnsafeIndex with custom impls for std::ops::Index and std::ops::IndexMut that use .get_unchecked() / .get_unchecked_mut() internally
      
      (Playground example: https://play.rust-lang.org/?version=stable&mode=release&edition=2024&gist=163c07cefb3645b74d6ceea40ac6295e)
      1. 1
        
        dist1ll 4 days ago | on: Why do I find Rust inadequate for text compression codecs?
        
        It’s a decent workaround, but imo still too cumbersome. For example it’s hard to experimentally disable bounds checks for certain code sections during prototyping, because you would have to add/remove these assignments constantly.
        
        Also, I imagine a bounds_unchecked block to also disable bounds checks in a viral way, so that any part of the call graph that is downstream from this block also has bounds checks removed, including third-party libraries.
    2. 2
      
      intelfx edited 3 days ago | on: Why do I find Rust inadequate for text compression codecs?
      
      This would have been really nice to have in Rust. I’m not sure about making it viral, though — that would require monomorphising all callees over a hidden parameter, which is likely going to be a nightmare for a lot of reasons.
      
      I also dream of a block-style arithmetics mode override (checked/wrapping/saturating), something like:
      
      let x = wrapping! { a + b } + c
      
      This would have been so much nicer than the .add_*() syntax which becomes really ugly really fast.
      1. 3
        
        ekuber 3 days ago | on: Why do I find Rust inadequate for text compression codecs?
        
        Writing such a macro would be relatively straightforward, and would be surprised if it doesn’t exist already.

1

kornel edited 5 days ago | on: The Einstein AI model (why AI won't bring a "compressed 21st century")

I’m cautiously optimistic. There are ways in which AI could help advance science without being the smartest researcher itself. It can be a tool that multiplies people’s capabilities.

We advance science by standing on shoulders of giants, and AI can help people get quicker to the shoulder level. Even if asking good novel questions is a human thing, answering them involves building upon the existing knowledge.

We have experts in some fields, but it’s rare for people to be great is many fields at once. Even if AI just interpolates the existing knowledge, it can still help make new connections.

We have brilliant biologists who aren’t great programmers, and vice versa, but problems like protein folding require both. Often it doesn’t even have to be that cutting edge. We can start by not wasting researchers’ time spent on researching syntax for tables in LaTeX.

1

k749gtnc9l3w 5 days ago | on: The Einstein AI model (why AI won't bring a "compressed 21st century")

If we are talking about proven capabilities and real problems, statistical learning is already impressively good at translation, so search across the terminology shifts could benefit. Doesn’t even require the more iffy untethered-generative stuff that only mostly works!

If we are talking about worst case, a lot of current development is controlled by people who are avoiding the advanced-tool-for-highly-competent-humans framing, so some shiny-appliance-wrapping can destroy the potential of larger commercial offerings for anything moving the progress forward.

There is a hope that even in that case a bit of academia optimisation will continue so that at the LaTeX level of ease-of-use the monthly-PhD-student-salary-price level of hardware becomes enough to get the things that work with freeware-distributed models.

15

kornel 5 days ago | on: Zig's dot star syntax (value.*)

The C’s prefix syntax has been used for so long that .* looks weird, but a postfix operator is the right place for a dereference. It chains nicely, and if you squint, it’s consistent with field access syntax that is also a form of dereference.

11

snej 5 days ago | on: Zig's dot star syntax (value.*)

The “.” is weird, though — it makes it look like a struct field. Pascal’s postfix “^” operator is clearer.

(An article explaining pointer dereferencing seems maybe a bit basic for lobste.rs…?)
1. 6
  
  xq 5 days ago | on: Zig's dot star syntax (value.*)
  
  Think about it as just “field access, but with a wildcard”
  
  Instead of .field you do .* which matches all fields, and suddenly it feels more natural than another “special syntax”
  1. 3
    
    matklad 5 days ago | on: Zig's dot star syntax (value.*)
    
    According to this mental model, .? should return arbitrary one field :D
    1. 2
      
      xiaq 4 days ago | on: Zig's dot star syntax (value.*)
      
      It should return all fields whose names are one character :)
2. 2
  
  edwardloveall 5 days ago | on: Zig's dot star syntax (value.*)
  
  I was actually coming here to say that I really appreciate the well-written, intro-level post. I’ve been programming for almost 15 years professionally but I’ve spent all that time in high-level languages and am still really “getting” pointers. Programming is such an enormous field, there’s always something to learn. 🙂
3. 1
  
  kyrias 5 days ago | on: Zig's dot star syntax (value.*)
  
  Do you feel the same way about calling methods with .?
  1. 1
    
    snej 5 days ago | on: Zig's dot star syntax (value.*)
    
    No, a method is an element of the pointed-to struct, just like a field access. It’s the use of “.” without any field that seems weird to me.
    1. 2
      
      dfawcus 5 days ago | on: Zig's dot star syntax (value.*)
      
      Funny, I quite like it. If I ever create a language of my own I’d probably use that.
      
      Maybe it is just from being exposed to Go, with it’s “ptr.(type)” and “ptr.(int)” forms for type switch and assertion which make it seem a bit more comfortable.
6

doug-moen 5 days ago | on: Zig's dot star syntax (value.*)

Adding to snej’s comment, in Pascal you write p^ to dereference a pointer and r.f to select a field from a record. If you have a pointer to a record, you can write pr^.f. It’s simple, orthogonal and logical.

The C language made the ergonomic mistake of using a prefix operator to dereference a pointer (*p). To get a field from a struct pointer, this means you write (*rp).f. That sucks so badly that C has a short-form for this: you can write rp->f instead. It’s an over-complicated bad design.

Zig collapses the C operators . and -> into a single . operator, and replaces C’s one-character prefix pointer-deref operator * with a two-character postfix operator .*. This is a bit more ergonomic than C, but it’s still a bad design, when compared to Pascal.
1. 4
  
  masklinn 5 days ago | on: Zig's dot star syntax (value.*)
  
  Not to defend prefix dereference, but it’s not like you can’t fix C’s mistakes. Rust supports rp.f just fine by auto-dereferencing until it finds the right type and doesn’t need -> or an explicit deref for trivial accesses
  
  The order of operations issue still exists, but to a much lesser extent.
2. 3
  
  jackdk 5 days ago | on: Zig's dot star syntax (value.*)
  
  Huh, I wonder if that Pascal design is why the Haskell lens packages uses ^. as an alias for view. As in, record ^. foo . bar . baz = view (foo . bar . baz) record accesses a value nested through several layers of record.
3. 3
  fanf 5 days ago | on: Zig's dot star syntax (value.*)
  BCPL has two forms of the dereference operator, prefix unary and infix binary. The latter does double duty for structure access and array indexing.
  
  C BCPL *ptr !ptr ptr->field ptr!field ptr[n] ptr!n
  
  In pre-K&R C before the type safety of structures was improved, C’s ptr->field was a lot more like BCPL’s ptr!field, but with a nicer way to declare the field offsets and types. But like BCPL, the left hand operand could be anything. For example, in the 6th edition kernel there are expressions (after macro expansion) like 0177560->integ for accessing hardware registers.
  
  Late 1960s versions of BCPL had the rather awkward lv and rv operators for taking addresses and indirection; they became @ and ! in the early 1970s. https://www.softwarepreservation.org/projects/BCPL/
  
  The lv / rv thing was from Strachey’s idea of lvalues and rvalues. The CPL project seems to have struggled a lot with references, list processing, and record types. The whole effort fell apart before they worked out a solution.
  
  Dunno how much users in the USA kept up with revisions to BCPL after Martin Richards returned from Cambridge Massachusetts to Cambridge England.
2

xigoi 4 days ago | on: Zig's dot star syntax (value.*)

I like how Nim uses p[] for dereference. It’s like indexing an array, but without an index.

21

ansible-rs 8 days ago | on: Avoid the nightmare bicycle

Actually, the popcorn button on microwaves can potentially be offering functionality beyond what a time and power setting can accomplish. Some microwaves have ambient moisture sensors, and some have microphones.

Technology Connections tells you all about it: https://www.youtube.com/watch?v=Limpr1L8Pss

What would be cool is if the full functionality of these sensors was available to the end user, who had a way to create and modify the preset functions. I don’t know if the market is ready for that however.

5

kevincox 8 days ago | on: Avoid the nightmare bicycle

Yes. I think this falls into a different category of features. Also in most cases this setting is still power level X and then run for Y seconds after first steam detected, which could be programed by a user fairly easily.

Also most microwaves these days don’t actually have a steam sensor unfortunately. The majority of popcorn buttons just have a preset time.
3

kornel 7 days ago | on: Avoid the nightmare bicycle

OTOH all bags of microwave popcorn have “DO NOT USE THE POPCORN BUTTON” on them. Collectively, the microwave makers have failed to create a predictable, consistent popcorn button. The smart ones are smart in different ways, the dumb ones are dumb in their own ways, and it’s hard to even tell which is which.
1. 1
  
  fanf 7 days ago | on: Avoid the nightmare bicycle
  
  I bet you watched this Technology Connections video https://youtu.be/Limpr1L8Pss
1

freddyb 8 days ago | on: Avoid the nightmare bicycle

You’re aware the article is not about popcorn or popcorn buttons on microwaves, aren’t you? :=
1. 12
  
  ansible-rs 8 days ago | on: Avoid the nightmare bicycle
  
  Yes, I know.
  
  But we have to be careful with our analogies. And sometimes… sometimes, there is more thought and effort into creating such a “nightmare bicycle” user interface than there may appear to be on the surface. And maybe there is a good reason for some design choices.
  
  Not always, of course. As /u/kevincox mentions, some microwaves don’t have a moisture sensor, but present the same user interface as one that does. Which is very unfortunate.

3

pyj 11 days ago | on: Is Rust a good fit for business apps?

you land in a space where everything has at least 7 different crates available, but half of them are actually a toy projects and most of them were not maintained for the last 5 years

The other side of the dependency problem I’m curious about is how teams efficiently get crates through security and legal review and how they handle updates to those crates.

12

kornel 11 days ago | on: Is Rust a good fit for business apps?

I’m biased, but I think crates.io search is bad (does not have active maintenance as a ranking signal), and contributes to the perception. When looking through the lens of https://lib.rs it’s much better.

As for legal and security: with tooling. cargo-deny for licensing. cargo-vet to ensure that deps are either from trusted authors, or had their code reviewed.
1. 4
  
  atk 11 days ago | on: Is Rust a good fit for business apps?
  
  I don’t know, maintenance matters for a HTTP library, it doesn’t matter for a strictyaml parser. One of those is constantly evolving, and the other is a standard which is immutable and won’t change. There’s an argument to be made that Rust is itself evolving and so dependencies should evolve with it, but a lot of the time it’s fine if some things were written and worked on over the course of a few months and then left. There’s plenty of perfectly serviceable crates out there like that. I encounter them all the time when doing embedded stuff. The ADS1015 is a chip with a know interface and once you write a working driver for it, you’re done. Now embedded HAL might come out with a 2.0, but until then, you don’t need to touch it.
  1. 5
    
    ssokolow 11 days ago | on: Is Rust a good fit for business apps?
    
    …which reminds me of this quote from a video on the advantage of Rust being so focused on correctness and “fearless upgrades”.
    
    They’re not abandoned; they’re done.
    
    – No Boilerplate, Stop writing Rust @ https://www.youtube.com/watch?v=Z3xPIYHKSoI
  2. 1
    
    kornel edited 11 days ago | on: Is Rust a good fit for business apps?
    
    That’s optimizing the general case for an edge case.
    
    There are some old “done” crates, but literally 99.5% of old crates are junk. With such a terrible false positive rate it’s hard to even find the timeless ones.
    
    These are real numbers — I’ve actually checked. I’ve searched for crates that are old but good, using multiple methods, and I could not find more than about 100 that may be useful. 150 is scraping the bottom of the barrel. That may sound like a significant number, but that was among 31000 old crates. That’s 200 junk crates for every non-junk one. The timeless ones aren’t even that good — they’re mostly unoptimized classic hash functions, simple macros, and opinionated wrappers for std functions.
    
    Sturgeon’s law holds true for the Rust ecosystem too. In my estimates, 11% of crates are good, 25% maybe okay, and the quality goes sharply down from there. It’s full of “hello world”s, abandoned forks, unfinished prototypes, namesquatting and various grades of spammy behavior.
    
    Rust has more “done” crates, but most of them still release some minor patch every few years. It’s really not a big ask to make a release once every 3-5 years just to mark a crate as still usable. At least there’s an excuse to bump Rust edition.
    
    Avoiding hard decisions about ranking to give a chance to every edge-case crate doesn’t work. You just get search results full of crap, and ironically the crap crowds-out the good crates, making them harder to find.
    1. 2
      
      atk edited 10 days ago | on: Is Rust a good fit for business apps?
      
      Is it really the edge case? I am not entirely convinced.
      
      The fact that 99.5% of crates are junk doesn’t really matter in this discussion since that’s true for most code written and the only difference is that here the barrier to entry (to publishing) is very low.
      
      I think if crates got marked “WIP” by default and non-WIP crates got prioritised over WIP ones in search results, you could probably solve most of these problems. Kindly ask people to consider whether they consider their crate ready for general use or not, and I think most of these unfinished crates would just never drop their WIP marker.
      1. 2
        
        fanf 10 days ago | on: Is Rust a good fit for business apps?
        
        If only there were a version numbering scheme that came with a WIP marker built in!
        
        1
        
        atk 8 days ago | on: Is Rust a good fit for business apps?
        
        Pre 1.0 when using semver isn’t quite the same as WIP, it’s a useful region where “nothing is compatible with anything else” which means you can just keep bumping the minor version. The alternative is to always bump the major version but this would now imply “not WIP” which might also not be correct.
2. 1
  
  pyj 10 days ago | on: Is Rust a good fit for business apps?
  
  The tooling is nice, but my curiosity is more along the lines of “this is how our programmers work with the legal and security departments to get and keep our package list approved.” Poking around, it looks like what I really want is an article about operationalizing something like OpenSSF’s Securing Software Repositories in a corporate environment.

6

kornel 11 days ago | on: Is Rust a good fit for business apps?

I think it’s mostly fair assessment.

Rust requires more of implementation details to be visible, like the locks, dynamic dispatch, or mutability of references. Compared to Java or Go, this is a pedantic level.

However, I still like strictness of Rust’s type system. Especially exclusive ownership makes APIs clearer and more robust – I never need to worry if an object my function is working on could be mutated from somewhere else.

The advantage in multi-threading isn’t about async at all, or mere ability to use many cores. It’s about whole-program guarantee there’s no accidental sharing and data races that could lead to data corruption, even in complex programs. This is a hard problem, and in Go it takes a lot of vigilance where you have both first-class multithreading and non-thread-safe built-in collections.

10

kornel 11 days ago | on: Thoughts about rustdoc

I like it, but it has some flaws:

it has no room for chapters with non-reference docs, like tutorials, getting started. That can be stuffed into module-level docs, but that’s not ideal.
it sorts types alphabetically, not in order of importance, nor even order of definition. All types in a module are thrown into one list. That makes it hard to find how to use it if you don’t already know what to search for. In practice libraries have some main type you use to initialize them, but good luck finding it among all the minor error types, newtype wrapper types, iterator types, and all kinds of helpers.
it takes a skill to understand the trait implementation sections. There’s a lot of boilerplate and noise there. Traits repeat all their methods even when this is redundant. It’s important to know if a type is an Iterator, but not list the same builtin 100 iter methods every time.
Apart from blanket impls (which are boring noise with too much prominence), there’s no UI distinction between standard traits, crate-local traits, foreign traits. These are usually implemented for very different reasons.
if you see Pattern arg in std, you won’t know it can be a closure with various arguments. Figuring this out needs diving deep into the trait impls and their bounds.
it doesn’t handle big types with lots of methods well. There is no explicit support for grouping them (str has search methods, has splitting, case changes, but they’re all mixed in the nav). It doesn’t even de-emphasise deprecated or nightly methods. I’d like _mut() and non-mut grouped as two flavors of the same method. I’d like to see “static” functions distinguished from self methods.

It’s like cargo – good enough, and all the value is in having it consistently for every crate.

1

kaycebasques edited 11 days ago | on: Thoughts about rustdoc

it sorts types alphabetically, not in order of importance, nor even order of definition

We debate this all the time on pigweed.dev. Among Pigweed contributors opinion is roughly split: one half prefers organizing alphabetically, the other half prefers organizing by order-of-importance.

If I was benevolent dictator of rustdoc I personally would stick to alphabetical organization and would never allow alternate organization schemas. As you said, the value is the consistency. Alphabetical may be sub-optimal in terms of finding what you need as quickly as possible, but it’s a system that can be unambiguously enforced in every library in the entire ecosystem. That’s a really powerful level of consistency. Also, the most logical organization for the crate owners is sometimes not the most logical organization for crate users, and in those cases the order-of-importance organization may actually be less effective than the highly predictable alphabetical organization. From the perspective of hypothetical rustdoc benevolent dictator, there’s no way for me to guarantee that the order-of-importance that the crate owner has decided upon is the same as the order-of-importance for crate users. At least, much more difficult to guarantee than alphabetical organization.

From Diataxis:

Reference material is useful when it is consistent. Standard patterns are what allow us to use reference material effectively. Your job is to place the material that your user needs know where they expect to find it, in a format that they are familiar with.

Very interesting to see where exactly rustdoc falls short from someone who has obviously looked over many real, non-trivial API references extensively. Thanks for the details.
1. 5
  
  kornel 11 days ago | on: Thoughts about rustdoc
  
  I think a lot of appeal of alphabetical sorting is merely a comfort of sticking with something we’ve been always doing, even though the original reason for it — making it possible to search on paper — is gone. It’s more a consistency to have consistency, than to solve a user problem.
  
  Rustdoc has an instant search. It can match more than just the prefix, and even supports name aliases. It’s almost a disservice to users to make a UI that suggests they can search lists manually, limited to just the prefix (is it under file_open, open_file, load_file, try_get_file?).
  
  Alpha sorting creates implicit grouping of names with common prefixes, but that has systemic failures — new, from_, with_, and builder constructors are all over the place. as_, into_ and to_ are scattered. try_ methods are divorced from their panicking alternatives.
  
  As a library author, I think I have a pretty good idea of what is most important in my library. Even if there can be different views, it’s not going to be the type starting with A. I’d rather have the ordering usually helpful than consistently irrelevant.
  1. 3
    
    fanf 10 days ago | on: Thoughts about rustdoc
    
    In my experience, a conceptually-grouped list takes a lot of thought to do well, and requires careful maintenance. It’s often the case that there are several categorizations that cut at different angles, so functions might need to be listed in multiple groups. Generally I like to see that kind of thing in handwritten overview section, in addition to a comprehensive automatically maintained alphabetically sorted reference section.
    
    I like to add cross-references to closely-related functions. If functions are grouped, maybe the groups should be explicit tags in each function’s description, so a reader can jump to the list of functions tagged in the same group.
    
    Reference documentation can be too DRY. If two different ways of organizing it are at odds, the tools should probably help us to produce both ways, not just one.
  2. 2
    
    kaycebasques edited 10 days ago | on: Thoughts about rustdoc
    
    merely a comfort of sticking with something we’ve been always doing
    
    The value IMO is that everyone is familiar with this organizational scheme and can recognize it very quickly, à la Don’t Make Me Think.
    
    As a library author, I think I have a pretty good idea of what is most important in my library
    
    Yes, this is a fair point. I wrote the library, I know the core use cases.
    
    How do you imagine the mechanics of order-of-importance organization working though? Marking up each API item with an attribute would be toilsome. I guess rustdoc could spit out a flat JSON list of all API items, and then it’s just a matter of re-organizing the list…
    
    (Or a nested dict, to indicate grouping of API items)
    1. 1
      
      kaycebasques 10 days ago | on: Thoughts about rustdoc
      
      That same Diataxis page I linked to before does however make an argument along your lines:
      
      The way a map corresponds to the territory it represents helps us use the former to find our way through the latter. It should be the same with documentation: the structure of the documentation should mirror the structure of the product, so that the user can work their way through them at the same time.
      
      It doesn’t mean forcing the documentation into an unnatural structure. What’s important is that the logical, conceptual arrangement of and relations within the code should help make sense of the documentation.

7

kornel edited 14 days ago | on: Standard proposal for memory-Safety in C (2024)

This proposal is very shallow and full of wishful thinking.

It doesn’t get into any of the hard problems that break static analysis in C, but at the same time leaves fixing its soundness holes and severe restrictions to be “(hopefully)” solved with static analysis later. It says they want fast compilation speed and “avoid expensive or complicated static analysis”, without any plan to deliver that. It’s a “TODO: fix undecidability”.

The STATIC subset is restricted beyond any usefulness. It allows less than constexpr.

The DYNAMIC subset is still very limited (no memory management, only fixed-size arrays), but adds pervasive run-time checks to everything, and no language constructs to avoid them. restrict is banned instead of being required. There’s no word about use-after-free.

2

dfawcus 14 days ago | on: Standard proposal for memory-Safety in C (2024)

Their definition of “memory safety” appears to simply be “spacial safety” (bounds checks):

We consider a memory-safe operation an operation that (under certain preconditions specified below) can be shown at compile-time not to cause any run-time out-of-bounds accesses, i.e. accesses outside the bounds of the objects accessible via reachable pointers.

As such, given the bounds check clang changes which are currently being upstreamed, this proposal strikes me as already obsolete. I intend to try applying that at work once it arrives in an upstream version of the compiler, and see what cost and impact it has for us.

As to some of the details of the proposal, certainly in previous instances of use of “[static 1]” for pointers, I’ve found that checks based on it seem to be quite limited, and easily skipped by the compiler. I can’t recall if it was clang or gcc which had the better (but limited) support.

10

dataangel 15 days ago | on: zlib-rs is faster than C

Always good to get more confirmation that Rust can go toe to toe with C, but without an analysis of why I assume it comes down to some minor algorithmic difference or trickery that the C implementation just lacks, rather than an artifact of the language design.

17

moltonel edited 15 days ago | on: zlib-rs is faster than C

Not sure what you would like to see (dis)proven here, with enough man-hours spent optimizing you can find the optimal machine code with either language.

A more interesting question is how much of that time is wasted on sidequests like eliminating UB or bound checks. The differentiator is rarely the theoretical max performance of C/Rust, but the performance you can achieve with a certain amount of work. And that’s clearly an artifact of language design.
1. 8
  
  brucehoult 15 days ago | on: zlib-rs is faster than C
  
  Or side quests such as convincing the borrow checker that you do, in fact, know what you’re doing?
  1. 20
    
    wrs 15 days ago | on: zlib-rs is faster than C
    
    At least you know when you’re done with that side quest, unlike finding the UB you missed…
  2. 11
    
    kornel 14 days ago | on: zlib-rs is faster than C
    
    Once you learn how to work with ownership+borrowing, it’s not a big deal.
    
    There is a lot of headbanging and frustration in the beginning when learning Rust, especially for C users who instinctively try to use Rust references as they would use pointers, but that goes away with practice.
  3. 1
    
    moltonel 14 days ago | on: zlib-rs is faster than C
    
    All languages have side quests like this. They differ in the skill/time required to diagnose/fix them, in how often they come up, in how far they take you from the direct path… It’s hard to make a definitive assessment and everybody’s adventure will be different, but I think more and more people see the advantages of Rust.
2. 4
  
  dataangel 14 days ago | on: zlib-rs is faster than C
  
  Yes but nearly all claims of “faster than C” come down to comparing two different algorithms rather than a language difference. A disproof of this would be something like being able to attribute the difference to better codegen that was allowed due to Rust’s aliasing semantics, but it’s almost never that.
  1. 3
    
    moltonel 14 days ago | on: zlib-rs is faster than C
    
    It’s almost never that because there’s almost never a difference in codegen, and when there is one it’s tiny.
    
    But focusing on that is IMHO missing the point of these stories: that using a safer language doesn’t mean you have to give up performance, and that given finite resources you might even get better performance with Rust because the better algorithms take more time / skill / maintenance to get right with C than Rust.
11

zesterer 14 days ago | on: zlib-rs is faster than C

I think the thing being done here is not pointless navel-gazing about which language is faster. It’s just getting rid of the “but C is faster and my application is performance-intensive above all other concerns so I should use C” argument. Choosing a language based on presumed performance alone is already a bad judgement call, but at least that bad judgement call will err on the side of memory safety now.
1. 3
  
  dataangel 14 days ago | on: zlib-rs is faster than C
  
  It doesn’t dispel anything unless the algorithms are actually the same, that’s the point. If you write a bad algorithm in C, and I write a better one in Python and get better performance, that’s not evidence Python is faster than C.
  1. 4
    
    zesterer 14 days ago | on: zlib-rs is faster than C
    
    The relative difficulty of making the code faster matters. With enough coercing almost any compiled language can be made to be almost arbitrarily fast. But keeping the code maintainable and not spending forever on it as well? That’s a tougher nut to crack.
2. 2
  
  cblake 14 days ago | on: zlib-rs is faster than C
  
  , but at least that bad judgement call will err on the side of memory safety now.
  
  Prediction is hard. As a proof one could do better than than an ancient C impl without memory unsafety, Nim had Zippy almost 5 years ago. Deflate/inflate have been very slow/far from state of the art almost since their inception. Their popularity (or the uptake of any given implementation), like popularity of anything, is not so easy to predict. Personally, I have preferred either pixz or zstd for multiple decades and was very grateful when kernel.org finally started distributing .xz tar balls that could be decompressed in parallel which is only very recently. So, we’ll see what happens.
  1. 2
    
    zesterer 14 days ago | on: zlib-rs is faster than C
    
    I don’t think it’s the popularity of zlib-rs that matters here.

17

david_chisnall 15 days ago | on: Smart Pointers Can't Solve Use-After-Free

The article is a bit more subtle than the title suggests:

Smart pointers don’t solve iterator invalidation.

Iterator invalidation is a very hard problem. GC’d languages can have memory-safe iterator invalidation problems that cause surprising behaviour. If you have a tree of buckets and deletion causes a bucket to be disconnected from the tree, then your iterator can keep holding a reference to that bucket but now can’t find the next bucket and will do the wrong thing.

You could implement memory-safe but not mutation-safe iterators in C++ in a few ways. For example, imagine if std::vector used a std::shared_ptr to its underlying buffer and represented iterators as a std::shared_ptr to the same buffer and an index. Insertion may cause the buffer to be reallocated but you will keep iterating over the range.

I have never been a fan of how C++ uses iterators. They are disconnected from the underlying type. Objective-C’s fast enumeration protocol is interesting. You pass the collection a structure for storing iteration state (which includes a buffer for storing objects) and it populates this with a pointer to a buffer, a length, and a pointer to mutation state. Normally this is implemented by a version counter in the collection that is incremented on mutations that would invalidate iterators. Iteration is a thing that collections do, rather than iterators being a thing that visits a collection.

Rust solves this by preventing mutation of the data structure while iterators exist. I can’t remember the details, but I think this also precludes some safe operations (for example, in C++ it’s fine to mutate elements, swap elements around, or replace an element in a collection it just isn’t safe to change the shape of the collection itself).

Swift, I believe, solves this by making most collections immutable CoW, so you can mutate a CoW copy of a collection during iteration and that’s fast, but you can never modify the original collection (though you can then replace the old collection with your new CoW copy and free the old one, which may or may not involve actually freeing any memory).

12

kornel edited 15 days ago | on: Smart Pointers Can't Solve Use-After-Free

It’s pretty cool how many things lifetimes + borrowing fix from general rules.

Rust doesn’t have anything special for iterators. It fixes the problem by having lifetimes (the iterator can’t outlive the collection) and shared-immutable vs exclusive-mutable access. While the iterator borrows for shared access, you can’t move or mutate the collection. When iterator borrows mutably, it gets exclusive access. Mutable iterators can hand out mutable references, so you can still swap elements while you iterate, in some cases.

Locked MutexGuard borrows from Mutex<T>, and you borrow &T from the MutexGuard. It naturally ensures that the mutex can’t be destroyed while anything has a lock on it, and bare &T reference can’t be used after the mutex is locked again. This works even tough Rust-the-language has no idea what locks are.
1. 12
  oconnor663 edited 15 days ago | on: Smart Pointers Can't Solve Use-After-Free
  There’s even wacky Cell magic you can use to modify elements while you iterate over the collection, even if the collection itself doesn’t use Cell (playground link):
  
  let mut v = vec![1, 1, 1, 1, 1]; let slice = Cell::from_mut(&mut v[..]).as_slice_of_cells(); for (i, e) in slice.into_iter().enumerate() { for j in i..slice.len() { slice[j].set(slice[j].get() * 2); } print!("{} ", e.get()); } println!();
  
  That prints “2 4 8 16 32”. I could totally imagine &Cell<T> being a first-class Rust reference type, in the same way that &pinned T might be someday. It’s not at all clear to me that it would actually be worth its weight as a feature, but I think it’s neat.
  1. 3
    
    muvlon 15 days ago | on: Smart Pointers Can't Solve Use-After-Free
    
    This is pretty neat! But it relies on the fact that you can borrow Vec as just a plain slice, right? So you can use as_slice_of_cells.
    
    For a more complex collection like a tree, I think you would either have to make the Cell part of the datastructure or to extend Cell with a as_tree_of_cells function. Or maybe some advanced GhostCell shenanigans?
    1. 5
      
      oconnor663 15 days ago | on: Smart Pointers Can't Solve Use-After-Free
      
      Yeah some sort of cast from &mut Collection<T> to &Collection<Cell<T>>. I’m not sure how to make that generic. The expensive approach that works today is to build a whole new collection of references:
      
      let mut set1 = BTreeMap::new(); for i in 0..10 { set1.insert(i, i); } let mut set2 = BTreeMap::new(); for (&i, val) in &mut set1 { set2.insert(i, Cell::from_mut(val)); }
6

oconnor663 15 days ago | on: Smart Pointers Can't Solve Use-After-Free

Rust solves this by preventing mutation of the data structure while iterators exist. I can’t remember the details, but I think this also precludes some safe operations

Rust has two separate problems to worry about here. The first is whether resizing a collection might invalidate iterators, as in C++. The second is whether iterators could be used to violate the no-mutable-aliasing rules in any way. I think the second thing ends up being a stricter requirement than the first thing in most (all?) cases. It can be confusing because mutable aliasing violations “feel” safe, in that they are mostly legal in C/C++, but they are genuinely unsound in Rust, because of all the no-alias annotations the compiler passes to LLVM.
1. 6
  
  muvlon 15 days ago | on: Smart Pointers Can't Solve Use-After-Free
  
  You can disentangle the mutability of the collection itself from the mutability of the values inside by adding interior mutability -so Collection<Cell<T>> or Collection<Mutex<T>> for example. Then, you can borrow and iterate over the collection immutably but get mutable access to its element from the iterator.
2. 3
  
  rpjohnst 15 days ago | on: Smart Pointers Can't Solve Use-After-Free
  
  Yes- though those two things are intertwined, because the no-mutable-aliasing rule is the very same mechanism that prevents you from resizing the vector while iterating over it.
  
  If you had a more relaxed rule that just tried to prevent iterator invalidation, but it didn’t prevent you from getting mutable aliases to its elements, you could turn around and use a Vec<Vec<T>> to invalidate an iterator into an inner Vec.
  1. 2
    
    oconnor663 14 days ago | on: Smart Pointers Can't Solve Use-After-Free
    
    Very true. My favorite example of this is using an enum, because you don’t even need heap allocation to do it: https://stackoverflow.com/a/67608729/823869
4

snej edited 15 days ago | on: Smart Pointers Can't Solve Use-After-Free

Maybe we should add Iterator management as a third (fourth?) hard thing in computer science.

I keep running into it (just last week in fact) as part of implementing notifications: this object has a list of observers, and when its state changes it has to iterate that list and call all the observers. Only it’s entirely possible that one of those callbacks will remove the observer from my list. Plus this has to be thread-safe…

(Note that invalidating the Iterator won’t help here. It has to be possible to add/remove observers in a re-entrant call during iteration. I’ve taken to copying the list and iterating the copy, but that has some undesirable behaviors with multiple threads, to wit, an observer can be called after it’s been removed.)
1. 1
  
  david_chisnall 14 days ago | on: Smart Pointers Can't Solve Use-After-Free
  
  It’s very common in C++ to implement something that’s really an ad-hoc monad for these things: iterate over a collection and build a list of changes that you want to apply, then apply the changes. Haskell programmers probably laugh at us.
  1. 3
    
    tonyarkles 14 days ago | on: Smart Pointers Can't Solve Use-After-Free
    
    Or the crazy erase(remove_if( pattern where it moves all of the “don’t need to be erased” elements to the start of the list and returns an iterator to the first element to remove… and if you happen to forget that you have to call it like foo.erase(remove_if(…), foo.end()) and accidentally call foo.erase(remove_if(…)) you end up calling a valid overload of erase() that doesn’t at all do what you expect…
    
    Not that I’ve been burned before…
2. 1
  
  Diana 15 days ago | on: Smart Pointers Can't Solve Use-After-Free
  
  I feel like iterator difficulties are mostly tied to concurrent access/write problems. It is all in the same “context” but in fact, you can see how it “leaks”.
  1. 7
    
    ssokolow 15 days ago | on: Smart Pointers Can't Solve Use-After-Free
    
    Whenever I see this sort of discussion, I’m reminded of Manish Goregaokar’s The Problem With Single-threaded Shared Mutability from 2015… though it references earlier things like Niko Matsakis’s On the connection between memory management and data-race freedom from 2013.

3

jcspencer 18 days ago | on: Ring is unmaintained

Slightly off-topic, but I have been very keen to see the rustcrypto project evolve, (and hopefully one day see formal verification).

Their use of strong traits (e.g. aead) make it very easy to enforce safety constraints directly in the type system. For example, these traits have already been used to wrap ring itself to allow easy swap-outs of cryptographic backends.

I often find libraries that will only accept, say, use of a file-backed private key. Instead, having libraries accept a generic signer trait implementation allows agility between cryptographic backends, such as different hardware devices or crypto libraries.

With cryptographic libraries in Rust being far from a settled debate, I strong believe there’s massive value in settling on standardized trait implementations for cryptographic libraries in the crate ecosystem to make these kinds of switches easier going forward.

7

kornel 17 days ago | on: Ring is unmaintained

OTOH, I strongly dislike the traits. They make usage of basic primitives more difficult. Their interface is spread over multiple crates, and everything goes through layers of indirection. It’s super annoying if I just need sha256, but need to import traits and convert from a GenericArray.

They also make semver-major changes too often for the ecosystem to keep up. A Signer trait is more trouble than it’s worth when you end up with a v0.9 implementation not working with a v0.11 interface.
1. 2
  
  jcspencer 16 days ago | on: Ring is unmaintained
  
  That’s fair, I can’t argue with that - I’ve faced those same issues myself.
1

jcspencer 18 days ago | on: Ring is unmaintained

For example, the yubikey.rs crate exposes a signer trait implementation for keys backed by the Yubikey PIV interface. This means that my code for Kerberos signing doesn’t need to concern itself with how the key is stored; all it needs is a struct that implements signer.

1

zesterer 18 days ago | on: Ring is unmaintained

I think this sort of thing, beyond anything technical, is the biggest argument for growing the standard library (or, at least, those official ordained and maintained by the rust-lang org). I can argue about the technical merits of Rust to my Rust-sceptic colleagues all day, but it’s difficult to deny that a worrying number of important dependencies in the ecosystem have a low bus factor.

8

Diana 17 days ago | on: Ring is unmaintained

I don’t understand how the standard library is any different?

Python, as one example, has several part of its standard library that are unmaintained and falling backward. Same for erlang. How is being part of the standard library helping?
1. 7
  
  kornel 17 days ago | on: Ring is unmaintained
  
  Indeed, being in the standard library doesn’t guarantee active maintenance.
  
  There aren’t enough people in the core team to work on everything. There still has to be someone with expertise and dedication to improve the code.
  
  There aren’t that many people available for reviewing pull requests either. The standard library is focused on preserving backward compatibility and portability, so changes take more time and it’s harder to contribute more than an occasional drive-by tweak.
  
  In Rust, mpsc channels were an example of this: they were in the standard library, technically maintained, but nobody actively worked on improving the implementation. Eventually the std code got replaced with an implementation developed outside of the standard library.
4

skade 18 days ago | on: Ring is unmaintained

This just puts a bit of rusty paint on the problem. The Rust project had very little capacity for library maintenance, the bus factor would not differ substantially. Indeed, for that reason, the project has more moved libraries out rather than in.
1. 6
  
  rtpg 17 days ago | on: Ring is unmaintained
  
  Coming from another language ecosystem, I think this doesn’t acknowledge an extremely common scenario: library maintainer is AWOL, and people show up with patches, and it’s just that there is nobody there to hit merge and do a release.
  
  There are huge piles of projects where there are a bunch of people who are capable, reviewing each other’s changes, and saying “yes this works, I have a fork and we have it working” etc etc etc, but the release can’t happen because one person holds the keys.
  
  Having a large team who holds release keys is generally something that can work out! For every “the maintainer is the only one who could actually make a call on this”-style issue there are hundreds of “maintainer just is done with the project” situations.
  
  I’m not calling to have eminent domain, but having lang teams be able to adopt packages (with maintainer consent) and allowing for a light touch can totally work, especially in the age of Github & co.
2. 4
  
  zesterer 18 days ago | on: Ring is unmaintained
  
  Bus factor and capacity are different things. I’m not expecting development to go faster or anything: but when something really, really critical pops up that needs fixing with haste, a larger pool of people ready to merge a fix and deploy a release matters. Ossification and glacial development is fine, not having an official patch for a CVE because the one dev is (quite understandably!) on holiday, less so.
  1. 7
    
    steveklabnik 18 days ago | on: Ring is unmaintained
    
    The standard library only releases every six weeks. The more stuff that’s in the standard library, the more stuff that would maybe call for an emergency release, which is far heavier weight and far more disruptive to the ecosystem. People already think Rust releases too often.
    1. 4
      
      moltonel 17 days ago | on: Ring is unmaintained
      
      Also, a crypto vulnerability is likely to have a high priority. If have to urgently release/deploy, I’d much rather update a single crate than the whole compiler and stdlib.
    2. 1
      
      rtpg edited 17 days ago | on: Ring is unmaintained
      
      Not trying to be snarky, but is the complaint about too many releases (and operational churn)? Or too much breakage (from stuff like the “partial ordering for sorts can trigger panics now” saga)?
      
      Like one, you do land on “we should bundle up releases”, but the other is a different flavor of thing, right? Though I mention the sorting panic thing only because it’s the only case I have even heard of.
      
      Saying all that, while I like fat standard libraries, it does feel like the weaker version of “it’s in the rust-lang org but still packaged separately” gets things 95% of the way there for these kinds of issues, IMO.
      1. 5
        
        steveklabnik 17 days ago | on: Ring is unmaintained
        
        Operational and “reputational” churn. The time crate is the only breakage I’ve experienced personally in a long time. The process for releasing rustc is involved, the process of releasing a crate is “cargo publish.”
        
        I already regularly hear complaints from very conservative type folks who look at the six week cadence and complain about how often releases happen. Adding more to the mix regularly increases the perception that Rust is still mega unstable, when that’s just not the case.
        
        1
        
        rtpg 17 days ago | on: Ring is unmaintained
        
        Can defeinitely see rustc’s released process being involved vs cargo. Having multiple trusted people be able to run cargo publish is the valuable thing here, moreso than it being in the standard library itself.
        
        I do think it’s important to dig into what the reason for the cadence complaints really are though. I feel like the biggest lesson to be taken from Chrome showing up and doing releases all the time is that smaller releases (-> more release) are easier to manage in theory. It would be a shame if those complaints are downstream of other issues that are, in a word, “more” fixable?
        
        This is all context free of course. At least in Python land complaints of things moving too fast that I hear tend to be broken down into “too much new stuff to learn” (a reason I guess, but one I don’t care about) and “old code is not working anymore too often” combined with “package maintainers disappearing” (which are realer issues, and can be handled).
        
        5
        
        steveklabnik 17 days ago | on: Ring is unmaintained
        
        It’s just systems people who are used to C89 and so they see backwards compatible but new features as “breaking” because they want to stay on old compilers but compile new code.
        
        1
        
        rtpg 17 days ago | on: Ring is unmaintained
        
        OK I see the case now. The maximally generous version of this is “well there is some issue X that is stopping my upgrade but everything around me is moving so fast that it throws me forward” (in Node this is common because packages agressively upper bound node versions even when they don’t need to, leading to a bunch of incidental things making upgrades hard), but tbh I feel like Rust isn’t doing it that much. Especially nowadays where people seem pretty serious about not using stuff from nightly etc.
        
        4
        
        steveklabnik 16 days ago | on: Ring is unmaintained
        
        The maximally generous version of this is “well there is some issue X that is stopping my upgrade but everything around me is moving so fast that it throws me forward”
        
        I agree, but the “issue x” is mostly “i want to use a dependency and it uses some rust language feature that is newer than my current stable version” and the “stopping my upgrade” is simply “I don’t want to” not “I tried it and it didn’t work.” That is, it’s perceived instability, rather than actual instability. For these folks, “stable” means “unchanging” not “backwards compatible.”
2

algesten 18 days ago | on: Ring is unmaintained

The shift from RSA to ECDSA is, relatively speaking, quite recent and we just had Microsoft announce their crazy qubit capabilities meaning quantum resistant cryptos are maybe sooner required than we previously thought. I think crypto is exactly the kind of thing that needs to be kept out of a standard library to avoid churn/deprecation.

Some officially blessed crate, sure.
1. 3
  
  jcspencer 18 days ago | on: Ring is unmaintained
  
  I’ll leave a top-level comment on it, but aside from having a single “blessed” library, I think consensus on cryptographic trait implementations for backends would be fantastic.
  
  High-level libraries that can be built to consume those traits, low-level implementations that implement them; would lead to easier agility when the “blessed” library is no longer the go-to.
2

moltonel 17 days ago | on: Ring is unmaintained

The idea that absorbing something like ring into stdlib would automatically give it new maintainers feels like wishful thinking. Getting maintenance done is a combination of having the required skill (and for crypto, the bar is high) and accepting that it’s not someone else’s problem.

The “someone else’s problem” feeling can be pretty high for code in a stdlib. Whereas here, in the crate ecosystem, when the ring maintainer gave up we got an advisory that prompted new maintainers to step in. Looks like FOSS is working.

10

kornel 19 days ago | on: Docker limits unauthenticated pulls to 10/hr/ip from Docker Hub, from March 1

Kids these days, complaining they can’t reinstall an OS more than 10 times per hour.

In my days, if I wanted a fresh Linux install, I had to go and buy a CD-ROM and spend two days installing it.

2

dvogel edited 19 days ago | on: Linus Torvalds Clearly Lays Out Linux Maintainer Roles - Or Not - Around Rust Code

So when you change the C interfaces, the Rust people will have to deal with the fallout, and will have to fix the Rust bindings.

Does Linus accept patches that break rust drivers? Isn’t that the implication here? Subsystem maintainer can break C interfaces. How does it become clear that the rust code has been broken? If not via code moving upstream then isn’t the subsystem maintainer effectively blocked from pushing upstream until the rust maintainers fix all of the code using those modified C interfaces? Is Linus being sly here, essentially expecting these people to work together collaboratively or neither will be able to push upstream?

5

vifon 19 days ago | on: Linus Torvalds Clearly Lays Out Linux Maintainer Roles - Or Not - Around Rust Code

And no, I don’t actually think it needs to be all that black-and-white. I’ve stated the above in very black-and-white terms (“becoming a maintainer of the Rust bindings too” vs “don’t want to deal with Rust at all”), but in many cases I suspect it will be a much less harsh of a line, where a subsystem maintainer may be aware of the Rust bindings, and willing to work with the Rust side, but perhaps not hugely actively involved.

This part seems to clarify it’s mostly an extreme example for the sake of argument.
4

mordae 19 days ago | on: Linus Torvalds Clearly Lays Out Linux Maintainer Roles - Or Not - Around Rust Code

If Rust code get broken, it will get fixed during the stabilization part of the release cycle, I guess.
2

jstoja 19 days ago | on: Linus Torvalds Clearly Lays Out Linux Maintainer Roles - Or Not - Around Rust Code

I assume that it’s likely that breaking a C interface that will break rust drivers will also break C drivers.
1. 2
  
  kornel 19 days ago | on: Linus Torvalds Clearly Lays Out Linux Maintainer Roles - Or Not - Around Rust Code
  
  Maintainers of C subsystems fix the C drivers themselves.
  
  That’s why there was controversy whether the Rust side can really really be broken, or whether maintainers will end up being forced to learn Rust to fix the Rust drivers too.

2

kornel edited 21 days ago | on: This ESP32 antenna array can see WiFi

The implementation is really clever. They get timing of all the Wi-Fi chips so precisely aligned, because they run them all from the same oscillator.

And because these are real Wi-Fi chips, not merely 2.4Ghz antennas, they can look for a specific network. Brilliant.

1

kornel edited 22 days ago | on: What's in a ring buffer? And using them in Rust

Rust’s VecDeque exposes its contents as two slices. With memory mapping tricks it’s possible to map the same page of memory twice, and get a ring buffer that looks like one contiguous slice (there are some libraries for this).