This bit at the end seems particularly generalizably interesting beyond Rust:
Reifying the borrow checker into types
There is another theme running through here: moving the borrow checker analysis out from the compiler’s mind and into types that can be expressed. Right now, all types always represent fully initialized, unborrowed values. There is no way to express a type that captures the state of being in the midst of iterating over something or having moved one or two fields but not all of them. These changes address that gap.⁹
[…]
I remember years ago presenting Rust at some academic conference and a friendly professor telling me, “In my experience, you always want to get that state into the type system”. I think that professor was right, though I don’t regret not prioritizing it (always a million things to do, better to ask what is the right next step now than to worry about what step might’ve been better in the past). Anyway, I wish I could remember who that was!
std:🧵:spawn
That emoji-fier could use some self-restraint. :-) (It should say std::thread::spawn.)
I had to add my own emoji shortcode support to a little pulldown-cmark-based thing I wrote and not doing emoji processing on code fell naturally out of the difference between text and code tokens in the event stream it emits.
(Though I did notice that its smart quotes support isn’t smart enough to turn itself off when using <code>foo</code> instead of `foo` unless the raw HTML is part of a raw HTML block rather than an inline element… but then I’m considering disabling pulldown-cmark’s smart quotes and copying its implementation out into a postprocessing pass anyway, so I can hook things like :e': → é in before the smart quotes run.)
That emoji-fier could use some self-restraint. :-) (It should say std::thread::spawn.)
I’ve started using emojis as variable names in my code recently. I believe it’s a really bad idea, but … sometimes … I just can’t … resist … the temptation.
It looks like the self-referential example implies that a move won’t necessarily be a simple memcpy anymore, but might include some glue to “fix-up” the interior references after the memcpy.
That’s an interesting approach that’s orthogonal to !Move types (another way we could get safe interior references). I wonder how it would work with user-provided reference types (the footnote about a dedicated unsafe Deref trait might hint at something)
The pointers in headers and body point directly into the allocation text contains a pointer to, not at text itself. When you copy the Message struct that allocation stays in the same place.
I suspect Niko’s plan involves keeping move as memcpy, because that assumption is very baked into the ecosystem. Which either means self-referential structs will be !Move, or that they will have to always “look like” Message with no pointers pointing to “themselves” just to things they own.
Ah! You’re right I had somehow missed that. String is on the heap so moving the struct doesn’t move its buffer
Not allowing interior references on the stack feels limiting. Like you said move as memcpy is backed into the ecosystem (e.g. realloc of a vector will just memcpy its buffer), so I guess this is what we’re getting.
The new trait he’s talking about looks like some variant of StableDeref then
This is the thing that worries me quite a bit, since this is overloading what it means to ‘reference’ a path. Today, a reference like &'a Foo implies referencing data that cannot be moved during the lifetime of the reference.
However, this proposed design flips that on its head and instead says ‘yes, the referencee may move, and we’re not permitted to bind to the value of the referencee itself, but to some nebulous notion of data that it owns’.
I have no idea how one could feasibly express that in the type system, it’s much more complex than our existing notions of referencing that make use of the ‘referencee cannot move’ rule to paper over such differences.
Which either means self-referential structs will be !Move, or that they will have to always “look like” Message with no pointers pointing to “themselves” just to things they own.
Thinking back about this, isn’t there a third possibility opened by the &'self.field syntax?
Pointers to “themselves” could be transparently replaced by “offset from the base address of the struct” by the implementation, alleviating the need for “fixing-up” the pointer on move.
as long as actually requesting the address/pointer returns “the correct answer”, I think this should be fine. Now I think this was discussed in the past and there’s a withoutboat post somewhere about why offsets don’t work. I’d need to re-read it to know if their objection still applies in the presence of the &'self.field syntax
EDIT: found it… to the untrained eye, it looks like the special syntax + sufficiently restrictive rules around subtyping of these lifetimes (e.g. not being able to escape the original struct when appearing in generic contexts) would work
This bit at the end seems particularly generalizably interesting beyond Rust:
That emoji-fier could use some self-restraint. :-) (It should say
std::thread::spawn.)…wow. I wonder how that came about.
I had to add my own emoji shortcode support to a little
pulldown-cmark-based thing I wrote and not doing emoji processing on code fell naturally out of the difference between text and code tokens in the event stream it emits.(Though I did notice that its smart quotes support isn’t smart enough to turn itself off when using
<code>foo</code>instead of`foo`unless the raw HTML is part of a raw HTML block rather than an inline element… but then I’m considering disablingpulldown-cmark’s smart quotes and copying its implementation out into a postprocessing pass anyway, so I can hook things like:e': → éin before the smart quotes run.)I’ve started using emojis as variable names in my code recently. I believe it’s a really bad idea, but … sometimes … I just can’t … resist … the temptation.
It looks like the self-referential example implies that a
movewon’t necessarily be a simplememcpyanymore, but might include some glue to “fix-up” the interior references after thememcpy.That’s an interesting approach that’s orthogonal to
!Movetypes (another way we could get safe interior references). I wonder how it would work with user-provided reference types (the footnote about a dedicated unsafe Deref trait might hint at something)The example doesn’t actually require fix-up code… but I’m not sure how the type system is supposed to know that. It’s moving a
The pointers in
headersandbodypoint directly into the allocationtextcontains a pointer to, not attextitself. When you copy theMessagestruct that allocation stays in the same place.I suspect Niko’s plan involves keeping move as memcpy, because that assumption is very baked into the ecosystem. Which either means self-referential structs will be
!Move, or that they will have to always “look like”Messagewith no pointers pointing to “themselves” just to things they own.Ah! You’re right I had somehow missed that. String is on the heap so moving the struct doesn’t move its buffer
Not allowing interior references on the stack feels limiting. Like you said move as memcpy is backed into the ecosystem (e.g. realloc of a vector will just memcpy its buffer), so I guess this is what we’re getting.
The new trait he’s talking about looks like some variant of
StableDerefthenThis is the thing that worries me quite a bit, since this is overloading what it means to ‘reference’ a path. Today, a reference like
&'a Fooimplies referencing data that cannot be moved during the lifetime of the reference.However, this proposed design flips that on its head and instead says ‘yes, the referencee may move, and we’re not permitted to bind to the value of the referencee itself, but to some nebulous notion of data that it owns’.
I have no idea how one could feasibly express that in the type system, it’s much more complex than our existing notions of referencing that make use of the ‘referencee cannot move’ rule to paper over such differences.
Thinking back about this, isn’t there a third possibility opened by the
&'self.fieldsyntax? Pointers to “themselves” could be transparently replaced by “offset from the base address of the struct” by the implementation, alleviating the need for “fixing-up” the pointer on move.as long as actually requesting the address/pointer returns “the correct answer”, I think this should be fine. Now I think this was discussed in the past and there’s a withoutboat post somewhere about why offsets don’t work. I’d need to re-read it to know if their objection still applies in the presence of the
&'self.fieldsyntaxEDIT: found it… to the untrained eye, it looks like the special syntax + sufficiently restrictive rules around subtyping of these lifetimes (e.g. not being able to escape the original struct when appearing in generic contexts) would work