Note that Rust does this automatically for you since Rust 1.18, released in 2017. By coincidence, the example case used, (u8, u16, u8), is exactly same in the Rust release note.
As far as I know, there is nothing in the spec that guarantees struct order layout, and the only way to really observe it is to use the unsafe package which is not covered by the Go 1 compatibility promise. So technically it could.
That said, changing it now would break too much code that uses structs with cgo/assembly/syscalls, so I doubt it will happen any time soon. If it ever does, I’d expect it will come with a similar annotation to Rust’s #[repr(C)], at the very least.
Automatic reordering has advantages and disadvantages. We know the problem with padding, but it has the advantage when you’re dealing with order-sensitive data interfaces.
I see the problem rather in regard to the lack of education about his topic. In clang, you can enable diagnostic padding information with -Wpadding, but be aware that some cases are unresolvable.
I found this article a bit flat, to be honest. The go-to resource, in my opinion, is the lost art of structure packing, which also does a deep dive into other languages (C++, Go, Rust, etc.).
C/C++ require order sensitivity because they don’t have any other way of doing stable ABIs. If you add a field to a C struct then you don’t change the layouts of any other fields. The offsets of structure fields are baked into binaries and so it would break things if you modified field layout. Stable C ABIs often reserve a few unused fields at the end so you can add fields without breaking things. Go does not have this problem because the Go ecosystem doesn’t support pre-compiled libraries (dynamic or static): you compile all of your dependencies at the same time.
Worse, in C, you can take the address of structure fields and then do arithmetic to get other structure fields. This means that the offset of any structure field can leak into the rest of the program in ways that the compiler can’t track. Other code may assume that two structures have the same partial layout. Go does not have this problem unless you use the Unsafe package (and you generally don’t).
The remaining reasons to care about structure layout are related to caches:
If you have concurrent mutation, you want to ensure that things that are modified together are in the same cache line and things that are not modified together are in separate cache lines. Go discourages concurrent mutation.
If you have very large objects then you want structure fields that are accessed together to be in the same cache lines (or, at least, cache lines that are accessed together).
I suspect that these cases are sufficiently rare that the default of just packing the structures by arranging the fields by size (or in a more GC-friendly layout) would give better results than relying on user layout.
There is one remaining bad reason for fixed layout: C/C++ interop. If your interop layer relies on matching C structure layouts in your language then you need a mechanism for doing this layout. This is a terrible idea because there are a lot of corner cases (bitfields, packed structures, and so on) where it will be wrong and you have no compile-time checking that it’s correct. The programmer is responsible for keeping two machine-parsable definitions of the same thing in sync. Unfortunately, Go does seem to encourage that.
This would leave me potentially conflicted. In one hand, I value readability more than performance, or at least first than performance, so I’d be more interesting in keeping the struct field’s declaration in a order that makes sense from a business logic/problem Domain point of view. On the other hand, that’s a significant optimization.
I really like the Rust strategy, mentioned here, better. Optimize automatically for me, and give me a way to control the order directly if I need to.
Note that Rust does this automatically for you since Rust 1.18, released in 2017. By coincidence, the example case used, (u8, u16, u8), is exactly same in the Rust release note.
Does anyone know if Go could do this? And was it ever considered?
As far as I know, there is nothing in the spec that guarantees struct order layout, and the only way to really observe it is to use the unsafe package which is not covered by the Go 1 compatibility promise. So technically it could.
That said, changing it now would break too much code that uses structs with cgo/assembly/syscalls, so I doubt it will happen any time soon. If it ever does, I’d expect it will come with a similar annotation to Rust’s
#[repr(C)], at the very least.Here’s an issue where these things have been considered: https://github.com/golang/go/issues/36606
That seems rather brutal for binary compatibility or when interacting with C. I assume it can be turned off on a struct by struct basis?
Yes,
#[repr(C)], it’s right there in the linked release notes :-)And all the details (and other representations) are documented: https://doc.rust-lang.org/reference/type-layout.html#representations
Tools exist that can help find and optimise struct orders for you. I just found https://github.com/orijtech/structslop.
Automatic reordering has advantages and disadvantages. We know the problem with padding, but it has the advantage when you’re dealing with order-sensitive data interfaces.
I see the problem rather in regard to the lack of education about his topic. In clang, you can enable diagnostic padding information with -Wpadding, but be aware that some cases are unresolvable.
I found this article a bit flat, to be honest. The go-to resource, in my opinion, is the lost art of structure packing, which also does a deep dive into other languages (C++, Go, Rust, etc.).
C/C++ require order sensitivity because they don’t have any other way of doing stable ABIs. If you add a field to a C
structthen you don’t change the layouts of any other fields. The offsets of structure fields are baked into binaries and so it would break things if you modified field layout. Stable C ABIs often reserve a few unused fields at the end so you can add fields without breaking things. Go does not have this problem because the Go ecosystem doesn’t support pre-compiled libraries (dynamic or static): you compile all of your dependencies at the same time.Worse, in C, you can take the address of structure fields and then do arithmetic to get other structure fields. This means that the offset of any structure field can leak into the rest of the program in ways that the compiler can’t track. Other code may assume that two structures have the same partial layout. Go does not have this problem unless you use the
Unsafepackage (and you generally don’t).The remaining reasons to care about structure layout are related to caches:
I suspect that these cases are sufficiently rare that the default of just packing the structures by arranging the fields by size (or in a more GC-friendly layout) would give better results than relying on user layout.
There is one remaining bad reason for fixed layout: C/C++ interop. If your interop layer relies on matching C structure layouts in your language then you need a mechanism for doing this layout. This is a terrible idea because there are a lot of corner cases (bitfields, packed structures, and so on) where it will be wrong and you have no compile-time checking that it’s correct. The programmer is responsible for keeping two machine-parsable definitions of the same thing in sync. Unfortunately, Go does seem to encourage that.
This would leave me potentially conflicted. In one hand, I value readability more than performance, or at least first than performance, so I’d be more interesting in keeping the struct field’s declaration in a order that makes sense from a business logic/problem Domain point of view. On the other hand, that’s a significant optimization.
I really like the Rust strategy, mentioned here, better. Optimize automatically for me, and give me a way to control the order directly if I need to.