I would love to read more about experiences with Zig after the honeymoon phase. I find Zig super promising and doing some very impressive things, but I’d be interested in how it holds up during the daily grind. Public packages (a la crates.io) and API documentation (a la rustdoc vía docs.rs) are things that I can easily do without for hobby projects or while I’m still getting into a language but would be missed on an extended project.
Additionally, I’d love to read more Zig articles by people without prior C/C++/Rust experience. One of the super powers of Rust is that it holds your hand through treacherous territory without fear of segfaults or accidentally leading to major classes of bugs causing CVEs. There have been so many times I do something in Rust and the compiler says no, and I fight for a while thinking, “No! This is safe!” Only to realize at some point that I was wrong after all. I’d be a little nervous that in other languages I’d make the same mistake, but it’d go uncaught…or would it? Maybe other static analysis tools would be just as good, or maybe Zig doesn’t lead to the same design corners that lead to these types of issues? That’s what I’d love to read.
There have been so many times I do something in Rust and the compiler says no, and I fight for a while thinking, “No! This is safe!” Only to realize at some point that I was wrong after all. I’d be a little nervous that in other languages I’d make the same mistake, but it’d go uncaught…or would it?
You should try firsthand. I think that the experience would be valuable even if you don’t plan to use Zig in the future. In general Zig as a language, and even more so as an ecosystem, will try not to put you in a situation where the right solution to a problem is something extremely complicated because you need to thread a needle through a lot of language features and custom abstractions designed by library authors. There are some quirky things but it’s usually all small-scale stuff that can be understood in isolation.
In general I feel that your comment here rings true from my perspective:
maybe Zig doesn’t lead to the same design corners that lead to these types of issues?
That said it’s the age old conundrum that also C and C++ had: not having a ton of sophisticated abstraction everywhere helps prevent some problems but it can cause others and vice versa. Different people can make either approach work and I think it’s worth trying both even if you have a clear affinity towards one or the other.
Personally, before trying Zig, I’ve been a Python / Go / C# developer and I was very happy to be able to learn how bare metal programming works with Zig. I liked Go a lot and, while I don’t regret my C# experience, it helped confirm my preference for simplicity over powerful language features and abstractions.
I guess this is also why I never really got much into Rust, while Zig clicked immediately in my brain.
From the perspective of learning systems programming, I think that here too there is a conundrum where you have to choose your set of tradeoffs: either the compiler protects you from unsafe operations, but in doing so it prevents you from going off the beaten path, or you have a compiler that can’t protect you from everything but that in exchange allows you to see the system more clearly instead of hiding some corners of it from you.
As an example in the Zig event loop code there’s a lot of very clever stuff with intrusive data structures, and suspend/resume for async frames is a very simple but powerful way of handling coroutines. Learning about this really helped me expand my understanding of async/await from a systems programming perspective, but at the same time the compiler won’t catch all misuses of suspend/resume at compile time. On the other hand, if we look at Rust, Futures are much more hard to misuse but I wonder how many people understand their inner workings past “Futures need to be Pinned before you can poll them”.
Keep going, guys. Every time I read about Zig I get a bit closer to deciding to try it. (Srsly.) it helps that my current project uses very little memory allocation, so Zig’s lack of a global allocator won’t be as big a deal. (Previosly I have been turned off by the need to pass or store allocator references all over the place.)
For end-product projects there’s really nothing wrong with setting a global allocator and using it everywhere! You can even use that single point of reference to swap in test allocator in tests so you can check for memory leaks in all of your unit and integration tests. You might want to be more flexible, composable, and unopinionated (or maybe strategically opinionated) with allocator strategies if you’re writing a library.
The scenario that worries me is where I’ve got a subsystem that doesn’t allocate memory. Then I modify something down inside it so that it does need to allocate. Now I have to plumb an allocator reference through umpteen layers of call stack (or structs) in all control flow paths that reach the affected function.
Maybe far fetched, but it gets uglier the bigger the codebase gets. I’ve had to do this before (not with allocators, but other types of state) in big layer-cake codebases like Chromium. It’s not rocket science but it’s a pain.
I guess I’m not used to thinking of “performs memory allocation” as a color of function.
Then I modify something down inside it so that it does need to allocate.
To me this would be a smell, a hint that the design may want to be rethought so as not to have the possibility of allocation failure. Many code paths do have the possibility of allocation failure, but if you have an abstraction that eliminates that possibility, you’ve opened up more possible users of that abstraction. Adding the first possibility of allocation failure is in fact a big design decision in my opinion - one that warrants the friction of having to refactor a bunch of function prototypes.
As I’ve programmed more Zig, I’ve found that things that need to allocate tend to be grouped together, and likewise with things that don’t need to allocate. Plus there are some really nice things you can do to reserve memory and then use a code path that cannot fail. As an example, this is an extremely common pattern:
Here we use ensureUnusedCapacity to make it so that the following code can use the “assumeCapacity” forms of appending to various data structures. This makes error handling simpler since most of the logic does not need to handle failure (note the lack of the word try after those first 3 lines). This pattern can be especially helpful when there is a resource that (unfortunately) lacks a cheap or simple way to deallocate it. If you reserve the other resources such as memory up front, and then leave that weird resource at the end, you don’t have to handle failure.
A note on safety: “assumeCapacity” functions are runtime safety-protected with the usual note that you can opt out of runtime safety checks on a per-scope basis.
the design may want to be rethought so as not to have the possibility of allocation failure.
True, I’ve been forgetting about allocation failures because I code for big iron, like phones and Raspberry Pis 😉 … but I do want my current project to run on li’l embedded devices.
The ensureUnusedCapacity trick is neat. But doesn’t it assume that the allocator has no per-block overhead? Otherwise the heap may have A+B+C bytes free, but it’s not possible to allocate 3 blocks of sizes A, B and C because each heap block has an n-byte header. (Or there may not be a free block big enough to hold C bytes.)
Speaking of allocators: the Zig docs say that implementations need to satisfy “the Allocator interface”, but don’t say what an “interface” is. That is in fact the only mention of the word “interface” in the docs.
I’m guessing that Zig supports Go-like interfaces, but this seems to be undocumented, which is weird for such a significant language feature…?
The Allocator interface is actually not part of the language at all! It’s a standard library concept. Zig does not support Go-like interfaces. It in fact does not have any OOP features. There are a few patterns you can use to get OOP-like abstractions, and the Allocator interface is one of them.
The ensureUnusedCapacity trick is neat. But doesn’t it assume that the allocator has no per-block overhead?
The ensureUnusedCapacity usage I showed above is using array lists and hash maps :)
It’s not function coloring (or if it is, it doesn’t feel like it), because in your interfacing function you can fairly trivially catch the enomem and return a null value that will correspond to allocation failure… And you return that, (or panic on alloc failure, if you prefer), in either case you don’t have to change your stack at all. Since allocators are factories, it’s pretty easy to set it up so that there is an overrideable default allocator.
As an example, here is some sample code that mocks libc’s calloc and malloc (and free) with zig’s stdlib “testing.allocator” which gives you memory leak analysis in code that is designed to use the libc functions, note that testing.allocator, of course, has zig’s standard memory allocator signature, but with the correct interface that doesn’t have to travel up the call stack and muck up the rest of the code, that is expecting something that looks like libc’s calloc/malloc (which of course doesn’t have error return values, but communicates failure with null):
I actually feel that we should compare Zig to Go, because it is closer in concept to what I think Go wanted to be
Eh.. kinda, yes, as in Go “wanted to be a modern C”, but it seems like fully automatic memory management was a really big part of their vision of “modern”.
Go compiles to native code. Besides the performance benefit, it also means that, like C/++, it produces a regular binary file anyone can simply run without having to install anything. This also makes binding to C APIs a lot easier.
At the language level, Go and Java disagree violently about inheritance and error handling.
i always feel like Go is a lovechild of C and Scheme : compiles to native, compiler can detect stack allocatable activation records as in C, with a huge standard library, but the compiler can also generate functions whose activation records are heap allocated for situations needing a construction of a closure for runtime realization of lexical scoping like Scheme (and , unlike Java’s half-hearted version).
There’s also a “X for me but not for thee” division between how go’s stdlib is implemented under the hood versus user code. That philosophy is non-existent in zig, and often I find myself cribbing code snippets or concepts from stdlib in zig.
I would love to read more about experiences with Zig after the honeymoon phase. I find Zig super promising and doing some very impressive things, but I’d be interested in how it holds up during the daily grind. Public packages (a la crates.io) and API documentation (a la rustdoc vía docs.rs) are things that I can easily do without for hobby projects or while I’m still getting into a language but would be missed on an extended project.
Additionally, I’d love to read more Zig articles by people without prior C/C++/Rust experience. One of the super powers of Rust is that it holds your hand through treacherous territory without fear of segfaults or accidentally leading to major classes of bugs causing CVEs. There have been so many times I do something in Rust and the compiler says no, and I fight for a while thinking, “No! This is safe!” Only to realize at some point that I was wrong after all. I’d be a little nervous that in other languages I’d make the same mistake, but it’d go uncaught…or would it? Maybe other static analysis tools would be just as good, or maybe Zig doesn’t lead to the same design corners that lead to these types of issues? That’s what I’d love to read.
You should try firsthand. I think that the experience would be valuable even if you don’t plan to use Zig in the future. In general Zig as a language, and even more so as an ecosystem, will try not to put you in a situation where the right solution to a problem is something extremely complicated because you need to thread a needle through a lot of language features and custom abstractions designed by library authors. There are some quirky things but it’s usually all small-scale stuff that can be understood in isolation.
In general I feel that your comment here rings true from my perspective:
That said it’s the age old conundrum that also C and C++ had: not having a ton of sophisticated abstraction everywhere helps prevent some problems but it can cause others and vice versa. Different people can make either approach work and I think it’s worth trying both even if you have a clear affinity towards one or the other.
Personally, before trying Zig, I’ve been a Python / Go / C# developer and I was very happy to be able to learn how bare metal programming works with Zig. I liked Go a lot and, while I don’t regret my C# experience, it helped confirm my preference for simplicity over powerful language features and abstractions.
I guess this is also why I never really got much into Rust, while Zig clicked immediately in my brain.
From the perspective of learning systems programming, I think that here too there is a conundrum where you have to choose your set of tradeoffs: either the compiler protects you from unsafe operations, but in doing so it prevents you from going off the beaten path, or you have a compiler that can’t protect you from everything but that in exchange allows you to see the system more clearly instead of hiding some corners of it from you.
As an example in the Zig event loop code there’s a lot of very clever stuff with intrusive data structures, and suspend/resume for async frames is a very simple but powerful way of handling coroutines. Learning about this really helped me expand my understanding of async/await from a systems programming perspective, but at the same time the compiler won’t catch all misuses of suspend/resume at compile time. On the other hand, if we look at Rust, Futures are much more hard to misuse but I wonder how many people understand their inner workings past “Futures need to be Pinned before you can poll them”.
Keep going, guys. Every time I read about Zig I get a bit closer to deciding to try it. (Srsly.) it helps that my current project uses very little memory allocation, so Zig’s lack of a global allocator won’t be as big a deal. (Previosly I have been turned off by the need to pass or store allocator references all over the place.)
For end-product projects there’s really nothing wrong with setting a global allocator and using it everywhere! You can even use that single point of reference to swap in test allocator in tests so you can check for memory leaks in all of your unit and integration tests. You might want to be more flexible, composable, and unopinionated (or maybe strategically opinionated) with allocator strategies if you’re writing a library.
Good point! I tend to write libraries, though.
The scenario that worries me is where I’ve got a subsystem that doesn’t allocate memory. Then I modify something down inside it so that it does need to allocate. Now I have to plumb an allocator reference through umpteen layers of call stack (or structs) in all control flow paths that reach the affected function.
Maybe far fetched, but it gets uglier the bigger the codebase gets. I’ve had to do this before (not with allocators, but other types of state) in big layer-cake codebases like Chromium. It’s not rocket science but it’s a pain.
I guess I’m not used to thinking of “performs memory allocation” as a color of function.
To me this would be a smell, a hint that the design may want to be rethought so as not to have the possibility of allocation failure. Many code paths do have the possibility of allocation failure, but if you have an abstraction that eliminates that possibility, you’ve opened up more possible users of that abstraction. Adding the first possibility of allocation failure is in fact a big design decision in my opinion - one that warrants the friction of having to refactor a bunch of function prototypes.
As I’ve programmed more Zig, I’ve found that things that need to allocate tend to be grouped together, and likewise with things that don’t need to allocate. Plus there are some really nice things you can do to reserve memory and then use a code path that cannot fail. As an example, this is an extremely common pattern:
https://github.com/ziglang/zig/blob/f81b2531cb4904064446f84a06f6e09e4120e28a/src/AstGen.zig#L9745-L9784
Here we use
ensureUnusedCapacityto make it so that the following code can use the “assumeCapacity” forms of appending to various data structures. This makes error handling simpler since most of the logic does not need to handle failure (note the lack of the wordtryafter those first 3 lines). This pattern can be especially helpful when there is a resource that (unfortunately) lacks a cheap or simple way to deallocate it. If you reserve the other resources such as memory up front, and then leave that weird resource at the end, you don’t have to handle failure.A note on safety: “assumeCapacity” functions are runtime safety-protected with the usual note that you can opt out of runtime safety checks on a per-scope basis.
True, I’ve been forgetting about allocation failures because I code for big iron, like phones and Raspberry Pis 😉 … but I do want my current project to run on li’l embedded devices.
The ensureUnusedCapacity trick is neat. But doesn’t it assume that the allocator has no per-block overhead? Otherwise the heap may have A+B+C bytes free, but it’s not possible to allocate 3 blocks of sizes A, B and C because each heap block has an n-byte header. (Or there may not be a free block big enough to hold C bytes.)
Speaking of allocators: the Zig docs say that implementations need to satisfy “the Allocator interface”, but don’t say what an “interface” is. That is in fact the only mention of the word “interface” in the docs.
I’m guessing that Zig supports Go-like interfaces, but this seems to be undocumented, which is weird for such a significant language feature…?
The Allocator interface is actually not part of the language at all! It’s a standard library concept. Zig does not support Go-like interfaces. It in fact does not have any OOP features. There are a few patterns you can use to get OOP-like abstractions, and the Allocator interface is one of them.
The ensureUnusedCapacity usage I showed above is using array lists and hash maps :)
Will I get in trouble if i start calling it the allocator factory?
Lol
It’s not function coloring (or if it is, it doesn’t feel like it), because in your interfacing function you can fairly trivially catch the enomem and return a null value that will correspond to allocation failure… And you return that, (or panic on alloc failure, if you prefer), in either case you don’t have to change your stack at all. Since allocators are factories, it’s pretty easy to set it up so that there is an overrideable default allocator.
As an example, here is some sample code that mocks libc’s calloc and malloc (and free) with zig’s stdlib “testing.allocator” which gives you memory leak analysis in code that is designed to use the libc functions, note that testing.allocator, of course, has zig’s standard memory allocator signature, but with the correct interface that doesn’t have to travel up the call stack and muck up the rest of the code, that is expecting something that looks like libc’s calloc/malloc (which of course doesn’t have error return values, but communicates failure with null):
https://gist.github.com/ityonemo/fb1f9aca32feb56ad46dd5caab76a765
We should, though. Although, it should be something handled like generics.
It’s not, though (see sibling reply with code example).
Can always tell when someone is talking about Rust or Zig with a C or C++ background, heh
Eh.. kinda, yes, as in Go “wanted to be a modern C”, but it seems like fully automatic memory management was a really big part of their vision of “modern”.
Yeah, I never understand comparing Go to C, Go competes more with Java at the “mid level garbage collection care about performance a bit” tier
I think it’s because it’s small and imperative, and also doesn’t hide pointers from you.
Go compiles to native code. Besides the performance benefit, it also means that, like C/++, it produces a regular binary file anyone can simply run without having to install anything. This also makes binding to C APIs a lot easier.
At the language level, Go and Java disagree violently about inheritance and error handling.
Yes
There is no reason Java can’t compile to native as well. And with GraalVM it is becoming more and more prevalent.
I assume the connection to C is primarily because of Ken Thompson’s involvement.
i always feel like Go is a lovechild of C and Scheme : compiles to native, compiler can detect stack allocatable activation records as in C, with a huge standard library, but the compiler can also generate functions whose activation records are heap allocated for situations needing a construction of a closure for runtime realization of lexical scoping like Scheme (and , unlike Java’s half-hearted version).
There’s also a “X for me but not for thee” division between how go’s stdlib is implemented under the hood versus user code. That philosophy is non-existent in zig, and often I find myself cribbing code snippets or concepts from stdlib in zig.
Metacommentary: wow, this site’s design is busy.
It’s definitely not minimalist but I wouldn’t call it overwhelming.
On mobile, it’s fine. On desktop, there are these very text-ful and colorful sidebars.
Yeah, it’s a bit too much on desktop.