This is something I was hoping Carbon would have addressed in a satisfactory way, but, alas, they do the same thing Zig does, so Zig is once again the language which will have to innovate to solve this problem.
The video explains that “Parameter Reference Optimization” is a serious design flaw in Zig. It’s a footgun that leads to subtle bugs that can’t be seen by grug-brained developers. It’s unsound. Other system languages like C, C++, Rust do not have this footgun, so people who come to Zig from those other languages will fall into the trap.
It seems like the obvious fix (but not mentioned in the video?) is to use C semantics by default for passing large parameters (copy the value before making the function call, then pass a reference to the copy). Only use the “parameter reference optimization” when the optimizing compiler can prove that no aliasing will result if the copy is elided. This optimization strategy is sometimes called “copy elision”.
At the 8 minute mark, a bug in List.add() from the Zig library is shown. The bug is caused by the “Parameter Reference Optimization” feature. To fix the bug requires the developer to add an extra copy of ‘item’ to the function body, which causes a performance hit. If, instead, Zig used copy elision as a sound compiler optimization, then you wouldn’t have this subtle bug, you wouldn’t take a performance hit to fix the bug, and the extra copy of the ‘item’ parameter would only happen in cases where it was needed (the compiler adds the copy at the caller site to prevent parameter aliasing).
For the other buggy feature in the video, “Result Location Semantics”, I make the same comments. Fixing the bug in rotate (without changing the function signature) requires the developer to make an explicit copy of the parameter before using it, which again is a performance hit. Instead of making people write code like this, you should generate compiler temporaries by default, and elide the copies as an optimization only when it is sound to do so.
You guys understand these issues better than I do, so what am I missing? Alias analysis is hard, so the standard approach I describe will have worse performance than what the Zig compiler does today (on par with C/C++), and you want better performance than C?
I wouldn’t say that the issue is solved in other languages:
in C++, you can accidentally copy large heap objects. C++ semantics of implicit deep copy is not a useful one.
Rust solve correctness problems around parameter passing and aliasing via borrow checker, but that is complicated (at least, as a part of the whole package that is Rust), and doesn’t solve perf problems — Rust does a lot of stack-to-stack copies. It also lacks placement.
C doesn’t protect you from aliasing when passing arguments, and has useless copies where the caller has to defensively copy parameter.
Then, there’s this annoying question of “should you pass small things by reference or by value? And what is small, by the way?”.
My personal take here is that “fn f(param: Foo) passes parameter param by immutable reference” is the right high-level semantics:
not mutating is the common case, it shouldn’t require any sigils
some things can’t be copied (e.g., their address is meaningful), but you still want to be able to pass them as parameters without extra ceremony
whether “foo” is passed by value in registers or by pointer is a detail that should depend on target CPU. Language semantics should allow either.
The question is rather how to implement that semantics without running afoul of aliasing, which is what Carbon, Zig, and Hylo (formerly Val) are trying to do.
Thank you for the video suggestion, I am not a Zig programmer and the blog post was a bit too terse for me – I could not really follow. The video is super clear and got me very excited about Zig, but folks if you expect a solution besides a clear exposition of the problems you will be disappointed :D we’ll have to wait a while for that I guess.
I’ve been getting in to Zig in the past few weeks and I really have to say: It’s a joy.
Would forcing the compiler to always pass a struct by value with a flag ever be considered? Or does that go harder against the grain of other language principles?
That kind of solution doesn’t really work because it doesn’t compose … Now libraries would have to add test coverage for being compiled with both flags.
And if the libraries have dependencies, then they can’t rely on the semantics of those either. You end up with a 2^N test matrix for N libraries.
The flags also don’t appear in the source code, which is a problem for readability.
Compiler flags can be used for optimizations, which by definition are semantics-preserving. Pass by value vs. by reference isn’t a semantics-preserving change – it’s basically the opposite semantics!
And is that expected behavior, or a bug that will be fixed? If it will be fixed, it doesn’t seem like a big deal, unless the point is that the bug reveals the underlying behavior of things being passed by reference under the hood?
Structs, unions, and arrays can sometimes be more efficiently passed as a reference, since a copy could be arbitrarily expensive depending on the size. When these types are passed as parameters, Zig may choose to copy and pass by value, or pass by reference, whichever way Zig decides will be faster. This is made possible, in part, by the fact that parameters are immutable.
So it’s not undefined behavior in the C sense, it’s merely implementation defined behavior. But this is one hell of a footgun. I’m very surprised this simple program has two possible behaviors. I would expect that the compiler would sometimes pass by reference, but conservatively ensure that the fact it was doing so was never visible, thus making a copy in this case.
So I’ve read through this multiple times, and it sounds like Zig chooses by some opaque criteria to pass by “large” values by what c++ would call a const reference. But it does this whether or not the type is mutable, and thus it is not semantically equivalent to copy by value despite being syntactically identical. This seems like a language bug.
if p0 is changed to some_type& you can hopefully see how the program behaviour would change, and possibly not in a good way. Because this seems to be an decision made by the compiler that does not occur 100% of the time, it seems that this could result in different versions of the compiler, or different callsites even (depending on inlining, etc) changing behaviour.
Now I believe swift does do automatic conversion of value types to reference parameters, but that’s because value types in swift are semantically immutable (I think the moral equivalent to this particular code in swift would require UnsafeMutablePointer which is a pretty big red flag).
I’d like to understand better how PRO is different from the usual ABI behavior of “things larger than a register (or two) get stored on the stack and passed by address”. I assume the difference is that in C for example the compiler has to provide the illusion of value semantics, so in the general case that stack location needs to be a fresh copy, and you pay for that. Are there other differences?
The difference is that, in C/C++/most other languages, the copy that’s passed by reference is just that: a copy. You then rely on copy elision to avoid doing the copy for cases where the original cannot be mutated after the call.
In C, when you do:
struct SomeBigStruct x;
// Initialise x in some way.
someFn(x);
The compiler transforms it into something roughly equivalent to:
struct SomeBigStruct x;
// Initialise x in some way.
struct SomeBigStruct __tmpCopy = x;
someFn(&__tmpCopy);
This guarantees the behaviour that the callee sees a non-aliased copy. When optimisations run, the first thing that will happen is that scalar replacement of aggregates will split x apart into a load of independent fields represented with SSA variables. If something takes the address of x, then x will be reconstituted, but if not then those SSA variables will not exist in any defined location until the call to someFn, at which point they’ll be stored into __tmpCopy. In general, if some pointers to x escape (or, more accurately, if the compiler cannot prove that pointers to x have not escaped) then both x and __tmpCopy will exist in the optimised code. Otherwise, x will be eliminated.
This is a sound semantics-preserving transform. The Zig thing is only if your semantics are ‘nondeterminstically, sometimes, things passed by value may alias other things’.
Swift does something interesting here and has a bunch of value types that implement copy-on-write behaviour, so you can pass them by reference and end up with a copy if you mutate them.
Uh, is this a concern of people (bloggers) in general? lobste.rs isn’t a big site like Reddit, HN, or Slashdot (back in the day). I guess I don’t understand.
I believe the intent is to link out to active discussions of the article, and the author is promoting those two options as ones they would support. If it did get discussed on either some fediverse page or lobste.rs they would add links. If there were a discussion on Reddit they would not.
I’m not watching Zig proactively and closely (yet!), but since I heard about it and its promises a long time ago, it got me hooked and made me root for it (-: Now, that said, what strikes me as a surprise is this… and please correct my potentially incorrect understanding.
One of the Zig’s USPs is this (taken straight from the project homepage):
A Simple Language
Focus on debugging your application rather than debugging your programming language knowledge.
No hidden control flow.
No hidden memory allocations.
No preprocessor, no macros.
That’s how I remember Zig from the day I met it. It tries to be WISIWZD, i.e. What I See Is What Zig Does (-: Does sometimes-pass-by-ref optimisation contradict that promise? The code semantic and programme’s behaviour changes (and thus potentially bugs creep in) basing on a compilers decision and idea how complex or large passed arguments are.
(Oh, and it’s so wonderful that Zig developers openly admit and discuss these issues. Kudos!)
Highly recommend this talk from Software You Can Love, Vancouver 2023, which dives deeper into this topic: ATTACK of the KILLER FEATURES - Martin Wickham.
This is something I was hoping Carbon would have addressed in a satisfactory way, but, alas, they do the same thing Zig does, so Zig is once again the language which will have to innovate to solve this problem.
The video explains that “Parameter Reference Optimization” is a serious design flaw in Zig. It’s a footgun that leads to subtle bugs that can’t be seen by grug-brained developers. It’s unsound. Other system languages like C, C++, Rust do not have this footgun, so people who come to Zig from those other languages will fall into the trap.
It seems like the obvious fix (but not mentioned in the video?) is to use C semantics by default for passing large parameters (copy the value before making the function call, then pass a reference to the copy). Only use the “parameter reference optimization” when the optimizing compiler can prove that no aliasing will result if the copy is elided. This optimization strategy is sometimes called “copy elision”.
At the 8 minute mark, a bug in
List.add()from the Zig library is shown. The bug is caused by the “Parameter Reference Optimization” feature. To fix the bug requires the developer to add an extra copy of ‘item’ to the function body, which causes a performance hit. If, instead, Zig used copy elision as a sound compiler optimization, then you wouldn’t have this subtle bug, you wouldn’t take a performance hit to fix the bug, and the extra copy of the ‘item’ parameter would only happen in cases where it was needed (the compiler adds the copy at the caller site to prevent parameter aliasing).For the other buggy feature in the video, “Result Location Semantics”, I make the same comments. Fixing the bug in
rotate(without changing the function signature) requires the developer to make an explicit copy of the parameter before using it, which again is a performance hit. Instead of making people write code like this, you should generate compiler temporaries by default, and elide the copies as an optimization only when it is sound to do so.You guys understand these issues better than I do, so what am I missing? Alias analysis is hard, so the standard approach I describe will have worse performance than what the Zig compiler does today (on par with C/C++), and you want better performance than C?
I wouldn’t say that the issue is solved in other languages:
Then, there’s this annoying question of “should you pass small things by reference or by value? And what is small, by the way?”.
And then there’s: https://outerproduct.net/boring/2021-05-07_abi-wrong.html
My personal take here is that “
fn f(param: Foo)passes parameterparamby immutable reference” is the right high-level semantics:The question is rather how to implement that semantics without running afoul of aliasing, which is what Carbon, Zig, and Hylo (formerly Val) are trying to do.
Rust has added fixes/optimizations to LLVM to eliminate redundant stack-to-stack copies.
Thank you for the video suggestion, I am not a Zig programmer and the blog post was a bit too terse for me – I could not really follow. The video is super clear and got me very excited about Zig, but folks if you expect a solution besides a clear exposition of the problems you will be disappointed :D we’ll have to wait a while for that I guess.
I’ve been getting in to Zig in the past few weeks and I really have to say: It’s a joy. Would forcing the compiler to always pass a struct by value with a flag ever be considered? Or does that go harder against the grain of other language principles?
That kind of solution doesn’t really work because it doesn’t compose … Now libraries would have to add test coverage for being compiled with both flags.
And if the libraries have dependencies, then they can’t rely on the semantics of those either. You end up with a 2^N test matrix for N libraries.
The flags also don’t appear in the source code, which is a problem for readability.
Compiler flags can be used for optimizations, which by definition are semantics-preserving. Pass by value vs. by reference isn’t a semantics-preserving change – it’s basically the opposite semantics!
Adding a compiler flag to decide semantics as fundamental as “are variables passed by pointer or by copy” sounds like a terrible idea.
In case you don’t understand (like initially myself):
Go https://zig-play.dev/ and paste in:
Result:
wtf: 5, while theawas assumed to be a copy.Ah so “anything” rather than everything.
good catch! I’ve updated the article title (can’t change the lobster one though)
And is that expected behavior, or a bug that will be fixed? If it will be fixed, it doesn’t seem like a big deal, unless the point is that the bug reveals the underlying behavior of things being passed by reference under the hood?
it is UB (I think), not a bug (by design), just how Zig works for now
The zig language has a spec IIRC. So if this behaviour isn’t in the spec then it would be a bug, no?
Oh, it’s explained in the link at the top of the post. Which I followed earlier but (impressively) failed to notice what it said.
https://ziglang.org/documentation/0.11.0/#Pass-by-value-Parameters
So it’s not undefined behavior in the C sense, it’s merely implementation defined behavior. But this is one hell of a footgun. I’m very surprised this simple program has two possible behaviors. I would expect that the compiler would sometimes pass by reference, but conservatively ensure that the fact it was doing so was never visible, thus making a copy in this case.
The optimisation in itself is reasonable. The fact that it’s not transparent, i.e., it doesn’t preserve value semantics, seems to be the bug.
So I’ve read through this multiple times, and it sounds like Zig chooses by some opaque criteria to pass by “large” values by what c++ would call a const reference. But it does this whether or not the type is mutable, and thus it is not semantically equivalent to copy by value despite being syntactically identical. This seems like a language bug.
Call it a language bug, or call it implementation-defined behavior. I agree that it is insidious and bug-prone.
Aren’t function arguments in Zig always const anyway?
The problem as I understand it from the article is that the way Zig is implemented (using C++ syntax here)
Could actually be implemented as (implicitly, at the compilers discretion):
Which is only safe if some_type is itself immutable. But if some_type is not immutable you get different behavior for:
if p0 is changed to some_type& you can hopefully see how the program behaviour would change, and possibly not in a good way. Because this seems to be an decision made by the compiler that does not occur 100% of the time, it seems that this could result in different versions of the compiler, or different callsites even (depending on inlining, etc) changing behaviour.
Now I believe swift does do automatic conversion of value types to reference parameters, but that’s because value types in swift are semantically immutable (I think the moral equivalent to this particular code in swift would require UnsafeMutablePointer which is a pretty big red flag).
To be clear, the title is a bit hyperbolic. “Everything” does not include primitive types (my first alarmed assumption upon reading it).
sorry about that! I pondered on this, and decided to have a shorter title instead of an accurate one.
Zig passes result location as reference too. I don’t know if primitive types apply there, since it’s not documented :(
I’d like to understand better how PRO is different from the usual ABI behavior of “things larger than a register (or two) get stored on the stack and passed by address”. I assume the difference is that in C for example the compiler has to provide the illusion of value semantics, so in the general case that stack location needs to be a fresh copy, and you pay for that. Are there other differences?
The difference is that, in C/C++/most other languages, the copy that’s passed by reference is just that: a copy. You then rely on copy elision to avoid doing the copy for cases where the original cannot be mutated after the call.
In C, when you do:
The compiler transforms it into something roughly equivalent to:
This guarantees the behaviour that the callee sees a non-aliased copy. When optimisations run, the first thing that will happen is that scalar replacement of aggregates will split
xapart into a load of independent fields represented with SSA variables. If something takes the address ofx, thenxwill be reconstituted, but if not then those SSA variables will not exist in any defined location until the call tosomeFn, at which point they’ll be stored into__tmpCopy. In general, if some pointers toxescape (or, more accurately, if the compiler cannot prove that pointers toxhave not escaped) then bothxand__tmpCopywill exist in the optimised code. Otherwise,xwill be eliminated.This is a sound semantics-preserving transform. The Zig thing is only if your semantics are ‘nondeterminstically, sometimes, things passed by value may alias other things’.
Swift does something interesting here and has a bunch of value types that implement copy-on-write behaviour, so you can pass them by reference and end up with a copy if you mutate them.
At the end of the page:
Uh, is this a concern of people (bloggers) in general? lobste.rs isn’t a big site like Reddit, HN, or Slashdot (back in the day). I guess I don’t understand.
I believe the intent is to link out to active discussions of the article, and the author is promoting those two options as ones they would support. If it did get discussed on either some fediverse page or lobste.rs they would add links. If there were a discussion on Reddit they would not.
I guess.
That makes sense. The “site has been slashdot’ed” meant that the server was overloaded, and that was obviously a bad thing.
I especially don’t understand it because the author submitted the story themselves on lobste.rs.
Maybe it’s a meta-joke?
yes! not a joke though. I forgot to re-upload the site with the link after Step 3
It’s too late for that. I joined 5 years ago, and even then I remember lobsters linkbacks in this fashion
it has now been lobstered! check the article again!
“This post has not been Lobstered.” means that the post has not been lobstered! The text is applied automatically!
I also don’t post on the YC site.
I’m not watching Zig proactively and closely (yet!), but since I heard about it and its promises a long time ago, it got me hooked and made me root for it (-: Now, that said, what strikes me as a surprise is this… and please correct my potentially incorrect understanding.
One of the Zig’s USPs is this (taken straight from the project homepage):
That’s how I remember Zig from the day I met it. It tries to be WISIWZD, i.e. What I See Is What Zig Does (-: Does sometimes-pass-by-ref optimisation contradict that promise? The code semantic and programme’s behaviour changes (and thus potentially bugs creep in) basing on a compilers decision and idea how complex or large passed arguments are.
(Oh, and it’s so wonderful that Zig developers openly admit and discuss these issues. Kudos!)
a bit. the trade-off is better performance
thanks :)