Why am I not surprised! This should not be considered acceptable behavior for a standard library.
I mean, it’s impossible to avoid in C: varargs don’t tell the callee what arguments were passed. I was wondering if you could switch between the open functions with variadic macros somehow, but if there’s a way I can’t see it offhand. So it’s not up to the implementation; POSIX would have to be changed to specify open2 and open3.
On FreeBSD and, I believe, most other systems, the kernel interface is not variadic. Only the libc shim is. In C++, you’d implement this with a default last argument and so the caller would always pass 0 unless the argument were added explicitly. I believe Rust also has a mechanism for doing this, but C does not.
This interface dates back to K&R C, when parameter types (and number) were not part of the formal type of functions. Back then, it was completely acceptable to treat some of the later arguments as optional. All arguments were passed on the stack and if you didn’t need later ones then you never read them and so the caller could skip them when not needed.
I guess we should be thankful that open seems to be the only place where that has leaked into POSIX.
For extra fun, there’s no requirement in C that variadic and not-variadic functions have the same calling convention and so treating this as a function with explicit arguments for FFI is technically undefined behaviour.
For extra fun, there’s no requirement in C that variadic and not-variadic functions have the same calling convention and so treating this as a function with explicit arguments for FFI is technically undefined behaviour.
This isn’t even just a theoretical pitfall. I ran into it in practice when porting a client’s macOS device driver kernel extension from x86-64 to arm64. (M1 Macs) The original developers had done some questionable function pointer casting and I think called a non-variadic function via a variadic function pointer (or vice versa) which was disguised by a round-trip via void* or something along those lines. This worked out OK on x86, but had me scratching my head over a mysterious crash on arm64 until I cottoned on to what was happening.
We also got bitten by this in CHICKEN. This is when we switched from trying to pass arguments to function pointers in the “standard” C way to explicitly allocating an argument vector and passing that as a single argument to the function pointer.
Before, we were using an assembly stub on some platforms to allow us to apply functions to arbitrary numbers of arguments, and a hacky C macro that recursively expanded to up to 1024 or so arguments in a big switch. So, in the end, it was a net positive: the code is a lot cleaner and simpler, and not even slower than the old code. But going from our old implementation to the new one required a massive amount of programming. Luckily this was done in the context of a commercial project, so we had the funding to do it properly.
The interesting thing is that we got by with the old argument passing convention for a decade and a half without ever running into trouble over this. This includes platforms like x86, x86_64, SPARC, PowerPC and other ARM variants. It’s only when porting to Apple’s version of the ARM ABI that we ran into this bug.
There’s some interesting back-story to this. Arm originally proposed a variadic calling convention that was more efficient (especially for the common case where a variadic function is a wrapper around a function that that an explicit va_list). Both the Perl and Python interpreters assumed that you could invoke a function as variadic as long as it had fewer than 4 arguments (I think), without casting the function pointer to the correct type. I learned about this because I was proposing a variadic calling convention for CHERI where all variadic arguments lived on the stack, which turned out to be quite a bit more efficient than the MIPS default (which couldn’t be made to work with CHERI anyway). The feedback from Arm was that there was too much code that made that assumption and they’d had to back away from it in their main ABI. Apple went ahead and shipped the original proposal. I’m not sure if they thought the iOS ecosystem was sufficiently controlled that they could afford to require source code changes or if they just didn’t have time to make the switch (they shipped AArch64 really early). Either way, they’ve caused a bunch of cleanup in invalid-but-happened-to work C code.
I knew there’s be at least one example I’d forgotten! Fortunately, that’s not an API that I’ve had to use very often, unlike open which is pretty much essential for most *NIX software.
#define COUNTER(a1, a2, a3, n, ...) n
#define CONCAT(a, b) CONCAT_(a, b)
#define CONCAT_(a, b) a ## b
#define OPEN_2(a, b) open(a, b, 0)
#define OPEN_3 open
#define my_open(...) CONCAT(OPEN_, COUNTER(__VA_ARGS__, 3, 2, 1, a))(__VA_ARGS__)
Even better, you can call it “open” not “my_open” to just “fix open”. { You may need to put other references to the open function in ()s, i.e. “open” -> “(open)”, to be robust to a situation where open is already so wrapped. }
I mean, it’s impossible to avoid in C: varargs don’t tell the callee what arguments were passed. I was wondering if you could switch between the open functions with variadic macros somehow, but if there’s a way I can’t see it offhand. So it’s not up to the implementation; POSIX would have to be changed to specify open2 and open3.
On FreeBSD and, I believe, most other systems, the kernel interface is not variadic. Only the libc shim is. In C++, you’d implement this with a default last argument and so the caller would always pass 0 unless the argument were added explicitly. I believe Rust also has a mechanism for doing this, but C does not.
This interface dates back to K&R C, when parameter types (and number) were not part of the formal type of functions. Back then, it was completely acceptable to treat some of the later arguments as optional. All arguments were passed on the stack and if you didn’t need later ones then you never read them and so the caller could skip them when not needed.
I guess we should be thankful that open seems to be the only place where that has leaked into POSIX.
For extra fun, there’s no requirement in C that variadic and not-variadic functions have the same calling convention and so treating this as a function with explicit arguments for FFI is technically undefined behaviour.
The kernel interface isn’t variadic. The syscall always has 3 arguments, but is declared variadic is C for convenience.
This isn’t even just a theoretical pitfall. I ran into it in practice when porting a client’s macOS device driver kernel extension from x86-64 to arm64. (M1 Macs) The original developers had done some questionable function pointer casting and I think called a non-variadic function via a variadic function pointer (or vice versa) which was disguised by a round-trip via
void*or something along those lines. This worked out OK on x86, but had me scratching my head over a mysterious crash on arm64 until I cottoned on to what was happening.It turns out that at least on Apple’s arm64 platforms, the variadic arguments are all passed on the stack, even if they would be passed in a register were they non-optional.
We also got bitten by this in CHICKEN. This is when we switched from trying to pass arguments to function pointers in the “standard” C way to explicitly allocating an argument vector and passing that as a single argument to the function pointer.
Before, we were using an assembly stub on some platforms to allow us to apply functions to arbitrary numbers of arguments, and a hacky C macro that recursively expanded to up to 1024 or so arguments in a big switch. So, in the end, it was a net positive: the code is a lot cleaner and simpler, and not even slower than the old code. But going from our old implementation to the new one required a massive amount of programming. Luckily this was done in the context of a commercial project, so we had the funding to do it properly.
The interesting thing is that we got by with the old argument passing convention for a decade and a half without ever running into trouble over this. This includes platforms like x86, x86_64, SPARC, PowerPC and other ARM variants. It’s only when porting to Apple’s version of the ARM ABI that we ran into this bug.
There’s some interesting back-story to this. Arm originally proposed a variadic calling convention that was more efficient (especially for the common case where a variadic function is a wrapper around a function that that an explicit va_list). Both the Perl and Python interpreters assumed that you could invoke a function as variadic as long as it had fewer than 4 arguments (I think), without casting the function pointer to the correct type. I learned about this because I was proposing a variadic calling convention for CHERI where all variadic arguments lived on the stack, which turned out to be quite a bit more efficient than the MIPS default (which couldn’t be made to work with CHERI anyway). The feedback from Arm was that there was too much code that made that assumption and they’d had to back away from it in their main ABI. Apple went ahead and shipped the original proposal. I’m not sure if they thought the iOS ecosystem was sufficiently controlled that they could afford to require source code changes or if they just didn’t have time to make the switch (they shipped AArch64 really early). Either way, they’ve caused a bunch of cleanup in invalid-but-happened-to work C code.
There’s also
fcntl’sF_GETFL,F_GETFD, andF_GETOWN.I knew there’s be at least one example I’d forgotten! Fortunately, that’s not an API that I’ve had to use very often, unlike
openwhich is pretty much essential for most *NIX software.It’s quite simple:
Even better, you can call it “open” not “my_open” to just “fix
open”. { You may need to put other references to theopenfunction in ()s, i.e. “open” -> “(open)”, to be robust to a situation where open is already so wrapped. }Yes! I should have clarified I mean the behavior specified (or in this case, left unspecified) in POSIX.