It’s quite interesting to read this in the context of Rachel’s recent post about not pulling in libraries too quickly. I’m not quite sure I can make a precise point, and I’m not saying there’s a direct contradiction, but that it’s important to somehow combine the two insights.
Well, the point of this post, as an extension of that post, is that a lot of libraries do not handle all of those edge cases, and that at a certain level of deployment (Rachel has operated in some very large server fleets, from her other writings), you end up dealing with all of these edge cases. So, you may not want to use a library that doesn’t handle them gracefully
If you’re thinking “this means I need a pointer, and a counter, and I have to do this in a loop while bumping the pointer forward and clawing the counter downward”, you’re on the right track.
Never understood why libc/POSIX didn’t provide a way to do this. Other languages do, and it’s a common enough case, and touchy enough to do well in C, that it seems like a weird omission.
I thought C actually does protect you from this if you’re using stdio, and it’s the raw POSIX I/O functions that can and will fuck you over. On Unix, it’s easy to conflate them because stdio doesn’t have as much magic as other platforms like text/record conversion… except the protection from PC lusering.
The problem is if the system simply can’t keep up, a higher level might be able to do something about it but the lower function would only just keep trying to shovel data into it. Good think the Rust impl there returns on interrupted… trying again may very well be the right thing to do but this function probably won’t know. Just a similar decision will sometimes have to be made with a partial write too lest you end up with buffer overflows since they have nowhere to go.
but…. yeah it is often useful enough to write_all that i do it that way plenty often too.
For blocking sockets, if the system can’t keep up it will necessarily block at that point, which is usually what you want. As long as you only retry if you either got EINTR or made nonzero progress, you should be safe.
Of course the fun level is increased if you live in nonblocking land, but then you’re way more involved than you could expect libc to be already.
Also reminds me of this talk by Dan Luu: https://www.deconstructconf.com/2019/dan-luu-files
Didn’t know there was a talk; he also wrote an excellent essay (and follow-up) that I recommend.
It’s quite interesting to read this in the context of Rachel’s recent post about not pulling in libraries too quickly. I’m not quite sure I can make a precise point, and I’m not saying there’s a direct contradiction, but that it’s important to somehow combine the two insights.
Well, the point of this post, as an extension of that post, is that a lot of libraries do not handle all of those edge cases, and that at a certain level of deployment (Rachel has operated in some very large server fleets, from her other writings), you end up dealing with all of these edge cases. So, you may not want to use a library that doesn’t handle them gracefully
Never understood why libc/POSIX didn’t provide a way to do this. Other languages do, and it’s a common enough case, and touchy enough to do well in C, that it seems like a weird omission.
I thought C actually does protect you from this if you’re using stdio, and it’s the raw POSIX I/O functions that can and will fuck you over. On Unix, it’s easy to conflate them because stdio doesn’t have as much magic as other platforms like text/record conversion… except the protection from PC lusering.
stdio does protect you, but stdio doesn’t have sockets, and mixing abstraction levels on the same open file makes everything terrible.
The problem is if the system simply can’t keep up, a higher level might be able to do something about it but the lower function would only just keep trying to shovel data into it. Good think the Rust impl there returns on interrupted… trying again may very well be the right thing to do but this function probably won’t know. Just a similar decision will sometimes have to be made with a partial write too lest you end up with buffer overflows since they have nowhere to go.
but…. yeah it is often useful enough to write_all that i do it that way plenty often too.
For blocking sockets, if the system can’t keep up it will necessarily block at that point, which is usually what you want. As long as you only retry if you either got EINTR or made nonzero progress, you should be safe.
Of course the fun level is increased if you live in nonblocking land, but then you’re way more involved than you could expect libc to be already.