Perhaps it flew over my head in the examples but how does the signal call know when the server is done (presumably) copying the relevant buffer data into a non-shared region for later processing? The shared region has separate areas for the incoming vs outgoing buffers so it seems both sides can write messages simultaneously without locking. But without a lock, what does the signal func use to determine when to stop blocking?
This is synchronous and blocking. The thread will not continue execution until the server end has said it is ok (the shmifsrv_video_step code). The server can defer this in order to prioritise other clients or to stagger releases to mitigate the ‘thundering herd’ problem.
I think this passage is meant to explain it but I am struggling to unpack the meaning here.
Context: the article was mostly written for ongoing discussions with a 3rd party and why I didn’t post it here, so some of the OS dependent parts were omitted. The same goes for how aux data need to be tagged into some events for GPU buffer handles and clipboard contents (DuplicateHandle for Windows which is the better option, forced to use a Socket elsewhere).
The safe route is named semaphores (thanks OSX) - otherwise three futexes: one for waiting when the ring-buffer is full and client doesn’t want to error-handle themselves, one for audio and one for video. Audio has a deeper number of smaller buffers and also rarely locks.
In experiments using it as a kernel interface it was both easier and snappier to use it as a custom blocking syscall.
DuplicateHandle for Windows which is the better option, forced to use a Socket elsewhere
Note that “modern” Linux has gotten some more much needed functionality to duplicate file descriptors without having to do the old socket + sendmsg() dance.
You still have the ugly ‘int + sparse-allocation + ulimit’ problems of regular file descriptors, and it’s in the wrong direction. For DuplicateHandle you get what the handle is in the namespace of the recipient process (and immediate failure notification), so you can fill that out in the event as it is enqueued or hold the event
The way pidfd_getfd seems designed (sleep-deprived disclaimer) you’d need to encode the process local descriptor in the event, keep it open until the other end gets to getfd.2, then ack so that it’s safe to close without racing - on top of giving ptrace credentials and handle deferred failure. That gets more painful with partial fails (multiplanar GPU handles with fences consuming 4+ descriptors for one logical object when you’re at > ulimit - n).
Yup the roundtripping would be painful, that’s a valid complaint. However, for the other direction there is SECCOMP_IOCTL_NOTIF_ADDFD, which will directly install a copy of the file descriptor into the target process fd table (gosh, will they ever stop overloading ioctl()).
A few years ago I had the very same problem when writing a little utility program, that intercepts (and without resorting to ptrace) all open()-style syscalls, transparently replaces them with completely different files and returns them to the target process. Indeed, the opened and faked fd is only valid in the interceptor process and there is no time for roundtripping, since the intercepted process is currently doing a syscall. The solution was using seccomp User Notify and you can immediately close the fd afterwards, so there is no problem with any race conditions. As a bonus you don’t need to handle anything on the intercepted process, so this works for any arbitrary programs, even the ones out of your control.
The only caveat I see is that this requires a pre-defined relationship between the two processes (usually that means the process installing the fd needs to be the parent of the other process), which I suppose is a deal-breaker for your usecase. Surely we should have covered all usecases of DuplicateHandle on the Linux side at some point… :D
Perhaps it flew over my head in the examples but how does the signal call know when the server is done (presumably) copying the relevant buffer data into a non-shared region for later processing? The shared region has separate areas for the incoming vs outgoing buffers so it seems both sides can write messages simultaneously without locking. But without a lock, what does the signal func use to determine when to stop blocking?
I think this passage is meant to explain it but I am struggling to unpack the meaning here.
Context: the article was mostly written for ongoing discussions with a 3rd party and why I didn’t post it here, so some of the OS dependent parts were omitted. The same goes for how aux data need to be tagged into some events for GPU buffer handles and clipboard contents (DuplicateHandle for Windows which is the better option, forced to use a Socket elsewhere).
The safe route is named semaphores (thanks OSX) - otherwise three futexes: one for waiting when the ring-buffer is full and client doesn’t want to error-handle themselves, one for audio and one for video. Audio has a deeper number of smaller buffers and also rarely locks.
In experiments using it as a kernel interface it was both easier and snappier to use it as a custom blocking syscall.
Note that “modern” Linux has gotten some more much needed functionality to duplicate file descriptors without having to do the old socket +
sendmsg()dance.One of them is pidfd_getfd(), which should be roughly equivalent to what you need from DuplicateHandle()?
You still have the ugly ‘int + sparse-allocation + ulimit’ problems of regular file descriptors, and it’s in the wrong direction. For DuplicateHandle you get what the handle is in the namespace of the recipient process (and immediate failure notification), so you can fill that out in the event as it is enqueued or hold the event
The way pidfd_getfd seems designed (sleep-deprived disclaimer) you’d need to encode the process local descriptor in the event, keep it open until the other end gets to getfd.2, then ack so that it’s safe to close without racing - on top of giving ptrace credentials and handle deferred failure. That gets more painful with partial fails (multiplanar GPU handles with fences consuming 4+ descriptors for one logical object when you’re at > ulimit - n).
Yup the roundtripping would be painful, that’s a valid complaint. However, for the other direction there is SECCOMP_IOCTL_NOTIF_ADDFD, which will directly install a copy of the file descriptor into the target process fd table (gosh, will they ever stop overloading
ioctl()).A few years ago I had the very same problem when writing a little utility program, that intercepts (and without resorting to
ptrace) allopen()-style syscalls, transparently replaces them with completely different files and returns them to the target process. Indeed, the opened and faked fd is only valid in the interceptor process and there is no time for roundtripping, since the intercepted process is currently doing a syscall. The solution was using seccomp User Notify and you can immediately close the fd afterwards, so there is no problem with any race conditions. As a bonus you don’t need to handle anything on the intercepted process, so this works for any arbitrary programs, even the ones out of your control.The only caveat I see is that this requires a pre-defined relationship between the two processes (usually that means the process installing the fd needs to be the parent of the other process), which I suppose is a deal-breaker for your usecase. Surely we should have covered all usecases of
DuplicateHandleon the Linux side at some point… :D