I have been reading through this and I feel like POSIX MQs are really underutilized. Perhaps it’s because they don’t use the file API.
Does anyone have thoughts on using POSIX MQs?
One issue with them on Linux, at least, is that they have smallish default maximums: 8kb max message size, max 10 messages in flight at a time.
The bigger issue though imo is that they fit in an awkward gap between stream abstractions like pipes or sockets, and full-featured message queue systems. Most people who want simple local IPC just use pipes or unix sockets (even if this requires a bit of DIY protocol around message delimeters), and people who want full-on queueing usually want queue state that persists over reboots, at least the option for network distribution, etc., so they use zeromq or similar.
I’m looking at POSIX mqueue’s for better concurrency control than pipes but with less ceremony than sockets. Seems like that might be its sweet spot. Also, on FreeBSD, PIPE_BUF is way too small (512 bytes). I might whip up some test programs to see how well they go.
I don’t think I’ve ever actually heard of them before, though I’m definitely planning on looking into using them for a few things now…
They seem like a very nice tool. It’s interesting that they support priority natively, and can handle both non-blocking modes and blocking with a timeout. To clarify what mjn says, they have a 8kb default max size. Looks like the actual max size is somewhere around 16MB, which makes them more than big enough for my use cases.
I wonder what the performance is like.
Yeah, the max size (and number of messages) is configurable, but it’s a kernel parameter rather than something accessible from userspace (so the program can’t request the change itself, at least if it doesn’t run as root). Which is probably fine if you’re writing something fairly specialized, but it really reduces the number of cases I’d consider using them. I don’t want to write software where the install instructions have to tell users how to tweak their kernel parameters (and package managers don’t like to package those kinds of packages either).
That’s interesting. I wonder how that relates to containers/cgroups/etc. Can a docker container be spun up with a dedicated message size specific to that container?
I think I partially answered my own question. It appears that POSIX message queues are part of the IPC namespace, what I’m not sure about is if /proc/sys/fs/mqueue/msgsize_max is per-container.
Yeah, the max size (and number of messages) is configurable, but it’s a kernel parameter rather than something accessible from userspace
Is this the case? mq_open takes an mq_attr which lets one specify the size and max messages. There are upper bounds but they seem quite high from what I can gather.
Of the various /proc/sys/fs/mqueue/ parameters: you can override msgsize_default and msg_default with the parameters to mq_open, but only up to the ceilings specified by msgsize_max and msg_max.
But on my Debian machine, the *_default and *_max parameters are the same, 8192 message size and 10 messages, so in practice you can’t actually request anything larger than the default, without tweaking settings in /proc. It’s possible other distributions ship different defaults; I’ve only checked Debian.