1. 49

  2. 11

    Nice! I use a similar trick for nginx virtual hosting:

      # Proxy to a backend server based on the hostname.
      if (-d /var/lib/nginx/vhosts/$host) {
        proxy_pass http://unix:/var/lib/nginx/vhosts/$host/server.sock;

    This lets you add and remove virtual hosts without touching nginx config. Like the bind broker, the /var/lib/nginx/vhosts/$host/ directories also let you apply the unix permissions model.

    This maps IP ownership (which doesn’t really exist in unix) to the filesystem namespace (which does have working permissions).

    I wish more programs supported unix domain sockets for exactly this reason. Well, that, and because path allocation is easier to coordinate than port allocation.

    Then I broke out the big cheat stick and just spliced the sockets together. In reality, we’d have to set up a read/copy/write loop for each end to copy traffic between them. That’s not very interesting to read though.

    So, today I learned about SO_SPLICE. Awesome! But why is it necessary to set up a read/copy/write loop? Isn’t that what SO_SPLICE does for you?

    1. 4

      The splicing code doesn’t even work on OpenBSD, because it’s not (yet) possible to splice sockets from different domains. I wouldn’t want the answer key to be so easy to steal. Bonus points if you fix the kernel instead.

      1. 2

        Thanks, I missed that bit. In setsockopt(2) it says “both sockets must be of the same type,” but nothing about family, so I assumed any two SOCK_STREAM sockets could be spliced.

      1. 4

        Though I can see the cuteness of this I cannot help but think a shell script that seds in the users allocated IP/runas and spit out and reloads the configuration snippets for the admins choice of inetd server seems easier.

        You would also get a bunch of functionality (datagram, fork’ing, ACLs, rate limiting, logging, …) for free and the user could choose to ignore ‘networking’ (ie. accept()) as everything is a STDIO operation.

        The configuration is also simple enough to reject/ignore/audit ‘unsafe’ configurations.

        1. 4

          Foreword: I suck at C and have never used it professionally. I started programming professionally using C# + ASP.NET; my classes mostly used that, too. I’m digging through this because I am curious about it. Please be gentle. :P

          Question: I dug through many man pages and header files and SO_SPLICE seems to be absent from the possible values for setsockopt on Linux, at the very least on my system; would a splice() call work out to the same end result on my system, assuming I substituted all the kevent related stuff for the apparently more portable poll or select?

          1. 3

            Yeah, SO_SPLICE is OpenBSD-specific. I think it’s a relatively recent addition; here are slides from a 2013 talk about adding it.

            And yes, splice() is the (somewhat older) Linux equivalent. However, like with OpenBSD’s SO_SPLICE, it doesn’t currently work out of the box for this particular case: you can’t splice between a TCP socket and a Unix domain socket. That’s unsupported for two reasons. First, but this one can be worked around, splice() doesn’t support splicing between two ends that are both non-pipes; from the man page “one of the file descriptors must refer to a pipe”. But you can splice two sockets with a little extra hassle by creating a named pipe and splicing socket -> pipe -> socket. However the more fatal problem is that splice() doesn’t support splicing from Unix domain sockets on the input side at all, it can only splice from a pipe to a unix socket but not vice-versa (though this isn’t documented).