1. 10
  1.  

  2. 4

    …or just use UNIX sockets.

    1. 1

      But that only works for comms b/w procs on the same machine!

      1. 1

        Which is what the article is about (as well as ephemeral ports).

        1. 1

          I thought it was about using WebSockets. Did I miss something?

          1. 6

            No more than the article is about Ruby.

            Ephemeral port exhaustion only happens when using TCP, if you are proxying to localhost then UNIX or anon sockets are a far better option; they also have less overhead.

            1. 2

              I was wondering, is there any downside of binding to UNIX sockets instead of regular TCP ones?

              1. 4

                Other than it being a host local only socket, not really though portability to Windows might be important to you. Maybe you are fond of running tcpdump to packet capture the chit-chat between the front and backends and UNIX sockets would prevent this though if you are doing this you probably are just as okay with using strace instead.

                From a developer perspective instead of connecting to a TCP port you just connect to a file on your disk, the listener when binding to a UNIX socket creates that file, nothing else is different. The only confusing gotcha is that you cannot ‘re-bind’ if the UNIX socket file on the filesystem already exists; for example the situation when your code bombed out and was unable to mop up. Two ways to handle this:

                1. unlink() (delete) any previous stale UNIX socket file before bind()ing (or starting your code); most do this, as do I
                2. use abstract UNIX sockets which works functionally identical but does not create files on the filesystem so no need to unlink. You need to take care though on the naming of the socket as all the bytes in sun_path contribute to the reference name, not just the bytes up to the NUL termination

                Personally what I have found works with teams (for an HTTP service) is for development the backend presentation is a traditional HTTP server listening over TCP enabling everyone to just use cURL, their browser directly or whatever they like. In production though, a flag is set (well I just test if STDIN is a network socket) to go into UNIX socket/FastCGI mode.

                As JavaScript/Node.js is a effectively a lingua franca around here, this is what that looks like:

                $ cat src/server.js | grep --interesting-bits
                const http = require('http');
                const fcgi = require('node-fastcgi');
                
                const handler = function(req, res){
                  ...
                };
                
                const server = fcgi.isService()
                  ? fcgi.createServer(handler).listen()
                  : http.createServer(handler).listen(8000);
                
                server.on('...', function(){
                  ...
                });
                
                $ cat /etc/systemd/system/sockets.target.wants/myapp.socket 
                [Unit]
                Description=MyApp Server Socket
                
                [Socket]
                ListenStream=/run/myapp.sock
                SocketUser=www-data
                SocketGroup=www-data
                SocketMode=0660
                Accept=false
                
                [Install]
                WantedBy=sockets.target
                
                $ cat /etc/systemd/system/myapp.service
                [Unit]
                Description=MyApp Server
                Before=nginx.service
                
                [Service]
                WorkingDirectory=/opt/myorg/myapp
                ExecStartPre=/bin/sh -c '/usr/bin/touch npm-debug.log && /bin/chown myapp:myapp npm-debug.log'
                ExecStart=/usr/bin/multiwatch -f 3 -- /usr/bin/nodejs src/server.js
                User=myapp
                StandardInput=socket
                StandardOutput=null
                #StandardError=null
                Restart=on-failure
                ExecReload=/bin/kill -HUP $MAINPID
                ExecStop=/bin/kill -TERM $MAINPID
                
                [Install]
                WantedBy=multi-user.target
                

                The reason for multiwatch in production is you get forking and high-availability reloads. Historically I would have also used runit and spawn-fcgi but systemd has made this no longer necessary.

              2. 1

                Agreed.

            2. 1

              Local load balancing is the motivating example, but I wrote it highlight the general problem when load balancing between a large number of connections between a small number of backends (potentially external machines).

              UNIX sockets might be a reasonable solution to the particular problem in the post. It’s not something I’ve tried with HAProxy before though, so I’m not sure how practical it would be.

        2. 3

          It’s briefly mentioned in the article, but be very wary of the number of sockets in TIME_WAIT state. It’s bitten me more than I’d like admit.

          1. 2

            FreeBSD has a tunable for “no local time wait”

            1. 1

              The Linux kernel also has the following sysctl tunables which affect the behavior of sockets in TIME_WAIT (or more accurately, the kernel’s willingness to reuse them):

              • net.ipv4.tcp_tw_reuse
              • net.ipv4.tcp_tw_recycle (apparently removed in Linux 4.12+)

              I have used the former without issue, but my understanding is they are best avoided as e.g., stateful packet filers will be confused.