1. 24
  1.  

  2. 8

    The Go code looks to be overly complicated.

    One small improvement would be to use Go 1.16’s embed directive:

    //go:embed static/*
    var staticFiles embed.FS
    var staticHandler = func() http.Handler {
    	fSys, err := fs.Sub(staticFiles, "static")
    	if err != nil {
    		panic(err)
    	}
    	return http.FileServer(http.FS(fSys))
    }()
    

    In this example staticHandler is a http.Handler that serves everything inside the ./static directory, relative to the source file. The contents of the folder are embedded into the binary at compile time. This handler is understood by the standard library.

    1. 2

      The contents of the folder are embedded into the binary at compile time.

      That violates the constraints the author states for the program though? You are supposed to be able to serve an arbitrary path specified at program startup.

      1. 3

        I based my comment off the first line (in bold) of “The idea” section:

        why aren’t there any programs that I can download that serve literally just one file

        They also later state that a benefit of using Go is that it “Produce[s] static binaries”. I interpreted this to mean a single, dependency-free, binary is desirable for easy deployment.

    2. 4

      Looks like there is a typo in the last benchmark table,

      Request/sec	34.76	3773
      Trasfer/sec (MB)	37.34	3690
      

      if not then there is 100x improvement in bandwith for the use of hyper::body::Bytes. Also, this is some of the most “unconventional” Go and Rust code I saw in a while. What is the purpose of this article anyways, showing how to build a custom HTTP server to serve a single file and concluding that Rust is the answer? When deciding what language to pick to implement a similar tool those toy benchmarks do not help a thing. I do not get why this got that many upvotes.

      1. 3

        The Go here is just wild. (Also, fasthttp should be a nonstarter.) I think this is functionally equivalent, and also much faster.

        edit: binding to a Unix domain socket to avoid the network stack, and using the io.Copy method, easily gets you to 100k RPS on sufficient hardware.

        1. 3

          Okay, I didn’t read to the very end, so forgive me if that’s been touched in the article. But I also haven’t found the word “sendfile” in the text… Thing is, reading a file in memory and pushing it in chunks through a socket in user-land code is probably slower than using sendfile which is a way to tell the kernel to take an open file descriptor and send its contents to another open file descriptor (or a socket) as fast as possible, and that doesn’t block the user-land code.

          1. 2

            Go will actually utilize sendfile(2) under the hood when it makes sense: https://golang.org/src/net/sendfile_linux.go

            Unfortunately the way the author is reading the file or stdin into a buffer means that this optimization will likely not be used. Instead one could os.Open the file and then io.Copy to the ResponseWriter. This pattern is exposed by the http.FileServer implementation with a lot of other niceties. For the stdin use case I believe opening /dev/stdin as a file would work.

            Similar optimizations have been proposed for Rust; however, I’m less familiar with the state of them: https://github.com/rust-lang/rust/issues/60689

            1. 2

              Forgive my poor Unix-fu, but how would you expose sendfile to port 5000 like the author does with their programs?

              1. 2

                sendfile is a function, so you would just bind a socket as normal (which creates a file descriptor) and open the file (which also creates a file descriptor) and then run sendfile() on the 2.

                https://man7.org/linux/man-pages/man2/sendfile.2.html

                1. 1

                  bind a socket as normal

                  This is the part where my poor Unix knowledge comes in. What commands do you use to bind a socket and open the file?

                  1. 2

                    In any operating system, when you open a file or a socket, you get back some kind of handle. In UNIX, this is a file descriptor, which is an int that is an index into the file descriptor table. The sendfile system call takes a file descriptor for a file and a file descriptor for a socket and then sends a range of the file to the socket. This avoids a copy: the disk controller DMAs data into the buffer cache, the network stack then instructs the NIC to DMA the data out (it may need to encrypt it, but at least on FreeBSD that can also happen in the kernel and with some NICs that can happen on the device).

                    1. 1

                      Thanks!