increasing the default number of upload jobs in the Cachix CLI to 8. You can always push that number even higher with the –jobs option.
I’m surprised that this makes sense. Is this a common experience with CDNs? I get how 2-3 streams could be an improvement on high latency links because of TCP inefficiencies and jitter, but 8 or higher sounds extreme.
It depends on the size of the files, does it not? If you’re uploading a bunch of smaller files, having many more in flight may allow for a quicker overall upload due to overheads.
Release notes state “Improved throughput for large files, which can now be uploaded in multiple parts in parallel.” which seems to suggest that even a single 5GB file would now be able to profit from optimizations for multi-file uploads.
You can compare network stream performance between single and multi connection transfers using speedtest.net (I see a 600Mbps difference between the two on my home connection) - I find it unintuitive as well, but the folks at work that know more than I do about the internals of the internet assure me that there is some technical reason this is true.
One point this comes up is with QSFP and OSFP connections (quad SFP and octuple SFP). For instance, a 400Gbit network link might actually be an 8x50Gbit. Each flow - the five tuple (protocol, source port, source IP, dest port, dest IP) - can only go on one of the four/eight links.
Unfortunately I’m only familiar with this when the QSFP connection is on the host. I don’t know if or how it matters for intermediate routers.
Hey folks, Cachix founder here. Here are a few links for getting started:
https://nix.dev Nix tutorials
https://devenv.sh Fast, Declarative, Reproducible, and Composable Developer Environments
Let me know if you have any questions. Domen
I’m surprised that this makes sense. Is this a common experience with CDNs? I get how 2-3 streams could be an improvement on high latency links because of TCP inefficiencies and jitter, but 8 or higher sounds extreme.
It depends on the size of the files, does it not? If you’re uploading a bunch of smaller files, having many more in flight may allow for a quicker overall upload due to overheads.
The first example is 14min -> 2min for “Pushing a single 5GB file”, so it doesn’t seem to be about a bunch of smaller files.
Release notes state “Improved throughput for large files, which can now be uploaded in multiple parts in parallel.” which seems to suggest that even a single 5GB file would now be able to profit from optimizations for multi-file uploads.
You can compare network stream performance between single and multi connection transfers using speedtest.net (I see a 600Mbps difference between the two on my home connection) - I find it unintuitive as well, but the folks at work that know more than I do about the internals of the internet assure me that there is some technical reason this is true.
One point this comes up is with QSFP and OSFP connections (quad SFP and octuple SFP). For instance, a 400Gbit network link might actually be an 8x50Gbit. Each flow - the five tuple (protocol, source port, source IP, dest port, dest IP) - can only go on one of the four/eight links.
Unfortunately I’m only familiar with this when the QSFP connection is on the host. I don’t know if or how it matters for intermediate routers.