Distributed builds is why I made lazyssh. It’s a little basic right now, but I’m not sure how much I’ll improve on what’s there. It does just about everything I want.
I use it at $work to start beefy EC2 instances for building our NixOS deployments, and also for ARM builds that can run on the newer generation EC2 instance types. It sits in between Nix and the actual machine as an SSH jump host.
I used to also use it with VirtualBox locally, but am trying to cut down on non-sandboxed apps on a clean macOS install.
lazyssh looks sick as! I’d been thinking of writing something similar, and it’s good to know that I don’t have to. Hopefully you get the robustness dialed in.
Interesting overview of the Nix approaches, so thanks for sharing. My curiosity has been piqued by all the NixOS posts I’ve been seeing even though I don’t run it myself. Any reports from folks running a bunch of devices like the one from this post on their home network? What are you using them to do?
My home router/firewall/dhcp/ipsec server (atom x86-64) and file/media/proxy/cache/print server (celeron x86-64) are OpenBSD, so when I bought a Beaglebone Black (ARMv7) to toy with, it was fun to go down a rabbit hole pretending I was @tedu (his 2014 post) to learn about diskless(8) and pxeboot(8) and how to netboot via uboot. This ended being pure experimentation since the actual parallelized work I do at home is on a single beefy Linux workstation (hard requirement on Nvidia GPU for now) and I’m not a professional sysadmin. The BBB sits disconnected in a drawer, but the setup lives on as the mere handful of config line changes required to set up tftpd(8) on the file server and point dhcpd(8) to it from the router, so I gained a more complete understanding of those as a neat side effect of experimenting. At some point in the next couple years I’m going to want to play with a RISC-V SoC, but that’s going to mean looking at Linux again unless I magically become competent to write my own drivers.
I set up the workstation and chromebox as remote builders for all systems, just as @steinuil did in the post. I’m using the rpi for running Jellyfin (music) and Nextcloud (for sharing calendars and files with my spouse), and setting up the chromebox to be an IPFS node for sharing research data. The laptop and workstation are using home-manager for syncing my dev environment configurations, but I do most of the dev/data analysis in the workstation (which has gigabit connections to the internet), and while the laptop is often more than enough for dev, my home connection is way too slow for anything network-intensive (so, it serves as a glorified SSH client =P)
They are all wired together using zerotier, and services running in the machines are bound to the zerotier interface, which ends up creating a pretty nice distributed LAN.
I don’t have my configs in public (booo!), because I’ve not been too good on maintaining secrets out of the configs. But @cadey posts are a treasure trove of good ideas, and I also enjoyed this post and accompanying repo as sources of inspiration.
That’s a fair point. Part of using nixops was about exploring how to use it later for other kinds of deployment (clouds), and it is a bit excessive for my use case (especially since I use nixops to deploy locally in the laptop =P).
A lot of my nix experience so far is seeing multiple implementations of similar concepts, but I also feel like I can refactor and try other approaches without borking my systems (too much).
I offload builds to my desktop at home using guix offload. The only problem I’ve run into so far is not a lot of things seem able to cross compile (for instance, the linux kernel for aarch-64-linux).
Distributed builds is why I made lazyssh. It’s a little basic right now, but I’m not sure how much I’ll improve on what’s there. It does just about everything I want.
I use it at $work to start beefy EC2 instances for building our NixOS deployments, and also for ARM builds that can run on the newer generation EC2 instance types. It sits in between Nix and the actual machine as an SSH jump host.
I used to also use it with VirtualBox locally, but am trying to cut down on non-sandboxed apps on a clean macOS install.
lazysshlooks sick as! I’d been thinking of writing something similar, and it’s good to know that I don’t have to. Hopefully you get the robustness dialed in.Interesting overview of the Nix approaches, so thanks for sharing. My curiosity has been piqued by all the NixOS posts I’ve been seeing even though I don’t run it myself. Any reports from folks running a bunch of devices like the one from this post on their home network? What are you using them to do?
My home router/firewall/dhcp/ipsec server (atom x86-64) and file/media/proxy/cache/print server (celeron x86-64) are OpenBSD, so when I bought a Beaglebone Black (ARMv7) to toy with, it was fun to go down a rabbit hole pretending I was @tedu (his 2014 post) to learn about diskless(8) and pxeboot(8) and how to netboot via uboot. This ended being pure experimentation since the actual parallelized work I do at home is on a single beefy Linux workstation (hard requirement on Nvidia GPU for now) and I’m not a professional sysadmin. The BBB sits disconnected in a drawer, but the setup lives on as the mere handful of config line changes required to set up tftpd(8) on the file server and point dhcpd(8) to it from the router, so I gained a more complete understanding of those as a neat side effect of experimenting. At some point in the next couple years I’m going to want to play with a RISC-V SoC, but that’s going to mean looking at Linux again unless I magically become competent to write my own drivers.
I just converted my last non-NixOS machine yesterday, so I’ll share my experience =]
I currently have 5 machines running NixOS and deployed using NixOps (to a network called
ekumen):I set up the workstation and chromebox as remote builders for all systems, just as @steinuil did in the post. I’m using the rpi for running Jellyfin (music) and Nextcloud (for sharing calendars and files with my spouse), and setting up the chromebox to be an IPFS node for sharing research data. The laptop and workstation are using home-manager for syncing my dev environment configurations, but I do most of the dev/data analysis in the workstation (which has gigabit connections to the internet), and while the laptop is often more than enough for dev, my home connection is way too slow for anything network-intensive (so, it serves as a glorified SSH client =P)
They are all wired together using zerotier, and services running in the machines are bound to the zerotier interface, which ends up creating a pretty nice distributed LAN.
I don’t have my configs in public (booo!), because I’ve not been too good on maintaining secrets out of the configs. But @cadey posts are a treasure trove of good ideas, and I also enjoyed this post and accompanying repo as sources of inspiration.
I don’t really see the value nixops provides over nixos-rebuild which can work over ssh.
That’s a fair point. Part of using nixops was about exploring how to use it later for other kinds of deployment (clouds), and it is a bit excessive for my use case (especially since I use nixops to deploy locally in the laptop =P).
A lot of my nix experience so far is seeing multiple implementations of similar concepts, but I also feel like I can refactor and try other approaches without borking my systems (too much).
On the Pi from the post I run:
I offload builds to my desktop at home using guix offload. The only problem I’ve run into so far is not a lot of things seem able to cross compile (for instance, the linux kernel for aarch-64-linux).
It’s amazing to me that two layers of virtualisaion is still a significant speedup. Shows how far we’ve come.