A shame to build this kind of thing with LVM2. ZFS could have been designed for this use case: It can create a block device, snapshot it, create clones (in constant time), modify them by applying more layers, and store only the deltas.
The licensing issues around ZFS make it a non-starter for a lot of places.
There are no licensing issues around ZFS, there are licensing issues round ZFS on Linux. That sounds like a reason to build your infrastructure on FreeBSD, rather than a reason to use LVM2.
As I understand it, fly.io uses a lot of Linux-specific technologies - Amazon’s Firecracker, Wireguard, eBPF, etc. It’s not impossible to build something that runs Docker containers based on FreeBSD, but switching fly.io to FreeBSD would not be a trivial undertaking.
If I understood the blog post correctly, they’re running the containers in a separate VM. There’s no reason why that VM couldn’t be Linux even if the VM that’s exporting the block devices is FreeBSD.
What’s the freebsd container ecosystem look like? Vanishingly small compared to what is available for Linux. Picking freebsd to get zfs in a container runtime system is the tail wagging the dog. As nice as zfs is you can still build cool stuff with LVM2/DM.
Honest question about dishonesty: if a company never advertises they use ZFS, how can licensing issues affect them? How would anyone detect they used ZFS?
The licensing issues only apply to open-source distribution. A company can use whatever they want as long as they don’t distribute the software under a license that clashes with that of ZFS.
Disgruntled staff leaking info? Server logs from package downloads? Discovery in an unrelated lawsuit?
Anyone used fly? It looks interesting but I’d appreciate any first-hand experience from crustaceans
I do. It works very well for some applications, and the team is extremely responsive when someone runs into the occasional glitch.
A cursory look at https://community.fly.io should give you a good intuition on what works well on Fly and what needs a little bit of love.