Does anyone have any interesting ZFS-based projects they’re working on?
I was thinking of doing a tightly integrated filebrowser that has a REST API that can handle ZFS send streams. And a JSON API to query dataset and object properties and such…
I never got around to it, but I had a look a few years ago and it would be really interesting to embed SQLite directly on top of the ZFS transactional object layer, bypassing the ZPL (possibly integrating with it for a shared namespace, but not for file I/O). I’d love to see a SQL database layer or other non-filesystem abstractions built on top of the lower layers of ZFS. At the start, ZFS had the ZPL and ZVOL layers: one presents a POSIX filesystem (though with NFSv4 ACLs instead of POSIX ones), the other presents a block device abstraction. 14 years later, ZFS still has only these two abstractions.
Device removal exists for some usecases, specifically mirrored vdevs.
The CDDL flaming is so predictable at this point that it hurts to argue, so I’ll hold off for the most part. Yes, Oracle is bad because they haven’t relicensed ZFS under the GPL. However, the CDDL enabled the open source components of Solaris to be extricated from Oracle and allowed innovation to continue to happen in the open when Oracle closed off Solaris.
One example of OpenZFS’ innovation: we finally have an open source encryption alternative to LUKS that can do snapshot backups to untrusted devices. It’s totally changed my backup workflow. I patiently waited for ZFS encryption to start setting up encrypted-by-default Linux machines with snapshots and transparent backups, and my patience was rewarded. OpenZFS 0.8 changed how I set up machines.
Would you prefer if people didn’t complain about CDDL whenever ZFS is brought up? Because Oracle and the CDDL is literally the main thing which takes an otherwise super impressive project and turns it into a project which has absolutely no practical applicability.
Is it even a good thing at this point that innovation is “allowed to continue” on a DoA filesystem, rather than focusing effort on relevant filesystems?
The bias in this comment is just so painful I don’t even know where to start. I’ve been using ZFS on FreeBSD happily for nearly a decade now, the idea that the filesystem is DoA is just propagandist nonsense.
Not really, since I’m not the only one using FreeBSD for a storage server running ZFS, and haven’t been for a very long time. Just because you aren’t using it doesn’t mean it’s not widely used. FreeNAS is very popular among home NAS builders, mostly because of ZFS. Get outside your bubble.
I wonder if there’s a misconception that supporting OpenZFS is supporting Oracle, which is explicitly not the case considering that OpenZFS deliberately has diverged from Oracle to implement things like non proprietary encryption.
I think there were a few reasons cited for that in the GitHub discussions. One was that the specs for Oracle’s ZFS encryption weren’t available, another was that Oracle’s key management was too complex.
Personally, I know that supporting ZFS isn’t necessarily supporting Oracle. However, continuing development on ZFS means continuing development on a project which is intentionally license poisoned by Oracle to cripple Linux, which is bad enough in itself.
intentionally license poisoned by Oracle to cripple Linux
Do you mean the fact that it originally came out of the Solaris codebase? That was Sun’s call, not Oracle’s. That “Fork Yeah!” video I linked in the top level comment has a nice overview of that bit of history.
FWIW, it also explains that the majority of the ZFS team (and teams for many other Solaris subsystems) immediately quit after Oracle acquired Sun and closed off Solaris. It seems like most of the Solaris team wanted development to be in the open, which is orthogonal to how Oracle does business.
Personally, I’m not seeing the harm in supporting the project. The license is not libre, but this is probably a historical artifact of the competition between Solaris and Linux. Linux won in a lot of regards, but the team behind ZFS doesn’t seem like they’re carrying that historical baggage. If they were, ZFS would have died with Solaris and really would be irrelevant.
There’s no (legal) problem using it on Windows or macOS either. The problem is not the CDDL, it’s the GPL. The CDDL does not impose any restrictions on what you can link it with. The GPL does.
That’s not entirely fair IMO. You can’t GPL-licensed code and integrate it into a license which is more restrictive than the GPL, which is entirely reasonable. The issue is that the CDDL is more restrictive than the GPL, so CDDL-licensed code can’t use GPL-licensed code, so ZFS can’t use Linux code.
turns it into a project which has absolutely no practical applicability.
I’m really confused, are you arguing that ZFS doesn’t work on widely used operating systems? FreeBSD and Linux are pretty widely used.
I was also shooting for a technical discussion about the filesystem instead of bikeshedding the license. There’s a lot of technically interesting things that zero-trust data storage enables, such as cloud storage providers that can’t see your data at rest. I think that’s much more interesting to discuss than this CDDL vs. GPL boilerplate. For example, I’ve got some ideas for web-based ZFS replication projects to make sharing files between different people with ZFS pools easier.
The cddl is similar to the MPL in that it is weak file based copy left. The sizeable difference is the MPLv2 has an explicit exception to allow it to be relicensed as GPL.
Device removal exists for some usecases, specifically mirrored vdevs.
I recently tried removing a mirrored vdev from my pool and it worked flawlessly. Pretty nice feature - all data was migrated to the other vdevs in the pool. I’m currently going through my pool and replacing old drives with newer drives after testing them. I am tempted to go from 3 mirrored vdevs (2 TB each) to 2 mirrored vdevs (8TB each) without losing anything but the time required for testing, or going with 3 vdevs again.
Are you a current ZFS user, or are those particular reasons that you don’t use ZFS?
After years of waiting, with the release of ZoL 0.8.0, I finally moved all-but-one of my machines from LUKS+btrfs to encrypted ZFS. Four out of five, and so far so good. I am close to, but not yet at the point of, flat-out recommending it as a default to my friends who run desktop Linux. The only features I miss so far are:
The way RAID expansion works under btrfs.
LUKS having had time to be integrated cleanly into various utilities shipped with desktop Linux distros.
I am very thankful than RAID-Z expansion is in the works, and I hope my faith in the OpenZFS team will be rewarded the way it was with encryption. But much like how so much of ZFS feels “right”, the way btrfs handles adding drives feels like the way it should have always been, with all file systems.
I like that NixOS makes it pretty clear that my ZFS module is properly built for the exact kernel version I’m running, FWIW. I’ve had lots of success deploying on the order of tens of currently reliable NixOS machines with ZFS.
Someone needs to go full RMS on this project and just reimplement the whole thing from scratch. No more CDDL, but all the benefits of ZFS. A man can dream…
That would be btrfs. The length people go to because of “wrong open source license” or “not-invented-here-syndrome” is mind bending. More power to them, but it’s non-trivial.
It is, but CDDL binaries cannot legally be shipped with GPL binaries. Some distros work around this by shipping the source code and compiling on the customer device.
Does anyone have any interesting ZFS-based projects they’re working on?
I was thinking of doing a tightly integrated filebrowser that has a REST API that can handle ZFS send streams. And a JSON API to query dataset and object properties and such…
I never got around to it, but I had a look a few years ago and it would be really interesting to embed SQLite directly on top of the ZFS transactional object layer, bypassing the ZPL (possibly integrating with it for a shared namespace, but not for file I/O). I’d love to see a SQL database layer or other non-filesystem abstractions built on top of the lower layers of ZFS. At the start, ZFS had the ZPL and ZVOL layers: one presents a POSIX filesystem (though with NFSv4 ACLs instead of POSIX ones), the other presents a block device abstraction. 14 years later, ZFS still has only these two abstractions.
Still under CDDL, still can’t shrink pools.
Device removal exists for some usecases, specifically mirrored vdevs.
The CDDL flaming is so predictable at this point that it hurts to argue, so I’ll hold off for the most part. Yes, Oracle is bad because they haven’t relicensed ZFS under the GPL. However, the CDDL enabled the open source components of Solaris to be extricated from Oracle and allowed innovation to continue to happen in the open when Oracle closed off Solaris.
One example of OpenZFS’ innovation: we finally have an open source encryption alternative to LUKS that can do snapshot backups to untrusted devices. It’s totally changed my backup workflow. I patiently waited for ZFS encryption to start setting up encrypted-by-default Linux machines with snapshots and transparent backups, and my patience was rewarded. OpenZFS 0.8 changed how I set up machines.
Would you prefer if people didn’t complain about CDDL whenever ZFS is brought up? Because Oracle and the CDDL is literally the main thing which takes an otherwise super impressive project and turns it into a project which has absolutely no practical applicability.
Is it even a good thing at this point that innovation is “allowed to continue” on a DoA filesystem, rather than focusing effort on relevant filesystems?
The bias in this comment is just so painful I don’t even know where to start. I’ve been using ZFS on FreeBSD happily for nearly a decade now, the idea that the filesystem is DoA is just propagandist nonsense.
DoA on the commonly used operating systems. There, happy?
Not really, since I’m not the only one using FreeBSD for a storage server running ZFS, and haven’t been for a very long time. Just because you aren’t using it doesn’t mean it’s not widely used. FreeNAS is very popular among home NAS builders, mostly because of ZFS. Get outside your bubble.
I wonder if there’s a misconception that supporting OpenZFS is supporting Oracle, which is explicitly not the case considering that OpenZFS deliberately has diverged from Oracle to implement things like non proprietary encryption.
I think there were a few reasons cited for that in the GitHub discussions. One was that the specs for Oracle’s ZFS encryption weren’t available, another was that Oracle’s key management was too complex.
Personally, I know that supporting ZFS isn’t necessarily supporting Oracle. However, continuing development on ZFS means continuing development on a project which is intentionally license poisoned by Oracle to cripple Linux, which is bad enough in itself.
Do you mean the fact that it originally came out of the Solaris codebase? That was Sun’s call, not Oracle’s. That “Fork Yeah!” video I linked in the top level comment has a nice overview of that bit of history.
FWIW, it also explains that the majority of the ZFS team (and teams for many other Solaris subsystems) immediately quit after Oracle acquired Sun and closed off Solaris. It seems like most of the Solaris team wanted development to be in the open, which is orthogonal to how Oracle does business.
Personally, I’m not seeing the harm in supporting the project. The license is not libre, but this is probably a historical artifact of the competition between Solaris and Linux. Linux won in a lot of regards, but the team behind ZFS doesn’t seem like they’re carrying that historical baggage. If they were, ZFS would have died with Solaris and really would be irrelevant.
There’s no (legal) problem using it on Windows or macOS either. The problem is not the CDDL, it’s the GPL. The CDDL does not impose any restrictions on what you can link it with. The GPL does.
That’s not entirely fair IMO. You can’t GPL-licensed code and integrate it into a license which is more restrictive than the GPL, which is entirely reasonable. The issue is that the CDDL is more restrictive than the GPL, so CDDL-licensed code can’t use GPL-licensed code, so ZFS can’t use Linux code.
I’m really confused, are you arguing that ZFS doesn’t work on widely used operating systems? FreeBSD and Linux are pretty widely used.
I was also shooting for a technical discussion about the filesystem instead of bikeshedding the license. There’s a lot of technically interesting things that zero-trust data storage enables, such as cloud storage providers that can’t see your data at rest. I think that’s much more interesting to discuss than this CDDL vs. GPL boilerplate. For example, I’ve got some ideas for web-based ZFS replication projects to make sharing files between different people with ZFS pools easier.
I don’t care to argue about it, but I think the camp that’s unhappy about the CDDL is pretty huge.
I’m not even that picky. I’d settle for MIT, BSD, or even MPL.
The cddl is similar to the MPL in that it is weak file based copy left. The sizeable difference is the MPLv2 has an explicit exception to allow it to be relicensed as GPL.
I meant to say MPLv2. Twas a typo.
I recently tried removing a mirrored vdev from my pool and it worked flawlessly. Pretty nice feature - all data was migrated to the other vdevs in the pool. I’m currently going through my pool and replacing old drives with newer drives after testing them. I am tempted to go from 3 mirrored vdevs (2 TB each) to 2 mirrored vdevs (8TB each) without losing anything but the time required for testing, or going with 3 vdevs again.
Are you a current ZFS user, or are those particular reasons that you don’t use ZFS?
After years of waiting, with the release of ZoL 0.8.0, I finally moved all-but-one of my machines from LUKS+btrfs to encrypted ZFS. Four out of five, and so far so good. I am close to, but not yet at the point of, flat-out recommending it as a default to my friends who run desktop Linux. The only features I miss so far are:
I am very thankful than RAID-Z expansion is in the works, and I hope my faith in the OpenZFS team will be rewarded the way it was with encryption. But much like how so much of ZFS feels “right”, the way btrfs handles adding drives feels like the way it should have always been, with all file systems.
DKMS is a bit of a pain. Which distro do you typically work with?
I like that NixOS makes it pretty clear that my ZFS module is properly built for the exact kernel version I’m running, FWIW. I’ve had lots of success deploying on the order of tens of currently reliable NixOS machines with ZFS.
Someone needs to go full RMS on this project and just reimplement the whole thing from scratch. No more CDDL, but all the benefits of ZFS. A man can dream…
That would be btrfs. The length people go to because of “wrong open source license” or “not-invented-here-syndrome” is mind bending. More power to them, but it’s non-trivial.
So is OpenZFS fully FOSS? I feel like I remember some kind of licensing weirdness.
It is, but CDDL binaries cannot legally be shipped with GPL binaries. Some distros work around this by shipping the source code and compiling on the customer device.