I always wonder why people still repeat all these ZFS myths like the need for ECC or lots of RAM - its really silly.
About ECC … lets make something clear … ALL filesystems benefit from having ECC memory but if you do not have ECC memory the ZFS filesystem is the SAFEST to use on such systems.
About RAM … if you have small about of RAM the only thing you will lack is ZFS cache in RAM. Nothing more. Of course a lot RAM is needed if you want to use ZFS with deduplication but that is useful only in very special cases. Most people do not use/enable deduplication on ZFS - sometime only for several datasets. You do not have to enable deduplication for entire pool.
I have used 2 x 2 TB ZFS mirror with 512 MB RAM on FreeBSD along with serving there a lot of other services then just SAMBA or NFS. It also server Syncthing, KODI and other services and the only time when it got rebooted was when there was power outage (lack of electricity) as this system did not had UPS emergency power.
I now run 2 x 5 TB ZFS mirror with 4 GB RAM and 2 GB still remains free. I could run that 2 x 5 TB also with only 512 MB of RAM but I got that 4 GB RAM stick for free with motherboard so I use it here.
Also all ‘bad’ or ‘tragic’ stories on ZFS I know are on Linux systems. I have never heard any such stories on FreeBSD.
If you want to read more about storage on FreeBSD then check this: https://is.gd/bsdstg
I have used 2 x 2 TB ZFS mirror with 512 MB RAM on FreeBSD along with serving there a lot of other services then just SAMBA or NFS
On what kind of storage, over what kind of network? ZFS’ copy-on-write nature leads to a lot of on-disk fragmentation and the persistence model causes write amplification. This means RAM is useful for two things:
More RAM lets ZFS buffer writes so that it can handle bursty loads better. In very write-heavy workloads you’ll often see things freeze periodically as ZFS gets to the point where it must flush buffers and back pressure propagates through the system. You can alleviate this a lot with a separate log device on fast flash. This should be large enough that it can handle as much data as your pool can write in a 10-second burst write.
More RAM lets it avoid needing to do fragmented reads to recover files, because more of the working set can fit in RAM.
If you’re using a relatively slow network (e.g. 100Mb/s) then even a fairly modest pair of drives should be able to keep up with reads. ZFS is very good at read-ahead on predictable workloads so if you’re mostly streaming files from disk then the RAM requirement is related to be bandwidth delay product of the disks and so can be pretty small. If you’re doing a lot of random reads and your pool is spinning rust then that can dominate performance unless most reads are satisfied from the ARC (in which case it will still dominate your performance, just much later).
If your pool is on SSDs, then the ARC is much less important. A typical cheap SSD can do around 30MiB/s of random reads and with a ZFS mirror can read from both disks in parallel, so you can probably handle serving read-heavy workloads over gigabit ethernet without the ARC mattering too much. If you’re on spinning rust then the ARC lets it burst read large contiguous chunks of files and keep them in cache while a lot of other things are being fetched.
TL;DR: ZFS doesn’t stop working if you have less than the rough guidelines for the amount of RAM it needs, it just gets slower. Whether this slowdown matters depends a lot on whether your network or your disks are the bottleneck. Most of the advice on RAM:Disk ratios is written by people building out large multi-user file servers over very fast networks.
OLOy required memetest86 photos to process a warranty
I assume this is a typo but it would be amazing if a manufacturer really required you to run the memetest (the creation of which was inspired by one of my posts during the Twitch Installs Arch Linux event).
I haven’t used ZFS, is it good for a laptop as well as a server? Should I install ZFS on all my systems?
Generally, the rule of thumb for ZFS is that you should have 1 GiB of RAM per 1 TiB of storage, multiplied by 2-4 if you are turning on deduplication. You can get away with a bit less if the storage is SSD than if it’s spinning rust - ZFS does a lot of buffering to mitigate fragmentation. I’ve not had a laptop that didn’t meet those requirements and the advantage of ZFS on a laptop over anything other than APFS are huge (relative to APFS they’re slightly less large):
O(1) snapshots, so you’re protected against accidental deletion. There are some nice tools that manage snapshots and let you do the gradual-decay thing. You can also do the NetApp-style thing with snapshots and have them automatically mounted inside the .zfs directory of the filesystem so as a user without the ability to run ZFS commands you can still copy files from a snapshot that you accidentally deleted.
Fast filesystem creation if you want individual units of checkpointing / restoring or different persistency guarantees (for example, I turn of sync on a filesystem mounted on ~/build so that it lies to my compiler / linker about persistence of data written there - if my machine crashes I may lose data there, but there’s nothing there I can’t recreate by simply blowing away the build and running it again, so I don’t care).
Delegated administration, so you can do the things above without elevating privilege, but only for the filesystems owned by you.
Easy backups with zfs send (if your NAS also uses ZFS then replicating your local disk, including snapshots, to the NAS is really easy).
Per-block checksums so you can detect on-disk corruption (doesn’t help against in-memory corruption, sadly, as this article points out).
I haven’t used anything other than ZFS by choice for over 10 years on everything from a laptop with 20 GiB if disk and 1 GiB of RAM to servers with 512 GiB of RAM and many TiBs of disks. If you’re doing docker / containerd things, there’s a nice ZFS snapshotter that works really nicely with ZFS’s built-in snapshots.
I stumbled upon this rant by torvalds about ECC and how it fade away in the consumer space.
ddr5 will help somewhat, by requiring an on-chip ECC system. Problem is, it’s not helping with the data<->cpu bus and it looks like you will also not get reports in your OS about ECC errors, as you would do with normal ECC.
I always wonder why people still repeat all these ZFS myths like the need for ECC or lots of RAM - its really silly.
About ECC … lets make something clear … ALL filesystems benefit from having ECC memory but if you do not have ECC memory the ZFS filesystem is the SAFEST to use on such systems.
About RAM … if you have small about of RAM the only thing you will lack is ZFS cache in RAM. Nothing more. Of course a lot RAM is needed if you want to use ZFS with deduplication but that is useful only in very special cases. Most people do not use/enable deduplication on ZFS - sometime only for several datasets. You do not have to enable deduplication for entire pool.
I have used 2 x 2 TB ZFS mirror with 512 MB RAM on FreeBSD along with serving there a lot of other services then just SAMBA or NFS. It also server Syncthing, KODI and other services and the only time when it got rebooted was when there was power outage (lack of electricity) as this system did not had UPS emergency power.
I now run 2 x 5 TB ZFS mirror with 4 GB RAM and 2 GB still remains free. I could run that 2 x 5 TB also with only 512 MB of RAM but I got that 4 GB RAM stick for free with motherboard so I use it here.
Also all ‘bad’ or ‘tragic’ stories on ZFS I know are on Linux systems. I have never heard any such stories on FreeBSD.
If you want to read more about storage on FreeBSD then check this: https://is.gd/bsdstg
Regards.
On what kind of storage, over what kind of network? ZFS’ copy-on-write nature leads to a lot of on-disk fragmentation and the persistence model causes write amplification. This means RAM is useful for two things:
If you’re using a relatively slow network (e.g. 100Mb/s) then even a fairly modest pair of drives should be able to keep up with reads. ZFS is very good at read-ahead on predictable workloads so if you’re mostly streaming files from disk then the RAM requirement is related to be bandwidth delay product of the disks and so can be pretty small. If you’re doing a lot of random reads and your pool is spinning rust then that can dominate performance unless most reads are satisfied from the ARC (in which case it will still dominate your performance, just much later).
If your pool is on SSDs, then the ARC is much less important. A typical cheap SSD can do around 30MiB/s of random reads and with a ZFS mirror can read from both disks in parallel, so you can probably handle serving read-heavy workloads over gigabit ethernet without the ARC mattering too much. If you’re on spinning rust then the ARC lets it burst read large contiguous chunks of files and keep them in cache while a lot of other things are being fetched.
TL;DR: ZFS doesn’t stop working if you have less than the rough guidelines for the amount of RAM it needs, it just gets slower. Whether this slowdown matters depends a lot on whether your network or your disks are the bottleneck. Most of the advice on RAM:Disk ratios is written by people building out large multi-user file servers over very fast networks.
I assume this is a typo but it would be amazing if a manufacturer really required you to run the memetest (the creation of which was inspired by one of my posts during the Twitch Installs Arch Linux event).
Consumer systems need ECC RAM as much as servers!
I haven’t used ZFS, is it good for a laptop as well as a server? Should I install ZFS on all my systems?
Generally, the rule of thumb for ZFS is that you should have 1 GiB of RAM per 1 TiB of storage, multiplied by 2-4 if you are turning on deduplication. You can get away with a bit less if the storage is SSD than if it’s spinning rust - ZFS does a lot of buffering to mitigate fragmentation. I’ve not had a laptop that didn’t meet those requirements and the advantage of ZFS on a laptop over anything other than APFS are huge (relative to APFS they’re slightly less large):
.zfsdirectory of the filesystem so as a user without the ability to run ZFS commands you can still copy files from a snapshot that you accidentally deleted.~/buildso that it lies to my compiler / linker about persistence of data written there - if my machine crashes I may lose data there, but there’s nothing there I can’t recreate by simply blowing away the build and running it again, so I don’t care).zfs send(if your NAS also uses ZFS then replicating your local disk, including snapshots, to the NAS is really easy).I haven’t used anything other than ZFS by choice for over 10 years on everything from a laptop with 20 GiB if disk and 1 GiB of RAM to servers with 512 GiB of RAM and many TiBs of disks. If you’re doing docker / containerd things, there’s a nice ZFS snapshotter that works really nicely with ZFS’s built-in snapshots.
TL;DR: Yes.
I stumbled upon this rant by torvalds about ECC and how it fade away in the consumer space. ddr5 will help somewhat, by requiring an on-chip ECC system. Problem is, it’s not helping with the data<->cpu bus and it looks like you will also not get reports in your OS about ECC errors, as you would do with normal ECC.
i would also like to know this as Im about to install it on my laptop
tl;dr use ECC