1. 7

  2. 4

    Until the firmware of the flash device in question no longer wants to be recognized as a device. Or the filesystem is corrupted beyond repair.

    Multiple backups in multiple places. Script it.

    1. 3

      Yeah. This only covers a narrow set of problems (loss of single block inside a large file) and it’s quite a bit of work to use. cp is much easier and provides even better protection. And against more errors (even rm).

      Reed-Solomon is interesting if you’re trying to transmit data over a particularly lossy link and need to make something of the garbage you receive. I’d still use more advanced checksums like sha512 for integrity, but sha512 doesn’t provide an easy way to reconstruct the correct data after failure.

      1. 3

        Usenet pirates have been using Parchive for years now - as the comments note, it has the advantage of limiting how much recovery info you have to slog around by splitting up the parity files into increasingly larger sizes; if you’re only missing a few blocks you can download a tiny par file rather than the entire thing.

        Usenet & PAR2 actually make a really decent backup strategy for stuff you don’t want to disappear. Some providers are starting to offer retention rates upward of seven years - and when you can upload for around $0.20 - $0.14 per gigabyte…

    2. 1

      ZFS (with raidz2) is the best solution I’ve found. It does the equivalent of RAID as well as checksumming to detect silent corruption; you can survive a simultaneous controller failure, drive failure, and random corruption across every drive (and I have). It’s not a replacement for off-site backup (nothing else can stop a fire destroying all your disks at once), and I guess not quite for offline backup (though the well-designed interface and good documentation make sysadmin screwups much less likely than alternatives like linux md, IME, and automatic regular snapshotting covers most rm -rf-like cases), but it covers most cases including those you’d want to use Reed-Solomon for.