Very cool! This is conceptually similar to borg, in case you haven’t seen that one and are interested in other ideas.
In particular, you might look at buzhash or something similar, for chunking up large files. Also, in my archive tool, I found that blake-3 can be about 5x faster than SHA-2, so you might want to poke at that, since hashing may consume a lot of the wall clock time once you’re doing incremental backups.
Very cool! This is conceptually similar to borg, in case you haven’t seen that one and are interested in other ideas.
In particular, you might look at buzhash or something similar, for chunking up large files. Also, in my archive tool, I found that blake-3 can be about 5x faster than SHA-2, so you might want to poke at that, since hashing may consume a lot of the wall clock time once you’re doing incremental backups.