Sane atomic file I/O is totally broken: https://danluu.com/deconstruct-files/
As Linux is idolized by many, it is great to see so many articles that hint at how bad Linux really is.
The illusion needs to be broken, for progress to happen.
From the linked article: “OpenBSD and NetBSD behaved the same as Linux (true error status dropped, retrying causes success response, data lost)”
So while I agree Linux isn’t ideal, and is still broken in many respects, to claim it’s only Linux is also incorrect.
to claim it’s only Linux
to claim it’s only Linux
But I never claimed it is only Linux?
Linux just happens to be way more popular than the other systems mentioned.
I don’t think it’s idolized, at least not as infallible. If anything, it is idolized because it is open source and people can do these analyses. There are zealots preaching the Linux superiority for one reason or other but they’re mostly focused on the Microsoft hate, in my experience. I think i was personally never such zealot and I use Linux simply because it works best for me out all of the options.
But your experiencemay vary and maybe I’m wrong. Just saying that the fact that you point out flaws in a thing does not mean the thing is bad.
Turning on direct io is sometimes useful for performance too. When writing image files onto disks, I usually want oflag=direct in order to avoid replacing all the useful stuff in my filesystem cache with the contents of the image file. This IME doesn’t make those kinds of transfers slow down, provided a large enough bs is used (more than 128k is overkill on every device I’ve seen).
I can also recommend reading the recent paper Can Applications Recover from fsync Failures? where the authors explore failing fsync due to block I/O errors and how applications fail to handle these errors correctly.
Thanks for sharing, interesting talk!
In a recent question I mailed into the 2.5 Admins podcast, Jim Salter of Ars Technica had the IMO excellent suggestion of using ‘pv’ for most simple tasks where you’d normally reach for dd, since it just does the right thing and doesn’t require specifying a block size at all.
I’m not sure enough about the intricacies of your task, but this could come in handy if it makes sense for you.
I know I’ve stopped worrying about block size when zeroing out flash and SD drives.
I’ve been using (GNU) dd like this lately: dd if=/dev/zero of=/swap bs=1M count=10k oflag=sync status=progress
dd if=/dev/zero of=/swap bs=1M count=10k oflag=sync status=progress