1. 40
  1. 7

    I’m still not sure how they pulled it off with a rename(2) that’s not atomic on the default file system. Was the test suite run on UFS?

    1. 6

      Checking the POSIX spec, it doesn’t add a requirement that rename is atomic, it references the fact that the C specification requires that it is atomic. The only reference to rename being atomic is in the RATIONALE section, which says:

      This rename() function is equivalent for regular files to that defined by the ISO C standard. Its inclusion here expands that definition to include actions on directories and specifies behavior when the new parameter names a file that already exists. That specification requires that the action of the function be atomic.

      However, neither the C99 nor C11 specifications (I can’t find a copy of C89) require rename to be atomic. The text appears unmodified. The full spec (skipping return value and function signature) from 7.21.4.2 of the C11 spec is:

      The rename function causes the file whose name is the string pointed to by old to be henceforth known by the name given by the string pointed to by new. The file named old is no longer accessible by that name. If a file named by the string pointed to by new exists prior to the call to the rename function, the behavior is implementation-defined

      No mention of atomicity, no mention of any ordering with respect to other threads or other concurrent entities interacting with the filesystem.

      As far as I can tell, there is actually no requirement that rename be atomic for POSIX compliance. POSIX mentions atomicity only in the RATIONALE section as a property inherited from ISO C and in the DESCRIPTION it explicitly says that in the case of conflicts between ISO C and POSIX then ISO C’s definition takes precedence.

      That said, I’m surprised to learn that XNU’s rename isn’t atomic, because the documentation for Cocoa’s -writeToFile:atomically: states that it uses a rename operation to perform the atomic filesystem update.

      1. 1

        writeToFile:atomically

        Perhaps this changed by now, but it was broken in 2011: https://stackoverflow.com/questions/4662115/is-an-atomic-rename-possible-in-mac-vfs-hfs

        With APFS, rename should be atomic.

        1. 1

          Hmm, this doesn’t look like it’s a violation of atomicity: the rename completes atomically if the ACL permits deletion, it fails if the ACL doesn’t permit deletion.

      2. 1

        Maybe there’s no test for that in the test suite? If compliance is simply a matter of “passes the tests”, you might be able to get away with breaking the standard in several ways and still be seen as compliant. Or maybe there’s some leeway and it doesn’t need to be 100% passing all the tests to get certified?

      3. 2

        We were promised 1/10th of the $200 million, or $20 million in stock, on completion. $10 million to me, $5 million to Ed, and $5 million to Karen Crippes,

        This is very far removed from the compensation I am used to seeing, is this normal and I’m just oblivious?

        1. 7

          A separate issue, but apparently they didn’t receive those stocks. (Very much a tangent, but maybe of interest.)

          1. 5

            I was at Apple at the time, in the macOS division, and this was very far from normal, at least in my experience.

            I think in this case it has something to do with the very large expense of not doing this, the mind numbing grunge of the actual work, and the very small number of people actually qualified to do it. Kind of a perfect storm.

            1. 1

              It looks quite high but it’s not unusual for companies to give occasional one-off bonuses that may be a multiple of their normal compensation to employees who were instrumental in shipping products that brought in large amounts of revenue. From the article, it sounds as if the author was someone who was right at the top of the engineering track. Levels.fyi doesn’t have data from people that high up in any of the big tech companies except Google (where it has a single data point for an E9 engineer making around $4.5/year). Two levels below that at Apple they have someone making over $1m/year.

              I’m not sure exactly how you can extrapolate across the industry from this. At Microsoft, the base salary scales roughly linearly with level but the stock and bonus amounts scale with some polynomial factor (on the assumption that the more influence you have over the overall success of the company, the more your total compensation should reflect this).

              Even accounting for inflation, this looks like it’s a large but not unbelievable bonus amount for one of the most senior engineers for completion of a project that saved the company a much larger amount. That said, it sounds as if the team were never actually paid this bonus, so who knows? I guess the lesson is that if you’re promised a large bonus in advance, get it in writing.

              1. 1

                Answered in the very next paragraph

                I got the $10 million, because it was going to be my job on the line, and potentially, my ability to work in industry at a high level, ever again, in the future.