I agree with most of the points in this article, but I want to point out that, at least nowadays, the folks in group 2 tend to also be concerned with exactly these issues too. Some of the malfeasances (like redistributing a(b+c) -> ab+ac) completely destroy reasonability; others (like fma contraction, intermediate results in higher precision) tend not to in practice, but while they may have been desirable in the past (kahan in particular strongly advocated for the latter, and is directly responsible for the fact that c works that way), today, numerical computing is moving in the direction of strong reproducibility.
EDIT: of course, kahan very well knew how to write good numerical code without high-precision intermediate results. But he thought it was a good idea because it would be less likely to blow up in the face of code written by people who weren’t very good with numerics (ie, people in group 3). Which makes the presentation here a bit funny.
Great text explaining WTF is IEEE floating point, why it was created and why it is so strange.
I moved that cool Stradivari quote to the title.
Also related: quote from Graydon Hoare (author of Rust):
There are two very distinct historical threads – accounting uses and scientific uses – that literally build different machines for a long time! Business machines and science machines are separate product lines. COBOL and FORTRAN. The IBM “360” project (along with PL/I) is, among other things, a merger of the two product lines into a single one. We still see echoes of this division in (say) the presence of separate binary floating point (science) and integer-and-BCD (business) instruction sets and datatypes
One of Kahan’s papers which was not linked in the comments is mindless.pdf, whose title I never remember. I only remember the conclusion: there is not yet a “mindless” automatic error analysis algorithm for IEEE 754 numerical operations, and there probably won’t be.
I agree with most of the points in this article, but I want to point out that, at least nowadays, the folks in group 2 tend to also be concerned with exactly these issues too. Some of the malfeasances (like redistributing a(b+c) -> ab+ac) completely destroy reasonability; others (like fma contraction, intermediate results in higher precision) tend not to in practice, but while they may have been desirable in the past (kahan in particular strongly advocated for the latter, and is directly responsible for the fact that c works that way), today, numerical computing is moving in the direction of strong reproducibility.
EDIT: of course, kahan very well knew how to write good numerical code without high-precision intermediate results. But he thought it was a good idea because it would be less likely to blow up in the face of code written by people who weren’t very good with numerics (ie, people in group 3). Which makes the presentation here a bit funny.
Great text explaining WTF is IEEE floating point, why it was created and why it is so strange.
I moved that cool Stradivari quote to the title.
Also related: quote from Graydon Hoare (author of Rust):
https://graydon2.dreamwidth.org/304682.html
One of Kahan’s papers which was not linked in the comments is mindless.pdf, whose title I never remember. I only remember the conclusion: there is not yet a “mindless” automatic error analysis algorithm for IEEE 754 numerical operations, and there probably won’t be.