I think a similar thing can be said about distributed systems in that the extra dimension of complexity is timing and communication between the individual parts. This is a classic case of a heisenbug where having a single thread attached and blocked on a debugger will allow other threads to proceed and the race condition or data race is removed. We’ve alleviated the problem with things like formal proofs of protocols and thread sanatizer but they’re still incredibly hard bugs to understand and debug.
The link gives a 404 for me. Archive.org has it cached:
https://web.archive.org/web/20161111141506/http://ai.stanford.edu/~zayd/why-is-machine-learning-hard.html
I think lobsters automatically archives the link on submission. There’s a convenient “cached” link right next to the flag and hide button.
I think a similar thing can be said about distributed systems in that the extra dimension of complexity is timing and communication between the individual parts. This is a classic case of a heisenbug where having a single thread attached and blocked on a debugger will allow other threads to proceed and the race condition or data race is removed. We’ve alleviated the problem with things like formal proofs of protocols and thread sanatizer but they’re still incredibly hard bugs to understand and debug.