I like this article but I think that the fundamental challenge is something deeper (although related). It’s that there’s a trade-off between overall performance and ability to reason about performance. Take memory caching: it makes spatially local access patterns a lot faster, but at the cost of mov instructions varying over orders of magnitude in time cost.
Relatedly, extra work can be good. Garbage collection is extra work for the machine… but can actually improve performance if it’s cache-aware.
The call-count examples are a case of this trade-off. Usually, we can think in microseconds as interchangeable and additive. The nonlinear congestion behavior is rare. When it’s not in play, it makes sense to add a few more microseconds (under normal conditions) of work in exchange for simplicity, correctness, or maintainability. But when congestion happens, other metrics best approximated by call counts become a lot more meaningful.
We’re used to systems that are extremely fast under typical use cases, but whose performance as a function over all cases is complicated and often counterintuitive, if not opaque.
I like this article but I think that the fundamental challenge is something deeper (although related). It’s that there’s a trade-off between overall performance and ability to reason about performance. Take memory caching: it makes spatially local access patterns a lot faster, but at the cost of mov instructions varying over orders of magnitude in time cost.
Relatedly, extra work can be good. Garbage collection is extra work for the machine… but can actually improve performance if it’s cache-aware.
The call-count examples are a case of this trade-off. Usually, we can think in microseconds as interchangeable and additive. The nonlinear congestion behavior is rare. When it’s not in play, it makes sense to add a few more microseconds (under normal conditions) of work in exchange for simplicity, correctness, or maintainability. But when congestion happens, other metrics best approximated by call counts become a lot more meaningful.
We’re used to systems that are extremely fast under typical use cases, but whose performance as a function over all cases is complicated and often counterintuitive, if not opaque.