How to get reliable performance regression seems hard.
We run a major enterprise application and we track performance as general throughput and latencies. Somehow having a bigger more complex system seem, to me, to make for easier performance regression monitoring. All the differences from measurement to measurement kind of evens out.
Obligatory plug for ASV as a benchmark runner, tracker and regression-finder; and my Nix plugin which generalises it to non-Python projects :)
It also suggests it could speed up even more with some allocation optimization / custom allocator?