I hope @ltratt sees this: the desires expressed in this post are directly aligned with his research.
Interesting post, thanks. I used to work for a software company, doing performance analysis. It was Java, C++, and database components in that case, for a large, heavy, server-side web app.
I was hoping to see something of the same sort of context here; maybe it wouldn’t change the problem or recommendations, but it would educate on the type of program in which this issue was causing problems for the author. While I had to be aware of JIT behaviour in our Java app, it had been pretty clear to me that the real problems lay elsewhere, in bloat encouraged by some terrible development practices.
So, maybe I would encourage a second part to this post, where knowing that JIT causes some inconsistency, what he did to profile network activity and code execution to improve the state of things.