I wonder whether running an end-to-end test suite before forking would warm the memory enough to show similar results with traditional forking app servers like unicorn. Shopify probably has enough data to tailor a warming test suite to cover most if not all code paths and not take forever.
But this would be quite impractical, as you’d very significantly slow down rollout of a new version, and fast deploys are really key in incident management among other things.
Maintaining such a “warmup” suite would also be quite a pain.
This is also speculative, but a test suite has lots of potential to cover code paths infrequently seen in production, plus there’s the resource use of the test code itself.
When you take this through to it’s logical conclusion you arrive at PGO. Because the best results likely come from measuring how often certain code paths are executed in production. We just need the developers of ruby to come up with a way to actually feed that data to the jit.
I wonder whether running an end-to-end test suite before forking would warm the memory enough to show similar results with traditional forking app servers like unicorn. Shopify probably has enough data to tailor a warming test suite to cover most if not all code paths and not take forever.
Eric Wong suggested something similar when I submitted the first version of reforking upstream: https://yhbt.net/unicorn-public/aecd9142-94cf-b195-34f3-bea4870ed9c8@shopify.com/T/#m30f8059af3b0d7f2dbcbbca12aff261b0cb9e1fb
But this would be quite impractical, as you’d very significantly slow down rollout of a new version, and fast deploys are really key in incident management among other things.
Maintaining such a “warmup” suite would also be quite a pain.
This is also speculative, but a test suite has lots of potential to cover code paths infrequently seen in production, plus there’s the resource use of the test code itself.
When you take this through to it’s logical conclusion you arrive at PGO. Because the best results likely come from measuring how often certain code paths are executed in production. We just need the developers of ruby to come up with a way to actually feed that data to the jit.
PGO is Profile Guided Optimization for folks like me who didn’t know.