Maybe today is the day I finally try my code on PyPy. Maybe.
Likely not though. The fear of having to figure out what would go wrong while customers are down just sounds like something I’d rather not risk. I think I’ll have to try PyPy on a new project where I can start building with it hand in hand.
If you’re building a service, I would guess you have multiple machines offering your service, in which case you could slowly roll it out across the fleet. It’s time consuming and requires paying attention, but with nginx or haproxy infront you can take a machine that is misbehaving out of rotation easily.
Wikipedia did that with PHP/HHVM. There’s a writeup here; see the section “How we made the switch” about halfway down. In their case it was a fairly extended rollout, though, in part because the worry wasn’t just about stability, but compatibility with a codebase that had only been tested on the original PHP implementation. So they first asked some volunteers to opt-in to using the HHVM servers for their edits, reporting bugs and inconsistencies as they found them. Then a few months later they started slowly rolling it out machine-by-machine to the general public, and monitoring for any remaining anomalies.
From my perspective, this is is conceptually no different than any code change. You have some canaries you smoke test against, then role it out across your fleet verifying that it work at each step. It’s a bit scarier because you don’t know all the changes, but the actual steps are all the same.
I can get behind that thinking. I don’t have a use case for it given the monolithic codebase we’re running at work, but the new stuff I’m porting that too could eventually run in a PyPy roll-out.