1. 7

  2. 3

    I remember years ago, working with a bunch of benchmarks and as we got them faster the amount of variance between runs went up continuously - which you expect, to an extent - but we had one test that had way more variance than any of the others.

    It turned out it was calling Math.random() to control behavior. Unsurprisingly this had a significant impact - the fix was just to have it use a deterministic RNG instead (specifically starting from the same seed, etc)