Counter-based RNGs make it much easier to calculate skip-ahead states – adding 0,1,2,3,4 is more obvious and faster than two multiplications and an add – so they are a lot easier to vectorize than PCG!
I haven’t looked at counter-based RNGs in detail; I decided a few years ago that PCG is good enough (TM) so I could just use my standard implementation whenever I need some random numbers. But I can’t stop tinkering with it, and occasionally it attacks me with a cunning idea that I need to blog about.
Fast/parallel PRNG do have some use in LLM quantization (and digital signal processing in general): https://arxiv.org/abs/2406.11235. See Algorithm 2 and Algorithm 3. Would be even cooler if you can generate random Gaussian samples instead of uniformly random bits.
I wonder, how does this compare to counter-based pRNGs (e.g. Philox)?
Counter-based RNGs make it much easier to calculate skip-ahead states – adding 0,1,2,3,4 is more obvious and faster than two multiplications and an add – so they are a lot easier to vectorize than PCG!
I haven’t looked at counter-based RNGs in detail; I decided a few years ago that PCG is good enough (TM) so I could just use my standard implementation whenever I need some random numbers. But I can’t stop tinkering with it, and occasionally it attacks me with a cunning idea that I need to blog about.
Just curious, what’s the use case for generating random numbers at such speed? Simulations?
Yeah, or randomized testing. But tbh I did it because it’s a fun hack.
Fast/parallel PRNG do have some use in LLM quantization (and digital signal processing in general): https://arxiv.org/abs/2406.11235. See Algorithm 2 and Algorithm 3. Would be even cooler if you can generate random Gaussian samples instead of uniformly random bits.
Disclaimer: I’m one of the authors.
Gaussian random number generators seem complicated :-)