|Mersenne Twister Considered Harmful|
|Written by Mike James|
|Wednesday, 16 October 2019|
The Mersenne Twister is a very common random number generator. It is used in C, Python, Mathematica, Excel, PHP, Ruby ... A new paper suggests it shouldn't be.
More cartoon fun at xkcd a webcomic of romance,sarcasm, math, and languag
"Once language implementors decided to use the Mersenne Twister, the choice was doomed to be cemented for a very long time. Also, the problems of the Mersenne Twister are not immediately detectable in everyday applications, and most users really interested in the quality of their PRNG will make an informed choice, rather than relying on the stock PRNG of whichever programming environment they are using. So there was no strong motivation to move towards a better PRNG."
Part of the argument for not moving from the Mersenne Twister is that the tests it did fail were not important in the real world use of the generator. In particular, if you take the bits generated as uniformly distributed in the interval 0 to 1 and then apply a non-linear transformation to a different distribution then the defects go away. This is indeed the way that many simulations use pseudo random numbers but the new paper gives examples where this is not the case.
The bottom line of all of these examples is:
"F2-linear generators fail statistical tests which can have an impact on actual applications. They should be used with care, and only in context where there it is certain that strongly nonlinear operations will be applied to their output, so their defects are diluted or completely hidden. In particular, linear generators should never be used as general-purpose generators, unless their output is suitably scrambled by combining it with other, nonlinear generators, or by applying nonlinear maps. The current, dangerous ubiquity of the Mersenne Twister as basic PRNG in many environments is a historical artifact that we, as a community, should take care of. Moreover, several touted advantages of the Mersenne Twister, such as reparameterization, do not actually work properly when examined closely."
There also seems to be an easy fix to the problem - the use of a scrambler as in the author's own xoshiro256++ means that we have a generator which does not fail any known statistical test. The point is there are better ways of doing the job and we are not using them.
or email your comment to: firstname.lastname@example.org
|Last Updated ( Wednesday, 16 October 2019 )|