You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by "Luke Lu (JIRA)" <ji...@apache.org> on 2014/09/29 20:47:34 UTC

[jira] [Created] (HADOOP-11152) Better random number generator

Luke Lu created HADOOP-11152:
--------------------------------

             Summary: Better random number generator
                 Key: HADOOP-11152
                 URL: https://issues.apache.org/jira/browse/HADOOP-11152
             Project: Hadoop Common
          Issue Type: Improvement
            Reporter: Luke Lu


HDFS-7122 showed that naive ThreadLocal usage of simple LCG based j.u.Random creates unacceptable distribution of random numbers for block placement. Similarly, ThreadLocalRandom in java 7 (same static thread local with synchronized methods overridden) has the same problem. 

"Better" is defined as better quality and faster than j.u.Random (which is already much faster (20x) than SecureRandom).

People (e.g. Numerical Recipes) have shown that by combining LCG and XORShift we can have a better fast RNG. It'd be worthwhile to investigate a thread local version of these "better" RNG.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)