You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@commons.apache.org by ah...@apache.org on 2019/07/18 11:54:15 UTC
[commons-rng] branch master updated: Update SeedFactory use of
SecureRandom to non-blocking on Linux systems.
This is an automated email from the ASF dual-hosted git repository.
aherbert pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/commons-rng.git
The following commit(s) were added to refs/heads/master by this push:
new 70c0929 Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
70c0929 is described below
commit 70c09297f771771ee341303530c80ca174f10307
Author: aherbert <ah...@apache.org>
AuthorDate: Thu Jul 18 12:54:11 2019 +0100
Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
See https://tersesystems.com/blog/2015/12/17/the-right-way-to-use-securerandom/
---
.../java/org/apache/commons/rng/simple/internal/SeedFactory.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
index 8802622..170c978 100644
--- a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
+++ b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
@@ -68,8 +68,9 @@ public final class SeedFactory {
// Use a secure RNG so that different instances (e.g. in multiple JVM
// instances started in rapid succession) will have different seeds.
final SecureRandom seedGen = new SecureRandom();
- final long[] seed = NumberFactory.makeLongArray(
- seedGen.generateSeed(8 * XOR_SHIFT_1024_STATE_SIZE));
+ final byte[] bytes = new byte[8 * XOR_SHIFT_1024_STATE_SIZE];
+ seedGen.nextBytes(bytes);
+ final long[] seed = NumberFactory.makeLongArray(bytes);
// The XorShift1024StarPhi generator cannot recover from an all zero seed and
// will produce low quality initial output if initialised with some zeros.
// Ensure it is non zero at all array positions using a SplitMix64
Re: [commons-rng] branch master updated: Update SeedFactory use of
SecureRandom to non-blocking on Linux systems.
Posted by Alex Herbert <al...@gmail.com>.
On 18/07/2019 14:04, Gilles Sadowski wrote:
> Le jeu. 18 juil. 2019 à 14:21, Alex Herbert <al...@gmail.com> a écrit :
>>
>> On 18/07/2019 13:11, Gilles Sadowski wrote:
>>> Hello Alex.
>>>
>>> Le jeu. 18 juil. 2019 à 13:54, <ah...@apache.org> a écrit :
>>>> This is an automated email from the ASF dual-hosted git repository.
>>>>
>>>> aherbert pushed a commit to branch master
>>>> in repository https://gitbox.apache.org/repos/asf/commons-rng.git
>>>>
>>>>
>>>> The following commit(s) were added to refs/heads/master by this push:
>>>> new 70c0929 Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
>>>> 70c0929 is described below
>>>>
>>>> commit 70c09297f771771ee341303530c80ca174f10307
>>>> Author: aherbert <ah...@apache.org>
>>>> AuthorDate: Thu Jul 18 12:54:11 2019 +0100
>>>>
>>>> Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
>>>>
>>>> See https://tersesystems.com/blog/2015/12/17/the-right-way-to-use-securerandom/
>>> The above says:
>>> ---CUT---
>>> The generateSeed method is not needed for a Native PRNG of any type,
>>> but it IS useful to seed a user space PRNG such as SHA1PRNG.
>>> ----CUT---
>>>
>>> IIRC, I assumed that when initializing the SeedFactory RNG, we were in
>>> that case.
>> I was seeing it block for 2min:30s on my machine on first usage of the
>> SeedFactory. This occurs in the samplers package when running tests as
>> the CombinationSamplerTest uses a RandomSource without a hard coded seed.
>>
>> If I did this:
>>
>> mvn test -Dtest=CombinationSamplerTest
>>
>> It blocks.
>>
>> If I did this:
>>
>> mvn test -Dtest=CombinationSamplerTest
>> -Djava.security.egd=file:/dev/./urandom
>>
>> It ran with no problem.
>>
>> IIUC the generateSeed() method will default to /dev/random on Linux. The
>> old SeedFactory code only requested 64 bits. The new method is
>> requesting 16 * 64 bits = 1024 bits. This is resulting in blocking
>> behaviour.
>>
>> If we call nextBytes then the default SecureRandom seeds itself and
>> generates secure bytes. I can only assume that the number of bits to
>> self seed is much smaller for the default algorithm so does not block.
>> If it is SHA1 then it would be 160 bits.
>>
>> I am happy to go for another solution. What we want to avoid is to have
>> the SeedFactory block for minutes to create the default generator.
> That's fine; I think I got it now: We trust the system's algo to seed
> itself so that it can produce a sequence of 1024 bits that is good
> enough to be the seed for the SeedFactory RNG.
> Correct?
Yes. The article I linked states:
"NativePRNG (nextBytes() uses /dev/urandom, generateSeed() uses
/dev/random)"
This is the default on Linux. So nextBytes should not block and will
collect system entropy for the entire byte array.
Windows does not have a blocking problem. Here it states that "SHA1PRNG
is the preferred algorithm – however, the underlying NativeSeedGenerator
called by generateSeed() calls out to the Windows Crypto API." So the
entire byte array generated from nextBytes() will not be from the system
but the seeding should be.
>
> Gilles
>
>> Alex
>>
>>
>>> Regards,
>>> Gilles
>>>
>>>
>>>> ---
>>>> .../java/org/apache/commons/rng/simple/internal/SeedFactory.java | 5 +++--
>>>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
>>>> index 8802622..170c978 100644
>>>> --- a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
>>>> +++ b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
>>>> @@ -68,8 +68,9 @@ public final class SeedFactory {
>>>> // Use a secure RNG so that different instances (e.g. in multiple JVM
>>>> // instances started in rapid succession) will have different seeds.
>>>> final SecureRandom seedGen = new SecureRandom();
>>>> - final long[] seed = NumberFactory.makeLongArray(
>>>> - seedGen.generateSeed(8 * XOR_SHIFT_1024_STATE_SIZE));
>>>> + final byte[] bytes = new byte[8 * XOR_SHIFT_1024_STATE_SIZE];
>>>> + seedGen.nextBytes(bytes);
>>>> + final long[] seed = NumberFactory.makeLongArray(bytes);
>>>> // The XorShift1024StarPhi generator cannot recover from an all zero seed and
>>>> // will produce low quality initial output if initialised with some zeros.
>>>> // Ensure it is non zero at all array positions using a SplitMix64
>>>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [commons-rng] branch master updated: Update SeedFactory use of
SecureRandom to non-blocking on Linux systems.
Posted by Gilles Sadowski <gi...@gmail.com>.
Le jeu. 18 juil. 2019 à 14:21, Alex Herbert <al...@gmail.com> a écrit :
>
>
> On 18/07/2019 13:11, Gilles Sadowski wrote:
> > Hello Alex.
> >
> > Le jeu. 18 juil. 2019 à 13:54, <ah...@apache.org> a écrit :
> >> This is an automated email from the ASF dual-hosted git repository.
> >>
> >> aherbert pushed a commit to branch master
> >> in repository https://gitbox.apache.org/repos/asf/commons-rng.git
> >>
> >>
> >> The following commit(s) were added to refs/heads/master by this push:
> >> new 70c0929 Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
> >> 70c0929 is described below
> >>
> >> commit 70c09297f771771ee341303530c80ca174f10307
> >> Author: aherbert <ah...@apache.org>
> >> AuthorDate: Thu Jul 18 12:54:11 2019 +0100
> >>
> >> Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
> >>
> >> See https://tersesystems.com/blog/2015/12/17/the-right-way-to-use-securerandom/
> > The above says:
> > ---CUT---
> > The generateSeed method is not needed for a Native PRNG of any type,
> > but it IS useful to seed a user space PRNG such as SHA1PRNG.
> > ----CUT---
> >
> > IIRC, I assumed that when initializing the SeedFactory RNG, we were in
> > that case.
>
> I was seeing it block for 2min:30s on my machine on first usage of the
> SeedFactory. This occurs in the samplers package when running tests as
> the CombinationSamplerTest uses a RandomSource without a hard coded seed.
>
> If I did this:
>
> mvn test -Dtest=CombinationSamplerTest
>
> It blocks.
>
> If I did this:
>
> mvn test -Dtest=CombinationSamplerTest
> -Djava.security.egd=file:/dev/./urandom
>
> It ran with no problem.
>
> IIUC the generateSeed() method will default to /dev/random on Linux. The
> old SeedFactory code only requested 64 bits. The new method is
> requesting 16 * 64 bits = 1024 bits. This is resulting in blocking
> behaviour.
>
> If we call nextBytes then the default SecureRandom seeds itself and
> generates secure bytes. I can only assume that the number of bits to
> self seed is much smaller for the default algorithm so does not block.
> If it is SHA1 then it would be 160 bits.
>
> I am happy to go for another solution. What we want to avoid is to have
> the SeedFactory block for minutes to create the default generator.
That's fine; I think I got it now: We trust the system's algo to seed
itself so that it can produce a sequence of 1024 bits that is good
enough to be the seed for the SeedFactory RNG.
Correct?
Gilles
>
> Alex
>
>
> >
> > Regards,
> > Gilles
> >
> >
> >> ---
> >> .../java/org/apache/commons/rng/simple/internal/SeedFactory.java | 5 +++--
> >> 1 file changed, 3 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
> >> index 8802622..170c978 100644
> >> --- a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
> >> +++ b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
> >> @@ -68,8 +68,9 @@ public final class SeedFactory {
> >> // Use a secure RNG so that different instances (e.g. in multiple JVM
> >> // instances started in rapid succession) will have different seeds.
> >> final SecureRandom seedGen = new SecureRandom();
> >> - final long[] seed = NumberFactory.makeLongArray(
> >> - seedGen.generateSeed(8 * XOR_SHIFT_1024_STATE_SIZE));
> >> + final byte[] bytes = new byte[8 * XOR_SHIFT_1024_STATE_SIZE];
> >> + seedGen.nextBytes(bytes);
> >> + final long[] seed = NumberFactory.makeLongArray(bytes);
> >> // The XorShift1024StarPhi generator cannot recover from an all zero seed and
> >> // will produce low quality initial output if initialised with some zeros.
> >> // Ensure it is non zero at all array positions using a SplitMix64
> >>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [commons-rng] branch master updated: Update SeedFactory use of
SecureRandom to non-blocking on Linux systems.
Posted by Alex Herbert <al...@gmail.com>.
On 18/07/2019 13:11, Gilles Sadowski wrote:
> Hello Alex.
>
> Le jeu. 18 juil. 2019 à 13:54, <ah...@apache.org> a écrit :
>> This is an automated email from the ASF dual-hosted git repository.
>>
>> aherbert pushed a commit to branch master
>> in repository https://gitbox.apache.org/repos/asf/commons-rng.git
>>
>>
>> The following commit(s) were added to refs/heads/master by this push:
>> new 70c0929 Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
>> 70c0929 is described below
>>
>> commit 70c09297f771771ee341303530c80ca174f10307
>> Author: aherbert <ah...@apache.org>
>> AuthorDate: Thu Jul 18 12:54:11 2019 +0100
>>
>> Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
>>
>> See https://tersesystems.com/blog/2015/12/17/the-right-way-to-use-securerandom/
> The above says:
> ---CUT---
> The generateSeed method is not needed for a Native PRNG of any type,
> but it IS useful to seed a user space PRNG such as SHA1PRNG.
> ----CUT---
>
> IIRC, I assumed that when initializing the SeedFactory RNG, we were in
> that case.
I was seeing it block for 2min:30s on my machine on first usage of the
SeedFactory. This occurs in the samplers package when running tests as
the CombinationSamplerTest uses a RandomSource without a hard coded seed.
If I did this:
mvn test -Dtest=CombinationSamplerTest
It blocks.
If I did this:
mvn test -Dtest=CombinationSamplerTest
-Djava.security.egd=file:/dev/./urandom
It ran with no problem.
IIUC the generateSeed() method will default to /dev/random on Linux. The
old SeedFactory code only requested 64 bits. The new method is
requesting 16 * 64 bits = 1024 bits. This is resulting in blocking
behaviour.
If we call nextBytes then the default SecureRandom seeds itself and
generates secure bytes. I can only assume that the number of bits to
self seed is much smaller for the default algorithm so does not block.
If it is SHA1 then it would be 160 bits.
I am happy to go for another solution. What we want to avoid is to have
the SeedFactory block for minutes to create the default generator.
Alex
>
> Regards,
> Gilles
>
>
>> ---
>> .../java/org/apache/commons/rng/simple/internal/SeedFactory.java | 5 +++--
>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
>> index 8802622..170c978 100644
>> --- a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
>> +++ b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
>> @@ -68,8 +68,9 @@ public final class SeedFactory {
>> // Use a secure RNG so that different instances (e.g. in multiple JVM
>> // instances started in rapid succession) will have different seeds.
>> final SecureRandom seedGen = new SecureRandom();
>> - final long[] seed = NumberFactory.makeLongArray(
>> - seedGen.generateSeed(8 * XOR_SHIFT_1024_STATE_SIZE));
>> + final byte[] bytes = new byte[8 * XOR_SHIFT_1024_STATE_SIZE];
>> + seedGen.nextBytes(bytes);
>> + final long[] seed = NumberFactory.makeLongArray(bytes);
>> // The XorShift1024StarPhi generator cannot recover from an all zero seed and
>> // will produce low quality initial output if initialised with some zeros.
>> // Ensure it is non zero at all array positions using a SplitMix64
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [commons-rng] branch master updated: Update SeedFactory use of
SecureRandom to non-blocking on Linux systems.
Posted by Gilles Sadowski <gi...@gmail.com>.
Hello Alex.
Le jeu. 18 juil. 2019 à 13:54, <ah...@apache.org> a écrit :
>
> This is an automated email from the ASF dual-hosted git repository.
>
> aherbert pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/commons-rng.git
>
>
> The following commit(s) were added to refs/heads/master by this push:
> new 70c0929 Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
> 70c0929 is described below
>
> commit 70c09297f771771ee341303530c80ca174f10307
> Author: aherbert <ah...@apache.org>
> AuthorDate: Thu Jul 18 12:54:11 2019 +0100
>
> Update SeedFactory use of SecureRandom to non-blocking on Linux systems.
>
> See https://tersesystems.com/blog/2015/12/17/the-right-way-to-use-securerandom/
The above says:
---CUT---
The generateSeed method is not needed for a Native PRNG of any type,
but it IS useful to seed a user space PRNG such as SHA1PRNG.
----CUT---
IIRC, I assumed that when initializing the SeedFactory RNG, we were in
that case.
Regards,
Gilles
> ---
> .../java/org/apache/commons/rng/simple/internal/SeedFactory.java | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
> index 8802622..170c978 100644
> --- a/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
> +++ b/commons-rng-simple/src/main/java/org/apache/commons/rng/simple/internal/SeedFactory.java
> @@ -68,8 +68,9 @@ public final class SeedFactory {
> // Use a secure RNG so that different instances (e.g. in multiple JVM
> // instances started in rapid succession) will have different seeds.
> final SecureRandom seedGen = new SecureRandom();
> - final long[] seed = NumberFactory.makeLongArray(
> - seedGen.generateSeed(8 * XOR_SHIFT_1024_STATE_SIZE));
> + final byte[] bytes = new byte[8 * XOR_SHIFT_1024_STATE_SIZE];
> + seedGen.nextBytes(bytes);
> + final long[] seed = NumberFactory.makeLongArray(bytes);
> // The XorShift1024StarPhi generator cannot recover from an all zero seed and
> // will produce low quality initial output if initialised with some zeros.
> // Ensure it is non zero at all array positions using a SplitMix64
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org