You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Patricio Echagüe <pa...@gmail.com> on 2010/10/07 01:59:57 UTC

Threads blocked in org.apache.jackrabbit.core.id.SeededSecureRandom

Hi everyone, has anyone seen this behavior too?

I ran a couple of jstacks and after running a diff between different ones
taken at 5 secs interval, we've seen that this stack:

We have more than 50 threads waiting here.

"SeededSecureRandom" prio=10 tid=0x0000000008142800 nid=0x566b waiting for
monitor entry [0x0000000049bbb000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at
sun.security.provider.NativePRNG$RandomIO.implGenerateSeed(NativePRNG.java:201)
    - waiting to lock <0x00002aaacddf0940> (a java.lang.Object)
    at
sun.security.provider.NativePRNG$RandomIO.access$300(NativePRNG.java:108)
    at
sun.security.provider.NativePRNG.engineGenerateSeed(NativePRNG.java:102)
    at java.security.SecureRandom.generateSeed(SecureRandom.java:495)
    at
org.apache.jackrabbit.core.id.SeededSecureRandom.run(SeededSecureRandom.java:94)
    at java.lang.Thread.run(Thread.java:619)

any clue?

Thanks


-- 
Patricio.-

Re: Threads blocked in org.apache.jackrabbit.core.id.SeededSecureRandom

Posted by Patricio Echagüe <pa...@gmail.com>.
Thanks Thomas.

On Wed, Oct 20, 2010 at 4:27 AM, Thomas Müller <th...@day.com>wrote:

> Hi,
>
> This problem is fixed now in the trunk, see
> https://issues.apache.org/jira/browse/JCR-2795
>
> Regards,
> Thomas
>



-- 
Patricio.-

Re: Threads blocked in org.apache.jackrabbit.core.id.SeededSecureRandom

Posted by Thomas Müller <th...@day.com>.
Hi,

This problem is fixed now in the trunk, see
https://issues.apache.org/jira/browse/JCR-2795

Regards,
Thomas

Re: Threads blocked in org.apache.jackrabbit.core.id.SeededSecureRandom

Posted by Patricio Echagüe <pa...@gmail.com>.
Thomas, you are right. I saw the code, but still I cannot explain why these
57 threads are doing the same thing and been waiting for at least 5 seconds.

Our deployment is a Linux Red Hat Enterprise  2.6.18-194.el5 running in a 4
6-core machines (24 cores) with 34 GB RAM.

Jackrabbit 2.0

any suggestions?

2010/10/7 Patricio Echagüe <pa...@gmail.com>

> Thanks for the response.
>
> They are all different thread ids.
>
> What would it be the best way to avoid so many threads getting blocked
> there?
>
>
> On Wed, Oct 6, 2010 at 11:44 PM, Thomas Müller <th...@day.com>wrote:
>
>> Hi,
>>
>> > We have more than 50 threads waiting here.
>>
>> Is it the same thread number for each of the 50 stack traces, or is it
>> multiple distinct threads?
>>
>> Looking at the code, if getInstance() is called concurrently from
>> multiple threads, each time a new SeededSecureRandom is created, which
>> is not needed. It would be a bit better to ensure only one
>> SeededSecureRandom() instance is created. When I anyway change that, I
>> will also add a bit more entropy to the alternative seed algorithm,
>> similar to
>> http://code.google.com/p/h2database/source/browse/trunk/h2/src/main/org/h2/util/MathUtils.java#109
>>
>> Regards,
>> Thomas
>>
>
>
>
> --
> Patricio.-
>



-- 
Patricio.-

Re: Threads blocked in org.apache.jackrabbit.core.id.SeededSecureRandom

Posted by Patricio Echagüe <pa...@gmail.com>.
Thanks for the response.

They are all different thread ids.

What would it be the best way to avoid so many threads getting blocked
there?

On Wed, Oct 6, 2010 at 11:44 PM, Thomas Müller <th...@day.com>wrote:

> Hi,
>
> > We have more than 50 threads waiting here.
>
> Is it the same thread number for each of the 50 stack traces, or is it
> multiple distinct threads?
>
> Looking at the code, if getInstance() is called concurrently from
> multiple threads, each time a new SeededSecureRandom is created, which
> is not needed. It would be a bit better to ensure only one
> SeededSecureRandom() instance is created. When I anyway change that, I
> will also add a bit more entropy to the alternative seed algorithm,
> similar to
> http://code.google.com/p/h2database/source/browse/trunk/h2/src/main/org/h2/util/MathUtils.java#109
>
> Regards,
> Thomas
>



-- 
Patricio.-

Re: Threads blocked in org.apache.jackrabbit.core.id.SeededSecureRandom

Posted by Thomas Müller <th...@day.com>.
Hi,

> We have more than 50 threads waiting here.

Is it the same thread number for each of the 50 stack traces, or is it
multiple distinct threads?

Looking at the code, if getInstance() is called concurrently from
multiple threads, each time a new SeededSecureRandom is created, which
is not needed. It would be a bit better to ensure only one
SeededSecureRandom() instance is created. When I anyway change that, I
will also add a bit more entropy to the alternative seed algorithm,
similar to http://code.google.com/p/h2database/source/browse/trunk/h2/src/main/org/h2/util/MathUtils.java#109

Regards,
Thomas

Re: Threads blocked in org.apache.jackrabbit.core.id.SeededSecureRandom

Posted by Thomas Müller <th...@day.com>.
Hi,

> We have more than 50 threads waiting here.

Please note this is a background thread that doesn't block the application.

The thread to seeds the secure random singleton is slow, because
reading from /dev/random is very slow in your system. What kind of
system is it? I'm just curious. Jackrabbit uses a workaround for such
systems: if reading from /dev/random takes longer than 1 second, it
uses an alternative approach to seed the pseudo-random number
generator (see the source code).

> NativePRNG generation reads at intervals

Actually, it's only doing that once per SeededSecureRandom instance.
The maximum waiting time to generate a SeededSecureRandom instance is
a bit more than 1 second, and Jackrabbit only uses one instance.

Regards,
Thomas

Re: Threads blocked in org.apache.jackrabbit.core.id.SeededSecureRandom

Posted by Ian Boston <ie...@tfd.co.uk>.
NativePRNG generation reads at intervals from /dev/random and/or /dev/urandom depending on the OS.
That is a blocking read.
Some variants of Linux have been reported to be quite slow reading from there, there was discussion on dev@sling.a.o on this subject about 3 months back, have a search in  the markmail.org archive

You can configure the OS/JVm to limit the frequency of reads to those files or to re-configure the PRNG.

HTH
Ian
On 7 Oct 2010, at 00:59, Patricio Echagüe wrote:

> Hi everyone, has anyone seen this behavior too?
> 
> I ran a couple of jstacks and after running a diff between different ones
> taken at 5 secs interval, we've seen that this stack:
> 
> We have more than 50 threads waiting here.
> 
> "SeededSecureRandom" prio=10 tid=0x0000000008142800 nid=0x566b waiting for
> monitor entry [0x0000000049bbb000]
>   java.lang.Thread.State: BLOCKED (on object monitor)
>    at
> sun.security.provider.NativePRNG$RandomIO.implGenerateSeed(NativePRNG.java:201)
>    - waiting to lock <0x00002aaacddf0940> (a java.lang.Object)
>    at
> sun.security.provider.NativePRNG$RandomIO.access$300(NativePRNG.java:108)
>    at
> sun.security.provider.NativePRNG.engineGenerateSeed(NativePRNG.java:102)
>    at java.security.SecureRandom.generateSeed(SecureRandom.java:495)
>    at
> org.apache.jackrabbit.core.id.SeededSecureRandom.run(SeededSecureRandom.java:94)
>    at java.lang.Thread.run(Thread.java:619)
> 
> any clue?
> 
> Thanks
> 
> 
> -- 
> Patricio.-