You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Peter Chen <pe...@aicent.com> on 2009/11/30 10:57:04 UTC
How to solve the problem "java.lang.OutOfMemoryError: unable to create new native thread" in Tomcat5.5.26
Hi,
I meet one problem of OutOfMemoryError when I am running the
Tomcat5.5.26. The OS is Solaris 10 sparc, and the JVM version is
1.5.0.12, and following is the detail of stack information.
Nov 29, 2009 12:41:16 AM
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to create
new native thread) executing
org.apache.tomcat.util.net.LeaderFollowerWorkerThread@958b36,
terminating thread
Exception in thread "SIGTERM handler" java.lang.OutOfMemoryError: unable
to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:574)
at java.lang.Shutdown.runHooks(Shutdown.java:128)
at java.lang.Shutdown.sequence(Shutdown.java:173)
at java.lang.Shutdown.exit(Shutdown.java:218)
at java.lang.Terminator$1.handle(Terminator.java:35)
at sun.misc.Signal$1.run(Signal.java:195)
at java.lang.Thread.run(Thread.java:595)
Has anyone met this problem? Please give me some advice, thanks in
advance.
Re: How to solve the problem "java.lang.OutOfMemoryError: unable
to create new native thread" in Tomcat5.5.26
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Peter,
On 12/1/2009 1:51 AM, Peter Chen wrote:
> The memory given to the JVM is $JAVA_OPTS -Xms512m -Xmx1024m
> The memory of the Solaris system is: Memory size: 4096 Megabytes
>
> The result of executing command ulimit -a is as follows:
> -bash-3.00$ ulimit -a
> core file size (blocks, -c) unlimited
> data seg size (kbytes, -d) unlimited
> file size (blocks, -f) unlimited
> open files (-n) 256
> pipe size (512 bytes, -p) 10
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 29995
> virtual memory (kbytes, -v) unlimited
Hmm... no information about the threads-per-process. Try this in a C
program:
sysconf( _SC_THREAD_THREADS_MAX )
or something like that. You'll have to look at the manual page for
sysconf on your system to determine the exact parameter to use and the
exact meaning of the return value (though it should be somewhat standard).
You may also have to look in /usr/include/unistd.h or
/usr/include/bits/confname.h (where these SC_* constants are defined on
my system) to determine what the parameter should be.
The above might not work if POSIX threads are not being used.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAksWmKcACgkQ9CaO5/Lv0PAkQgCfeBmzlXP6LtV92qaMprK/vvPm
AZ8AnizN90vFlSUSdszqu1KzAMmDRiXL
=5a4X
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: How to solve the problem "java.lang.OutOfMemoryError: unable to create new native thread" in Tomcat5.5.26
Posted by Peter Chen <pe...@aicent.com>.
The memory given to the JVM is $JAVA_OPTS -Xms512m -Xmx1024m
The memory of the Solaris system is: Memory size: 4096 Megabytes
The result of executing command ulimit -a is as follows:
-bash-3.00$ ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
open files (-n) 256
pipe size (512 bytes, -p) 10
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 29995
virtual memory (kbytes, -v) unlimited
-bash-3.00$
-----Original Message-----
From: André Warnier [mailto:aw@ice-sa.com]
Sent: 2009年11月30日 18:38
To: Tomcat Users List
Subject: Re: How to solve the problem "java.lang.OutOfMemoryError: unable to create new native thread" in Tomcat5.5.26
Peter Chen wrote:
> Hi,
>
>
>
> I meet one problem of OutOfMemoryError when I am running the
> Tomcat5.5.26. The OS is Solaris 10 sparc, and the JVM version is
> 1.5.0.12, and following is the detail of stack information.
>
...
>
> SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to create
> new native thread) executing
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread@958b36,
>
...
> Has anyone met this problem? Please give me some advice, thanks in
> advance.
>
Well, it seem that you are running out of memory.
The helpful thing to do then, would be to provide some information about
how much memory is being given to your JVM to run in.
Such as : how much memory does your system have, and what are the
memory-related parameters used to start the JVM in which your Tomcat runs.
And maybe the output of a "ps" or "top" like command, showing how much
memory is really being used by Java.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: How to solve the problem "java.lang.OutOfMemoryError: unable to create new native thread" in Tomcat5.5.26
Posted by "Looijmans, Mike" <mi...@oce.com>.
> > SEVERE: Caught exception (java.lang.OutOfMemoryError:
> unable to create
> > new native thread) executing
> > org.apache.tomcat.util.net.LeaderFollowerWorkerThread@958b36,
> >
> ...
> > Has anyone met this problem? Please give me some advice, thanks in
> > advance.
> >
> Well, it seem that you are running out of memory.
That, or the underlying VM implementation just throws that exception
because it's close enough to the real problem.
So the OS reports "sorry, there's plenty memory but I cannot start your
thread because that would allocate some other resource that is running
out", and the VM translates this to "OutOfMemoryError" because a
"SorryICannotStartYourThreadError" isn't defined. Much like running out
of GDI resources in an AWT application would throw OutOfMemoryError,
simply because the Java VM cannot translate the information from the OS
because in the Java world, these things don't exist.
M.
<disclaimer: My message ends here>
This message and attachment(s) are intended solely for use by the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.
If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please notify the sender immediately by telephone and with a 'reply' message.
Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: How to solve the problem "java.lang.OutOfMemoryError: unable
to create new native thread" in Tomcat5.5.26
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
André,
On 11/30/2009 5:37 AM, André Warnier wrote:
> Peter Chen wrote:
>> Hi,
>>
>>
>>
>> I meet one problem of OutOfMemoryError when I am running the
>> Tomcat5.5.26. The OS is Solaris 10 sparc, and the JVM version is
>> 1.5.0.12, and following is the detail of stack information.
> ...
>
>>
>> SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to create
>> new native thread) executing
>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread@958b36,
>>
> ...
>> Has anyone met this problem? Please give me some advice, thanks in
>> advance.
>>
> Well, it seem that you are running out of memory.
Looks more like he's hit his per-process thread limit. The clue is in
the stack trace:
Exception in thread "SIGTERM handler" java.lang.OutOfMemoryError: unable
to create new native thread
at java.lang.Thread.start0(Native Method)
What's amusing is that it's happening in the signal handler, so who
knows why the JVM shutdown was happening in the first place.
It might be helpful to know what the OP's <Connector> configuration is,
and what resource limits are being applied to this process (see ulimit
on UNIX, or whatever the MS-Windows equivalent is).
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAksULU4ACgkQ9CaO5/Lv0PDjfwCgrWMxl7BpVALvxGEzVaTYZqlR
A8cAnArOXwNgEqq4PxgXrxfUJ6215tDx
=kYaX
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: How to solve the problem "java.lang.OutOfMemoryError: unable
to create new native thread" in Tomcat5.5.26
Posted by André Warnier <aw...@ice-sa.com>.
Peter Chen wrote:
> Hi,
>
>
>
> I meet one problem of OutOfMemoryError when I am running the
> Tomcat5.5.26. The OS is Solaris 10 sparc, and the JVM version is
> 1.5.0.12, and following is the detail of stack information.
>
...
>
> SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to create
> new native thread) executing
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread@958b36,
>
...
> Has anyone met this problem? Please give me some advice, thanks in
> advance.
>
Well, it seem that you are running out of memory.
The helpful thing to do then, would be to provide some information about
how much memory is being given to your JVM to run in.
Such as : how much memory does your system have, and what are the
memory-related parameters used to start the JVM in which your Tomcat runs.
And maybe the output of a "ps" or "top" like command, showing how much
memory is really being used by Java.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: How to solve the problem "java.lang.OutOfMemoryError: unable to create new native thread" in Tomcat5.5.26
Posted by "Rainer Frey (Inxmail GmbH)" <ra...@inxmail.de>.
On Monday 30 November 2009 10:57:04 Peter Chen wrote:
> Hi,
>
> I meet one problem of OutOfMemoryError when I am running the
> Tomcat5.5.26. The OS is Solaris 10 sparc, and the JVM version is
> 1.5.0.12, and following is the detail of stack information.
>
> Nov 29, 2009 12:41:16 AM
>
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
>
> SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to create
> new native thread) executing
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread@958b36,
While Andre is right that system memory size and JVM parameters are important,
this is not the usual OOM that heap space is full. Java is not able to create
a OS level thread for a new Java thread.
The most common cause seems to be that there is not enough memory to allocate
for the thread stack, either because there is not enough system memory
available, or the memory limit for the java process is reached.
You might be able to increase the memory limit for the process, if OS permits
that and you have enough physical memory. If the limit is indeed physical
memory, cou can of course increase that or try to move other processes away
from the machine.
Otherwise you need to provide java with enough memory for the new thread.
One approach is to reduce the stack size. There is a -Xss switch to java, but
I don't know Solaris, so I'm not sure this works or is sufficient. Possibly
the OS has also to be configured to use/allow a smaller stack size.
Another approach is to reduce the other memory usage of the java process, by
reducing heap (or perhaps permgen) size, so java can use more of its
permitted memory for thread stacks instead of heap. See
http://www.egilh.com/blog/archive/2006/06/09/2811.aspx and the links in the
article.
A completely different cause might be that Solaris imposes a limit of the
number of threads, regardless of the memory/stack size, per process. Perhaps
a Solaris expert has more info.
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: How to solve the problem "java.lang.OutOfMemoryError: unable to
create new native thread" in Tomcat5.5.26
Posted by Peter Crowther <pe...@melandra.com>.
2009/11/30 Peter Chen <pe...@aicent.com>:
> I meet one problem of OutOfMemoryError when I am running the
> Tomcat5.5.26. The OS is Solaris 10 sparc, and the JVM version is
> 1.5.0.12, and following is the detail of stack information.
>
> SEVERE: Caught exception (java.lang.OutOfMemoryError: unable to create
> new native thread) executing
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread@958b36,
>
> terminating thread
>
> Exception in thread "SIGTERM handler" java.lang.OutOfMemoryError: unable
> to create new native thread
>
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:574)
> at java.lang.Shutdown.runHooks(Shutdown.java:128)
> at java.lang.Shutdown.sequence(Shutdown.java:173)
> at java.lang.Shutdown.exit(Shutdown.java:218)
> at java.lang.Terminator$1.handle(Terminator.java:35)
> at sun.misc.Signal$1.run(Signal.java:195)
> at java.lang.Thread.run(Thread.java:595)
OK, so this seems to happen at a very particular point: when you're
shutting down your server via SIGTERM (so killing the process).
Something has registered a shutdown hook, and the call to create the
thread to run the hooks is failing.
Does this always happen, or is it intermittent? If it always happens,
presumably we'll very quickly know if we've found a fix, as you can
start the server, stop it, and see that the problem doesn't happen.
If it always happens, does increasing the amount of heap memory
assigned to the JVM help? It might do if the JVM's very low on heap
memory. Does *de*creasing the amount of heap memory help? It might
do if the JVM's very low on non-heap memory, as creating a new thread
requires space for things like the thread's stack, which are not part
of the heap.
What options are you starting the JVM with, and how much RAM do you
have on the box? I'm particularly interested in your heap and stack
sizes.
What connector options do you have set in your conf/server.xml? I'm
particularly interested in the number of threads you have configured,
and hence how much heap and non-heap memory is being used for threads.
Sorry to request so much information, but OOMEs aren't always the
easiest to debug!
- Peter
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org