You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Bill Davidson <bi...@gmail.com> on 2009/02/06 02:37:12 UTC
Fun with the JVM crashing.
I've submitted this to Sun a few times. No response. I was hoping
someone here might have an idea of what to look for.
Tomcat 6.0.18
RedHat 5.2Server
Sun JVM
#
# An unexpected error has been detected by Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00002b68e6a1db57, pid=10229, tid=1103006016
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (11.0-b16 mixed mode linux-amd64)
# Problematic frame:
# V [libjvm.so+0x564b57]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- T H R E A D ---------------
Current thread (0x000000004bfe5c00): JavaThread "CompilerThread0" daemon [_thread_in_native, id=10244, stack(0x0000000041ae8000,0x0000000041be9000)]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000000000000000
.....
Current CompileTask:
C2:2525 com.myCompany.servlets.sales.blah.some.method(Lcom/myCompany/servlets/someClass;Lcom/myCompany/servlets/otherClass;Lcom/oreilly/servlet/ParameterParser;)V (425 bytes)
......
-------------
I'm wondering if that CompileTask is what's causing the problem.
I'm thinking that the JVM shouldn't be getting SIGSEGV's.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Bill Davidson <bi...@gmail.com>.
Bill Davidson wrote:
> Caldarale, Charles R wrote:
>> Major changes went in between 6u7 and 6u10.
>
> Maybe I should try reverting to 6u7?
Dammit. Looking back through my old hs_err*.log files,
it also happened with 6u7.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Fun with the JVM crashing.
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Bill Davidson [mailto:billdsd@gmail.com]
> Subject: Re: Fun with the JVM crashing.
>
> Maybe I should try reverting to 6u7?
That would be my next step.
> I was thinking it might make the CompileTask issue go away.
Well it does, because the JVM won't use the compilers in interpreter mode. Consequently every byte code of every method is interpreted rather than being translated to native instructions. Performance is severely impacted.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Fun with the JVM crashing.
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Caldarale, Charles R
> Subject: RE: Fun with the JVM crashing.
>
> > Maybe I should try reverting to 6u7?
>
> That would be my next step.
You could install a 32-bit JVM; that would give you the option of running either -server or -client. If the 32-bit -server mode crashes, you can try -client instead, which historically has had fewer problems.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Bill Davidson <bi...@gmail.com>.
Caldarale, Charles R wrote:
> Major changes went in between 6u7 and 6u10.
Maybe I should try reverting to 6u7?
>> I'm wondering if -Xint might help.
>
> Only if you want performance to go into the toilet - that's
> interpreter mode, also know as really, really, abominably slow.
I was thinking it might make the CompileTask issue go away.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Fun with the JVM crashing.
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Bill Davidson [mailto:billdsd@gmail.com]
> Subject: Re: Fun with the JVM crashing.
>
> However, it has happened on three different boxes,
> always with the same CompileTask mentioned.
That pretty much eliminates hardware.
> BTW, it also happened with 6u10.
Major changes went in between 6u7 and 6u10.
> I'm wondering if -Xint might help.
Only if you want performance to go into the toilet - that's interpreter mode, also know as really, really, abominably slow.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Bill Davidson <bi...@gmail.com>.
Caldarale, Charles R wrote:
> Only if that shows up repeatedly.
It does.
>> I'm thinking that the JVM shouldn't be getting SIGSEGV's.
>
> You're right about that. However, it could also be an OS or
> hardware problem. You might want to run some serious memory
> tests on the box, just to eliminate the latter. Does this
> particular system use ECC memory, or just plain old
> parity-checking DIMMs?
Not sure. However, it has happened on three different boxes,
always with the same CompileTask mentioned.
> You can also try either moving up to the recently released
> JDK 6u12, or dropping back to 6u7. Lots of changes went in
> at 6u10, and something might still be broken in there. The
> C2 compiler is an extremely complex beast.
I'll try that. BTW, it also happened with 6u10. I'm wondering
if -Xint might help.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by grappler <al...@gmail.com>.
Christopher Schultz-2 wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Chuck,
>
> Caldarale, Charles R wrote:
>>> From: Bill Davidson [mailto:billdsd@gmail.com]
>>> Subject: Fun with the JVM crashing.
>>
>>> I'm thinking that the JVM shouldn't be getting SIGSEGV's.
>>
>> You're right about that. However, it could also be an OS or hardware
>> problem. You might want to run some serious memory tests on the box,
>> just to eliminate the latter.
>
> +100!
>
> What is the architecture? Do you have physical access? If it's x86 and
> you do have physical access, 100% you should run memtest86+ against it.
> Let it run all night.
>
> If it's a production server, replace it with one you trust and then
> figure out what the problem is with the old box.
>
> We had 6 app servers at [bloated CA name removed to protect the guilty]
> in production and 2 of them were giving us SIG11's. The solution was to
> throw them out and replace them with 2 new ones. One of those sucked,
> too, so we had to play server-roulette again before we got all 6 good
> ones.
>
> Good luck,
> - -chris
>
>
>
We had a similar problem. After running fine for a couple of years, Tomcat
on one of our servers started to die every 2 or so hours with the dreaded
SIGSEGV (0xb) errors. Since the identical versions of java as well as tomcat
were running on our other machines and they had no problems, it would seem
to point to a hardware problem on the crashing server.
We had our hosting company swap the memory out of that machine and it looks
good so far.
--
View this message in context: http://www.nabble.com/Fun-with-the-JVM-crashing.-tp21865004p23175427.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Konstantin Kolinko <kn...@gmail.com>.
> Caldarale, Charles R wrote:
>>> From: Bill Davidson [mailto:billdsd@gmail.com]
>>> Subject: Fun with the JVM crashing.
>>
>>> I'm thinking that the JVM shouldn't be getting SIGSEGV's.
>>
>> You're right about that. However, it could also be an OS or hardware
>> problem. You might want to run some serious memory tests on the box,
>> just to eliminate the latter.
>
Here is a good article/FAQ on Sig 11:
http://www.bitwizard.nl/sig11/
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Chuck,
Caldarale, Charles R wrote:
>> From: Bill Davidson [mailto:billdsd@gmail.com]
>> Subject: Fun with the JVM crashing.
>
>> I'm thinking that the JVM shouldn't be getting SIGSEGV's.
>
> You're right about that. However, it could also be an OS or hardware
> problem. You might want to run some serious memory tests on the box,
> just to eliminate the latter.
+100!
What is the architecture? Do you have physical access? If it's x86 and
you do have physical access, 100% you should run memtest86+ against it.
Let it run all night.
If it's a production server, replace it with one you trust and then
figure out what the problem is with the old box.
We had 6 app servers at [bloated CA name removed to protect the guilty]
in production and 2 of them were giving us SIG11's. The solution was to
throw them out and replace them with 2 new ones. One of those sucked,
too, so we had to play server-roulette again before we got all 6 good ones.
Good luck,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkmMg2sACgkQ9CaO5/Lv0PACwACgmYPdCqQLUoBxnJGtT/CLrsSQ
M5AAnRjFYbOp6xqfFj1/mc4/Qn/G/7to
=tvDm
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Fun with the JVM crashing.
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Bill Davidson [mailto:billdsd@gmail.com]
> Subject: Fun with the JVM crashing.
> I'm wondering if that CompileTask is what's causing the problem.
Only if that shows up repeatedly.
> I'm thinking that the JVM shouldn't be getting SIGSEGV's.
You're right about that. However, it could also be an OS or hardware problem. You might want to run some serious memory tests on the box, just to eliminate the latter. Does this particular system use ECC memory, or just plain old parity-checking DIMMs?
You can also try either moving up to the recently released JDK 6u12, or dropping back to 6u7. Lots of changes went in at 6u10, and something might still be broken in there. The C2 compiler is an extremely complex beast.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Alan Chaney <al...@compulsivecreative.com>.
Hi Bill
My development workstation is a 64 bit Ubuntu 8.04. I've had numerous
problems with SIGSEGV crashes when I run my applications under Eclipse
3.3, but the same machine also runs the same applications from the same
version(s) of tomcat (6.0.14,6.0.16 and 6.0.18) I found that what seemed
to affect the thing the most was 'loading up' the JVM. My app is
actually 5 different wars - I get the most problems when all of them are
loaded.
I found it so frustrating that I've actually procured a 2nd 64 bit
machine to try to see what common factors there are. For example, I was
going to see if Eclipse 3.4 was better than 3.3.
I tried b4 thru b16 and it seems to have made no difference.
No real answer here, just me giving you some more background data.
Regards
Alan Chaney
Bill Davidson wrote:
> I've submitted this to Sun a few times. No response. I was hoping
> someone here might have an idea of what to look for.
>
> Tomcat 6.0.18
> RedHat 5.2Server
> Sun JVM
>
> #
> # An unexpected error has been detected by Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00002b68e6a1db57, pid=10229, tid=1103006016
> #
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (11.0-b16 mixed mode
> linux-amd64)
> # Problematic frame:
> # V [libjvm.so+0x564b57]
> #
> # If you would like to submit a bug report, please visit:
> # http://java.sun.com/webapps/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
>
> --------------- T H R E A D ---------------
>
> Current thread (0x000000004bfe5c00): JavaThread "CompilerThread0"
> daemon [_thread_in_native, id=10244,
> stack(0x0000000041ae8000,0x0000000041be9000)]
>
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
> si_addr=0x0000000000000000
>
> .....
>
> Current CompileTask:
> C2:2525
> com.myCompany.servlets.sales.blah.some.method(Lcom/myCompany/servlets/someClass;Lcom/myCompany/servlets/otherClass;Lcom/oreilly/servlet/ParameterParser;)V
> (425 bytes)
>
> ......
>
> -------------
>
> I'm wondering if that CompileTask is what's causing the problem.
> I'm thinking that the JVM shouldn't be getting SIGSEGV's.
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
> !DSPAM:498b962676632009820482!
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Bill Davidson <bi...@gmail.com>.
Rainer Jung wrote:
> In case you only want to exclude few methods from compilation,
>
> -XX:CompileCommand=exclude,com/myCompany/servlets/sales/blah/some,method
>
> will do it.
Since we put this change into production, we haven't seen the problem.
Thank you so much.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Bill Davidson <bi...@gmail.com>.
Rainer Jung wrote:
> You can track compilation with -XX:+PrintCompilation.
[...various other options...]
Wow. I never realized you could do stuff like that. Neat.
Thanks!
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Fun with the JVM crashing.
Posted by Rainer Jung <ra...@kippdata.de>.
On 06.02.2009 02:37, Bill Davidson wrote:
> I've submitted this to Sun a few times. No response. I was hoping
> someone here might have an idea of what to look for.
>
> Tomcat 6.0.18
> RedHat 5.2Server
> Sun JVM
>
> #
> # An unexpected error has been detected by Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00002b68e6a1db57, pid=10229, tid=1103006016
> #
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (11.0-b16 mixed mode
> linux-amd64)
> # Problematic frame:
> # V [libjvm.so+0x564b57]
> #
> # If you would like to submit a bug report, please visit:
> # http://java.sun.com/webapps/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
>
> --------------- T H R E A D ---------------
>
> Current thread (0x000000004bfe5c00): JavaThread "CompilerThread0" daemon
> [_thread_in_native, id=10244, stack(0x0000000041ae8000,0x0000000041be9000)]
>
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
> si_addr=0x0000000000000000
>
> .....
>
> Current CompileTask:
> C2:2525
> com.myCompany.servlets.sales.blah.some.method(Lcom/myCompany/servlets/someClass;Lcom/myCompany/servlets/otherClass;Lcom/oreilly/servlet/ParameterParser;)V
> (425 bytes)
>
> ......
>
> -------------
>
> I'm wondering if that CompileTask is what's causing the problem.
> I'm thinking that the JVM shouldn't be getting SIGSEGV's.
You can track compilation with -XX:+PrintCompilation.
Once you have an idea which class and method produces the crash, you can
disable compilation for it with a file called .hotspot_compiler. It
contains entries like
exclude com/myCompany/servlets/sales/blah/some method
and has to be put into the working directory of the JVM before starting
it. If you want to use a different file with such command or a different
place, you can use
-XX:CompileCommandFile=/my/hotspot/compiler/file
In case you only want to exclude few methods from compilation,
-XX:CompileCommand=exclude,com/myCompany/servlets/sales/blah/some,method
will do it.
See
http://futuretask.blogspot.com/2005/01/java-tip-7-use-hotspotcompiler-file-to.html
or
http://forums-beta.sun.com/thread.jspa?messageID=4273845
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org