You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by tarunk <ta...@gmail.com> on 2020/01/20 08:50:22 UTC
JVM crashed with SIGSEGV with stack trace to
OptimizedObjectOutputStream.writeObject
Hi All,
In our 12 nodes ignite cluster one of the node jvm get crashes sometimes
with below error.
Is this some know issue with Ignite ? or some other memory/jvm issue.
We are using openjdk 1.8 ("1.8.0_232") provided by RedHat.
A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fef7f1d3eb0, pid=42623, tid=0x00007feea62c8700
#
# JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build
1.8.0_222-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# J 62683 C2
org.apache.ignite.internal.marshaller.optimized.OptimizedObjectOutputStream.writeObject0(Ljava/lang/Object;)V
(331 bytes) @ 0x00007fef7f1d3eb0 [0x00007fef7f1d3e00+0xb0]
#
# Failed to write core dump. Core dumps have been disabled. To enable core
dumping, try "ulimit -c unlimited" before starting Java again
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: JVM crashed with SIGSEGV with stack trace to OptimizedObjectOutputStream.writeObject
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I don't think I ever saw this issue happening with 2.7.0 users.
Regards,
--
Ilya Kasnacheev
вт, 28 янв. 2020 г. в 15:10, tarunk <ta...@gmail.com>:
> Thanks Ilya,
>
> Sorry I missed to update on your response. We are using Ignite 2.7.0 with
> openjdk 1.8 (RedHat distribution). Is there any existing Ignite bug with
> this version ?
>
> We are not yet eagerly looking to upgrade Java 11.
> Do you think Ignite or Java upgrade handle this ?
> Regarding different hotspot compiler version, its being built and deployed
> on Linus RHEL mahines so should be server version only.
> Not sure if we can check server/client mode on prod directly as this might
> give unknown performance impact and there is no definite pattern when this
> error come.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: JVM crashed with SIGSEGV with stack trace to
OptimizedObjectOutputStream.writeObject
Posted by tarunk <ta...@gmail.com>.
Thanks Ilya,
Sorry I missed to update on your response. We are using Ignite 2.7.0 with
openjdk 1.8 (RedHat distribution). Is there any existing Ignite bug with
this version ?
We are not yet eagerly looking to upgrade Java 11.
Do you think Ignite or Java upgrade handle this ?
Regarding different hotspot compiler version, its being built and deployed
on Linus RHEL mahines so should be server version only.
Not sure if we can check server/client mode on prod directly as this might
give unknown performance impact and there is no definite pattern when this
error come.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: JVM crashed with SIGSEGV with stack trace to OptimizedObjectOutputStream.writeObject
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
We are using Unsafe API in this method, which may lead to JVM crashes not
caused by any bugs in JVM.
However, in this case I don't see any other reports about JVM crash in this
method. What specific version of Ignite are you using? Have you tried using
different HotSpot compiler, such as client vs. server? Have you considered
upgrading to Java 11?
Regards,
--
Ilya Kasnacheev
вт, 21 янв. 2020 г. в 08:54, tarunk <ta...@gmail.com>:
> Thanks Ilya,
>
> The node is not same always where it comes, and it keep changing.
>
> We got all nodes hardware/OS checked by respective teams, also raised the
> case with RedHat and got the response this should be service issue as only
> specific service is crashing.
>
> Curios to know why we got this raised as Compiler type crash as per
>
> https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/crashes001.html
> This seems to be matching 5.1.2 Crash in Compiled Code from above doc as
> the
> type of problematic frame is "J".
>
> Thanks
> Tarun
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: JVM crashed with SIGSEGV with stack trace to
OptimizedObjectOutputStream.writeObject
Posted by tarunk <ta...@gmail.com>.
Thanks Ilya,
The node is not same always where it comes, and it keep changing.
We got all nodes hardware/OS checked by respective teams, also raised the
case with RedHat and got the response this should be service issue as only
specific service is crashing.
Curios to know why we got this raised as Compiler type crash as per
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/crashes001.html
This seems to be matching 5.1.2 Crash in Compiled Code from above doc as the
type of problematic frame is "J".
Thanks
Tarun
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: JVM crashed with SIGSEGV with stack trace to OptimizedObjectOutputStream.writeObject
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
If it's always the same physical machine, I would check hardware and/or OS.
I don't remember seeing this exception and I don't see why you would only
see it on 1 node.
Regards,
--
Ilya Kasnacheev
пн, 20 янв. 2020 г. в 11:50, tarunk <ta...@gmail.com>:
> Hi All,
>
> In our 12 nodes ignite cluster one of the node jvm get crashes sometimes
> with below error.
> Is this some know issue with Ignite ? or some other memory/jvm issue.
> We are using openjdk 1.8 ("1.8.0_232") provided by RedHat.
>
> A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007fef7f1d3eb0, pid=42623,
> tid=0x00007feea62c8700
> #
> # JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build
> 1.8.0_222-b10)
> # Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-amd64
> compressed oops)
> # Problematic frame:
> # J 62683 C2
>
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectOutputStream.writeObject0(Ljava/lang/Object;)V
> (331 bytes) @ 0x00007fef7f1d3eb0 [0x00007fef7f1d3e00+0xb0]
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core
> dumping, try "ulimit -c unlimited" before starting Java again
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>