You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by qingyang li <li...@gmail.com> on 2014/06/17 06:24:40 UTC
encounter jvm problem when integreation spark with mesos
hi, I encounter jvm problem when integreation spark with mesos,
here is the log when i run "spark-shell":
-48ce131dc5af
14/06/17 12:24:55 INFO HttpServer: Starting HTTP Server
14/06/17 12:24:55 INFO SparkUI: Started Spark Web UI at
http://bigdata001:4040
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f94f4843d21, pid=5956, tid=140277175580416
#
# JRE version: OpenJDK Runtime Environment (7.0_51-b02) (build
1.7.0_51-mockbuild_2014_01_15_01_39-b00)
# Java VM: OpenJDK 64-Bit Server VM (24.45-b08 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# V [libjvm.so+0x5e5d21] JNI_CreateJavaVM+0x6551
#
# Core dump written. Default location:
/home/zjw/spark/spark-0.9.0-incubating-bin-hadoop2/core or core.5956
#
# An error report file with more information is saved as:
# /tmp/jvm-5956/hs_error.log
#
# If you would like to submit a bug report, please include
# instructions on how to reproduce the bug and visit:
# http://icedtea.classpath.org/bugzilla
#
bin/spark-shell: line 101: 5956 Aborted (core dumped)
$FWDIR/bin/spark-class $OPTIONS org.apache.spark.repl.Main "$@"
Re: encounter jvm problem when integreation spark with mesos
Posted by qingyang li <li...@gmail.com>.
here is the core stack info:
-----------------------------
(gdb) bt
#0 0x00007fc0153fc925 in raise () from /lib64/libc.so.6
#1 0x00007fc0153fe105 in abort () from /lib64/libc.so.6
#2 0x00007fc014d78405 in os::abort(bool) ()
from /home/zjw/jdk1.7/jdk1.7.0_51/jre/lib/amd64/server/libjvm.so
#3 0x00007fc014ef7347 in VMError::report_and_die() ()
from /home/zjw/jdk1.7/jdk1.7.0_51/jre/lib/amd64/server/libjvm.so
#4 0x00007fc014d7cd8f in JVM_handle_linux_signal ()
from /home/zjw/jdk1.7/jdk1.7.0_51/jre/lib/amd64/server/libjvm.so
#5 <signal handler called>
#6 0x00007fc014b96ce9 in jni_GetByteArrayElements ()
from /home/zjw/jdk1.7/jdk1.7.0_51/jre/lib/amd64/server/libjvm.so
#7 0x00007fbff70f002c in GetByteArrayElements (env=<value optimized out>,
jobj=0x7fbf8c000f80)
at /home/zjw/jdk1.7/jdk1.7.0_51//include/jni.h:1668
#8 construct<mesos::FrameworkInfo> (env=<value optimized out>,
jobj=0x7fbf8c000f80)
at ../../src/java/jni/construct.cpp:123
#9 0x00007fbff70f51c8 in
Java_org_apache_mesos_MesosSchedulerDriver_initialize (env=0x7fc010d189e8,
thiz=0x7fbfd94f6830) at
../../src/java/jni/org_apache_mesos_MesosSchedulerDriver.cpp:528
2014-06-17 16:12 GMT+08:00 andy petrella <an...@gmail.com>:
> Yep but no real resolution nor advances on this topic, since finally we've
> chosen to stick with a "compatible" version of Mesos (0.14.1 ftm).
> But I'm still convince it has to do with native libs clash :-s
>
> aℕdy ℙetrella
> about.me/noootsab
> [image: aℕdy ℙetrella on about.me]
>
> <http://about.me/noootsab>
>
>
> On Tue, Jun 17, 2014 at 9:57 AM, qingyang li <li...@gmail.com>
> wrote:
>
> > somebody else has also encountered such problem:
> >
> >
> http://mail-archives.apache.org/mod_mbox/spark-user/201404.mbox/%3CAFC0D60983129F4F9FBAD571AA422C9A5AF8F098@MAIL-MBX1.ad.renci.org%3E
> >
> >
> > 2014-06-17 12:31 GMT+08:00 Andrew Ash <an...@andrewash.com>:
> >
> > > Hi qingyang,
> > >
> > > This looks like an issue with the open source version of the Java
> runtime
> > > (called OpenJDK) that causes the JVM to fail. Can you try using the
> JVM
> > > released by Oracle and see if it has the same issue?
> > >
> > > Thanks!
> > > Andrew
> > >
> > >
> > > On Mon, Jun 16, 2014 at 9:24 PM, qingyang li <liqingyang1985@gmail.com
> >
> > > wrote:
> > >
> > > > hi, I encounter jvm problem when integreation spark with mesos,
> > > > here is the log when i run "spark-shell":
> > > > -48ce131dc5af
> > > > 14/06/17 12:24:55 INFO HttpServer: Starting HTTP Server
> > > > 14/06/17 12:24:55 INFO SparkUI: Started Spark Web UI at
> > > > http://bigdata001:4040
> > > > #
> > > > # A fatal error has been detected by the Java Runtime Environment:
> > > > #
> > > > # SIGSEGV (0xb) at pc=0x00007f94f4843d21, pid=5956,
> > tid=140277175580416
> > > > #
> > > > # JRE version: OpenJDK Runtime Environment (7.0_51-b02) (build
> > > > 1.7.0_51-mockbuild_2014_01_15_01_39-b00)
> > > > # Java VM: OpenJDK 64-Bit Server VM (24.45-b08 mixed mode linux-amd64
> > > > compressed oops)
> > > > # Problematic frame:
> > > > # V [libjvm.so+0x5e5d21] JNI_CreateJavaVM+0x6551
> > > > #
> > > > # Core dump written. Default location:
> > > > /home/zjw/spark/spark-0.9.0-incubating-bin-hadoop2/core or core.5956
> > > > #
> > > > # An error report file with more information is saved as:
> > > > # /tmp/jvm-5956/hs_error.log
> > > > #
> > > > # If you would like to submit a bug report, please include
> > > > # instructions on how to reproduce the bug and visit:
> > > > # http://icedtea.classpath.org/bugzilla
> > > > #
> > > > bin/spark-shell: line 101: 5956 Aborted (core
> dumped)
> > > > $FWDIR/bin/spark-class $OPTIONS org.apache.spark.repl.Main "$@"
> > > >
> > >
> >
>
Re: encounter jvm problem when integreation spark with mesos
Posted by andy petrella <an...@gmail.com>.
Yep but no real resolution nor advances on this topic, since finally we've
chosen to stick with a "compatible" version of Mesos (0.14.1 ftm).
But I'm still convince it has to do with native libs clash :-s
aℕdy ℙetrella
about.me/noootsab
[image: aℕdy ℙetrella on about.me]
<http://about.me/noootsab>
On Tue, Jun 17, 2014 at 9:57 AM, qingyang li <li...@gmail.com>
wrote:
> somebody else has also encountered such problem:
>
> http://mail-archives.apache.org/mod_mbox/spark-user/201404.mbox/%3CAFC0D60983129F4F9FBAD571AA422C9A5AF8F098@MAIL-MBX1.ad.renci.org%3E
>
>
> 2014-06-17 12:31 GMT+08:00 Andrew Ash <an...@andrewash.com>:
>
> > Hi qingyang,
> >
> > This looks like an issue with the open source version of the Java runtime
> > (called OpenJDK) that causes the JVM to fail. Can you try using the JVM
> > released by Oracle and see if it has the same issue?
> >
> > Thanks!
> > Andrew
> >
> >
> > On Mon, Jun 16, 2014 at 9:24 PM, qingyang li <li...@gmail.com>
> > wrote:
> >
> > > hi, I encounter jvm problem when integreation spark with mesos,
> > > here is the log when i run "spark-shell":
> > > -48ce131dc5af
> > > 14/06/17 12:24:55 INFO HttpServer: Starting HTTP Server
> > > 14/06/17 12:24:55 INFO SparkUI: Started Spark Web UI at
> > > http://bigdata001:4040
> > > #
> > > # A fatal error has been detected by the Java Runtime Environment:
> > > #
> > > # SIGSEGV (0xb) at pc=0x00007f94f4843d21, pid=5956,
> tid=140277175580416
> > > #
> > > # JRE version: OpenJDK Runtime Environment (7.0_51-b02) (build
> > > 1.7.0_51-mockbuild_2014_01_15_01_39-b00)
> > > # Java VM: OpenJDK 64-Bit Server VM (24.45-b08 mixed mode linux-amd64
> > > compressed oops)
> > > # Problematic frame:
> > > # V [libjvm.so+0x5e5d21] JNI_CreateJavaVM+0x6551
> > > #
> > > # Core dump written. Default location:
> > > /home/zjw/spark/spark-0.9.0-incubating-bin-hadoop2/core or core.5956
> > > #
> > > # An error report file with more information is saved as:
> > > # /tmp/jvm-5956/hs_error.log
> > > #
> > > # If you would like to submit a bug report, please include
> > > # instructions on how to reproduce the bug and visit:
> > > # http://icedtea.classpath.org/bugzilla
> > > #
> > > bin/spark-shell: line 101: 5956 Aborted (core dumped)
> > > $FWDIR/bin/spark-class $OPTIONS org.apache.spark.repl.Main "$@"
> > >
> >
>
Re: encounter jvm problem when integreation spark with mesos
Posted by qingyang li <li...@gmail.com>.
somebody else has also encountered such problem:
http://mail-archives.apache.org/mod_mbox/spark-user/201404.mbox/%3CAFC0D60983129F4F9FBAD571AA422C9A5AF8F098@MAIL-MBX1.ad.renci.org%3E
2014-06-17 12:31 GMT+08:00 Andrew Ash <an...@andrewash.com>:
> Hi qingyang,
>
> This looks like an issue with the open source version of the Java runtime
> (called OpenJDK) that causes the JVM to fail. Can you try using the JVM
> released by Oracle and see if it has the same issue?
>
> Thanks!
> Andrew
>
>
> On Mon, Jun 16, 2014 at 9:24 PM, qingyang li <li...@gmail.com>
> wrote:
>
> > hi, I encounter jvm problem when integreation spark with mesos,
> > here is the log when i run "spark-shell":
> > -48ce131dc5af
> > 14/06/17 12:24:55 INFO HttpServer: Starting HTTP Server
> > 14/06/17 12:24:55 INFO SparkUI: Started Spark Web UI at
> > http://bigdata001:4040
> > #
> > # A fatal error has been detected by the Java Runtime Environment:
> > #
> > # SIGSEGV (0xb) at pc=0x00007f94f4843d21, pid=5956, tid=140277175580416
> > #
> > # JRE version: OpenJDK Runtime Environment (7.0_51-b02) (build
> > 1.7.0_51-mockbuild_2014_01_15_01_39-b00)
> > # Java VM: OpenJDK 64-Bit Server VM (24.45-b08 mixed mode linux-amd64
> > compressed oops)
> > # Problematic frame:
> > # V [libjvm.so+0x5e5d21] JNI_CreateJavaVM+0x6551
> > #
> > # Core dump written. Default location:
> > /home/zjw/spark/spark-0.9.0-incubating-bin-hadoop2/core or core.5956
> > #
> > # An error report file with more information is saved as:
> > # /tmp/jvm-5956/hs_error.log
> > #
> > # If you would like to submit a bug report, please include
> > # instructions on how to reproduce the bug and visit:
> > # http://icedtea.classpath.org/bugzilla
> > #
> > bin/spark-shell: line 101: 5956 Aborted (core dumped)
> > $FWDIR/bin/spark-class $OPTIONS org.apache.spark.repl.Main "$@"
> >
>
Re: encounter jvm problem when integreation spark with mesos
Posted by Andrew Ash <an...@andrewash.com>.
Hi qingyang,
This looks like an issue with the open source version of the Java runtime
(called OpenJDK) that causes the JVM to fail. Can you try using the JVM
released by Oracle and see if it has the same issue?
Thanks!
Andrew
On Mon, Jun 16, 2014 at 9:24 PM, qingyang li <li...@gmail.com>
wrote:
> hi, I encounter jvm problem when integreation spark with mesos,
> here is the log when i run "spark-shell":
> -48ce131dc5af
> 14/06/17 12:24:55 INFO HttpServer: Starting HTTP Server
> 14/06/17 12:24:55 INFO SparkUI: Started Spark Web UI at
> http://bigdata001:4040
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007f94f4843d21, pid=5956, tid=140277175580416
> #
> # JRE version: OpenJDK Runtime Environment (7.0_51-b02) (build
> 1.7.0_51-mockbuild_2014_01_15_01_39-b00)
> # Java VM: OpenJDK 64-Bit Server VM (24.45-b08 mixed mode linux-amd64
> compressed oops)
> # Problematic frame:
> # V [libjvm.so+0x5e5d21] JNI_CreateJavaVM+0x6551
> #
> # Core dump written. Default location:
> /home/zjw/spark/spark-0.9.0-incubating-bin-hadoop2/core or core.5956
> #
> # An error report file with more information is saved as:
> # /tmp/jvm-5956/hs_error.log
> #
> # If you would like to submit a bug report, please include
> # instructions on how to reproduce the bug and visit:
> # http://icedtea.classpath.org/bugzilla
> #
> bin/spark-shell: line 101: 5956 Aborted (core dumped)
> $FWDIR/bin/spark-class $OPTIONS org.apache.spark.repl.Main "$@"
>