You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2019/01/29 06:16:04 UTC
[GitHub] aashudwivedi removed a comment on issue #13853: Error while running
the mxnet spark examples and test cases
aashudwivedi removed a comment on issue #13853: Error while running the mxnet spark examples and test cases
URL: https://github.com/apache/incubator-mxnet/issues/13853#issuecomment-457561018
The complete stack trace of the error is :
```Testing started at 6:07 PM ...
/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java -Djava.library.path=/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/native/osx-x86_64-cpu/target "-javaagent:/Applications/IntelliJ IDEA CE.app/Contents/lib/idea_rt.jar=55436:/Applications/IntelliJ IDEA CE.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Users/ashutdwi/Library/Application Support/IdeaIC2018.2/Scala/lib/runners.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/tools.jar:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/target/test-classes:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/target/classes:/Users/ashutdwi/.m2/repository/org/apache/mxnet/mxnet-core/INTERNAL/mxnet-core-INTERNAL.jar:/Users/ashutdwi/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-mllib_2.11/1.6.3/spark-mllib_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-core_2.11/1.6.3/spark-core_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/org/apache/avro/avro-mapred/1.7.7/avro-mapred-1.7.7-hadoop2.jar:/Users/ashutdwi/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7.jar:/Users/ashutdwi/.m2/repository/org/apache/avro/avro/1.7.7/avro-1.7.7.jar:/Users/ashutdwi/.m2/repository/org/apache/avro/avro-ipc/1.7.7/avro-ipc-1.7.7-tests.jar:/Users/ashutdwi/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/Users/ashutdwi/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/Users/ashutdwi/.m2/repository/com/twitter/chill_2.11/0.5.0/chill_2.11-0.5.0.jar:/Users/ashutdwi/.m2/repository/com/esotericsoftware/kryo/kryo/2.21/kryo-2.21.jar:/Users/ashutdwi/.m2/repository/com/esotericsoftware/reflectasm/reflectasm/1.07/reflectasm-1.07-shaded.jar:/Users/ashutdwi/.m2/repository/com/esotericsoftware/minlog/minlog/1.2/minlog-1.2.jar:/Users/ashutdwi/.m2/repository/org/objenesis/objenesis/1.2/objenesis-1.2.jar:/Users/ashutdwi/.m2/repository/com/twitter/chill-java/0.5.0/chill-java-0.5.0.jar:/Users/ashutdwi/.m2/repository/org/apache/xbean/xbean-asm5-shaded/4.4/xbean-asm5-shaded-4.4.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-client/2.2.0/hadoop-client-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-common/2.2.0/hadoop-common-2.2.0.jar:/Users/ashutdwi/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/ashutdwi/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/ashutdwi/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/Users/ashutdwi/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/ashutdwi/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/Users/ashutdwi/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/ashutdwi/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/Users/ashutdwi/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-auth/2.2.0/hadoop-auth-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/Users/ashutdwi/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.2.0/hadoop-hdfs-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-app/2.2.0/hadoop-mapreduce-client-app-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/2.2.0/hadoop-mapreduce-client-common-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-yarn-client/2.2.0/hadoop-yarn-client-2.2.0.jar:/Users/ashutdwi/.m2/repository/com/google/inject/guice/3.0/guice-3.0.jar:/Users/ashutdwi/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/ashutdwi/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/ashutdwi/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-grizzly2/1.9/jersey-test-framework-grizzly2-1.9.jar:/Users/ashutdwi/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-core/1.9/jersey-test-framework-core-1.9.jar:/Users/ashutdwi/.m2/repository/javax/servlet/javax.servlet-api/3.0.1/javax.servlet-api-3.0.1.jar:/Users/ashutdwi/.m2/repository/com/sun/jersey/jersey-client/1.9/jersey-client-1.9.jar:/Users/ashutdwi/.m2/repository/com/sun/jersey/jersey-grizzly2/1.9/jersey-grizzly2-1.9.jar:/Users/ashutdwi/.m2/repository/org/glassfish/grizzly/grizzly-http/2.1.2/grizzly-http-2.1.2.jar:/Users/ashutdwi/.m2/repository/org/glassfish/grizzly/grizzly-framework/2.1.2/grizzly-framework-2.1.2.jar:/Users/ashutdwi/.m2/repository/org/glassfish/gmbal/gmbal-api-only/3.0.0-b023/gmbal-api-only-3.0.0-b023.jar:/Users/ashutdwi/.m2/repository/org/glassfish/external/management-api/3.0.0-b012/management-api-3.0.0-b012.jar:/Users/ashutdwi/.m2/repository/org/glassfish/grizzly/grizzly-http-server/2.1.2/grizzly-http-server-2.1.2.jar:/Users/ashutdwi/.m2/repository/org/glassfish/grizzly/grizzly-rcm/2.1.2/grizzly-rcm-2.1.2.jar:/Users/ashutdwi/.m2/repository/org/glassfish/grizzly/grizzly-http-servlet/2.1.2/grizzly-http-servlet-2.1.2.jar:/Users/ashutdwi/.m2/repository/org/glassfish/javax.servlet/3.1/javax.servlet-3.1.jar:/Users/ashutdwi/.m2/repository/com/sun/jersey/jersey-json/1.9/jersey-json-1.9.jar:/Users/ashutdwi/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/ashutdwi/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/ashutdwi/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.8.3/jackson-jaxrs-1.8.3.jar:/Users/ashutdwi/.m2/repository/org/codehaus/jackson/jackson-xc/1.8.3/jackson-xc-1.8.3.jar:/Users/ashutdwi/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.9/jersey-guice-1.9.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/2.2.0/hadoop-yarn-server-common-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/2.2.0/hadoop-mapreduce-client-shuffle-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.2.0/hadoop-mapreduce-client-core-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.2.0/hadoop-yarn-common-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.2.0/hadoop-mapreduce-client-jobclient-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/hadoop/hadoop-annotations/2.2.0/hadoop-annotations-2.2.0.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-launcher_2.11/1.6.3/spark-launcher_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-network-common_2.11/1.6.3/spark-network-common_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-network-shuffle_2.11/1.6.3/spark-network-shuffle_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar:/Users/ashutdwi/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.4.4/jackson-annotations-2.4.4.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-unsafe_2.11/1.6.3/spark-unsafe_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/net/java/dev/jets3t/jets3t/0.7.1/jets3t-0.7.1.jar:/Users/ashutdwi/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/ashutdwi/.m2/repository/org/apache/curator/curator-recipes/2.4.0/curator-recipes-2.4.0.jar:/Users/ashutdwi/.m2/repository/org/apache/curator/curator-framework/2.4.0/curator-framework-2.4.0.jar:/Users/ashutdwi/.m2/repository/org/apache/curator/curator-client/2.4.0/curator-client-2.4.0.jar:/Users/ashutdwi/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar:/Users/ashutdwi/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/Users/ashutdwi/.m2/repository/com/google/guava/guava/14.0.1/guava-14.0.1.jar:/Users/ashutdwi/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/Users/ashutdwi/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/Users/ashutdwi/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/ashutdwi/.m2/repository/org/slf4j/jul-to-slf4j/1.7.10/jul-to-slf4j-1.7.10.jar:/Users/ashutdwi/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.10/jcl-over-slf4j-1.7.10.jar:/Users/ashutdwi/.m2/repository/com/ning/compress-lzf/1.0.3/compress-lzf-1.0.3.jar:/Users/ashutdwi/.m2/repository/org/xerial/snappy/snappy-java/1.1.2.6/snappy-java-1.1.2.6.jar:/Users/ashutdwi/.m2/repository/net/jpountz/lz4/lz4/1.3.0/lz4-1.3.0.jar:/Users/ashutdwi/.m2/repository/org/roaringbitmap/RoaringBitmap/0.5.11/RoaringBitmap-0.5.11.jar:/Users/ashutdwi/.m2/repository/commons-net/commons-net/2.2/commons-net-2.2.jar:/Users/ashutdwi/.m2/repository/com/typesafe/akka/akka-remote_2.11/2.3.11/akka-remote_2.11-2.3.11.jar:/Users/ashutdwi/.m2/repository/com/typesafe/akka/akka-actor_2.11/2.3.11/akka-actor_2.11-2.3.11.jar:/Users/ashutdwi/.m2/repository/com/typesafe/config/1.2.1/config-1.2.1.jar:/Users/ashutdwi/.m2/repository/io/netty/netty/3.8.0.Final/netty-3.8.0.Final.jar:/Users/ashutdwi/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/ashutdwi/.m2/repository/org/uncommons/maths/uncommons-maths/1.2.2a/uncommons-maths-1.2.2a.jar:/Users/ashutdwi/.m2/repository/com/typesafe/akka/akka-slf4j_2.11/2.3.11/akka-slf4j_2.11-2.3.11.jar:/Users/ashutdwi/.m2/repository/org/json4s/json4s-jackson_2.11/3.2.10/json4s-jackson_2.11-3.2.10.jar:/Users/ashutdwi/.m2/repository/org/json4s/json4s-core_2.11/3.2.10/json4s-core_2.11-3.2.10.jar:/Users/ashutdwi/.m2/repository/org/json4s/json4s-ast_2.11/3.2.10/json4s-ast_2.11-3.2.10.jar:/Users/ashutdwi/.m2/repository/org/scala-lang/scalap/2.11.0/scalap-2.11.0.jar:/Users/ashutdwi/.m2/repository/com/sun/jersey/jersey-server/1.9/jersey-server-1.9.jar:/Users/ashutdwi/.m2/repository/asm/asm/3.1/asm-3.1.jar:/Users/ashutdwi/.m2/repository/com/sun/jersey/jersey-core/1.9/jersey-core-1.9.jar:/Users/ashutdwi/.m2/repository/org/apache/mesos/mesos/0.21.1/mesos-0.21.1-shaded-protobuf.jar:/Users/ashutdwi/.m2/repository/io/netty/netty-all/4.0.29.Final/netty-all-4.0.29.Final.jar:/Users/ashutdwi/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/Users/ashutdwi/.m2/repository/io/dropwizard/metrics/metrics-core/3.1.2/metrics-core-3.1.2.jar:/Users/ashutdwi/.m2/repository/io/dropwizard/metrics/metrics-jvm/3.1.2/metrics-jvm-3.1.2.jar:/Users/ashutdwi/.m2/repository/io/dropwizard/metrics/metrics-json/3.1.2/metrics-json-3.1.2.jar:/Users/ashutdwi/.m2/repository/io/dropwizard/metrics/metrics-graphite/3.1.2/metrics-graphite-3.1.2.jar:/Users/ashutdwi/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.4.4/jackson-databind-2.4.4.jar:/Users/ashutdwi/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.4.4/jackson-core-2.4.4.jar:/Users/ashutdwi/.m2/repository/com/fasterxml/jackson/module/jackson-module-scala_2.11/2.4.4/jackson-module-scala_2.11-2.4.4.jar:/Users/ashutdwi/.m2/repository/com/thoughtworks/paranamer/paranamer/2.6/paranamer-2.6.jar:/Users/ashutdwi/.m2/repository/org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar:/Users/ashutdwi/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/ashutdwi/.m2/repository/org/tachyonproject/tachyon-client/0.8.2/tachyon-client-0.8.2.jar:/Users/ashutdwi/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar:/Users/ashutdwi/.m2/repository/org/tachyonproject/tachyon-underfs-hdfs/0.8.2/tachyon-underfs-hdfs-0.8.2.jar:/Users/ashutdwi/.m2/repository/org/tachyonproject/tachyon-underfs-s3/0.8.2/tachyon-underfs-s3-0.8.2.jar:/Users/ashutdwi/.m2/repository/org/tachyonproject/tachyon-underfs-local/0.8.2/tachyon-underfs-local-0.8.2.jar:/Users/ashutdwi/.m2/repository/net/razorvine/pyrolite/4.9/pyrolite-4.9.jar:/Users/ashutdwi/.m2/repository/net/sf/py4j/py4j/0.9/py4j-0.9.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-streaming_2.11/1.6.3/spark-streaming_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-sql_2.11/1.6.3/spark-sql_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-catalyst_2.11/1.6.3/spark-catalyst_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/org/codehaus/janino/janino/2.7.8/janino-2.7.8.jar:/Users/ashutdwi/.m2/repository/org/codehaus/janino/commons-compiler/2.7.8/commons-compiler-2.7.8.jar:/Users/ashutdwi/.m2/repository/org/apache/parquet/parquet-column/1.7.0/parquet-column-1.7.0.jar:/Users/ashutdwi/.m2/repository/org/apache/parquet/parquet-common/1.7.0/parquet-common-1.7.0.jar:/Users/ashutdwi/.m2/repository/org/apache/parquet/parquet-encoding/1.7.0/parquet-encoding-1.7.0.jar:/Users/ashutdwi/.m2/repository/org/apache/parquet/parquet-generator/1.7.0/parquet-generator-1.7.0.jar:/Users/ashutdwi/.m2/repository/org/apache/parquet/parquet-hadoop/1.7.0/parquet-hadoop-1.7.0.jar:/Users/ashutdwi/.m2/repository/org/apache/parquet/parquet-format/2.3.0-incubating/parquet-format-2.3.0-incubating.jar:/Users/ashutdwi/.m2/repository/org/apache/parquet/parquet-jackson/1.7.0/parquet-jackson-1.7.0.jar:/Users/ashutdwi/.m2/repository/org/apache/spark/spark-graphx_2.11/1.6.3/spark-graphx_2.11-1.6.3.jar:/Users/ashutdwi/.m2/repository/com/github/fommil/netlib/core/1.1.2/core-1.1.2.jar:/Users/ashutdwi/.m2/repository/net/sourceforge/f2j/arpack_combined_all/0.1/arpack_combined_all-0.1.jar:/Users/ashutdwi/.m2/repository/org/scalanlp/breeze_2.11/0.11.2/breeze_2.11-0.11.2.jar:/Users/ashutdwi/.m2/repository/org/scalanlp/breeze-macros_2.11/0.11.2/breeze-macros_2.11-0.11.2.jar:/Users/ashutdwi/.m2/repository/net/sf/opencsv/opencsv/2.3/opencsv-2.3.jar:/Users/ashutdwi/.m2/repository/com/github/rwl/jtransforms/2.4.0/jtransforms-2.4.0.jar:/Users/ashutdwi/.m2/repository/org/spire-math/spire_2.11/0.7.4/spire_2.11-0.7.4.jar:/Users/ashutdwi/.m2/repository/org/spire-math/spire-macros_2.11/0.7.4/spire-macros_2.11-0.7.4.jar:/Users/ashutdwi/.m2/repository/org/apache/commons/commons-math3/3.4.1/commons-math3-3.4.1.jar:/Users/ashutdwi/.m2/repository/org/jpmml/pmml-model/1.1.15/pmml-model-1.1.15.jar:/Users/ashutdwi/.m2/repository/org/jpmml/pmml-agent/1.1.15/pmml-agent-1.1.15.jar:/Users/ashutdwi/.m2/repository/org/jpmml/pmml-schema/1.1.15/pmml-schema-1.1.15.jar:/Users/ashutdwi/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.7/jaxb-impl-2.2.7.jar:/Users/ashutdwi/.m2/repository/com/sun/xml/bind/jaxb-core/2.2.7/jaxb-core-2.2.7.jar:/Users/ashutdwi/.m2/repository/javax/xml/bind/jaxb-api/2.2.7/jaxb-api-2.2.7.jar:/Users/ashutdwi/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:/Users/ashutdwi/.m2/repository/args4j/args4j/2.33/args4j-2.33.jar:/Users/ashutdwi/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar:/Users/ashutdwi/.m2/repository/org/apache/logging/log4j/log4j-core/2.11.1/log4j-core-2.11.1.jar:/Users/ashutdwi/.m2/repository/org/apache/logging/log4j/log4j-api/2.11.1/log4j-api-2.11.1.jar:/Users/ashutdwi/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar:/Users/ashutdwi/.m2/repository/org/slf4j/slf4j-log4j12/1.7.7/slf4j-log4j12-1.7.7.jar:/Users/ashutdwi/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/ashutdwi/.m2/repository/org/scalatest/scalatest_2.11/3.0.4/scalatest_2.11-3.0.4.jar:/Users/ashutdwi/.m2/repository/org/scalactic/scalactic_2.11/3.0.4/scalactic_2.11-3.0.4.jar:/Users/ashutdwi/.m2/repository/org/scala-lang/modules/scala-xml_2.11/1.0.5/scala-xml_2.11-1.0.5.jar:/Users/ashutdwi/.m2/repository/org/scalacheck/scalacheck_2.11/1.13.5/scalacheck_2.11-1.13.5.jar:/Users/ashutdwi/.m2/repository/org/scala-sbt/test-interface/1.0/test-interface-1.0.jar:/Users/ashutdwi/.m2/repository/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.jar:/Users/ashutdwi/.m2/repository/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar:/Users/ashutdwi/.m2/repository/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.4/scala-parser-combinators_2.11-1.0.4.jar:/Users/ashutdwi/.m2/repository/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.jar" org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner -s org.apache.mxnet.spark.MXNetGeneralSuite -testName "run spark with MLP" -C org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestReporter -showProgressMessages true
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
19/01/25 18:07:52 INFO SparkContext: Running Spark version 1.6.3
19/01/25 18:07:52 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/01/25 18:07:52 INFO SecurityManager: Changing view acls to: ashutdwi
19/01/25 18:07:52 INFO SecurityManager: Changing modify acls to: ashutdwi
19/01/25 18:07:52 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(ashutdwi); users with modify permissions: Set(ashutdwi)
19/01/25 18:07:53 INFO Utils: Successfully started service 'sparkDriver' on port 55439.
19/01/25 18:07:53 INFO Slf4jLogger: Slf4jLogger started
19/01/25 18:07:53 INFO Remoting: Starting remoting
19/01/25 18:07:53 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@192.168.0.20:55440]
19/01/25 18:07:53 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 55440.
19/01/25 18:07:53 INFO SparkEnv: Registering MapOutputTracker
19/01/25 18:07:53 INFO SparkEnv: Registering BlockManagerMaster
19/01/25 18:07:53 INFO DiskBlockManager: Created local directory at /private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/blockmgr-5fb86e1b-8627-4a91-b3ac-a3c449e6a68b
19/01/25 18:07:53 INFO MemoryStore: MemoryStore started with capacity 2.4 GB
19/01/25 18:07:53 INFO SparkEnv: Registering OutputCommitCoordinator
19/01/25 18:07:53 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
19/01/25 18:07:53 INFO Utils: Successfully started service 'SparkUI' on port 4041.
19/01/25 18:07:53 INFO SparkUI: Started SparkUI at http://192.168.0.20:4041
19/01/25 18:07:53 INFO Executor: Starting executor ID driver on host localhost
19/01/25 18:07:53 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 55441.
19/01/25 18:07:53 INFO NettyBlockTransferService: Server created on 55441
19/01/25 18:07:54 INFO BlockManagerMaster: Trying to register BlockManager
19/01/25 18:07:54 INFO BlockManagerMasterEndpoint: Registering block manager localhost:55441 with 2.4 GB RAM, BlockManagerId(driver, localhost, 55441)
19/01/25 18:07:54 INFO BlockManagerMaster: Registered BlockManager
19/01/25 18:07:54 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 107.7 KB, free 2.4 GB)
19/01/25 18:07:54 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 9.8 KB, free 2.4 GB)
19/01/25 18:07:54 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:55441 (size: 9.8 KB, free: 2.4 GB)
19/01/25 18:07:54 INFO SparkContext: Created broadcast 0 from textFile at MXNetGeneralSuite.scala:35
19/01/25 18:07:54 INFO MXNetJVM: Try loading mxnet-scala from native path.
19/01/25 18:07:54 WARN MXNetJVM: MXNet Scala native library not found in path. Copying native library from the archive. Consider installing the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH), or specifying by Java cmd option -Djava.library.path=[lib path].
19/01/25 18:07:54 WARN MXNetJVM: LD_LIBRARY_PATH=/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/native/osx-x86_64-cpu/target
19/01/25 18:07:54 WARN MXNetJVM: java.library.path=/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/native/osx-x86_64-cpu/target
19/01/25 18:07:55 INFO Utils: Copying /Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/../assembly/osx-x86_64-cpu/target/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar to /private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3/userFiles-908ec2c6-1cd7-4248-a8d6-a29d63bfc3d1/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar
19/01/25 18:07:55 INFO SparkContext: Added file /Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/../assembly/osx-x86_64-cpu/target/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar at file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/assembly/osx-x86_64-cpu/target/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar with timestamp 1548419875041
19/01/25 18:07:55 INFO Utils: Copying /Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/target/mxnet-spark-INTERNAL.jar to /private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3/userFiles-908ec2c6-1cd7-4248-a8d6-a29d63bfc3d1/mxnet-spark-INTERNAL.jar
19/01/25 18:07:55 INFO SparkContext: Added file /Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/target/mxnet-spark-INTERNAL.jar at file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/target/mxnet-spark-INTERNAL.jar with timestamp 1548419875078
19/01/25 18:07:55 INFO FileInputFormat: Total input paths to process : 1
19/01/25 18:07:55 INFO MXNet: repartitioning training set to 2 partitions
19/01/25 18:07:55 INFO MXNet: Starting scheduler on 192.168.0.20:55443
19/01/25 18:07:55 INFO ParameterServer: Started process: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java -cp /private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3/userFiles-908ec2c6-1cd7-4248-a8d6-a29d63bfc3d1/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar:/private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3/userFiles-908ec2c6-1cd7-4248-a8d6-a29d63bfc3d1/mxnet-spark-INTERNAL.jar org.apache.mxnet.spark.ParameterServer --role=scheduler --root-uri=192.168.0.20 --root-port=55443 --num-server=1 --num-worker=2 --timeout=0 at 192.168.0.20:55443
19/01/25 18:07:55 INFO ParameterServer: Starting InputStream-Redirecter Thread for 192.168.0.20:55443
19/01/25 18:07:55 INFO ParameterServer: Starting ErrorStream-Redirecter Thread for 192.168.0.20:55443
19/01/25 18:07:55 WARN WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Symbol was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
19/01/25 18:07:55 INFO SparkContext: Starting job: foreachPartition at MXNet.scala:126
19/01/25 18:07:55 INFO DAGScheduler: Got job 0 (foreachPartition at MXNet.scala:126) with 1 output partitions
19/01/25 18:07:55 INFO DAGScheduler: Final stage: ResultStage 0 (foreachPartition at MXNet.scala:126)
19/01/25 18:07:55 INFO DAGScheduler: Parents of final stage: List()
19/01/25 18:07:55 INFO DAGScheduler: Missing parents: List()
19/01/25 18:07:55 INFO SparkContext: Starting job: foreachPartition at MXNet.scala:238
19/01/25 18:07:55 INFO DAGScheduler: Submitting ResultStage 0 (ParallelCollectionRDD[7] at parallelize at MXNet.scala:126), which has no missing parents
19/01/25 18:07:55 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 5.0 KB, free 2.4 GB)
19/01/25 18:07:55 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.5 KB, free 2.4 GB)
19/01/25 18:07:55 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:55441 (size: 2.5 KB, free: 2.4 GB)
19/01/25 18:07:55 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1006
19/01/25 18:07:55 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (ParallelCollectionRDD[7] at parallelize at MXNet.scala:126)
19/01/25 18:07:55 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
19/01/25 18:07:55 INFO DAGScheduler: Registering RDD 3 (repartition at MXNet.scala:251)
19/01/25 18:07:55 INFO DAGScheduler: Got job 1 (foreachPartition at MXNet.scala:238) with 2 output partitions
19/01/25 18:07:55 INFO DAGScheduler: Final stage: ResultStage 2 (foreachPartition at MXNet.scala:238)
19/01/25 18:07:55 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 1)
19/01/25 18:07:55 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 1)
19/01/25 18:07:55 INFO DAGScheduler: Submitting ShuffleMapStage 1 (MapPartitionsRDD[3] at repartition at MXNet.scala:251), which has no missing parents
19/01/25 18:07:55 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, partition 0,PROCESS_LOCAL, 2327 bytes)
19/01/25 18:07:55 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 4.3 KB, free 2.4 GB)
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" 19/01/25 18:07:55 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 2.4 KB, free 2.4 GB)
java.lang.NoClassDefFoundError: scala/collection/Seq
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: scala.collection.Seq
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)19/01/25 18:07:55 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on localhost:55441 (size: 2.4 KB, free: 2.4 GB)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
19/01/25 18:07:55 INFO SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:1006
19/01/25 18:07:55 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
19/01/25 18:07:55 INFO DAGScheduler: Submitting 8 missing tasks from ShuffleMapStage 1 (MapPartitionsRDD[3] at repartition at MXNet.scala:251)
19/01/25 18:07:55 INFO TaskSchedulerImpl: Adding task set 1.0 with 8 tasks
Exception in thread "Thread-34" java.lang.IllegalArgumentException: requirement failed: Failed to start ps scheduler process with exit code 1
at scala.Predef$.require(Predef.scala:224)
at org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$startPSSchedulerInner$1(MXNet.scala:159)
at org.apache.mxnet.spark.MXNet$$anonfun$startPSScheduler$1.apply(MXNet.scala:162)
at org.apache.mxnet.spark.MXNet$$anonfun$startPSScheduler$1.apply(MXNet.scala:162)
at org.apache.mxnet.spark.MXNet$MXNetControllingThread.run(MXNet.scala:38)
19/01/25 18:07:55 INFO Executor: Fetching file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/assembly/osx-x86_64-cpu/target/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar with timestamp 1548419875041
19/01/25 18:07:55 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, localhost, partition 0,PROCESS_LOCAL, 2377 bytes)
19/01/25 18:07:55 INFO TaskSetManager: Starting task 1.0 in stage 1.0 (TID 2, localhost, partition 1,PROCESS_LOCAL, 2377 bytes)
19/01/25 18:07:55 INFO TaskSetManager: Starting task 2.0 in stage 1.0 (TID 3, localhost, partition 2,PROCESS_LOCAL, 2377 bytes)
19/01/25 18:07:55 INFO TaskSetManager: Starting task 3.0 in stage 1.0 (TID 4, localhost, partition 3,PROCESS_LOCAL, 2377 bytes)
19/01/25 18:07:55 INFO TaskSetManager: Starting task 4.0 in stage 1.0 (TID 5, localhost, partition 4,PROCESS_LOCAL, 2377 bytes)
19/01/25 18:07:55 INFO TaskSetManager: Starting task 5.0 in stage 1.0 (TID 6, localhost, partition 5,PROCESS_LOCAL, 2377 bytes)
19/01/25 18:07:55 INFO TaskSetManager: Starting task 6.0 in stage 1.0 (TID 7, localhost, partition 6,PROCESS_LOCAL, 2377 bytes)
19/01/25 18:07:55 INFO Executor: Running task 0.0 in stage 1.0 (TID 1)
19/01/25 18:07:55 INFO Executor: Running task 1.0 in stage 1.0 (TID 2)
19/01/25 18:07:55 INFO Executor: Running task 2.0 in stage 1.0 (TID 3)
19/01/25 18:07:55 INFO Executor: Running task 3.0 in stage 1.0 (TID 4)
19/01/25 18:07:55 INFO Executor: Running task 4.0 in stage 1.0 (TID 5)
19/01/25 18:07:55 INFO Executor: Running task 5.0 in stage 1.0 (TID 6)
19/01/25 18:07:55 INFO Executor: Running task 6.0 in stage 1.0 (TID 7)
19/01/25 18:07:55 INFO Utils: /Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/assembly/osx-x86_64-cpu/target/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar has been previously copied to /private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3/userFiles-908ec2c6-1cd7-4248-a8d6-a29d63bfc3d1/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar
19/01/25 18:07:55 INFO Executor: Fetching file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/target/mxnet-spark-INTERNAL.jar with timestamp 1548419875078
19/01/25 18:07:55 INFO Utils: /Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/target/mxnet-spark-INTERNAL.jar has been previously copied to /private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3/userFiles-908ec2c6-1cd7-4248-a8d6-a29d63bfc3d1/mxnet-spark-INTERNAL.jar
19/01/25 18:07:55 INFO MXNet: Starting server ...
19/01/25 18:07:55 INFO HadoopRDD: Input split: file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/bin/train_full.txt.txt:33554432+33554432
19/01/25 18:07:55 INFO HadoopRDD: Input split: file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/bin/train_full.txt.txt:167772160+33554432
19/01/25 18:07:55 INFO HadoopRDD: Input split: file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/bin/train_full.txt.txt:134217728+33554432
19/01/25 18:07:55 INFO HadoopRDD: Input split: file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/bin/train_full.txt.txt:67108864+33554432
19/01/25 18:07:55 INFO HadoopRDD: Input split: file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/bin/train_full.txt.txt:201326592+33554432
19/01/25 18:07:55 INFO HadoopRDD: Input split: file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/bin/train_full.txt.txt:100663296+33554432
19/01/25 18:07:55 INFO HadoopRDD: Input split: file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/bin/train_full.txt.txt:0+33554432
19/01/25 18:07:55 INFO ParameterServer: Started process: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java -cp /private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3/userFiles-908ec2c6-1cd7-4248-a8d6-a29d63bfc3d1/mxnet-full_2.11-osx-x86_64-cpu-1.5.0-SNAPSHOT-src.jar:/private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3/userFiles-908ec2c6-1cd7-4248-a8d6-a29d63bfc3d1/mxnet-spark-INTERNAL.jar org.apache.mxnet.spark.ParameterServer --role=server --root-uri=192.168.0.20 --root-port=55443 --num-server=1 --num-worker=2 --timeout=0 at 192.168.0.20:55443
19/01/25 18:07:55 INFO ParameterServer: Starting InputStream-Redirecter Thread for 192.168.0.20:55443
19/01/25 18:07:55 INFO ParameterServer: Starting ErrorStream-Redirecter Thread for 192.168.0.20:55443
19/01/25 18:07:55 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
19/01/25 18:07:55 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
19/01/25 18:07:55 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
19/01/25 18:07:55 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
19/01/25 18:07:55 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
19/01/25 18:07:55 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
19/01/25 18:07:55 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
19/01/25 18:07:55 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
19/01/25 18:07:55 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
19/01/25 18:07:55 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/Seq
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: scala.collection.Seq
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
19/01/25 18:07:55 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.IllegalArgumentException: requirement failed: ps server process quit with exit code 1
at scala.Predef$.require(Predef.scala:224)
at org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:137)
at org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:126)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/01/25 18:07:55 INFO TaskSetManager: Starting task 7.0 in stage 1.0 (TID 8, localhost, partition 7,PROCESS_LOCAL, 2377 bytes)
19/01/25 18:07:55 INFO Executor: Running task 7.0 in stage 1.0 (TID 8)
19/01/25 18:07:55 INFO HadoopRDD: Input split: file:/Users/ashutdwi/ora/opensource/incubator-mxnet/scala-package/spark/bin/train_full.txt.txt:234881024+8528911
19/01/25 18:07:55 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.IllegalArgumentException: requirement failed: ps server process quit with exit code 1
at scala.Predef$.require(Predef.scala:224)
at org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:137)
at org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:126)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/01/25 18:07:55 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1 times; aborting job
19/01/25 18:07:55 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
19/01/25 18:07:55 INFO TaskSchedulerImpl: Cancelling stage 0
19/01/25 18:07:55 INFO DAGScheduler: ResultStage 0 (foreachPartition at MXNet.scala:126) failed in 0.428 s
19/01/25 18:07:55 INFO DAGScheduler: Job 0 failed: foreachPartition at MXNet.scala:126, took 0.483542 s
Exception in thread "Thread-35" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost): java.lang.IllegalArgumentException: requirement failed: ps server process quit with exit code 1
at scala.Predef$.require(Predef.scala:224)
at org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:137)
at org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:126)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:918)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:918)
at org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$startPSServersInner$1(MXNet.scala:126)
at org.apache.mxnet.spark.MXNet$$anonfun$startPSServers$1.apply(MXNet.scala:140)
at org.apache.mxnet.spark.MXNet$$anonfun$startPSServers$1.apply(MXNet.scala:140)
at org.apache.mxnet.spark.MXNet$MXNetControllingThread.run(MXNet.scala:38)
Caused by: java.lang.IllegalArgumentException: requirement failed: ps server process quit with exit code 1
at scala.Predef$.require(Predef.scala:224)
at org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:137)
at org.apache.mxnet.spark.MXNet$$anonfun$org$apache$mxnet$spark$MXNet$$startPSServersInner$1$1.apply(MXNet.scala:126)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/01/25 18:07:55 INFO BlockManagerInfo: Removed broadcast_1_piece0 on localhost:55441 in memory (size: 2.5 KB, free: 2.4 GB)
19/01/25 18:07:55 INFO ContextCleaner: Cleaned accumulator 1
19/01/25 18:07:56 INFO Executor: Finished task 7.0 in stage 1.0 (TID 8). 2254 bytes result sent to driver
19/01/25 18:07:56 INFO TaskSetManager: Finished task 7.0 in stage 1.0 (TID 8) in 832 ms on localhost (1/8)
19/01/25 18:07:57 INFO Executor: Finished task 1.0 in stage 1.0 (TID 2). 2254 bytes result sent to driver
19/01/25 18:07:57 INFO TaskSetManager: Finished task 1.0 in stage 1.0 (TID 2) in 2266 ms on localhost (2/8)
19/01/25 18:07:57 INFO Executor: Finished task 5.0 in stage 1.0 (TID 6). 2254 bytes result sent to driver
19/01/25 18:07:57 INFO Executor: Finished task 3.0 in stage 1.0 (TID 4). 2254 bytes result sent to driver
19/01/25 18:07:57 INFO Executor: Finished task 6.0 in stage 1.0 (TID 7). 2254 bytes result sent to driver
19/01/25 18:07:57 INFO TaskSetManager: Finished task 5.0 in stage 1.0 (TID 6) in 2297 ms on localhost (3/8)
19/01/25 18:07:57 INFO TaskSetManager: Finished task 3.0 in stage 1.0 (TID 4) in 2299 ms on localhost (4/8)
19/01/25 18:07:57 INFO TaskSetManager: Finished task 6.0 in stage 1.0 (TID 7) in 2296 ms on localhost (5/8)
19/01/25 18:07:57 INFO Executor: Finished task 2.0 in stage 1.0 (TID 3). 2254 bytes result sent to driver
19/01/25 18:07:57 INFO TaskSetManager: Finished task 2.0 in stage 1.0 (TID 3) in 2329 ms on localhost (6/8)
19/01/25 18:07:57 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 2254 bytes result sent to driver
19/01/25 18:07:57 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 2403 ms on localhost (7/8)
19/01/25 18:07:57 INFO Executor: Finished task 4.0 in stage 1.0 (TID 5). 2254 bytes result sent to driver
19/01/25 18:07:57 INFO TaskSetManager: Finished task 4.0 in stage 1.0 (TID 5) in 2411 ms on localhost (8/8)
19/01/25 18:07:57 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool
19/01/25 18:07:57 INFO DAGScheduler: ShuffleMapStage 1 (repartition at MXNet.scala:251) finished in 2.425 s
19/01/25 18:07:57 INFO DAGScheduler: looking for newly runnable stages
19/01/25 18:07:57 INFO DAGScheduler: running: Set()
19/01/25 18:07:57 INFO DAGScheduler: waiting: Set(ResultStage 2)
19/01/25 18:07:57 INFO DAGScheduler: failed: Set()
19/01/25 18:07:57 INFO DAGScheduler: Submitting ResultStage 2 (MapPartitionsRDD[8] at mapPartitions at MXNet.scala:209), which has no missing parents
19/01/25 18:07:57 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 7.2 KB, free 2.4 GB)
19/01/25 18:07:57 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 3.6 KB, free 2.4 GB)
19/01/25 18:07:57 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on localhost:55441 (size: 3.6 KB, free: 2.4 GB)
19/01/25 18:07:57 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1006
19/01/25 18:07:57 INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 2 (MapPartitionsRDD[8] at mapPartitions at MXNet.scala:209)
19/01/25 18:07:57 INFO TaskSchedulerImpl: Adding task set 2.0 with 2 tasks
19/01/25 18:07:57 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 9, localhost, partition 0,NODE_LOCAL, 2355 bytes)
19/01/25 18:07:57 INFO TaskSetManager: Starting task 1.0 in stage 2.0 (TID 10, localhost, partition 1,NODE_LOCAL, 2355 bytes)
19/01/25 18:07:57 INFO Executor: Running task 0.0 in stage 2.0 (TID 9)
19/01/25 18:07:57 INFO Executor: Running task 1.0 in stage 2.0 (TID 10)
19/01/25 18:07:57 INFO CacheManager: Partition rdd_8_0 not found, computing it
19/01/25 18:07:57 INFO CacheManager: Partition rdd_8_1 not found, computing it
19/01/25 18:07:57 INFO ShuffleBlockFetcherIterator: Getting 8 non-empty blocks out of 8 blocks
19/01/25 18:07:57 INFO ShuffleBlockFetcherIterator: Getting 8 non-empty blocks out of 8 blocks
19/01/25 18:07:57 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms
19/01/25 18:07:57 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 7 ms
19/01/25 18:08:01 WARN WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.NDArray was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
19/01/25 18:08:02 INFO MXNet: Launching worker ...
19/01/25 18:08:02 INFO MXNet: Batch 128
19/01/25 18:08:06 INFO MXNet: Launching worker ...
19/01/25 18:08:06 INFO MXNet: Batch 128
19/01/25 18:08:22 ERROR Executor: Exception in task 1.0 in stage 2.0 (TID 10)
org.apache.mxnet.MXNetError: [18:08:22] include/mxnet/kvstore.h:273: compile with USE_DIST_KVSTORE=1 to init parameter server's environment
Stack trace returned 5 entries:
[bt] (0) 0 libmxnet.so 0x0000000124656ab5 dmlc::StackTrace() + 261
[bt] (1) 1 libmxnet.so 0x000000012465686f dmlc::LogMessageFatal::~LogMessageFatal() + 47
[bt] (2) 2 libmxnet.so 0x0000000125cd87c3 MXInitPSEnv + 659
[bt] (3) 3 mxnet-scala 0x000000012114c86b Java_org_apache_mxnet_LibInfo_mxInitPSEnv + 331
[bt] (4) 4 ??? 0x00000001055ea667 0x0 + 4385056359
at org.apache.mxnet.Base$.checkCall(Base.scala:111)
at org.apache.mxnet.KVStoreServer$.init(KVStoreServer.scala:108)
at org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$setupKVStore(MXNet.scala:190)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:228)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:209)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:69)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:268)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/01/25 18:08:22 WARN TaskSetManager: Lost task 1.0 in stage 2.0 (TID 10, localhost): org.apache.mxnet.MXNetError: [18:08:22] include/mxnet/kvstore.h:273: compile with USE_DIST_KVSTORE=1 to init parameter server's environment
Stack trace returned 5 entries:
[bt] (0) 0 libmxnet.so 0x0000000124656ab5 dmlc::StackTrace() + 261
[bt] (1) 1 libmxnet.so 0x000000012465686f dmlc::LogMessageFatal::~LogMessageFatal() + 47
[bt] (2) 2 libmxnet.so 0x0000000125cd87c3 MXInitPSEnv + 659
[bt] (3) 3 mxnet-scala 0x000000012114c86b Java_org_apache_mxnet_LibInfo_mxInitPSEnv + 331
[bt] (4) 4 ??? 0x00000001055ea667 0x0 + 4385056359
at org.apache.mxnet.Base$.checkCall(Base.scala:111)
at org.apache.mxnet.KVStoreServer$.init(KVStoreServer.scala:108)
at org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$setupKVStore(MXNet.scala:190)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:228)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:209)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:69)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:268)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/01/25 18:08:22 ERROR TaskSetManager: Task 1 in stage 2.0 failed 1 times; aborting job
19/01/25 18:08:22 INFO TaskSchedulerImpl: Cancelling stage 2
19/01/25 18:08:22 INFO Executor: Executor is trying to kill task 0.0 in stage 2.0 (TID 9)
19/01/25 18:08:22 INFO TaskSchedulerImpl: Stage 2 was cancelled
19/01/25 18:08:22 INFO DAGScheduler: ResultStage 2 (foreachPartition at MXNet.scala:238) failed in 24.387 s
19/01/25 18:08:22 INFO DAGScheduler: Job 1 failed: foreachPartition at MXNet.scala:238, took 26.904674 s
Job aborted due to stage failure: Task 1 in stage 2.0 failed 1 times, most recent failure: Lost task 1.0 in stage 2.0 (TID 10, localhost): org.apache.mxnet.MXNetError: [18:08:22] include/mxnet/kvstore.h:273: compile with USE_DIST_KVSTORE=1 to init parameter server's environment
Stack trace returned 5 entries:
[bt] (0) 0 libmxnet.so 0x0000000124656ab5 dmlc::StackTrace() + 261
[bt] (1) 1 libmxnet.so 0x000000012465686f dmlc::LogMessageFatal::~LogMessageFatal() + 47
[bt] (2) 2 libmxnet.so 0x0000000125cd87c3 MXInitPSEnv + 659
[bt] (3) 3 mxnet-scala 0x000000012114c86b Java_org_apache_mxnet_LibInfo_mxInitPSEnv + 331
[bt] (4) 4 ??? 0x00000001055ea667 0x0 + 4385056359
at org.apache.mxnet.Base$.checkCall(Base.scala:111)
at org.apache.mxnet.KVStoreServer$.init(KVStoreServer.scala:108)
at org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$setupKVStore(MXNet.scala:190)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:228)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:209)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:69)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:268)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Driver stacktrace:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 2.0 failed 1 times, most recent failure: Lost task 1.0 in stage 2.0 (TID 10, localhost): org.apache.mxnet.MXNetError: [18:08:22] include/mxnet/kvstore.h:273: compile with USE_DIST_KVSTORE=1 to init parameter server's environment
Stack trace returned 5 entries:
[bt] (0) 0 libmxnet.so 0x0000000124656ab5 dmlc::StackTrace() + 261
[bt] (1) 1 libmxnet.so 0x000000012465686f dmlc::LogMessageFatal::~LogMessageFatal() + 47
[bt] (2) 2 libmxnet.so 0x0000000125cd87c3 MXInitPSEnv + 659
[bt] (3) 3 mxnet-scala 0x000000012114c86b Java_org_apache_mxnet_LibInfo_mxInitPSEnv + 331
[bt] (4) 4 ??? 0x00000001055ea667 0x0 + 4385056359
at org.apache.mxnet.Base$.checkCall(Base.scala:111)
at org.apache.mxnet.KVStoreServer$.init(KVStoreServer.scala:108)
at org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$setupKVStore(MXNet.scala:190)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:228)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:209)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:69)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:268)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:920)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:918)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:918)
at org.apache.mxnet.spark.MXNet.trainModel(MXNet.scala:238)
at org.apache.mxnet.spark.MXNet.fit(MXNet.scala:260)
at org.apache.mxnet.spark.MXNetGeneralSuite$$anonfun$3.apply(MXNetGeneralSuite.scala:63)
at org.apache.mxnet.spark.MXNetGeneralSuite$$anonfun$3.apply(MXNetGeneralSuite.scala:61)
at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186)
at org.scalatest.TestSuite$class.withFixture(TestSuite.scala:196)
at org.scalatest.FunSuite.withFixture(FunSuite.scala:1560)
at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:183)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:196)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:196)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:196)
at org.apache.mxnet.spark.MXNetGeneralSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(MXNetGeneralSuite.scala:30)
at org.scalatest.BeforeAndAfterEach$class.runTest(BeforeAndAfterEach.scala:221)
at org.apache.mxnet.spark.MXNetGeneralSuite.runTest(MXNetGeneralSuite.scala:30)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:229)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:229)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
at scala.collection.immutable.List.foreach(List.scala:381)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:229)
at org.scalatest.FunSuite.runTests(FunSuite.scala:1560)
at org.scalatest.Suite$class.run(Suite.scala:1147)
at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1560)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:233)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:233)
at org.scalatest.SuperEngine.runImpl(Engine.scala:521)
at org.scalatest.FunSuiteLike$class.run(FunSuiteLike.scala:233)
at org.apache.mxnet.spark.MXNetGeneralSuite.org$scalatest$BeforeAndAfterAll$$super$run(MXNetGeneralSuite.scala:30)
at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:210)
at org.apache.mxnet.spark.MXNetGeneralSuite.run(MXNetGeneralSuite.scala:30)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1340)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1334)
at scala.collection.immutable.List.foreach(List.scala:381)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1334)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1011)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1010)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1500)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
at org.scalatest.tools.Runner$.run(Runner.scala:850)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:131)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28)
Caused by: org.apache.mxnet.MXNetError: [18:08:22] include/mxnet/kvstore.h:273: compile with USE_DIST_KVSTORE=1 to init parameter server's environment
Stack trace returned 5 entries:
[bt] (0) 0 libmxnet.so 0x0000000124656ab5 dmlc::StackTrace() + 261
[bt] (1) 1 libmxnet.so 0x000000012465686f dmlc::LogMessageFatal::~LogMessageFatal() + 47
[bt] (2) 2 libmxnet.so 0x0000000125cd87c3 MXInitPSEnv + 659
[bt] (3) 3 mxnet-scala 0x000000012114c86b Java_org_apache_mxnet_LibInfo_mxInitPSEnv + 331
[bt] (4) 4 ??? 0x00000001055ea667 0x0 + 4385056359
at org.apache.mxnet.Base$.checkCall(Base.scala:111)
at org.apache.mxnet.KVStoreServer$.init(KVStoreServer.scala:108)
at org.apache.mxnet.spark.MXNet.org$apache$mxnet$spark$MXNet$$setupKVStore(MXNet.scala:190)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:228)
at org.apache.mxnet.spark.MXNet$$anonfun$1.apply(MXNet.scala:209)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$20.apply(RDD.scala:710)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:69)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:268)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/01/25 18:08:22 INFO SparkUI: Stopped Spark web UI at http://192.168.0.20:4041
19/01/25 18:08:22 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/01/25 18:08:22 INFO MemoryStore: MemoryStore cleared
19/01/25 18:08:22 INFO BlockManager: BlockManager stopped
19/01/25 18:08:22 INFO BlockManagerMaster: BlockManagerMaster stopped
19/01/25 18:08:22 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
19/01/25 18:08:22 INFO SparkContext: Successfully stopped SparkContext
19/01/25 18:08:22 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
19/01/25 18:08:22 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
19/01/25 18:08:22 INFO NativeLibraryLoader: Deleting /var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/mxnet2451649713761743801/mxnet-scala
19/01/25 18:08:22 INFO NativeLibraryLoader: Deleting /var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/mxnet2451649713761743801/libmxnet.so
19/01/25 18:08:22 INFO NativeLibraryLoader: Deleting /var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/mxnet2451649713761743801
19/01/25 18:08:22 INFO ShutdownHookManager: Shutdown hook called
19/01/25 18:08:22 INFO ShutdownHookManager: Deleting directory /private/var/folders/q5/fkq1x5gj5l73cznw53c263wr0000gn/T/spark-b3cbf154-c098-49dc-8232-3a97b7e43bc3
Process finished with exit code 0
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services