You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sebastien Rainville (JIRA)" <ji...@apache.org> on 2014/06/17 22:07:16 UTC
[jira] [Commented] (SPARK-2022) Spark 1.0.0 is failing if
mesos.coarse set to true
[ https://issues.apache.org/jira/browse/SPARK-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034294#comment-14034294 ]
Sebastien Rainville commented on SPARK-2022:
--------------------------------------------
I'm seeing the same behavior when trying to set spark.executor.extraLibraryPath:
in conf/spark-defaults.conf:
spark.executor.extraLibraryPath /usr/lib/hadoop/lib/native
the error message in stderr:
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0617 16:00:55.592289 27091 fetcher.cpp:73] Fetching URI 'hdfs://ca1-dcc1-0071:9200/user/sebastien/spark-1.0.0-bin-cdh4-sebr.tgz'
I0617 16:00:55.592428 27091 fetcher.cpp:99] Downloading resource from 'hdfs://ca1-dcc1-0071:9200/user/sebastien/spark-1.0.0-bin-cdh4-sebr.tgz' to '/u05/app/mesos/work/slaves/201311011608-1369465866-5050-9189-86/frameworks/20140416-011500-1369465866-5050-26096-0449/executors/9/runs/ba87d7b6-56c1-4892-9ed8-18fa8f8364d2/spark-1.0.0-bin-cdh4-sebr.tgz'
I0617 16:01:05.170714 27091 fetcher.cpp:61] Extracted resource '/u05/app/mesos/work/slaves/201311011608-1369465866-5050-9189-86/frameworks/20140416-011500-1369465866-5050-26096-0449/executors/9/runs/ba87d7b6-56c1-4892-9ed8-18fa8f8364d2/spark-1.0.0-bin-cdh4-sebr.tgz' into '/u05/app/mesos/work/slaves/201311011608-1369465866-5050-9189-86/frameworks/20140416-011500-1369465866-5050-26096-0449/executors/9/runs/ba87d7b6-56c1-4892-9ed8-18fa8f8364d2'
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0617 16:01:06.105363 27166 exec.cpp:131] Version: 0.18.0
I0617 16:01:06.112191 27175 exec.cpp:205] Executor registered on slave 201311011608-1369465866-5050-9189-86
Spark assembly has been built with Hive, including Datanucleus jars on classpath
Exception in thread "main" java.lang.NumberFormatException: For input string: "ca1-dcc1-0106.lab.mtl"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:492)
at java.lang.Integer.parseInt(Integer.java:527)
at scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229)
at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:135)
at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
here is the command in stdout:
Registered executor on ca1-dcc1-0106.lab.mtl
Starting task 9
Forked command at 27178
sh -c 'cd spark-1*; ./bin/spark-class org.apache.spark.executor.CoarseGrainedExecutorBackend -Djava.library.path=/usr/lib/hadoop/lib/native akka.tcp://spark@ca1-dcc1-0071.lab.mtl:32789/user/CoarseGrainedScheduler 201311011608-1369465866-5050-9189-86 ca1-dcc1-0106.lab.mtl 1'
Command exited with status 1 (pid: 27178)
> Spark 1.0.0 is failing if mesos.coarse set to true
> --------------------------------------------------
>
> Key: SPARK-2022
> URL: https://issues.apache.org/jira/browse/SPARK-2022
> Project: Spark
> Issue Type: Bug
> Components: Mesos
> Affects Versions: 1.0.0
> Reporter: Marek Wiewiorka
> Priority: Critical
>
> more stderr
> -------------------
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0603 16:07:53.721132 61192 exec.cpp:131] Version: 0.18.2
> I0603 16:07:53.725230 61200 exec.cpp:205] Executor registered on slave 201405220917-134217738-5050-27119-0
> Exception in thread "main" java.lang.NumberFormatException: For input string: "sparkseq003.cloudapp.net"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:492)
> at java.lang.Integer.parseInt(Integer.java:527)
> at scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229)
> at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
> at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:135)
> at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
> more stdout
> -----------------------
> Registered executor on sparkseq003.cloudapp.net
> Starting task 5
> Forked command at 61202
> sh -c '"/home/mesos/spark-1.0.0/bin/spark-class" org.apache.spark.executor.CoarseGrainedExecutorBackend -Dspark.mesos.coarse=true akka.tcp://spark@sparkseq001.cloudapp.net:40312/user/CoarseG
> rainedScheduler 201405220917-134217738-5050-27119-0 sparkseq003.cloudapp.net 4'
> Command exited with status 1 (pid: 61202)
--
This message was sent by Atlassian JIRA
(v6.2#6252)