You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by kyocum <ky...@illumina.com> on 2014/01/24 03:16:04 UTC
executor failed, cannot find compute-classpath.sh
Trying to run spark-shell from my laptop to a master node in a cluster. It
appears that if you've installed spark at loc A on a cluster and in loc B on
your local machine, the current app framework uses loc B as the path for
start up. Any way to config my way around this? TIA
The shell connects to the master and can connect to the workers:
Using Scala version 2.9.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_09)
Initializing interpreter...
Creating SparkContext...
14/01/23 18:07:36 INFO Slf4jEventHandler: Slf4jEventHandler started
14/01/23 18:07:36 INFO SparkEnv: Registering BlockManagerMaster
14/01/23 18:07:36 INFO DiskBlockManager: Created local directory at
/var/folders/d1/9h0hs71d0h112s1tk6vcbx540000gp/T/spark-local-20140123180736-bcf9
14/01/23 18:07:36 INFO MemoryStore: MemoryStore started with capacity 647.7
MB.
14/01/23 18:07:36 INFO ConnectionManager: Bound socket to port 59238 with id
= ConnectionManagerId(ilmn-coe-2.net,59238)
14/01/23 18:07:36 INFO BlockManagerMaster: Trying to register BlockManager
14/01/23 18:07:36 INFO BlockManagerMasterActor$BlockManagerInfo: Registering
block manager ilmn-coe-2.net:59238 with 647.7 MB RAM
14/01/23 18:07:36 INFO BlockManagerMaster: Registered BlockManager
14/01/23 18:07:36 INFO HttpBroadcast: Broadcast server started at
http://10.12.195.116:59239
14/01/23 18:07:36 INFO SparkEnv: Registering MapOutputTracker
14/01/23 18:07:36 INFO HttpFileServer: HTTP File server directory is
/var/folders/d1/9h0hs71d0h112s1tk6vcbx540000gp/T/spark-d683bfd4-333a-4610-be20-a7390aa8d0ba
14/01/23 18:07:36 INFO SparkUI: Started Spark Web UI at
http://ilmn-coe-2.net:4040
14/01/23 18:07:36 INFO Client$ClientActor: Connecting to master
spark://hnn05.net:7077...
2014-01-23 18:07:36.475 java[9793:6403] Unable to load realm info from
SCDynamicStore
Spark context available as sc.
14/01/23 18:07:36 INFO SparkDeploySchedulerBackend: Connected to Spark
cluster with app ID app-20140123180739-0021
14/01/23 18:07:36 INFO Client$ClientActor: Executor added:
app-20140123180739-0021/0 on worker-20140123164655-192.168.28.232-51898
(192.168.28.232:7077) with 4 cores
14/01/23 18:07:36 INFO SparkDeploySchedulerBackend: Granted executor ID
app-20140123180739-0021/0 on hostPort 192.168.28.232:7077 with 4 cores,
1024.0 MB RAM
14/01/23 18:07:36 INFO Client$ClientActor: Executor updated:
app-20140123180739-0021/0 is now RUNNING
14/01/23 18:07:36 INFO Client$ClientActor: Executor updated:
app-20140123180739-0021/0 is now FAILED (class java.io.IOException: Cannot
run program
"/Users/kyocum/spark/spark-0.8.1-incubating/bin/compute-classpath.sh" (in
directory "."): java.io.IOException: error=2, No such file or directory)
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: executor failed, cannot find compute-classpath.sh
Posted by Matei Zaharia <ma...@gmail.com>.
Hi Ken,
This is unfortunately a limitation of spark-shell and the way it works on the standalone mode. spark-shell sets an environment variable, SPARK_HOME, which tells Spark where to find its code installed on the cluster. This means that the path on your laptop must be the same as on the cluster, which is not the case. I recommend one of two things:
1) Either run spark-shell from a cluster node, where it will have the right path. (In general it’s also better for performance to have it close to the cluster)
2) Or, edit the spark-shell script and re-export SPARK_HOME right before it runs the Java command (ugly but will probably work).
Hopefully we’ll fix this in a future release.
Matei
On Jan 23, 2014, at 6:16 PM, kyocum <ky...@illumina.com> wrote:
> Trying to run spark-shell from my laptop to a master node in a cluster. It
> appears that if you've installed spark at loc A on a cluster and in loc B on
> your local machine, the current app framework uses loc B as the path for
> start up. Any way to config my way around this? TIA
>
> The shell connects to the master and can connect to the workers:
>
> Using Scala version 2.9.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_09)
> Initializing interpreter...
> Creating SparkContext...
> 14/01/23 18:07:36 INFO Slf4jEventHandler: Slf4jEventHandler started
> 14/01/23 18:07:36 INFO SparkEnv: Registering BlockManagerMaster
> 14/01/23 18:07:36 INFO DiskBlockManager: Created local directory at
> /var/folders/d1/9h0hs71d0h112s1tk6vcbx540000gp/T/spark-local-20140123180736-bcf9
> 14/01/23 18:07:36 INFO MemoryStore: MemoryStore started with capacity 647.7
> MB.
> 14/01/23 18:07:36 INFO ConnectionManager: Bound socket to port 59238 with id
> = ConnectionManagerId(ilmn-coe-2.net,59238)
> 14/01/23 18:07:36 INFO BlockManagerMaster: Trying to register BlockManager
> 14/01/23 18:07:36 INFO BlockManagerMasterActor$BlockManagerInfo: Registering
> block manager ilmn-coe-2.net:59238 with 647.7 MB RAM
> 14/01/23 18:07:36 INFO BlockManagerMaster: Registered BlockManager
> 14/01/23 18:07:36 INFO HttpBroadcast: Broadcast server started at
> http://10.12.195.116:59239
> 14/01/23 18:07:36 INFO SparkEnv: Registering MapOutputTracker
> 14/01/23 18:07:36 INFO HttpFileServer: HTTP File server directory is
> /var/folders/d1/9h0hs71d0h112s1tk6vcbx540000gp/T/spark-d683bfd4-333a-4610-be20-a7390aa8d0ba
> 14/01/23 18:07:36 INFO SparkUI: Started Spark Web UI at
> http://ilmn-coe-2.net:4040
> 14/01/23 18:07:36 INFO Client$ClientActor: Connecting to master
> spark://hnn05.net:7077...
> 2014-01-23 18:07:36.475 java[9793:6403] Unable to load realm info from
> SCDynamicStore
> Spark context available as sc.
> 14/01/23 18:07:36 INFO SparkDeploySchedulerBackend: Connected to Spark
> cluster with app ID app-20140123180739-0021
> 14/01/23 18:07:36 INFO Client$ClientActor: Executor added:
> app-20140123180739-0021/0 on worker-20140123164655-192.168.28.232-51898
> (192.168.28.232:7077) with 4 cores
> 14/01/23 18:07:36 INFO SparkDeploySchedulerBackend: Granted executor ID
> app-20140123180739-0021/0 on hostPort 192.168.28.232:7077 with 4 cores,
> 1024.0 MB RAM
> 14/01/23 18:07:36 INFO Client$ClientActor: Executor updated:
> app-20140123180739-0021/0 is now RUNNING
> 14/01/23 18:07:36 INFO Client$ClientActor: Executor updated:
> app-20140123180739-0021/0 is now FAILED (class java.io.IOException: Cannot
> run program
> "/Users/kyocum/spark/spark-0.8.1-incubating/bin/compute-classpath.sh" (in
> directory "."): java.io.IOException: error=2, No such file or directory)
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: executor failed, cannot find compute-classpath.sh
Posted by TimMalt <dw...@gmx.net>.
Hi,
has this issue been resolved? I am currently running into similar problems.
I am using spark-1.3.0-bin-hadoop2.4 on Windows and Ubuntu. I have setup all
path on my Windows machine in an identical manner as on my Ubuntu server
(using cygwin, so everything is somewhere under /usr/local/spark...).
However, when I run the PI example then I get this when starting the command
from cygwin:
bin/spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode
cluster --master spark://myServer:7077
file:///home/spark/spark-examples-1.3.0-hadoop2.4.0.jar
Driver successfully submitted as driver-20150416112240-0007
... waiting before polling master for driver state
... polling master for driver state
State of driver-20150416112240-0007 is ERROR
Exception from cluster was: org.apache.spark.SparkException: Process
List(/usr/local/spark-1.3.0-bin-hadoop2.4/bin/compute-classpath.sh) exited
with code 127
org.apache.spark.SparkException: Process
List(/usr/local/spark-1.3.0-bin-hadoop2.4/bin/compute-classpath.sh) exited
with code 127
When I use the exact same command on the server then the job runs just fine.
I confirmed that
/usr/local/spark-1.3.0-bin-hadoop2.4/bin/compute-classpath.sh runs on my
local machine and the server.
What is missing?
Thanks,
Tim
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859p22520.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: executor failed, cannot find compute-classpath.sh
Posted by Andrew Or <an...@databricks.com>.
Hi CJ,
Looks like I overlook a few lines in the spark shell case. It appears that
spark shell
explicitly overwrites
<https://github.com/apache/spark/blob/f4f46dec5ae1da48738b9b650d3de155b59c4674/repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala#L955>
"spark.home" to whatever "SPARK_HOME" is set to. I have filed
a JIRA to track this issue: https://issues.apache.org/jira/browse/SPARK-2454.
Until
then, you can workaround this by ensuring we don't set SPARK_HOME anywhere.
We currently do this in bin/spark-submit and bin/spark-class, so go ahead
and
remove these lines:
https://github.com/apache/spark/blob/f4f46dec5ae1da48738b9b650d3de155b59c4674/bin/spark-submit#L20
https://github.com/apache/spark/blob/f4f46dec5ae1da48738b9b650d3de155b59c4674/bin/spark-class#L31
In addition, make sure spark-submit no longer uses the SPARK_HOME variable
here:
https://github.com/apache/spark/blob/f4f46dec5ae1da48738b9b650d3de155b59c4674/bin/spark-submit#L44
Now, as you have done before, setting "spark.home" to your executor's spark
home
should do the job. I have verified that this solves the problem in my own
cluster.
To verify that your configs are in fact set, you can always run
bin/spark-submit (or
spark-shell, which calls spark-submit) with the --verbose flag.
Let me know if this fixes it. I will get to fixing the root problem soon.
Andrew
2014-07-10 18:43 GMT-07:00 cjwang <cj...@cjwang.us>:
> Andrew,
>
> Thanks for replying. I did the following and the result was still the
> same.
>
> 1. Added "spark.home /root/spark-1.0.0" to local conf/spark-defaults.conf,
> where "/root...." was the place in the cluster where I put Spark.
>
> 2. Ran "bin/spark-shell --master
> spark://sjc1-eng-float01.carrieriq.com:7077".
>
> 3. Sighed when I still saw the same error:
>
> 14/07/10 18:26:53 INFO AppClient$ClientActor: Executor updated:
> app-20140711012651-0007/5 is now FAILED (class java.io.IOException: Cannot
> run program "/Users/cwang/spark/bin/compute-classpath.sh" (in directory
> "."): error=2, No such file or directory)
>
> /Users/cwang/spark was my local SPARK_HOME, which is wrong.
>
> What did I do wrong? How do I know if the config file is taken?
>
> I am novice to Spark so spare with me.
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859p9378.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
Re: executor failed, cannot find compute-classpath.sh
Posted by cjwang <cj...@cjwang.us>.
Andrew,
Thanks for replying. I did the following and the result was still the same.
1. Added "spark.home /root/spark-1.0.0" to local conf/spark-defaults.conf,
where "/root...." was the place in the cluster where I put Spark.
2. Ran "bin/spark-shell --master
spark://sjc1-eng-float01.carrieriq.com:7077".
3. Sighed when I still saw the same error:
14/07/10 18:26:53 INFO AppClient$ClientActor: Executor updated:
app-20140711012651-0007/5 is now FAILED (class java.io.IOException: Cannot
run program "/Users/cwang/spark/bin/compute-classpath.sh" (in directory
"."): error=2, No such file or directory)
/Users/cwang/spark was my local SPARK_HOME, which is wrong.
What did I do wrong? How do I know if the config file is taken?
I am novice to Spark so spare with me.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859p9378.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: executor failed, cannot find compute-classpath.sh
Posted by Andrew Or <an...@databricks.com>.
Hi C.J.,
The PR Yana pointed out seems to fix this. However, it is not merged in
master yet, so for now I would recommend that you try the following
workaround: set "spark.home" to the executor's /path/to/spark. I provided
more detail here:
http://mail-archives.apache.org/mod_mbox/spark-user/201407.mbox/%3cCAMJOb8mYTzxrHWcaDOnVoOTw1TFrd9kJjOyj1=nKgmSK5Vsctg@mail.gmail.com%3e
Andrew
2014-07-10 1:57 GMT-07:00 cjwang <cj...@cjwang.us>:
> Not sure that was what I want. I tried to run Spark Shell on a machine
> other
> than the master and got the same error. The "192" was suppose to be a
> simple shell script change that alters SPARK_HOME before submitting jobs.
> Too bad it wasn't there anymore.
>
> The build described in the pull request (16440) seemed failed. So, I can't
> use it.
>
> I am looking for those shell script changes.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859p9277.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
Re: executor failed, cannot find compute-classpath.sh
Posted by cjwang <cj...@cjwang.us>.
Not sure that was what I want. I tried to run Spark Shell on a machine other
than the master and got the same error. The "192" was suppose to be a
simple shell script change that alters SPARK_HOME before submitting jobs.
Too bad it wasn't there anymore.
The build described in the pull request (16440) seemed failed. So, I can't
use it.
I am looking for those shell script changes.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859p9277.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: executor failed, cannot find compute-classpath.sh
Posted by Yana Kadiyska <ya...@gmail.com>.
https://github.com/apache/spark/pull/1244 I think is what you're looking for
On Wed, Jul 9, 2014 at 9:42 PM, cjwang <cj...@cjwang.us> wrote:
> The link:
>
> https://github.com/apache/incubator-spark/pull/192
>
> is no longer available. Could someone attach the solution or point me
> another location? Thanks.
>
> (I am using 1.0.0)
>
> C.J.
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859p9260.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: executor failed, cannot find compute-classpath.sh
Posted by cjwang <cj...@cjwang.us>.
The link:
https://github.com/apache/incubator-spark/pull/192
is no longer available. Could someone attach the solution or point me
another location? Thanks.
(I am using 1.0.0)
C.J.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859p9260.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: executor failed, cannot find compute-classpath.sh
Posted by Imran Rashid <im...@quantifind.com>.
if I understand your issue correctly, there is an open PR to deal w/ this
issue:
https://github.com/apache/incubator-spark/pull/192
though I dunno if its gonna get merged or not ...
On Thu, Jan 23, 2014 at 6:16 PM, kyocum <ky...@illumina.com> wrote:
> Trying to run spark-shell from my laptop to a master node in a cluster. It
> appears that if you've installed spark at loc A on a cluster and in loc B
> on
> your local machine, the current app framework uses loc B as the path for
> start up. Any way to config my way around this? TIA
>
> The shell connects to the master and can connect to the workers:
>
> Using Scala version 2.9.3 (Java HotSpot(TM) 64-Bit Server VM, Java
> 1.7.0_09)
> Initializing interpreter...
> Creating SparkContext...
> 14/01/23 18:07:36 INFO Slf4jEventHandler: Slf4jEventHandler started
> 14/01/23 18:07:36 INFO SparkEnv: Registering BlockManagerMaster
> 14/01/23 18:07:36 INFO DiskBlockManager: Created local directory at
>
> /var/folders/d1/9h0hs71d0h112s1tk6vcbx540000gp/T/spark-local-20140123180736-bcf9
> 14/01/23 18:07:36 INFO MemoryStore: MemoryStore started with capacity 647.7
> MB.
> 14/01/23 18:07:36 INFO ConnectionManager: Bound socket to port 59238 with
> id
> = ConnectionManagerId(ilmn-coe-2.net,59238)
> 14/01/23 18:07:36 INFO BlockManagerMaster: Trying to register BlockManager
> 14/01/23 18:07:36 INFO BlockManagerMasterActor$BlockManagerInfo:
> Registering
> block manager ilmn-coe-2.net:59238 with 647.7 MB RAM
> 14/01/23 18:07:36 INFO BlockManagerMaster: Registered BlockManager
> 14/01/23 18:07:36 INFO HttpBroadcast: Broadcast server started at
> http://10.12.195.116:59239
> 14/01/23 18:07:36 INFO SparkEnv: Registering MapOutputTracker
> 14/01/23 18:07:36 INFO HttpFileServer: HTTP File server directory is
>
> /var/folders/d1/9h0hs71d0h112s1tk6vcbx540000gp/T/spark-d683bfd4-333a-4610-be20-a7390aa8d0ba
> 14/01/23 18:07:36 INFO SparkUI: Started Spark Web UI at
> http://ilmn-coe-2.net:4040
> 14/01/23 18:07:36 INFO Client$ClientActor: Connecting to master
> spark://hnn05.net:7077...
> 2014-01-23 18:07:36.475 java[9793:6403] Unable to load realm info from
> SCDynamicStore
> Spark context available as sc.
> 14/01/23 18:07:36 INFO SparkDeploySchedulerBackend: Connected to Spark
> cluster with app ID app-20140123180739-0021
> 14/01/23 18:07:36 INFO Client$ClientActor: Executor added:
> app-20140123180739-0021/0 on worker-20140123164655-192.168.28.232-51898
> (192.168.28.232:7077) with 4 cores
> 14/01/23 18:07:36 INFO SparkDeploySchedulerBackend: Granted executor ID
> app-20140123180739-0021/0 on hostPort 192.168.28.232:7077 with 4 cores,
> 1024.0 MB RAM
> 14/01/23 18:07:36 INFO Client$ClientActor: Executor updated:
> app-20140123180739-0021/0 is now RUNNING
> 14/01/23 18:07:36 INFO Client$ClientActor: Executor updated:
> app-20140123180739-0021/0 is now FAILED (class java.io.IOException: Cannot
> run program
> "/Users/kyocum/spark/spark-0.8.1-incubating/bin/compute-classpath.sh" (in
> directory "."): java.io.IOException: error=2, No such file or directory)
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/executor-failed-cannot-find-compute-classpath-sh-tp859.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>