You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Junjie Qian <qi...@outlook.com> on 2016/02/17 17:20:11 UTC
Why no computations run on workers/slaves in cluster mode?
Hi all,
I am new to Spark, and have one problem that, no computations run on workers/slave_servers in the standalone cluster mode.
The Spark version is 1.6.0, and environment is CentOS. I run the example codes, e.g. https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/LinearRegression.scala#L117.
What I did: 1. setup slaves in ./conf/slaves, 2. setup the spark-env.sh file, 3. sbin/start-all.sh, 4. run the test program with spark-submit. Follow the link, http://spark.apache.org/docs/latest/spark-standalone.html.
Could anyone give some suggestions on this? Or the link to how setup this?
Many thanksJunjie Qian
Re: Why no computations run on workers/slaves in cluster mode?
Posted by Gourav Sengupta <go...@gmail.com>.
Hi,
Have you registered an application in the standalone cluster? This can also
happen if the data path that you are giving SPARK to access is only visible
in one system and not another. For example if I provide the data path as
"/abcd/*" and that path is available in only one system and not other then
the workers will only run from that system.
Regards,
Gourav Sengupta
On Wed, Feb 17, 2016 at 4:20 PM, Junjie Qian <qi...@outlook.com>
wrote:
> Hi all,
>
> I am new to Spark, and have one problem that, no computations run on
> workers/slave_servers in the standalone cluster mode.
>
> The Spark version is 1.6.0, and environment is CentOS. I run the example
> codes, e.g.
> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/LinearRegression.scala#L117
> .
>
> What I did: 1. setup slaves in ./conf/slaves, 2. setup the spark-env.sh
> file, 3. sbin/start-all.sh, 4. run the test program with spark-submit.
> Follow the link, http://spark.apache.org/docs/latest/spark-standalone.html
> .
>
> Could anyone give some suggestions on this? Or the link to how setup this?
>
> Many thanks
> Junjie Qian
>
Re: Why no computations run on workers/slaves in cluster mode?
Posted by Xiangrui Meng <me...@gmail.com>.
Did the test program finish and did you see any error messages from the
console? -Xiangrui
On Wed, Feb 17, 2016, 1:49 PM Junjie Qian <qi...@outlook.com> wrote:
> Hi all,
>
> I am new to Spark, and have one problem that, no computations run on
> workers/slave_servers in the standalone cluster mode.
>
> The Spark version is 1.6.0, and environment is CentOS. I run the example
> codes, e.g.
> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/LinearRegression.scala#L117
> .
>
> What I did: 1. setup slaves in ./conf/slaves, 2. setup the spark-env.sh
> file, 3. sbin/start-all.sh, 4. run the test program with spark-submit.
> Follow the link, http://spark.apache.org/docs/latest/spark-standalone.html
> .
>
> Could anyone give some suggestions on this? Or the link to how setup this?
>
> Many thanks
> Junjie Qian
>