Using spark in cluster mode

[masoom alam <ma...@wanclouds.net>]

Dear all

I want to setup spark in cluster mode. The problem is that each worker node
is looking for a file to process.....in its local directory.....is it
possible to setup some thing hdfs so that each worker node take  its part
of a file from hdfs....any good tutorials for this?

Thanks

Re: Using spark in cluster mode

[Jacek Laskowski <ja...@japila.pl>]

Hi,

Start here -> http://spark.apache.org/docs/latest/programming-guide.html#resilient-distributed-datasets-rdds
and then hop to
http://spark.apache.org/docs/latest/spark-standalone.html. Once done,
be back with your questions. I think it's gonna help a lot.

Pozdrawiam,
Jacek

--
Jacek Laskowski | http://blog.japila.pl | http://blog.jaceklaskowski.pl
Follow me at https://twitter.com/jaceklaskowski
Upvote at http://stackoverflow.com/users/1305344/jacek-laskowski


On Tue, Oct 20, 2015 at 5:48 PM, masoom alam <ma...@wanclouds.net> wrote:
> Dear all
>
> I want to setup spark in cluster mode. The problem is that each worker node
> is looking for a file to process.....in its local directory.....is it
> possible to setup some thing hdfs so that each worker node take  its part of
> a file from hdfs....any good tutorials for this?
>
> Thanks

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org