You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Jia Zou <ja...@gmail.com> on 2016/01/19 15:39:47 UTC

Can I configure Spark on multiple nodes using local filesystem on each node?

Dear all,

Can I configure Spark on multiple nodes without HDFS, so that output data
will be written to the local file system on each node?

I guess there is no such feature in Spark, but just want to confirm.

Best Regards,
Jia

Re: Can I configure Spark on multiple nodes using local filesystem on each node?

Posted by Pavel Plotnikov <pa...@team.wrike.com>.

Hi,

I'm using Spark in standalone mode without HDFS, and shared folder is
mounted on nodes via nfs. It looks like each node write data like in local
file system.

Regards,
Pavel

On Tue, Jan 19, 2016 at 5:39 PM Jia Zou <ja...@gmail.com> wrote:

> Dear all,
>
> Can I configure Spark on multiple nodes without HDFS, so that output data
> will be written to the local file system on each node?
>
> I guess there is no such feature in Spark, but just want to confirm.
>
> Best Regards,
> Jia
>