You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by shahid ashraf <sh...@trialx.com> on 2015/12/11 19:46:54 UTC
HDFS
hi Folks
I am using standalone cluster of 50 servers on aws. i loaded data on hdfs,
why i am getting Locality Level as ANY for data on hdfs, i have 900+
partitions.
--
with Regards
Shahid Ashraf
Re: HDFS
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Try to set the spark.locality.wait to a higher number and see if things
change. You can read more about the configuration properties from here
http://spark.apache.org/docs/latest/configuration.html#scheduling
Thanks
Best Regards
On Sat, Dec 12, 2015 at 12:16 AM, shahid ashraf <sh...@trialx.com> wrote:
> hi Folks
>
> I am using standalone cluster of 50 servers on aws. i loaded data on hdfs,
> why i am getting Locality Level as ANY for data on hdfs, i have 900+
> partitions.
>
>
> --
> with Regards
> Shahid Ashraf
>