You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by shahid ashraf <sh...@trialx.com> on 2015/12/11 19:46:54 UTC

HDFS

hi Folks

I am using standalone cluster of 50 servers on aws. i loaded data on hdfs,
 why i am getting Locality Level as ANY for data on hdfs, i have 900+
partitions.


-- 
with Regards
Shahid Ashraf

Re: HDFS

Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Try to set the spark.locality.wait to a higher number and see if things
change. You can read more about the configuration properties from here
http://spark.apache.org/docs/latest/configuration.html#scheduling

Thanks
Best Regards

On Sat, Dec 12, 2015 at 12:16 AM, shahid ashraf <sh...@trialx.com> wrote:

> hi Folks
>
> I am using standalone cluster of 50 servers on aws. i loaded data on hdfs,
>  why i am getting Locality Level as ANY for data on hdfs, i have 900+
> partitions.
>
>
> --
> with Regards
> Shahid Ashraf
>