You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Tarun Kumar <ta...@sumologic.com> on 2017/10/12 19:31:12 UTC

Tasks not getting scheduled on specified executor host

I am specifying executer host for my tasks via
org.apache.hadoop.fs.LocatedFileStatus#locations
(which is org.apache.hadoop.fs.BlockLocation[])'s hosts field, still tasks
are getting scheduled on different executor host.

- Speculative execution is off.
- Also confirmed that locality wait time out is not an issue in this case.
Tasks are immediately getting scheduled on some other host.

What can make it happen? Thanks!

Re: Tasks not getting scheduled on specified executor host

Posted by Tarun Kumar <ta...@sumologic.com>.

Any response on this one?

On Fri, Oct 13, 2017 at 1:01 AM, Tarun Kumar <ta...@sumologic.com> wrote:

> I am specifying executer host for my tasks via org.apache.hadoop.fs.LocatedFileStatus#locations
> (which is org.apache.hadoop.fs.BlockLocation[])'s hosts field, still
> tasks are getting scheduled on different executor host.
>
> - Speculative execution is off.
> - Also confirmed that locality wait time out is not an issue in this case.
> Tasks are immediately getting scheduled on some other host.
>
> What can make it happen? Thanks!
>