You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Thomas Graves (JIRA)" <ji...@apache.org> on 2017/05/26 16:38:04 UTC

[jira] [Created] (SPARK-20898) spark.blacklist.killBlacklistedExecutors doesn't work in YARN

Thomas Graves created SPARK-20898:
-------------------------------------

             Summary: spark.blacklist.killBlacklistedExecutors doesn't work in YARN
                 Key: SPARK-20898
                 URL: https://issues.apache.org/jira/browse/SPARK-20898
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.2.0
            Reporter: Thomas Graves


I was trying out the new spark.blacklist.killBlacklistedExecutors on YARN but it doesn't appear to work.  Everytime I get:

17/05/26 16:28:12 WARN BlacklistTracker: Not attempting to kill blacklisted executor id 4 since allocation client is not defined

Even though dynamic allocation is on.  Taking a quick look, I think the way it creates the blacklisttracker and passes the allocation client is wrong. The scheduler backend is 
 not set yet so it never passes the allocation client to the blacklisttracker correctly.  Thus it will never kill.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org