[jira] [Created] (SPARK-16138) YarnAllocator tries to cancel executor requests when we have non

["Peter Ableda (JIRA)" <ji...@apache.org>]

Peter Ableda created SPARK-16138:
------------------------------------

             Summary: YarnAllocator tries to cancel executor requests when we have non
                 Key: SPARK-16138
                 URL: https://issues.apache.org/jira/browse/SPARK-16138
             Project: Spark
          Issue Type: Bug
            Reporter: Peter Ableda
            Priority: Minor


Easy to reproduce: 
$ spark-shell
> sc.parallelize(1 to 10000).count()

After the computation finishes we have 0 pending tasks so we will need 0 executors. 
{code}
16/06/21 04:07:41 INFO yarn.YarnAllocator: Driver requested a total number of 0 executor(s).
16/06/21 04:07:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor containers
16/06/21 04:07:41 WARN yarn.YarnAllocator: Expected to find pending requests, but found none.
...
16/06/21 04:08:41 INFO yarn.YarnAllocator: Canceling requests for 0 executor containers
16/06/21 04:08:41 WARN yarn.YarnAllocator: Expected to find pending requests, but found none.
16/06/21 04:08:41 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 1.
16/06/21 04:08:48 INFO yarn.YarnAllocator: Driver requested a total number of 0 executor(s).
16/06/21 04:08:48 INFO yarn.YarnAllocator: Canceling requests for 0 executor containers
16/06/21 04:08:48 WARN yarn.YarnAllocator: Expected to find pending requests, but found none.
16/06/21 04:08:48 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 2.
{code}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org