You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by tg...@apache.org on 2016/06/23 14:00:47 UTC
spark git commit: [SPARK-16138] Try to cancel executor requests only
if we have at least 1
Repository: spark
Updated Branches:
refs/heads/master 5eef1e6c6 -> 5bf2889bf
[SPARK-16138] Try to cancel executor requests only if we have at least 1
## What changes were proposed in this pull request?
Adding additional check to if statement
## How was this patch tested?
I built and deployed to internal cluster to observe behaviour. After the change the invalid logging is gone:
```
16/06/22 08:46:36 INFO yarn.YarnAllocator: Driver requested a total number of 1 executor(s).
16/06/22 08:46:36 INFO yarn.YarnAllocator: Canceling requests for 1 executor container(s) to have a new desired total 1 executors.
16/06/22 08:46:36 INFO yarn.YarnAllocator: Driver requested a total number of 0 executor(s).
16/06/22 08:47:36 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 1.
```
Author: Peter Ableda <ab...@gmail.com>
Closes #13850 from peterableda/patch-2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5bf2889b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/5bf2889b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/5bf2889b
Branch: refs/heads/master
Commit: 5bf2889bfcfd776e7ee1369443a0474421a800bd
Parents: 5eef1e6
Author: Peter Ableda <ab...@gmail.com>
Authored: Thu Jun 23 09:00:31 2016 -0500
Committer: Tom Graves <tg...@yahoo-inc.com>
Committed: Thu Jun 23 09:00:31 2016 -0500
----------------------------------------------------------------------
.../main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/5bf2889b/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
----------------------------------------------------------------------
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
index 1b80071..dbdac33 100644
--- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
+++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
@@ -352,7 +352,7 @@ private[yarn] class YarnAllocator(
logInfo(s"Submitted container request (host: ${hostStr(request)}, capability: $resource)")
}
- } else if (missing < 0) {
+ } else if (numPendingAllocate > 0 && missing < 0) {
val numToCancel = math.min(numPendingAllocate, -missing)
logInfo(s"Canceling requests for $numToCancel executor container(s) to have a new desired " +
s"total $targetNumExecutors executors.")
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org