[GitHub] spark pull request #17813: [SPARK-20540][CORE] Fix unstable executor request...

[skonto <gi...@git.apache.org>]

Github user skonto commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17813#discussion_r146831076
  
    --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ---
    @@ -589,8 +605,18 @@ class CoarseGrainedSchedulerBackend(scheduler: TaskSchedulerImpl, val rpcEnv: Rp
           // take into account executors that are pending to be added or removed.
           val adjustTotalExecutors =
             if (!replace) {
    -          doRequestTotalExecutors(
    -            numExistingExecutors + numPendingExecutors - executorsPendingToRemove.size)
    +          requestedTotalExecutors = math.max(requestedTotalExecutors - executorsToKill.size, 0)
    +          if (requestedTotalExecutors !=
    --- End diff --
    
    Hey I have seen the message while testing dynamic allocation on mesos:
    
    ```17/10/25 13:58:44 DEBUG MesosCoarseGrainedSchedulerBackend: killExecutors(ArrayBuffer(1), false, false): Executor counts do not match:
    requestedTotalExecutors  = 0
    numExistingExecutors     = 2
    numPendingExecutors      = 0
    executorsPendingToRemove = 1
    ```
    What should I expect here?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org