[GitHub] spark pull request #19832: [SPARK-22628][CORE]Some situations��� the assignm...

[srowen <gi...@git.apache.org>]

Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19832#discussion_r205843695
  
    --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala ---
    @@ -671,10 +671,23 @@ private[deploy] class Master(
           // If the cores left is less than the coresPerExecutor,the cores left will not be allocated
           if (app.coresLeft >= coresPerExecutor) {
             // Filter out workers that don't have enough resources to launch an executor
    -        val usableWorkers = workers.toArray.filter(_.state == WorkerState.ALIVE)
    +        var usableWorkers = workers.toArray.filter(_.state == WorkerState.ALIVE)
               .filter(worker => worker.memoryFree >= app.desc.memoryPerExecutorMB &&
                 worker.coresFree >= coresPerExecutor)
               .sortBy(_.coresFree).reverse
    +
    +        if (spreadOutApps) {
    +          def hasContainWorkersCount(worker: WorkerInfo): Int = {
    --- End diff --
    
    Nit: this can just be `app.executors.values.count(_.worker == worker)`. It's simple enough too that you don't need a method.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org