[GitHub] [spark] jiangxb1987 commented on a change in pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

[GitBox <gi...@apache.org>]

jiangxb1987 commented on a change in pull request #29032:
URL: https://github.com/apache/spark/pull/29032#discussion_r455524790



##########
File path: core/src/main/scala/org/apache/spark/deploy/client/StandaloneAppClient.scala
##########
@@ -181,7 +182,8 @@ private[spark] class StandaloneAppClient(
         if (ExecutorState.isFinished(state)) {
           listener.executorRemoved(fullId, message.getOrElse(""), exitStatus, workerLost)
         } else if (state == ExecutorState.DECOMMISSIONED) {
-          listener.executorDecommissioned(fullId, message.getOrElse(""))
+          listener.executorDecommissioned(fullId,
+            ExecutorDecommissionInfo(message.getOrElse(""), isHostDecommissioned = workerLost))

Review comment:
       how is the flag `isHostDecommissioned` actually used?

##########
File path: core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala
##########
@@ -101,7 +101,8 @@ private[spark] trait TaskScheduler {
   /**
    * Process a decommissioning executor.
    */
-  def executorDecommission(executorId: String): Unit
+  def executorDecommission(

Review comment:
       nit: don't leave an empty implementation here.

##########
File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
##########
@@ -191,9 +191,9 @@ class CoarseGrainedSchedulerBackend(scheduler: TaskSchedulerImpl, val rpcEnv: Rp
         executorDataMap.get(executorId).foreach(_.executorEndpoint.send(StopExecutor))
         removeExecutor(executorId, reason)
 
-      case DecommissionExecutor(executorId) =>
+      case DecommissionExecutor(executorId, decommissionInfo) =>
         logError(s"Received decommission executor message ${executorId}.")

Review comment:
       do we want to also include the decommissionInfo in the error msg?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org