You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/09/14 07:06:11 UTC

[GitHub] [spark] cloud-fan commented on a change in pull request #29722: [SPARK-32850][CORE] Simplify the RPC message flow of decommission

cloud-fan commented on a change in pull request #29722:
URL: https://github.com/apache/spark/pull/29722#discussion_r487692255



##########
File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala
##########
@@ -95,8 +95,13 @@ private[spark] object CoarseGrainedClusterMessages {
   case class RemoveExecutor(executorId: String, reason: ExecutorLossReason)
     extends CoarseGrainedClusterMessage
 
-  case class DecommissionExecutor(executorId: String, decommissionInfo: ExecutorDecommissionInfo)
-    extends CoarseGrainedClusterMessage
+  // A message that sent from executor to driver to tell driver that the executor has been
+  // used. It's used for the case where decommission is triggered at executor (e.g., K8S)

Review comment:
       `has been used` -> `has been decommissioned`

##########
File path: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
##########
@@ -245,15 +245,19 @@ private[deploy] class Master(
       logError("Leadership has been revoked -- master shutting down.")
       System.exit(0)
 
-    case WorkerDecommission(id, workerRef) =>
-      logInfo("Recording worker %s decommissioning".format(id))
+    case WorkerDecommissioned(id, workerRef) =>
       if (state == RecoveryState.STANDBY) {
         workerRef.send(MasterInStandby)
       } else {
-        // We use foreach since get gives us an option and we can skip the failures.

Review comment:
       shall we keep this comment?

##########
File path: core/src/main/scala/org/apache/spark/ExecutorAllocationClient.scala
##########
@@ -109,14 +110,22 @@ private[spark] trait ExecutorAllocationClient {
    * @param executorId identifiers of executor to decommission
    * @param decommissionInfo information about the decommission (reason, host loss)
    * @param adjustTargetNumExecutors if we should adjust the target number of executors.
+   * @param triggeredByExecutor   whether the decommission is triggered by sending the
+   *                                 `DecommissionExecutor` from driver to executor

Review comment:
       then the name should be `triggeredByDriver`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org