[spark] branch master updated: [SPARK-26939][CORE][DOC] Fix some outdated comments about task schedulers

[sr...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

srowen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new ce3a157  [SPARK-26939][CORE][DOC] Fix some outdated comments about task schedulers
ce3a157 is described below

commit ce3a157f00e052ca4dc0c6d4ef282cb21588837e
Author: seancxmao <se...@gmail.com>
AuthorDate: Sat Feb 23 10:52:53 2019 -0600

    [SPARK-26939][CORE][DOC] Fix some outdated comments about task schedulers
    
    ## What changes were proposed in this pull request?
    This PR aims to fix some outdated comments about task schedulers.
    
    1. Change "ClusterScheduler" to "YarnScheduler" in comments of `YarnClusterScheduler`
    
    According to [SPARK-1140 Remove references to ClusterScheduler](https://issues.apache.org/jira/browse/SPARK-1140), ClusterScheduler is not used anymore.
    
    I also searched "ClusterScheduler" within the whole project, no other occurrences are found in comments or test cases. Note classes like `YarnClusterSchedulerBackend` or `MesosClusterScheduler` are not relevant.
    
    2. Update comments about `statusUpdate` from `TaskSetManager`
    `statusUpdate` has been moved to `TaskSchedulerImpl`. StatusUpdate event handling is delegated to `handleSuccessfulTask`/`handleFailedTask`.
    
    ## How was this patch tested?
    N/A. Fix comments only.
    
    Closes #23844 from seancxmao/taskscheduler-comments.
    
    Authored-by: seancxmao <se...@gmail.com>
    Signed-off-by: Sean Owen <se...@databricks.com>
---
 core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala    | 3 ++-
 .../org/apache/spark/scheduler/cluster/YarnClusterScheduler.scala      | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala b/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala
index b7bf069..453939a 100644
--- a/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala
+++ b/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala
@@ -38,7 +38,8 @@ import org.apache.spark.util.collection.MedianHeap
  * each task, retries tasks if they fail (up to a limited number of times), and
  * handles locality-aware scheduling for this TaskSet via delay scheduling. The main interfaces
  * to it are resourceOffer, which asks the TaskSet whether it wants to run a task on one node,
- * and statusUpdate, which tells it that one of its tasks changed state (e.g. finished).
+ * and handleSuccessfulTask/handleFailedTask, which tells it that one of its tasks changed state
+ *  (e.g. finished/failed).
  *
  * THREADING: This class is designed to only be called from code with a lock on the
  * TaskScheduler (e.g. its event handlers). It should not be called from other threads.
diff --git a/resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterScheduler.scala b/resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterScheduler.scala
index 96c9151..1f622a0 100644
--- a/resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterScheduler.scala
+++ b/resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterScheduler.scala
@@ -21,7 +21,7 @@ import org.apache.spark._
 import org.apache.spark.deploy.yarn.ApplicationMaster
 
 /**
- * This is a simple extension to ClusterScheduler - to ensure that appropriate initialization of
+ * This is a simple extension to YarnScheduler - to ensure that appropriate initialization of
  * ApplicationMaster, etc is done
  */
 private[spark] class YarnClusterScheduler(sc: SparkContext) extends YarnScheduler(sc) {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org