[GitHub] [spark] Ngone51 commented on a diff in pull request #36665: [SPARK-39287][CORE] TaskSchedulerImpl should quickly ignore task finished event if its task was finished state.

[GitBox <gi...@apache.org>]

Ngone51 commented on code in PR #36665:
URL: https://github.com/apache/spark/pull/36665#discussion_r885746852


##########
core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala:
##########
@@ -102,6 +102,10 @@ private[spark] class TaskResultGetter(sparkEnv: SparkEnv, scheduler: TaskSchedul
               (deserializedResult, size)
           }
 
+          // quickly return if the task has finished
+          if (scheduler.isFinishedTask(taskSetManager, tid)) {
+            return

Review Comment:
   This's only useful when you hit the race condition, which is only a corner case. And in SPARK-37300, I think our target is to fix the bug issue but not for improvement. I personally think this introduces more complexity compared to the benefit we could get.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org