[GitHub] spark pull request #22221: [SPARK-25231] : Fix synchronization of executor h...

[kiszk <gi...@git.apache.org>]

Github user kiszk commented on a diff in the pull request:

    https://github.com/apache/spark/pull/22221#discussion_r212833953
  
    --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
    @@ -465,7 +465,7 @@ private[spark] class TaskSchedulerImpl(
         var reason: Option[ExecutorLossReason] = None
         synchronized {
           try {
    -        taskIdToTaskSetManager.get(tid) match {
    +        Option(taskIdToTaskSetManager.get(tid)) match {
    --- End diff --
    
    Good catch. I like this direction.
    
    I have a question about the change of semantics. By removing `synchronization` at `accumUpdatesWithTaskIds()`, a pair of operations in this `synchronized` `get()` and `remove()` in `cleanupTaskState()` is not atomic regarding `get`.
    Is this change ok?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org