[GitHub] [incubator-celeborn] waitinfuture commented on a diff in pull request #1448: [CELEBORN-541][FOLLOWUP] handleGetReducerFileGroup occupy too much RPC thread cause other RPC can't been handled

["waitinfuture (via GitHub)" <gi...@apache.org>]

waitinfuture commented on code in PR #1448:
URL: https://github.com/apache/incubator-celeborn/pull/1448#discussion_r1173741913


##########
client/src/main/scala/org/apache/celeborn/client/commit/ReducePartitionCommitHandler.scala:
##########
@@ -98,8 +98,13 @@ class ReducePartitionCommitHandler(
     getReducerFileGroupRequest synchronized {
       stageEndShuffleSet.add(shuffleId)
     }
-    getReducerFileGroupRequest.remove(shuffleId)
-      .asScala.foreach(replyGetReducerFileGroup(_, shuffleId))
+
+    val requests = getReducerFileGroupRequest.remove(shuffleId)
+    // Set empty HashSet during register shuffle, but handleStageEnd may call
+    // un-registered shuffle, here still need a null check

Review Comment:
   The comment can be changed to :
   In case of stage with no shuffle data, ```registerShuffle``` will not be called, so here we still need to check null.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org