[GitHub] [spark] Ngone51 commented on a diff in pull request #42296: [SPARK-44635][CORE] Handle shuffle fetch failures in decommissions

["Ngone51 (via GitHub)" <gi...@apache.org>]

Ngone51 commented on code in PR #42296:
URL: https://github.com/apache/spark/pull/42296#discussion_r1321386699


##########
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala:
##########
@@ -1580,7 +1625,9 @@ object ShuffleBlockFetcherIterator {
    * Result of a fetch request that should be deferred for some reasons, e.g., Netty OOM

Review Comment:
   nit:
   ```suggestion
      * Result of a fetch request that should be deferred for some reasons, e.g., Netty OOM, decommission
   ```



##########
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala:
##########
@@ -1167,8 +1200,10 @@ final class ShuffleBlockFetcherIterator(
     // immediately, defer the request until the next time it can be processed.
 
     // Process any outstanding deferred fetch requests if possible.
+    // Skip when the address is the local BM Id (this may happen when shuffle migration is enabled)
     if (deferredFetchRequests.nonEmpty) {
-      for ((remoteAddress, defReqQueue) <- deferredFetchRequests) {
+      for ((remoteAddress, defReqQueue) <- deferredFetchRequests
+           if remoteAddress != blockManager.blockManagerId) {

Review Comment:
   nit 2 indents:
   ```suggestion
           if remoteAddress != blockManager.blockManagerId) {
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org