[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33378: [SPARK-32922][SHUFFLE][CORE] Fixes few issues when the executor tries to fetch push-merged blocks

[GitBox <gi...@apache.org>]

dongjoon-hyun commented on a change in pull request #33378:
URL: https://github.com/apache/spark/pull/33378#discussion_r670975288



##########
File path: core/src/main/scala/org/apache/spark/storage/PushBasedFetchHelper.scala
##########
@@ -197,9 +197,12 @@ private class PushBasedFetchHelper(
           localShuffleMergerBlockMgrId)
       }
     } else {
-      logDebug(s"Asynchronous fetch the push-merged-local blocks without cached merged dirs")
-      hostLocalDirManager.getHostLocalDirs(localShuffleMergerBlockMgrId.host,
-        localShuffleMergerBlockMgrId.port, Array(SHUFFLE_MERGER_IDENTIFIER)) {
+      // Push-based shuffle is only enabled when the ESS is enabled. If ESS is not enabled, then

Review comment:
       Shall we use `external shuffle service` instead of `ESS`? Someone uses ESS as external shuffle storage.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org