[GitHub] [spark] hiboyang edited a comment on pull request #34672: [SPARK-37394][CORE] Skip registering with ESS if a customized shuffle manager is configured

[GitBox <gi...@apache.org>]

hiboyang edited a comment on pull request #34672:
URL: https://github.com/apache/spark/pull/34672#issuecomment-1012360636


   Thanks folks for keeping working on this! Current ShuffleManager API is actually pretty good in Spark, agree we should make it as public API (maybe with small modification as discussed here).
   
   ShuffleBlockResolver is tightly coupled with current open source Spark shuffle implementation to get shuffle blocks, other shuffle implementation may not need it, e.g., Uber's RSS (Zeus) does not need it. It will be nice to make ShuffleBlockResolver optional in ShuffleManager, or even remove it from public API.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org