[GitHub] [spark] Kimahriman commented on pull request #35683: [SPARK-30835][SPARK-39018][CORE][YARN] Add support for YARN decommissioning when ESS is disabled

[GitBox <gi...@apache.org>]

Kimahriman commented on PR #35683:
URL: https://github.com/apache/spark/pull/35683#issuecomment-1126033055

   > Let us add shuffle service enabled == false as well for STORAGE_DECOMMISSION_SHUFFLE_BLOCKS_ENABLED, until this is supported in the context of shuffle blocks.
   
   We've started looking into handling decommissioning in our cluster, so just seeing this and catching up. It looks like currently decommissioning in yarn is completely disabled in this PR if the shuffle service is used, but based on this comment did you mean/would it make more sense to disable storage migration if the shuffle service is enabled, irrespective of the cluster manager, so you'd still get some clean decommissioning of executors in yarn even if the shuffle service is enabled? I'm guessing migrating shuffle blocks from the shuffle service won't be as simple of a task.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org