You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/05/31 03:30:05 UTC

[GitHub] [spark] zuston commented on pull request #35683: [SPARK-30835][SPARK-39018][CORE][YARN] Add support for YARN decommissioning when ESS is disabled

zuston commented on PR #35683:
URL: https://github.com/apache/spark/pull/35683#issuecomment-1141631786

   Thanks for @abhishekd0907 to submit this PR. I think this is a good improvement for the Spark jobs' stability, especially in the scenario where we implement elastic Yarn NMs on K8s.
   
   > we do not want to move shuffle blocks around if decomissioning is enabled when external shuffle service is in use.
   > 
   
   @mridulm 
   But in our internal cluster, we deployed the ess on Yarn nodemangers, and sometimes nm will  be gracefully decommissioned, it means that the ess will be invalid on these nms. 
   
   So why not allow trigger decommission event to executors when ess is in use?
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org