You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/03/29 06:27:03 UTC

[GitHub] [spark] abhishekd0907 commented on pull request #35683: [SPARK-30835][CORE][YARN] Add support for YARN decommissioning & pre-emption

abhishekd0907 commented on pull request #35683:
URL: https://github.com/apache/spark/pull/35683#issuecomment-1081453295


   > What happens to nodes which are getting decomissioned, which have shuffle data on them (but no active executors) ?
   
   @mridulm 
   If decommissioning is enabled, shuffle blocks are already migrated to another node before the executor exits. However, this is best effort. So shuffle data might be left if shuffle blocks migration is left incomplete or the executor doesn't exit cleanly. Can you please clarify what are the expectations for shuffle data when a node is put in decommissioning state?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org