[GitHub] [spark] mridulm commented on pull request #35906: [SPARK-33236][shuffle] Enable Push-based shuffle service to store state in NM level DB for work preserving restart

[GitBox <gi...@apache.org>]

mridulm commented on PR #35906:
URL: https://github.com/apache/spark/pull/35906#issuecomment-1202125264

   Updating PR based on discussion with @Ngone51: 
   
   Let us keep this as-is, since it is only relevant for yarn NM case, where rolling restart happens.
   As part of your ongoing testing @zhouyejoe, please do file follow up jira/pr's in case any issues are found - we want to make sure for 3.4 release this change will not cause issues.
   
   If you think we wont be able to stress test this before 3.4, let us flag guard it - if required, default to `true`, but allowing users to disable it. Thoughts @zhouyejoe ?
   
   (Please let me know if I missed anything from our discussion @Ngone51)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org