[GitHub] [spark] khogeland commented on issue #26687: [SPARK-30055][k8s] Allow configuration of restart policy for Kubernetes pods

[GitBox <gi...@apache.org>]

khogeland commented on issue #26687: [SPARK-30055][k8s] Allow configuration of restart policy for Kubernetes pods
URL: https://github.com/apache/spark/pull/26687#issuecomment-596726593
 
 
   Good call, @holdenk, I found a conflict here while digging through the dynamic allocation code. The restart policy itself shouldn't mess up the dynamic scaling, but the allocator maps executor IDs to pods 1:1 via the label selector, so randomizing the executor ID on startup doesn't work. I would need to add some hacky string manipulation on the ID to get the label back out. 
   
   Considering all the tricky edge cases that executors presents, and with https://github.com/apache/spark/pull/27568 handling the silent failure scenarios, I think it's best to reduce this PR's scope to the much more straightforward driver restart policy.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org