You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2026/06/16 15:20:23 UTC

[PR] [SPARK-57492] Use K8s Garbage Collection to delete executors only when restart is disabled [spark-kubernetes-operator]

dongjoon-hyun opened a new pull request, #708:
URL: https://github.com/apache/spark-kubernetes-operator/pull/708

   ### What changes were proposed in this pull request?
   
   This PR applies the `spark.kubernetes.executor.deleteOnTermination=false` override from
   SPARK-55352 only when the application does not restart, i.e. `RestartPolicy` is `Never` or
   unset, in addition to the existing static-allocation condition.
   
   ### Why are the changes needed?
   
   When `RestartPolicy=Always` (e.g. `examples/spark-history-server.yaml`), the driver pod can
   be recreated while old executor pods linger, so they accumulate instead of being garbage
   collected. Such applications must delete executors on termination explicitly.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   Pass the CIs with the updated `useGarbageCollectionToDeleteExecutors` unit test.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Claude Code (Claude Opus 4.8)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57492] Use K8s Garbage Collection to delete executors only when restart is disabled [spark-kubernetes-operator]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #708: [SPARK-57492] Use K8s Garbage Collection to delete executors only when restart is disabled
URL: https://github.com/apache/spark-kubernetes-operator/pull/708


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57492] Use K8s Garbage Collection to delete executors only when restart is disabled [spark-kubernetes-operator]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #708:
URL: https://github.com/apache/spark-kubernetes-operator/pull/708#issuecomment-4720848436

   Thank you, @peter-toth !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57492] Use K8s Garbage Collection to delete executors only when restart is disabled [spark-kubernetes-operator]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #708:
URL: https://github.com/apache/spark-kubernetes-operator/pull/708#issuecomment-4720855265

   Merged to main


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org