You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Henry Yu (JIRA)" <ji...@apache.org> on 2019/05/14 06:12:00 UTC
[jira] [Updated] (SPARK-27696) kubernetes driver pod not deleted
after finish.
[ https://issues.apache.org/jira/browse/SPARK-27696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henry Yu updated SPARK-27696:
-----------------------------
Description:
When submit to k8s, driver pod not deleted after job completion.
While k8s checks driver pod name not existing, It is especially painful when we use workflow tool to resubmit the failed spark job. (by the way, client always exist with 0 is another painful issue)
I have fix this with a new config spark.kubernetes.submission.deleteCompletedPod=true in our home maintained spark version.
Do you guys have more insights or I can make a pr on this issue?
was:
when submit to k8s, driver pod not deleted after job completion.
While k8s checks driver pod name not existing, It is especially painful when we use workflow tool to resubmit the failed spark job. (by the way, client always exist with 0 is another painful issue)
I have fix this with a new config spark.kubernetes.submission.deleteCompletedPod=true in our home maintained spark version.
> kubernetes driver pod not deleted after finish.
> -----------------------------------------------
>
> Key: SPARK-27696
> URL: https://issues.apache.org/jira/browse/SPARK-27696
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 2.4.0
> Reporter: Henry Yu
> Priority: Major
>
> When submit to k8s, driver pod not deleted after job completion.
> While k8s checks driver pod name not existing, It is especially painful when we use workflow tool to resubmit the failed spark job. (by the way, client always exist with 0 is another painful issue)
> I have fix this with a new config spark.kubernetes.submission.deleteCompletedPod=true in our home maintained spark version.
> Do you guys have more insights or I can make a pr on this issue?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org