[jira] [Resolved] (SPARK-33231) Make podCreationTimeout configurable

["Dongjoon Hyun (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-33231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun resolved SPARK-33231.
-----------------------------------
    Fix Version/s: 3.1.0
       Resolution: Fixed

Issue resolved by pull request 30155
[https://github.com/apache/spark/pull/30155]

> Make podCreationTimeout configurable
> ------------------------------------
>
>                 Key: SPARK-33231
>                 URL: https://issues.apache.org/jira/browse/SPARK-33231
>             Project: Spark
>          Issue Type: Improvement
>          Components: Kubernetes
>    Affects Versions: 3.0.0, 3.0.1, 3.1.0
>            Reporter: Holden Karau
>            Assignee: Holden Karau
>            Priority: Major
>             Fix For: 3.1.0
>
>
> Execution Monitor & Pod Allocator have differing views of the world which can lead to pod trashing.
> The executor monitor can be notified of an executor coming up before a snapshot is delivered to the PodAllocator. This can cause the executor monitor to believe it needs to delete a pod, and the pod allocator to believe that it needs to create a new pod. This happens if the podCreationTimeout is too low for the cluster. Currently podCreationTimeout can only be configured by increasing the batch delay but that has additional consequences leading to slower spin up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org