You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Holden Karau (Jira)" <ji...@apache.org> on 2020/10/23 18:37:00 UTC

[jira] [Created] (SPARK-33231) Make podCreationTimeout configurable

Holden Karau created SPARK-33231:
------------------------------------

             Summary: Make podCreationTimeout configurable
                 Key: SPARK-33231
                 URL: https://issues.apache.org/jira/browse/SPARK-33231
             Project: Spark
          Issue Type: Improvement
          Components: Kubernetes
    Affects Versions: 3.0.1, 3.0.0, 3.1.0
            Reporter: Holden Karau


Execution Monitor & Pod Allocator have differing views of the world which can lead to pod trashing.

The executor monitor can be notified of an executor coming up before a snapshot is delivered to the PodAllocator. This can cause the executor monitor to believe it needs to delete a pod, and the pod allocator to believe that it needs to create a new pod. This happens if the podCreationTimeout is too low for the cluster. Currently podCreationTimeout can only be configured by increasing the batch delay but that has additional consequences leading to slower spin up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org