You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Holden Karau (Jira)" <ji...@apache.org> on 2023/01/31 18:35:00 UTC

[jira] [Created] (SPARK-42260) Log when the K8s Exec Pods Allocator Stalls

Holden Karau created SPARK-42260:
------------------------------------

             Summary: Log when the K8s Exec Pods Allocator Stalls
                 Key: SPARK-42260
                 URL: https://issues.apache.org/jira/browse/SPARK-42260
             Project: Spark
          Issue Type: Improvement
          Components: Kubernetes
    Affects Versions: 3.4.0, 3.4.1
            Reporter: Holden Karau
            Assignee: Holden Karau


Sometimes if the K8s APIs are being slow the ExecutorPods allocator can stall and it would be good for us to log this (and how long we've stalled for) so folks can tell more clearly why Spark is unable to reach the desired target number of executors.

 

This is _somewhat_ related to SPARK-36664 which logs the time spent waiting for executor allocation but goes a step further for K8s and logs when we've stalled because we have too many pending pods.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org