(airflow) branch main updated: Clarify reattach order of `find_spark_job` in `SparkKubernetesOperator` (#64186)

[el...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 9c46b2176d9 Clarify reattach order of `find_spark_job` in `SparkKubernetesOperator` (#64186)
9c46b2176d9 is described below

commit 9c46b2176d9e4d1d24fddbb1152c9198a5732c3b
Author: Nataneljpwd <10...@users.noreply.github.com>
AuthorDate: Tue Mar 24 21:52:17 2026 +0200

    Clarify reattach order of `find_spark_job` in `SparkKubernetesOperator` (#64186)
---
 .../airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py    | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py b/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py
index 0d77361199d..6185b3053ef 100644
--- a/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py
+++ b/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py
@@ -265,6 +265,9 @@ class SparkKubernetesOperator(KubernetesPodOperator):
             #
             # Pending pods are considered to handle recent driver restarts without
             # prematurely failing the task.
+            # Pending pods are preferred over Running pods, as if a new pod is created
+            # that means the old pod is terminating (which is running state with deletion timestamp)
+            # and we always prefer a new pod over an old pod.
             pod = max(
                 pod_list,
                 key=lambda p: (