You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2026/08/27 01:19:00 UTC
[jira] [Assigned] (SPARK-59008) NullPointerException in ExecutorPodsLifecycleManager when pod is deleted between get() calls
[ https://issues.apache.org/jira/browse/SPARK-59008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun reassigned SPARK-59008:
-------------------------------------
Assignee: Venkata krishnan Sowrirajan
> NullPointerException in ExecutorPodsLifecycleManager when pod is deleted between get() calls
> --------------------------------------------------------------------------------------------
>
> Key: SPARK-59008
> URL: https://issues.apache.org/jira/browse/SPARK-59008
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 4.1.1
> Reporter: Venkata krishnan Sowrirajan
> Assignee: Venkata krishnan Sowrirajan
> Priority: Major
> Labels: pull-request-available
>
> ExecutorPodsLifecycleManager.removeExecutorFromK8s calls "podToDelete.get()" twice:
> {code:java}
> if (podToDelete.get() != null &&
> podToDelete.get.getMetadata.getDeletionTimestamp == null) {
> {code}
> podToDelete is a lazy PodResource handle, so each get() issues a separate API-server call. If the pod is deleted between the two calls, the second returns null and dereferencing getMetadata throws:
> {code:java}
> java.lang.NullPointerException: Cannot invoke "io.fabric8.kubernetes.api.model.Pod.getMetadata()"
> because the return value of "io.fabric8.kubernetes.client.dsl.PodResource.get()" is null
> at ExecutorPodsLifecycleManager.$anonfun$removeExecutorFromK8s$1(ExecutorPodsLifecycleManager.scala:222)
> at ExecutorPodsLifecycleManager.onFinalNonDeletedState(ExecutorPodsLifecycleManager.scala:194)
> at ExecutorPodsLifecycleManager.onNewSnapshots(ExecutorPodsLifecycleManager.scala:98)
> {code}
> Observed on a driver managing ~1000 executors during heavy pod churn. The exception aborts the K8s-side deletion for that executor (Spark-side accounting in removeExecutorFromSpark has already completed), leaving the pod to be reaped on a later resync and logging a spurious uncaught-exception error.
> Fix: fetch the pod once and test that single value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org