[jira] [Created] (SPARK-59606) Map `SparkCluster` pods by `spark.operator/spark-cluster-name` in `SparkClusterReconciler`

["Peter Toth (Jira)" <ji...@apache.org>]

Peter Toth created SPARK-59606:
----------------------------------

             Summary: Map `SparkCluster` pods by `spark.operator/spark-cluster-name` in `SparkClusterReconciler`
                 Key: SPARK-59606
                 URL: https://issues.apache.org/jira/browse/SPARK-59606
             Project: Spark
          Issue Type: Sub-task
          Components: Kubernetes
    Affects Versions: kubernetes-operator-1.1.0
            Reporter: Peter Toth


SparkClusterReconciler.prepareEventSources registers the Pod informer with basicLabelSecondaryToPrimaryMapper(LABEL_SPARK_APPLICATION_NAME), i.e. it maps a Pod back to its primary SparkCluster by reading the label spark.operator/spark-app-name.

SparkCluster pods never carry that label. SparkClusterResourceSpec adds only spark.operator/spark-role, spark.operator/spark-cluster-name and spark.operator/spark-version to the master and worker pod templates (lines 234-237 and 316-319), and Utils.sparkClusterResourceLabels adds spark.operator/spark-cluster-name plus commonManagedResourceLabels, which is only spark.operator/name. LABEL_SPARK_APPLICATION_NAME is set exclusively by Utils.sparkAppResourceLabels, for SparkApplication resources.

So Utils.basicLabelSecondaryToPrimaryMapper takes its `name == null` branch and returns Set.of() for every SparkCluster pod. The pods still match the informer's commonResourceLabelsStr() selector and are cached, but no primary ResourceID is produced, so a Pod event never triggers a SparkCluster reconciliation. An owner-reference mapper would not help either: cluster pods are owned by their StatefulSet, not by the SparkCluster, which is presumably why a label mapper is used.

Effect: a master or worker pod dying, becoming ready, or being evicted is not noticed until the next periodic reconcile, spark.kubernetes.operator.reconciler.intervalSeconds, 120 seconds by default. SparkApplication is unaffected; SparkAppReconciler uses the same mapper with the label its pods actually carry.

The fix is to use LABEL_SPARK_CLUSTER_NAME in SparkClusterReconciler.prepareEventSources, which is what PR #836 does for the new Kueue Workload informer three lines below.

Worth adding a test that asserts the mapper resolves a realistic cluster pod to its SparkCluster. The existing SparkClusterReconcilerTest only asserts the number of event sources and their resource classes, so it does not catch a wrong label: swapping the label in the Workload source leaves all 11 tests green.

Found while reviewing https://github.com/apache/spark-kubernetes-operator/pull/836




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org