You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/09/16 16:21:38 UTC

(spark-kubernetes-operator) branch main updated: [SPARK-59576] Support K8s `Normal` events for all `SparkApplication` and `SparkCluster` state transitions

This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git


The following commit(s) were added to refs/heads/main by this push:
     new 78ddddfb [SPARK-59576] Support K8s `Normal` events for all `SparkApplication` and `SparkCluster` state transitions
78ddddfb is described below

commit 78ddddfbe7a1dae5f25bc54ddc56c10cb1a985c6
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Wed Sep 16 09:21:30 2026 -0700

    [SPARK-59576] Support K8s `Normal` events for all `SparkApplication` and `SparkCluster` state transitions
    
    ### What changes were proposed in this pull request?
    
    This PR aims to publish K8s events for all `SparkApplication` and `SparkCluster` state transitions, not only for failure states. The reason of each event is the new state name.
    
    | Type | `SparkApplication` | `SparkCluster` |
    |---|---|---|
    | `Warning` | `SchedulingFailure`, `Failed`, `DriverEvicted`, `DriverStartTimedOut`, `DriverReadyTimedOut`, `ExecutorsStartTimedOut`, `RunningWithBelowThresholdExecutors`, `TerminatedWithoutReleaseResources` | `SchedulingFailure`, `Failed` |
    | `Normal` | All other states | All other states |
    
    An event is published only when the current state changes. Repeated transitions into the same state increment the count of one `Event` object.
    
    ### Why are the changes needed?
    
    To make the full lifecycle of Spark resources visible via `kubectl describe` and `kubectl get events`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No by default. When `spark.kubernetes.operator.events.enabled` is `true`, the operator also publishes the new `Normal` and `Warning` events above.
    
    ### How was this patch tested?
    
    Pass the CIs with the newly added unit tests.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5
    
    Closes #831 from dongjoon-hyun/SPARK-59576.
    
    Authored-by: Dongjoon Hyun <do...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 docs/config_properties.md                          |   2 +-
 docs/configuration.md                              |  28 +++++
 .../k8s/operator/config/SparkOperatorConf.java     |   3 +-
 .../spark/k8s/operator/utils/EventUtils.java       |  47 ++++++-
 .../spark/k8s/operator/utils/StatusRecorder.java   |  76 ++++++++---
 .../spark/k8s/operator/utils/EventUtilsTest.java   |  48 +++++++
 .../k8s/operator/utils/StatusRecorderTest.java     | 140 +++++++++++++++++++--
 7 files changed, 315 insertions(+), 29 deletions(-)

diff --git a/docs/config_properties.md b/docs/config_properties.md
index 0a701893..3807e36d 100644
--- a/docs/config_properties.md
+++ b/docs/config_properties.md
@@ -16,7 +16,7 @@
  | spark.kubernetes.operator.dynamicConfig.reloadIntervalSeconds | Long | 60 | false | Interval (in seconds) at which the dynamic config file is re-read. Used by the 'file' source. | 
  | spark.kubernetes.operator.dynamicConfig.selector | String | app.kubernetes.io/name=spark-kubernetes-operator,app.kubernetes.io/component=operator-dynamic-config-overrides | false | The selector str applied to dynamic config map. Used by the 'configMap' source. | 
  | spark.kubernetes.operator.dynamicConfig.source | String | configMap | false | Source of dynamic config overrides when spark.kubernetes.operator.dynamicConfig.enabled is true. Supported values: 'configMap' (default) watches a ConfigMap via a Kubernetes informer and requires RBAC to read ConfigMaps; 'file' periodically reloads a properties file mounted from a ConfigMap and requires no extra RBAC. | 
- | spark.kubernetes.operator.events.enabled | Boolean | false | true | When enabled, operator would publish Kubernetes Event objects into the namespace of the Spark resource they describe, making them visible to users of that namespace via 'kubectl describe' and 'kubectl get events'. Events are published when a resource enters a failure state, and when the operator cannot reconcile or update the status of a resource. These are Event resources in the core API group, unrelated to the inter [...]
+ | spark.kubernetes.operator.events.enabled | Boolean | false | true | When enabled, operator would publish Kubernetes Event objects into the namespace of the Spark resource they describe, making them visible to users of that namespace via 'kubectl describe' and 'kubectl get events'. Events are published when a resource transitions into a new state, as Warning for failure states and Normal for most other states, and when the operator cannot reconcile or update the status of a resource. T [...]
  | spark.kubernetes.operator.health.probePort | Integer | 19091 | false | The port used for health/readiness check probe status. | 
  | spark.kubernetes.operator.health.sentinelExecutorPoolSize | Integer | 3 | false | Size of executor service in Sentinel Managers to check the health of sentinel resources. | 
  | spark.kubernetes.operator.health.sentinelResourceReconciliationDelaySeconds | Integer | 60 | true | Allowed max time(seconds) between spec update and reconciliation for sentinel resources. | 
diff --git a/docs/configuration.md b/docs/configuration.md
index fd7ebeb6..10d9b052 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -51,6 +51,34 @@ operatorConfiguration:
     create: true
 ```
 
+## Kubernetes Events
+
+When `spark.kubernetes.operator.events.enabled` is `true`, the operator publishes Kubernetes
+`Event` objects about `SparkApplication` and `SparkCluster` resources into their namespaces. They
+are visible via `kubectl describe` and `kubectl get events`.
+
+Whenever a resource transitions into a new state, the operator publishes an event whose `reason`
+is the name of the new state and whose `message` is the state message. Repeated transitions into
+the same state are aggregated into a single `Event` object by incrementing its `count`.
+
+| Type | Reason |
+|---|---|
+| `Warning` | `SparkApplication`: `SchedulingFailure`, `Failed`, `DriverEvicted`, `DriverStartTimedOut`, `DriverReadyTimedOut`, `ExecutorsStartTimedOut`, `RunningWithBelowThresholdExecutors`, `TerminatedWithoutReleaseResources` |
+| `Warning` | `SparkCluster`: `SchedulingFailure`, `Failed` |
+| `Normal` | All other states, e.g. `Submitted`, `ScheduledToRestart`, `DriverRequested`, `DriverStarted`, `DriverReady`, `InitializedBelowThresholdExecutors`, `RunningHealthy`, `RunningWithPartialCapacity`, `Succeeded` and `ResourceReleased` |
+
+In addition, the operator publishes the following `Warning` events.
+
+| Reason | When |
+|---|---|
+| `ReconcileError` | A reconciliation throws. Only the first attempt of a failure episode publishes. |
+| `CleanupError` | A cleanup throws, so the resource cannot finish deleting. |
+| `StatusUpdateFailed` | A status patch is rejected. Transport-level errors are skipped. |
+
+The `reason` values are stable, while the `message` values may change between releases. Note that
+Kubernetes retains events only for a limited time (one hour by default), so the resource status
+remains the source of truth.
+
 ## Metrics
 
 Spark operator,
diff --git a/spark-operator/src/main/java/org/apache/spark/k8s/operator/config/SparkOperatorConf.java b/spark-operator/src/main/java/org/apache/spark/k8s/operator/config/SparkOperatorConf.java
index 96591816..8d4c955f 100644
--- a/spark-operator/src/main/java/org/apache/spark/k8s/operator/config/SparkOperatorConf.java
+++ b/spark-operator/src/main/java/org/apache/spark/k8s/operator/config/SparkOperatorConf.java
@@ -290,7 +290,8 @@ public final class SparkOperatorConf {
               "When enabled, operator would publish Kubernetes Event objects into the namespace "
                   + "of the Spark resource they describe, making them visible to users of that "
                   + "namespace via 'kubectl describe' and 'kubectl get events'. Events are "
-                  + "published when a resource enters a failure state, and when the operator "
+                  + "published when a resource transitions into a new state, as Warning for "
+                  + "failure states and Normal for most other states, and when the operator "
                   + "cannot reconcile or update the status of a resource. These are Event "
                   + "resources in the core API group, unrelated to the internal events that "
                   + "trigger reconciliation.")
diff --git a/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/EventUtils.java b/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/EventUtils.java
index 346cd0a0..a4a63ccf 100644
--- a/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/EventUtils.java
+++ b/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/EventUtils.java
@@ -19,10 +19,15 @@
 
 package org.apache.spark.k8s.operator.utils;
 
+import java.util.Set;
+
 import io.javaoperatorsdk.operator.api.event.EventRecord;
 import io.javaoperatorsdk.operator.api.event.EventType;
 import io.javaoperatorsdk.operator.api.event.ResourceEventRecorder;
 
+import org.apache.spark.k8s.operator.status.ApplicationStateSummary;
+import org.apache.spark.k8s.operator.status.BaseStateSummary;
+
 /**
  * Utility class for publishing Kubernetes events about Spark resources.
  *
@@ -53,8 +58,34 @@ public final class EventUtils {
   /** Maximum number of links followed when looking for the innermost cause of a failure. */
   private static final int MAX_CAUSE_DEPTH = 10;
 
+  /**
+   * States that are not failures but still deserve the attention of users, because nothing else
+   * reports them: {@code RunningWithBelowThresholdExecutors} persists without any timeout, and
+   * {@code TerminatedWithoutReleaseResources} is not meant for production use. A state that
+   * escalates to a failure on its own, such as {@code InitializedBelowThresholdExecutors} timing
+   * out into {@code ExecutorsStartTimedOut}, warns through that failure instead and stays normal.
+   */
+  private static final Set<BaseStateSummary> NON_FAILURE_WARNING_STATES =
+      Set.of(
+          ApplicationStateSummary.RunningWithBelowThresholdExecutors,
+          ApplicationStateSummary.TerminatedWithoutReleaseResources);
+
   private EventUtils() {}
 
+  /**
+   * Returns the type of the event published when a resource transitions into the given state.
+   * Failure states and the states listed in {@link #NON_FAILURE_WARNING_STATES} are warnings, every
+   * other state is normal.
+   *
+   * @param summary The state the resource has transitioned into.
+   * @return {@link EventType#WARNING} or {@link EventType#NORMAL}.
+   */
+  public static EventType eventTypeOf(BaseStateSummary summary) {
+    return summary.isFailure() || NON_FAILURE_WARNING_STATES.contains(summary)
+        ? EventType.WARNING
+        : EventType.NORMAL;
+  }
+
   /**
    * Publishes a warning event about a Spark resource.
    *
@@ -69,9 +100,23 @@ public final class EventUtils {
    * @param message The message, truncated to {@value #MAX_MESSAGE_LENGTH} characters.
    */
   public static void warn(ResourceEventRecorder recorder, String reason, String message) {
+    record(recorder, EventType.WARNING, reason, message);
+  }
+
+  /**
+   * Publishes an event of the given type about a Spark resource. The reason doubles as the event
+   * key, see {@link #warn(ResourceEventRecorder, String, String)}.
+   *
+   * @param recorder The event recorder bound to the resource.
+   * @param type The event type.
+   * @param reason A short CamelCase reason, as expected by Kubernetes.
+   * @param message The message, truncated to {@value #MAX_MESSAGE_LENGTH} characters.
+   */
+  public static void record(
+      ResourceEventRecorder recorder, EventType type, String reason, String message) {
     recorder.record(
         EventRecord.builder()
-            .type(EventType.WARNING)
+            .type(type)
             .reason(reason)
             .message(truncate(message))
             .key(reason)
diff --git a/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java b/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java
index e726436c..724fa1ab 100644
--- a/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java
+++ b/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java
@@ -23,7 +23,9 @@ import static java.net.HttpURLConnection.HTTP_CONFLICT;
 import static org.apache.spark.k8s.operator.config.SparkOperatorConf.API_RETRY_ATTEMPT_AFTER_SECONDS;
 import static org.apache.spark.k8s.operator.config.SparkOperatorConf.API_STATUS_PATCH_MAX_ATTEMPTS;
 
+import java.util.ArrayList;
 import java.util.List;
+import java.util.SortedMap;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
 
@@ -137,36 +139,76 @@ public class StatusRecorder<
         listener -> {
           listener.listenStatus(resource, prevStatus, resource.getStatus());
         });
-    recordFailureEvent(context, resource.getStatus());
+    recordTransitionEvents(context, prevStatus, resource.getStatus());
   }
 
   /**
-   * Publishes a warning event when the resource has just moved into a failure state.
+   * Publishes events when the resource has just transitioned into new states.
    *
-   * <p>This is the seam where the failures users actually hit surface: every reconcile step turns
-   * a rejected submission, a failed validation or an expired timeout into a state rather than
-   * throwing, so the reconciler error hooks never see them. Reaching this point means the patch
-   * succeeded and the status genuinely changed, so an event here fires once per transition.
+   * <p>This is the seam where every state transition surfaces, including the failures users
+   * actually hit: every reconcile step turns a rejected submission, a failed validation or an
+   * expired timeout into a state rather than throwing, so the reconciler error hooks never see
+   * them. Reaching this point means the patch succeeded and the status genuinely changed. The
+   * current state is compared as well, so a status change that keeps the current state, such as
+   * an attempt summary update, does not publish again.
    *
-   * <p>The state name is used as both reason and dedup key, so an app that keeps re-entering the
-   * same failure state bumps the count on one Event instead of accumulating one per attempt.
+   * <p>A single patch can carry more than one transition, for instance when a driver pod is
+   * observed as both started and ready. Each state appended since the previous status is published
+   * in order, while a state whose summary equals the one right before it in the same patch is
+   * skipped.
+   *
+   * <p>The event type follows {@link EventUtils#eventTypeOf(BaseStateSummary)}. The state name is
+   * used as both reason and dedup key, so an app that keeps re-entering the same state bumps the
+   * count on one Event instead of accumulating one per attempt.
    *
    * @param context Context of the resource whose status has just been patched.
+   * @param prevStatus The status before the patch, may be null.
    * @param status The status that was persisted.
    */
-  private void recordFailureEvent(BaseContext<CR> context, STATUS status) {
+  private void recordTransitionEvents(BaseContext<CR> context, STATUS prevStatus, STATUS status) {
     BaseState<?> currentState = status.getCurrentState();
-    if (currentState == null) {
+    if (currentState == null
+        || (prevStatus != null && currentState.equals(prevStatus.getCurrentState()))) {
       return;
     }
-    if (currentState.getCurrentStateSummary() instanceof BaseStateSummary summary
-        && summary.isFailure()) {
-      String message = currentState.getMessage();
-      EventUtils.warn(
-          context.getEventRecorder(),
-          summary.name(),
-          StringUtils.isBlank(message) ? summary.name() : message);
+    Object lastSummary = null;
+    for (BaseState<?> state : newStatesSince(prevStatus, status)) {
+      Object summary = state.getCurrentStateSummary();
+      if (summary instanceof BaseStateSummary stateSummary && !summary.equals(lastSummary)) {
+        String message = state.getMessage();
+        EventUtils.record(
+            context.getEventRecorder(),
+            EventUtils.eventTypeOf(stateSummary),
+            stateSummary.name(),
+            StringUtils.isBlank(message) ? stateSummary.name() : message);
+      }
+      lastSummary = summary;
+    }
+  }
+
+  /**
+   * Returns the states appended to the transition history since the previous status, in order.
+   * History keys keep increasing even when the history is trimmed, so the entries after the last
+   * key of the previous history are the new ones. Falls back to the current state when there is no
+   * previous history or no such entry, for instance when the status is reset.
+   *
+   * @param prevStatus The status before the patch, may be null.
+   * @param status The status that was persisted.
+   * @return The new states, never empty.
+   */
+  private List<BaseState<?>> newStatesSince(STATUS prevStatus, STATUS status) {
+    SortedMap<Long, ? extends BaseState<?>> history = status.getStateTransitionHistory();
+    if (prevStatus != null
+        && history != null
+        && prevStatus.getStateTransitionHistory() != null
+        && !prevStatus.getStateTransitionHistory().isEmpty()) {
+      long prevLastKey = prevStatus.getStateTransitionHistory().lastKey();
+      List<BaseState<?>> newStates = new ArrayList<>(history.tailMap(prevLastKey + 1).values());
+      if (!newStates.isEmpty()) {
+        return newStates;
+      }
     }
+    return List.of(status.getCurrentState());
   }
 
   /**
diff --git a/spark-operator/src/test/java/org/apache/spark/k8s/operator/utils/EventUtilsTest.java b/spark-operator/src/test/java/org/apache/spark/k8s/operator/utils/EventUtilsTest.java
index 4c17218a..0059f365 100644
--- a/spark-operator/src/test/java/org/apache/spark/k8s/operator/utils/EventUtilsTest.java
+++ b/spark-operator/src/test/java/org/apache/spark/k8s/operator/utils/EventUtilsTest.java
@@ -24,12 +24,17 @@ import static org.assertj.core.api.Assertions.assertThat;
 import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.verify;
 
+import java.util.Set;
+
 import io.javaoperatorsdk.operator.api.event.EventRecord;
 import io.javaoperatorsdk.operator.api.event.EventType;
 import io.javaoperatorsdk.operator.api.event.ResourceEventRecorder;
 import org.junit.jupiter.api.Test;
 import org.mockito.ArgumentCaptor;
 
+import org.apache.spark.k8s.operator.status.ApplicationStateSummary;
+import org.apache.spark.k8s.operator.status.ClusterStateSummary;
+
 class EventUtilsTest {
 
   private final ResourceEventRecorder recorder = mock(ResourceEventRecorder.class);
@@ -60,6 +65,49 @@ class EventUtilsTest {
     assertThat(event.key()).contains(EventUtils.REASON_STATUS_UPDATE_FAILED);
   }
 
+  @Test
+  void recordPublishesTheGivenType() {
+    EventUtils.record(recorder, EventType.NORMAL, "DriverRequested", "driver requested");
+
+    ArgumentCaptor<EventRecord> captor = ArgumentCaptor.forClass(EventRecord.class);
+    verify(recorder).record(captor.capture());
+    EventRecord event = captor.getValue();
+    assertThat(event.type()).isEqualTo(EventType.NORMAL);
+    assertThat(event.reason()).isEqualTo("DriverRequested");
+    assertThat(event.message()).isEqualTo("driver requested");
+    assertThat(event.key()).contains("DriverRequested");
+  }
+
+  @Test
+  void eventTypeOfApplicationStates() {
+    Set<ApplicationStateSummary> expectedWarnings =
+        Set.of(
+            ApplicationStateSummary.SchedulingFailure,
+            ApplicationStateSummary.Failed,
+            ApplicationStateSummary.DriverEvicted,
+            ApplicationStateSummary.DriverStartTimedOut,
+            ApplicationStateSummary.DriverReadyTimedOut,
+            ApplicationStateSummary.ExecutorsStartTimedOut,
+            ApplicationStateSummary.RunningWithBelowThresholdExecutors,
+            ApplicationStateSummary.TerminatedWithoutReleaseResources);
+    for (ApplicationStateSummary summary : ApplicationStateSummary.values()) {
+      EventType expected =
+          expectedWarnings.contains(summary) ? EventType.WARNING : EventType.NORMAL;
+      assertThat(EventUtils.eventTypeOf(summary)).as(summary.name()).isEqualTo(expected);
+    }
+  }
+
+  @Test
+  void eventTypeOfClusterStates() {
+    Set<ClusterStateSummary> expectedWarnings =
+        Set.of(ClusterStateSummary.SchedulingFailure, ClusterStateSummary.Failed);
+    for (ClusterStateSummary summary : ClusterStateSummary.values()) {
+      EventType expected =
+          expectedWarnings.contains(summary) ? EventType.WARNING : EventType.NORMAL;
+      assertThat(EventUtils.eventTypeOf(summary)).as(summary.name()).isEqualTo(expected);
+    }
+  }
+
   @Test
   void warnTruncatesAnOverLongMessage() {
     EventRecord event = recordWarning(EventUtils.REASON_CLEANUP_ERROR, "x".repeat(5000));
diff --git a/spark-operator/src/test/java/org/apache/spark/k8s/operator/utils/StatusRecorderTest.java b/spark-operator/src/test/java/org/apache/spark/k8s/operator/utils/StatusRecorderTest.java
index d0b326b6..5bfdabb1 100644
--- a/spark-operator/src/test/java/org/apache/spark/k8s/operator/utils/StatusRecorderTest.java
+++ b/spark-operator/src/test/java/org/apache/spark/k8s/operator/utils/StatusRecorderTest.java
@@ -71,9 +71,7 @@ class StatusRecorderTest {
     var resourceV2 = getSparkApplication("2");
     var resourceV3 = getSparkApplication("3");
 
-    BaseContext<SparkApplication> context = mock(BaseContext.class);
-    when(context.getResource()).thenReturn(testResource);
-    when(context.getClient()).thenReturn(client);
+    var context = contextFor(testResource);
     var basePath =
         "/apis/spark.apache.org/v1/namespaces/"
             + DEFAULT_NS
@@ -102,9 +100,7 @@ class StatusRecorderTest {
     var resourceV2 = getSparkApplication("2");
     var resourceV3 = getSparkApplication("3");
 
-    BaseContext<SparkApplication> context = mock(BaseContext.class);
-    when(context.getResource()).thenReturn(testResource);
-    when(context.getClient()).thenReturn(client);
+    var context = contextFor(testResource);
     var path =
         "/apis/spark.apache.org/v1/namespaces/"
             + DEFAULT_NS
@@ -149,7 +145,7 @@ class StatusRecorderTest {
   }
 
   @Test
-  void publishesNoEventForANonFailureTransition() {
+  void publishesANormalEventForANonFailureTransition() {
     var testResource = getSparkApplication("1");
     var context = contextFor(testResource);
     expectStatusPatch(testResource, getSparkApplication("2"));
@@ -160,8 +156,134 @@ class StatusRecorderTest {
             .appendNewState(
                 new ApplicationState(ApplicationStateSummary.DriverRequested, "driver requested")));
 
-    // Only failures are published, a Normal event per transition would be pure noise.
-    verifyNoInteractions(mockEventRecorder);
+    var event = captureRecordedEvent();
+    assertThat(event.type()).isEqualTo(EventType.NORMAL);
+    assertThat(event.reason()).isEqualTo(ApplicationStateSummary.DriverRequested.name());
+    assertThat(event.message()).isEqualTo("driver requested");
+    assertThat(event.key()).contains(ApplicationStateSummary.DriverRequested.name());
+  }
+
+  @Test
+  void publishesAWarningEventForANonFailureWarningState() {
+    var testResource = getSparkApplication("1");
+    var context = contextFor(testResource);
+    expectStatusPatch(testResource, getSparkApplication("2"));
+
+    statusRecorder.persistStatus(
+        context,
+        new ApplicationStatus()
+            .appendNewState(
+                new ApplicationState(
+                    ApplicationStateSummary.RunningWithBelowThresholdExecutors, "lost executors")));
+
+    var event = captureRecordedEvent();
+    assertThat(event.type()).isEqualTo(EventType.WARNING);
+    assertThat(event.reason())
+        .isEqualTo(ApplicationStateSummary.RunningWithBelowThresholdExecutors.name());
+  }
+
+  @Test
+  void publishesEveryTransition() {
+    var testResource = getSparkApplication("1");
+    var context = contextFor(testResource);
+    expectStatusPatch(testResource, getSparkApplication("2"));
+
+    var requested =
+        new ApplicationStatus()
+            .appendNewState(
+                new ApplicationState(ApplicationStateSummary.DriverRequested, "requested"));
+    statusRecorder.persistStatus(context, requested);
+    statusRecorder.persistStatus(
+        context,
+        requested.appendNewState(
+            new ApplicationState(ApplicationStateSummary.DriverStarted, "started")));
+
+    ArgumentCaptor<EventRecord> captor = ArgumentCaptor.forClass(EventRecord.class);
+    verify(mockEventRecorder, times(2)).record(captor.capture());
+    assertThat(captor.getAllValues())
+        .extracting(EventRecord::reason)
+        .containsExactly(
+            ApplicationStateSummary.DriverRequested.name(),
+            ApplicationStateSummary.DriverStarted.name());
+  }
+
+  @Test
+  void publishesEveryTransitionCarriedByASinglePatch() {
+    var testResource = getSparkApplication("1");
+    var context = contextFor(testResource);
+    expectStatusPatch(testResource, getSparkApplication("2"));
+
+    var requested =
+        new ApplicationStatus()
+            .appendNewState(
+                new ApplicationState(ApplicationStateSummary.DriverRequested, "requested"));
+    statusRecorder.persistStatus(context, requested);
+    // A driver pod observed as both started and ready appends two states before one patch.
+    statusRecorder.persistStatus(
+        context,
+        requested
+            .appendNewState(new ApplicationState(ApplicationStateSummary.DriverStarted, "started"))
+            .appendNewState(new ApplicationState(ApplicationStateSummary.DriverReady, "ready")));
+
+    ArgumentCaptor<EventRecord> captor = ArgumentCaptor.forClass(EventRecord.class);
+    verify(mockEventRecorder, times(3)).record(captor.capture());
+    assertThat(captor.getAllValues())
+        .extracting(EventRecord::reason)
+        .containsExactly(
+            ApplicationStateSummary.DriverRequested.name(),
+            ApplicationStateSummary.DriverStarted.name(),
+            ApplicationStateSummary.DriverReady.name());
+  }
+
+  @Test
+  void publishesOnceForRepeatedStatesInASinglePatch() {
+    var testResource = getSparkApplication("1");
+    var context = contextFor(testResource);
+    expectStatusPatch(testResource, getSparkApplication("2"));
+
+    var started =
+        new ApplicationStatus()
+            .appendNewState(
+                new ApplicationState(ApplicationStateSummary.DriverStarted, "started"));
+    statusRecorder.persistStatus(context, started);
+    // Both driver observers report the same failed driver pod, appending Failed twice.
+    statusRecorder.persistStatus(
+        context,
+        started
+            .appendNewState(new ApplicationState(ApplicationStateSummary.Failed, "driver failed"))
+            .appendNewState(new ApplicationState(ApplicationStateSummary.Failed, "driver failed")));
+
+    ArgumentCaptor<EventRecord> captor = ArgumentCaptor.forClass(EventRecord.class);
+    verify(mockEventRecorder, times(2)).record(captor.capture());
+    assertThat(captor.getAllValues())
+        .extracting(EventRecord::reason)
+        .containsExactly(
+            ApplicationStateSummary.DriverStarted.name(), ApplicationStateSummary.Failed.name());
+  }
+
+  @Test
+  void publishesNoEventWhenTheCurrentStateDidNotChange() {
+    var testResource = getSparkApplication("1");
+    var context = contextFor(testResource);
+    expectStatusPatch(testResource, getSparkApplication("2"));
+
+    var running =
+        new ApplicationStatus()
+            .appendNewState(
+                new ApplicationState(ApplicationStateSummary.RunningHealthy, "running"));
+    statusRecorder.persistStatus(context, running);
+    // The status changes and is patched, but the current state is the same one, so the resource
+    // has not transitioned again and must not be reported again.
+    var sameStateNewAttempt =
+        new ApplicationStatus(
+            running.getCurrentState(),
+            running.getStateTransitionHistory(),
+            running.getCurrentAttemptSummary(),
+            running.getCurrentAttemptSummary());
+    statusRecorder.persistStatus(context, sameStateNewAttempt);
+
+    verify(mockStatusListener, times(2)).listenStatus(any(), any(), any());
+    verify(mockEventRecorder, times(1)).record(any(EventRecord.class));
   }
 
   @Test


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org