(spark) branch branch-4.x updated: [SPARK-57952][CORE][TEST] Retry retryOnOOM a few times to reduce SorterSuite flaky OOM

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

HyukjinKwon pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 05158fc0a62b [SPARK-57952][CORE][TEST] Retry retryOnOOM a few times to reduce SorterSuite flaky OOM
05158fc0a62b is described below

commit 05158fc0a62b8817345c30ba33f99d47d83e71d4
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Mon Jul 6 17:27:37 2026 +0900

    [SPARK-57952][CORE][TEST] Retry retryOnOOM a few times to reduce SorterSuite flaky OOM
    
    ### What changes were proposed in this pull request?
    
    Make the test helper `retryOnOOM` (in `SparkFunSuite`) retry a few times (default 3) with
    a full `runGC()` between attempts, instead of running GC and retrying only once. The sole
    caller (`SorterSuite`) is updated to the new `retryOnOOM()(body)` form.
    
    ### Why are the changes needed?
    
    The `core` module test job intermittently aborts with an OOM (observed ~1 in 5 runs, e.g.
    in the branch-4.x Maven build):
    
    ```
    *** RUN ABORTED ***
      An exception or error caused a run to abort: Java heap space
      java.lang.OutOfMemoryError: Java heap space
      at org.apache.spark.util.collection.SorterSuite.$anonfun$new$12(SorterSuite.scala:145)
      at org.apache.spark.SparkFunSuite.retryOnOOM(SparkFunSuite.scala:122)
    ```
    
    `SorterSuite` allocates a single ~1 GB `byte[]` (heap is `-Xmx4g`). SPARK-57037 already
    wrapped that allocation in `retryOnOOM`, which does one `System.gc()` + one retry. But a
    single GC after the first failure does not always reclaim the previous test's large array
    in time, so the lone retry allocates against a still-pressured heap and OOMs again.
    
    Because the failure is intermittent (the job passes ~4 of 5 runs), 4 GB is not fundamentally
    too small — this is transient heap pressure, exactly the case where additional GC+retry
    attempts help. Retrying up to 3 times (with `runGC()`, which waits for a weak reference to
    clear, between attempts) gives the JVM more chances to reclaim memory.
    
    This is a test flakiness reduction, not a correctness change: a genuinely too-large or
    leaked allocation still fails once the attempts are exhausted, so real OOMs are not masked.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. Test-only change.
    
    ### How was this patch tested?
    
    `SorterSuite` (which exercises `retryOnOOM`) on a fork via GitHub Actions; confirmed the
    signature change compiles and the suite passes.
    
    **CI evidence**
    
    - Failing run (apache/spark, branch-4.x `build_maven`, `core` module — SorterSuite OOM):
      https://github.com/apache/spark/actions/runs/28747189185
    - Passing validation (fork, compiles `core` + runs `SorterSuite`):
      https://github.com/HyukjinKwon/spark-agent6/actions/runs/28766544512
    
    Note: the underlying OOM is intermittent (~1/5), so this reduces the flake probability;
    it is not claimed to statistically eliminate it.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (Opus 4.8)
    
    Closes #57029 from HyukjinKwon/ci-fix/agent6-oom-pr.
    
    Authored-by: Hyukjin Kwon <gu...@apache.org>
    Signed-off-by: Hyukjin Kwon <hy...@databricks.com>
    (cherry picked from commit 5ae5aa59a82b4d0a75b0412a32b0204a6ea2508f)
    Signed-off-by: Hyukjin Kwon <hy...@databricks.com>
---
 .../scala/org/apache/spark/SparkFunSuite.scala     | 29 ++++++++++++++++------
 .../apache/spark/util/collection/SorterSuite.scala |  4 +--
 2 files changed, 24 insertions(+), 9 deletions(-)

diff --git a/core/src/test/scala/org/apache/spark/SparkFunSuite.scala b/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
index a0f17f8af3f3..a18b1ec5dbf9 100644
--- a/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
+++ b/core/src/test/scala/org/apache/spark/SparkFunSuite.scala
@@ -113,13 +113,28 @@ abstract class SparkFunSuite
     }
   }
 
-  /** Run `body`; if it throws OutOfMemoryError, force a GC and retry once. */
-  protected def retryOnOOM[T](body: => T): T = {
-    try body
-    catch {
-      case _: OutOfMemoryError =>
-        runGC()
-        body
+  /**
+   * Run `body`; if it throws OutOfMemoryError, force a GC and retry, up to `maxAttempts`
+   * total attempts (default 3). A single GC+retry is not always enough in memory-constrained
+   * CI: a large array retained by a previous test may not be reclaimed by the first
+   * System.gc(), so the retried allocation OOMs again. Retrying a few times (with a full
+   * runGC() that waits for a weak reference to clear between attempts) gives the JVM
+   * additional chances to reclaim memory. This does not mask genuine leaks: an unbounded
+   * allocation still fails once the attempts are exhausted.
+   */
+  protected def retryOnOOM[T](maxAttempts: Int = 3)(body: => T): T = {
+    var attempt = 1
+    while (true) {
+      try {
+        return body
+      } catch {
+        case oom: OutOfMemoryError =>
+          if (attempt >= maxAttempts) throw oom
+          runGC()
+          attempt += 1
+      }
     }
+    // Unreachable: the loop either returns a value or rethrows the OOM.
+    throw new IllegalStateException("retryOnOOM exited its retry loop unexpectedly")
   }
 }
diff --git a/core/src/test/scala/org/apache/spark/util/collection/SorterSuite.scala b/core/src/test/scala/org/apache/spark/util/collection/SorterSuite.scala
index 2767769924bc..f654501ab436 100644
--- a/core/src/test/scala/org/apache/spark/util/collection/SorterSuite.scala
+++ b/core/src/test/scala/org/apache/spark/util/collection/SorterSuite.scala
@@ -141,8 +141,8 @@ class SorterSuite extends SparkFunSuite {
     val arrayToSortSize = 1091482190
     // Memory held by the previous test (e.g. the ~256 MB int array in "SPARK-5984
     // TimSort bug") may not be reclaimed before this >1 GB allocation, causing flaky
-    // OOM in CI. Force a GC and retry once on OOM.
-    val arrayToSort = retryOnOOM(new Array[Byte](arrayToSortSize))
+    // OOM in CI. Force a GC and retry a few times on OOM.
+    val arrayToSort = retryOnOOM()(new Array[Byte](arrayToSortSize))
     var sum: Int = -1
     for (i <- runLengths) {
       sum += i


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org