[jira] [Created] (SPARK-57991) Deflake BarrierTaskContextSuite wall-clock skew assertions on macOS-26

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

Hyukjin Kwon created SPARK-57991:
------------------------------------

             Summary: Deflake BarrierTaskContextSuite wall-clock skew assertions on macOS-26
                 Key: SPARK-57991
                 URL: https://issues.apache.org/jira/browse/SPARK-57991
             Project: Spark
          Issue Type: Bug
          Components: Spark Core, Tests
    Affects Versions: 5.0.0
            Reporter: Hyukjin Kwon


The `Build / Maven (Scala 2.13, JDK 21, MacOS-26)` scheduled workflow fails on the `core` module with:

{code}
BarrierTaskContextSuite:
- successively sync with allGather and barrier *** FAILED ***
  1078 was not less than or equal to 1000 (BarrierTaskContextSuite.scala:122)
{code}

Three tests in `BarrierTaskContextSuite` capture `System.currentTimeMillis()` in each task right after a `barrier()` / `allGather()` global sync and assert `times.max - times.min <= 1000`. That skew is bounded below by the barrier client's own polling granularity, not by sync correctness: `BarrierTaskContext.runBarrier` waits for the coordinator RPC in a `Thread.sleep(1000)` loop, so two tasks can observe the release up to ~1s apart from the poll interval alone, before any thread-scheduling or GC jitter on a busy CI host.

SPARK-49983 previously observed 1038 and only halved the pre-barrier sleep, which reduces arrival spread but cannot reduce the poll-granularity skew, so the flake persists on slower runners (macOS-26 arm64 observed 1078). Reproduced locally on macOS 26 arm64 / JDK 21: post-sync skew reached 994ms while idle.

Widen the tolerance to a documented constant that accounts for the poll interval plus jitter, keeping the test's intent (tasks stay loosely in lockstep; none races an extra sync ahead) without coupling to a razor-thin wall-clock margin.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org