You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "MartijnVisser (via GitHub)" <gi...@apache.org> on 2026/07/06 13:57:14 UTC

[PR] [FLINK-40074][tests] Avoid @TempDir cleanup race in SinkV2ITCase scaling test [flink]

MartijnVisser opened a new pull request, #28653:
URL: https://github.com/apache/flink/pull/28653

   ## What is the purpose of the change
   
   `SinkV2ITCase.writerAndCommitterExecuteInStreamingModeWithScaling` is the dominant flaky failure on the release-1.20 nightlies (19 of the last 25 failed runs, 37 test errors, across all JDK/profile legs). The test body always passes; the failure is a teardown race. The checkpoint directory is injected as a per-method JUnit `@TempDir`, but the class-shared MiniCluster keeps discarding checkpoint state asynchronously after `requestJobResult().get()` returns (on a successful FINISH the job's completed checkpoints are discarded via the CheckpointCoordinator/CheckpointsCleaner IO executor as fire-and-forget work). JUnit's temp-directory deletion then races with Flink's discard and fails with `IOException: Failed to delete temp directory` (suppressed `NoSuchFileException`, i.e. files vanishing mid-walk).
   
   The race is latent on master too (identical `@TempDir` wiring and `requestJobResult().get()`); it is fixed here first so the change can be backported to the release branches where it currently manifests (FLINK-40074).
   
   ## Brief change log
   
   - Manage the scaling test's checkpoint directory explicitly (`Files.createTempDirectory`) instead of injecting it via `@TempDir`.
   - Clean it up in a `finally` block with `FileUtils.deleteDirectoryQuietly`, which is safe against concurrent deletions and never lets a cleanup failure mask a real assertion failure.
   - No change to the test's assertions or job logic.
   
   ## Verifying this change
   
   This change is already covered by the existing test `SinkV2ITCase.writerAndCommitterExecuteInStreamingModeWithScaling`. Verified locally: the method passes (3/3 parameterized cases) and `spotless:check` is clean. On release-1.20, where the race manifests, the backported change ran green across repeated executions.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no (test-only change)
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (Claude Fable 5)
   
   Generated-by: Claude Fable 5
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40074][tests] Avoid @TempDir cleanup race in SinkV2ITCase scaling test [flink]

Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on PR #28653:
URL: https://github.com/apache/flink/pull/28653#issuecomment-4929862046

   @MartijnVisser , @spuru9 looks like this PR does not help
   another repro in master https://github.com/apache/flink/actions/runs/29041506045/job/86201515542#step:13:7316


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40074][tests] Avoid @TempDir cleanup race in SinkV2ITCase scaling test [flink]

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser commented on PR #28653:
URL: https://github.com/apache/flink/pull/28653#issuecomment-4897089258

   > Also 2.1 is left.
   
   We don't support 2.1 anymore, so it's 1.20, 2.2 and 2.3 at this moment


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40074][tests] Avoid @TempDir cleanup race in SinkV2ITCase scaling test [flink]

Posted by "spuru9 (via GitHub)" <gi...@apache.org>.
spuru9 commented on PR #28653:
URL: https://github.com/apache/flink/pull/28653#issuecomment-4894159375

   JIRA mention 1.20 as the affected version, could we updated it before merging the PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40074][tests] Avoid @TempDir cleanup race in SinkV2ITCase scaling test [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #28653:
URL: https://github.com/apache/flink/pull/28653#issuecomment-4893734089

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "6a99fec07503099fb14dc0e623f67358b91f3203",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6a99fec07503099fb14dc0e623f67358b91f3203",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 6a99fec07503099fb14dc0e623f67358b91f3203 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40074][tests] Avoid @TempDir cleanup race in SinkV2ITCase scaling test [flink]

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser merged PR #28653:
URL: https://github.com/apache/flink/pull/28653


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org