You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "MartijnVisser (via GitHub)" <gi...@apache.org> on 2026/10/20 14:56:14 UTC

[PR] [FLINK-34427][tests] Wait for slot manager close() in runTest [flink]

MartijnVisser opened a new pull request, #29533:
URL: https://github.com/apache/flink/pull/29533

   ## What is the purpose of the change
   
   Fixes the exit code 239 fork crash in `FineGrainedSlotManagerTest` (FLINK-34427). `runTest` returned without waiting for `close()`, so `afterAll` could shut the executor down under a running requirements check:
   
   ```
   ERROR org.apache.flink.util.FatalExitExceptionHandler [] - FATAL: Thread 'pool-429-thread-1' produced an uncaught exception. Stopping the process...
   java.util.concurrent.CompletionException: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@44a6cb42[...]
   	at org.apache.flink.runtime.resourcemanager.slotmanager.DefaultSlotStatusSyncer.allocateSlot(DefaultSlotStatusSyncer.java:148)
   ```
   
   This replaces #29301 by @snuyanzin, whose cancels in `suspend()` and `DefaultSlotStatusSyncer#close()` miss a check that is already running. The slot manager's own scheduling after `suspend()`, which @zentol raised on the ticket, is a separate change.
   
   ## Brief change log
   
     - `runTest` waits for `close()`. A failing `close()` now fails the test instead of ending the JVM through `FutureUtils#assertNoException`.
   
   ## Verifying this change
   
   This change is already covered by existing tests, such as `FineGrainedSlotManagerTest` and `AbstractFineGrainedSlotManagerITCase`. Measured in a Linux container pinned to one CPU, with surefire's JVM arguments and the CI log4j config:
   
     - With a `requestSlot` that blocks until the executor is shut down (at most 2 s), master (855597a5460) crashes with the CI stack trace in 10 of 10 JVMs and this change in 0 of 30 class runs. Same on release-2.3 and release-2.2 (JDK 17) and on release-1.20 (JDK 8).
     - Without the blocking `requestSlot`, master on JDK 17 crashes in 4 of 306 class runs, and with this change 0 of 800. The same loop on the previous base 5af0f51cfe4 gave 12 of 498 and 0 of 800. release-1.20 on JDK 8 crashes in 8 of 329, and with this change 0 of 1500.
     - The slot manager tests pass on all four branches.
   
   A test that fails while a check is still running still skips `close()` and can crash the fork the same way; I'll file a follow-up for that.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Code (Claude Opus 5.5, Claude Fable 5.1)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-34427][tests] Wait for slot manager close() in runTest [flink]

Posted by "XComp (via GitHub)" <gi...@apache.org>.
XComp commented on code in PR #29533:
URL: https://github.com/apache/flink/pull/29533#discussion_r4329914653


##########
flink-runtime/src/test/java/org/apache/flink/runtime/resourcemanager/slotmanager/FineGrainedSlotManagerTest.java:
##########
@@ -935,6 +939,141 @@ private void testAccessMetricValueDuringItsUnregister(
                 });
     }
 
+    /**
+     * Tests that {@code runTest} waits for {@code close()} while a requirements check is still

Review Comment:
   I'm not sure whether we should add these tests. They essentially test test code rather than production code. 🤔 I guess it's good to have created it for verification purposes but let's not keep it for the merge.



##########
flink-runtime/src/test/java/org/apache/flink/runtime/resourcemanager/slotmanager/FineGrainedSlotManagerTestBase.java:
##########
@@ -234,19 +230,22 @@ protected final void runTest(RunnableWithException testMethod) throws Exception
                                     resourceEventListenerBuilder.build(),
                                     blockedTaskManagerChecker));
 
-            testMethod.run();
-
-            CompletableFuture<Void> closeFuture = new CompletableFuture<>();
-            runInMainThread(
-                    () -> {
-                        try {
-                            slotManager.close();
-                        } catch (Exception e) {
-                            closeFuture.completeExceptionally(e);
-                        }
-                        closeFuture.complete(null);
-                    });
-            FutureUtils.assertNoException(closeFuture);
+            Throwable failure = null;
+            try {
+                testMethod.run();
+            } catch (Throwable t) {
+                failure = t;
+            }
+            // Wait for close(), so afterAll cannot shut the executor down under a running
+            // requirements check. A failing close() does not hide the test's own failure.
+            try {
+                FutureUtils.runAsync(slotManager::close, mainThreadExecutor).get();

Review Comment:
   ```suggestion
                   runInMainThreadAndWaitslotManager::close);
   ```
   nit: we could use the newly added utility method



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-34427][tests] Wait for slot manager close() in runTest [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #29533:
URL: https://github.com/apache/flink/pull/29533#issuecomment-6294053688

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "3757810755476d0259a2334baad863f39ec3dc5e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3757810755476d0259a2334baad863f39ec3dc5e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3757810755476d0259a2334baad863f39ec3dc5e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-34427][tests] Wait for slot manager close() in runTest [flink]

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser merged PR #29533:
URL: https://github.com/apache/flink/pull/29533


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-34427][tests] Wait for slot manager close() in runTest [flink]

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser commented on code in PR #29533:
URL: https://github.com/apache/flink/pull/29533#discussion_r4334493942


##########
flink-runtime/src/test/java/org/apache/flink/runtime/resourcemanager/slotmanager/FineGrainedSlotManagerTest.java:
##########
@@ -935,6 +939,141 @@ private void testAccessMetricValueDuringItsUnregister(
                 });
     }
 
+    /**
+     * Tests that {@code runTest} waits for {@code close()} while a requirements check is still

Review Comment:
   Dropped them. The numbers from `testRunTestWaitsForCloseWhileRequirementsCheckIsRunning` stay in the PR body.



##########
flink-runtime/src/test/java/org/apache/flink/runtime/resourcemanager/slotmanager/FineGrainedSlotManagerTestBase.java:
##########
@@ -234,19 +230,22 @@ protected final void runTest(RunnableWithException testMethod) throws Exception
                                     resourceEventListenerBuilder.build(),
                                     blockedTaskManagerChecker));
 
-            testMethod.run();
-
-            CompletableFuture<Void> closeFuture = new CompletableFuture<>();
-            runInMainThread(
-                    () -> {
-                        try {
-                            slotManager.close();
-                        } catch (Exception e) {
-                            closeFuture.completeExceptionally(e);
-                        }
-                        closeFuture.complete(null);
-                    });
-            FutureUtils.assertNoException(closeFuture);
+            Throwable failure = null;
+            try {
+                testMethod.run();
+            } catch (Throwable t) {
+                failure = t;
+            }
+            // Wait for close(), so afterAll cannot shut the executor down under a running
+            // requirements check. A failing close() does not hide the test's own failure.
+            try {
+                FutureUtils.runAsync(slotManager::close, mainThreadExecutor).get();

Review Comment:
   Done. The helper now takes a `RunnableWithException`, since `close()` throws.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-34427][tests] Wait for slot manager close() in runTest [flink]

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser commented on PR #29533:
URL: https://github.com/apache/flink/pull/29533#issuecomment-6311396788

   Filed FLINK-41059 for Chesnay's comment.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org