You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "lincoln-lil (via GitHub)" <gi...@apache.org> on 2025/07/10 13:36:26 UTC

[PR] [FLINK-38082][datastream] Avoid unexpected retries when user async function invokes completeExceptionally [flink]

lincoln-lil opened a new pull request, #26779:
URL: https://github.com/apache/flink/pull/26779

   ## What is the purpose of the change
   As discussed in https://lists.apache.org/thread/8gncfyy42cmj6xtyk4fqvpxwf22xk9x5, an issue that unexpected extra retries will be triggered when user async function invoke completeExceptionally in timeout.
   We can fix this by adding a start timestamp for future handler to help check if a retry still within configured timeout.
   
   ## Brief change log
   * adding a start timestamp for future handler to help check if a retry still within configured timeout.
   
   ## Verifying this change
   newly added test cases:
   * testProcessingTimeWithAlwaysTimeoutFunctionUnorderedWithRetry
   * testProcessingTimeWithAlwaysTimeoutFunctionOrderedWithRetry
   
   ## Does this pull request potentially affect one of the following parts:
     - Dependencies (does it add or upgrade a dependency): (no)
     - The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
     - The serializers: (no )
     - The runtime per-record code paths (performance sensitive): (no)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
     - The S3 file system connector: (no)
   
   ## Documentation
     - Does this pull request introduce a new feature? (no)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38082][datastream] Avoid unexpected retries when user async function invokes completeExceptionally [flink]

Posted by "AlanConfluent (via GitHub)" <gi...@apache.org>.

AlanConfluent commented on code in PR #26779:
URL: https://github.com/apache/flink/pull/26779#discussion_r2198899881


##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/operators/async/AsyncWaitOperatorTest.java:
##########
@@ -1333,6 +1333,50 @@ private void testProcessingTimeAlwaysTimeoutFunctionWithRetry(AsyncDataStream.Ou
         }
     }
 
+    @Test
+    void testProcessingTimeWithAlwaysTimeoutFunctionUnorderedWithRetry() throws Exception {
+        testProcessingTimeAlwaysTimeoutFunction(AsyncDataStream.OutputMode.UNORDERED);
+    }
+
+    @Test
+    void testProcessingTimeWithAlwaysTimeoutFunctionOrderedWithRetry() throws Exception {
+        testProcessingTimeAlwaysTimeoutFunction(AsyncDataStream.OutputMode.ORDERED);
+    }
+
+    private void testProcessingTimeAlwaysTimeoutFunction(AsyncDataStream.OutputMode mode)
+            throws Exception {
+        StreamTaskMailboxTestHarnessBuilder<Integer> builder =
+                new StreamTaskMailboxTestHarnessBuilder<>(
+                                OneInputStreamTask::new, BasicTypeInfo.INT_TYPE_INFO)
+                        .addInput(BasicTypeInfo.INT_TYPE_INFO);
+
+        try (StreamTaskMailboxTestHarness<Integer> testHarness =
+                builder.setupOutputForSingletonOperatorChain(
+                                new AsyncWaitOperatorFactory<Integer, Integer>(
+                                        new AlwaysTimeoutAsyncFunction(),
+                                        TIMEOUT,
+                                        10,
+                                        mode,
+                                        exceptionRetryStrategy))
+                        .build()) {
+
+            final long initialTime = 0L;
+            AtomicReference<Throwable> error = new AtomicReference<>();
+            testHarness.getStreamMockEnvironment().setExternalExceptionHandler(error::set);
+
+            try {
+                testHarness.processElement(new StreamRecord<>(1, initialTime + 1));
+                testHarness.processElement(new StreamRecord<>(2, initialTime + 2));
+                while (error.get() == null) {
+                    testHarness.processAll();
+                }
+            } catch (Exception e) {
+                error.set(e);
+            }
+            ExceptionUtils.assertThrowableWithMessage(error.get(), "Dummy timeout error");

Review Comment:
   If you counted the number of invocations, you could verify that it only ever tries once, even after the timeout exception.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [FLINK-38082][datastream] Avoid unexpected retries when user async function invokes completeExceptionally [flink]

Posted by "AlanConfluent (via GitHub)" <gi...@apache.org>.

AlanConfluent commented on code in PR #26779:
URL: https://github.com/apache/flink/pull/26779#discussion_r2201101003


##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/operators/async/AsyncWaitOperatorTest.java:
##########
@@ -1333,6 +1333,50 @@ private void testProcessingTimeAlwaysTimeoutFunctionWithRetry(AsyncDataStream.Ou
         }
     }
 
+    @Test
+    void testProcessingTimeWithAlwaysTimeoutFunctionUnorderedWithRetry() throws Exception {
+        testProcessingTimeAlwaysTimeoutFunction(AsyncDataStream.OutputMode.UNORDERED);
+    }
+
+    @Test
+    void testProcessingTimeWithAlwaysTimeoutFunctionOrderedWithRetry() throws Exception {
+        testProcessingTimeAlwaysTimeoutFunction(AsyncDataStream.OutputMode.ORDERED);
+    }
+
+    private void testProcessingTimeAlwaysTimeoutFunction(AsyncDataStream.OutputMode mode)
+            throws Exception {
+        StreamTaskMailboxTestHarnessBuilder<Integer> builder =
+                new StreamTaskMailboxTestHarnessBuilder<>(
+                                OneInputStreamTask::new, BasicTypeInfo.INT_TYPE_INFO)
+                        .addInput(BasicTypeInfo.INT_TYPE_INFO);
+
+        try (StreamTaskMailboxTestHarness<Integer> testHarness =
+                builder.setupOutputForSingletonOperatorChain(
+                                new AsyncWaitOperatorFactory<Integer, Integer>(
+                                        new AlwaysTimeoutAsyncFunction(),
+                                        TIMEOUT,
+                                        10,
+                                        mode,
+                                        exceptionRetryStrategy))
+                        .build()) {
+
+            final long initialTime = 0L;
+            AtomicReference<Throwable> error = new AtomicReference<>();
+            testHarness.getStreamMockEnvironment().setExternalExceptionHandler(error::set);
+
+            try {
+                testHarness.processElement(new StreamRecord<>(1, initialTime + 1));
+                testHarness.processElement(new StreamRecord<>(2, initialTime + 2));
+                while (error.get() == null) {
+                    testHarness.processAll();
+                }
+            } catch (Exception e) {
+                error.set(e);
+            }
+            ExceptionUtils.assertThrowableWithMessage(error.get(), "Dummy timeout error");

Review Comment:
   Great, thanks for the update.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org