[GitHub] [flink] pnowojski commented on a change in pull request #16905: [FLINK-23884][checkpoint] Fix the concurrent problem between triggering savepoint with drain and task finishing

[GitBox <gi...@apache.org>]

pnowojski commented on a change in pull request #16905:
URL: https://github.com/apache/flink/pull/16905#discussion_r695787852



##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/MultipleInputStreamTask.java
##########
@@ -244,6 +235,29 @@ protected void createInputProcessor(
         return resultFuture;
     }
 
+    private CompletableFuture<Boolean> triggerStopWithSavepointWithDrainAsync(
+            CheckpointMetaData checkpointMetaData, CheckpointOptions checkpointOptions) {
+        mainMailboxExecutor.execute(
+                () -> {
+                    setSynchronousSavepoint(checkpointMetaData.getCheckpointId(), true);
+                    operatorChain.getSourceTaskInputs().forEach(s -> s.getOperator().stop());
+                },
+                "stop chained Flip-27 source for stop-with-savepoint --drain");
+
+        CompletableFuture<Void> sourcesStopped =
+                FutureUtils.waitForAll(
+                        operatorChain.getSourceTaskInputs().stream()
+                                .map(s -> s.getOperator().getFinishedFuture())
+                                .collect(Collectors.toList()));

Review comment:
       Maybe it would be better to collect the futures inside the mailbox action `mainMailboxExecutor.submit()`? And keep the previous signature of `SourceOperator#stop()` returning `finished` future? 
   
   As it is we are here assuming `operatorChain.getSourceTaskInputs()` and `s.getOperator().getFinishedFuture()` are thread safe, which I think they are (because they are immutable), but it's not part of the guarantees.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org