[GitHub] [iceberg] pvary commented on a diff in pull request #5642: Flink: Fixed an issue where Flink batch entry was not accurate

[GitBox <gi...@apache.org>]

pvary commented on code in PR #5642:
URL: https://github.com/apache/iceberg/pull/5642#discussion_r960242991


##########
flink/v1.13/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergStreamWriter.java:
##########
@@ -83,9 +83,18 @@ public void dispose() throws Exception {
   @Override
   public void endInput() throws IOException {
     // For bounded stream, it may don't enable the checkpoint mechanism so we'd better to emit the
-    // remaining
-    // completed files to downstream before closing the writer so that we won't miss any of them.
+    // remaining completed files to downstream before closing the writer so that we won't miss any
+    // of them.
+    // Note that if the task is not closed after calling endInput, checkpoint may be triggered again
+    // causing files to be sent repeatedly, so we need to set write to null after the file is sent.
+
+    if (writer == null) {
+      return;
+    }
     emit(writer.complete());
+
+    // Set write to null to prevent multiple calls to the complete method of the same write
+    writer = null;

Review Comment:
   Thanks for the info @xuzhiwen1255 and @stevenzwu !



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org