Re: [PR] Flink: Fix duplicate commits in DynamicCommitter when Flink jobId changes on restart [iceberg]

["pvary (via GitHub)" <gi...@apache.org>]

pvary commented on code in PR #16011:
URL: https://github.com/apache/iceberg/pull/16011#discussion_r3315497391


##########
flink/v2.1/flink/src/test/java/org/apache/iceberg/flink/sink/dynamic/TestDynamicCommitter.java:
##########
@@ -512,8 +607,7 @@ void testTableBranchAtomicCommitForAppendOnlyData() throws Exception {
     // Two committables, one for each snapshot / table / branch.
     assertThat(table.snapshots()).hasSize(2);
 
-    Snapshot snapshot1 = Iterables.getFirst(table.snapshots(), null);
-    assertThat(snapshot1.snapshotId()).isEqualTo(table.refs().get("branch1").snapshotId());

Review Comment:
   So the new code could result in cases where the commit from checkpoint 2 is added to the table sooner than  a commit for checkpoint 1.
   
   I can accept that this is ok, if the commits are for different branches. The question is: do we have tests that ensure that if we commit them in the same cycle, they are committed in the expected order?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org