[GitHub] [iceberg] Simon0806 opened a new issue #1370: commit transaction failed when numSnapshots in retainLast function >= actual num of snapshot.

[GitBox <gi...@apache.org>]

Simon0806 opened a new issue #1370:
URL: https://github.com/apache/iceberg/issues/1370


   My Test Code is :
    ```
     // Generate **_3 snapshots._**
      table.newAppend()
           .appendFile(FILE_A)
           .commit();
       table.newAppend()
           .appendFile(FILE_B)
           .commit();
       table.newAppend()
           .appendFile(FILE_C)
           .commit();
   
       Transaction transaction = table.newTransaction();
       transaction.expireSnapshots()
           .expireOlderThan(System.currentTimeMillis())
           .retainLast(**_3_**)   // **_retain last 3 snapshots. will throw exception, when this is smaller than 3, it works._**
           .commit();
   
       transaction.commitTransaction(); 
   ```
   
   When executing this test， it raises error like this:
   
   ```
    java.lang.IllegalStateException: Cannot commit transaction: last operation has not committed
           at org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkState(Preconditions.java:508)
           at org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:203)
           at org.apache.iceberg.TestMergeAppend.testManifestMergeMinCount(TestMergeAppend.java:221)
   ```
   
   This is because the commit method in RemoveSnapshots.java only commit the operation when num of snapshots are different , like this:
   ```
             if (updated.snapshots().size() != base.snapshots().size()) {
               ops.commit(base, updated);
   ```
    When num in retainLast is larger than or equal with current num of snapshots, it will skip operation commit. So when doing transaction's commitTransaction, lastBase always equal with current because there is no any operation commit happened, so it throws the exception above.
   like this:
   ```
   @Override
     public void commitTransaction() {
       Preconditions.checkState(lastBase != current,
           "Cannot commit transaction: last operation has not committed"); 
   ...
   }
   ```
   I have tried to remove the if condition above, do commit every time and it seems work, (All UTs are passed). But I'm not sure the reason. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] rdblue closed issue #1370: commit transaction failed when numSnapshots in retainLast function >= actual num of snapshot.

[GitBox <gi...@apache.org>]

rdblue closed issue #1370:
URL: https://github.com/apache/iceberg/issues/1370


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org