You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "DavidCGarcia89 (via GitHub)" <gi...@apache.org> on 2023/03/21 11:05:17 UTC

[GitHub] [iceberg] DavidCGarcia89 opened a new issue, #7157: expireSnapshotId in Scala is deleting other Snapshots?

DavidCGarcia89 opened a new issue, #7157:
URL: https://github.com/apache/iceberg/issues/7157

   ### Query engine
   
   Spark 3.3.0
   
   
   ### Question
   
   I'm trying to delete specific snapshots ids with the objective of keep the first snapshot of each month plus the last 7 days. I have already the list of snapshots id to delete in the `snapshotsToDelete` variable, but when I apply this, it doesn't work as expected:
   
   `
   snapshotsToDelete.take(snapshotsToDelete.count().toInt).foreach { 
         snapshot_id => 
         println(s"Deleting snapshot_id: $snapshot_id");
         table.expireSnapshots.expireSnapshotId(snapshot_id.getLong(0)).commit();
       }
   `
   
   I'm printing those delete shapshots correctly, and it only prints those from the list, but it is deleting older snapshots too. Is this the expected behaviour (maybe there are dependencies between snapshots) or it is a bug?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] RussellSpitzer commented on issue #7157: expireSnapshotId in Scala is deleting other Snapshots?

Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on issue #7157:
URL: https://github.com/apache/iceberg/issues/7157#issuecomment-1478062177

   You are just missing the other defaults. Just because you specify a set of Id's doesn't mean the other properties are not used. The default for timing is 5 days, so just set that to infinity or a few years and you should be set.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] DavidCGarcia89 commented on issue #7157: expireSnapshotId in Scala is deleting other Snapshots?

Posted by "DavidCGarcia89 (via GitHub)" <gi...@apache.org>.
DavidCGarcia89 commented on issue #7157:
URL: https://github.com/apache/iceberg/issues/7157#issuecomment-1479355190

   @RussellSpitzer thank you! Here is the code to change the property.
   
       val maxSnapshotAgeMS = (5L * 365 * 24 * 60 * 60 * 1000).toString // 5 years retention
   
       table.updateProperties()
       .set("history.expire.max-snapshot-age-ms", maxSnapshotAgeMS)
       .commit()


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] DavidCGarcia89 closed issue #7157: expireSnapshotId in Scala is deleting other Snapshots?

Posted by "DavidCGarcia89 (via GitHub)" <gi...@apache.org>.
DavidCGarcia89 closed issue #7157: expireSnapshotId in Scala is deleting other Snapshots?
URL: https://github.com/apache/iceberg/issues/7157


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org