You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "xndai (via GitHub)" <gi...@apache.org> on 2026/07/24 21:29:25 UTC
[PR] Core: Fix time-travel snapshot lookup to not assume snapshot-log orde… [iceberg]
xndai opened a new pull request, #17360:
URL: https://github.com/apache/iceberg/pull/17360
…ring (#17358)
nullableSnapshotIdAsOfTime previously relied on snapshot-log entries being in chronological order, returning the last entry with timestamp <= requested. The spec does not guarantee ordering, and TableMetadata allows up to 1 minute of clock skew between entries. This could return the wrong snapshot when entries are out of order.
Fix by finding the entry with the maximum timestamp <= requested timestamp, which is correct regardless of iteration order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Fix time-travel snapshot lookup to not assume snapshot-log orde… [iceberg]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #17360:
URL: https://github.com/apache/iceberg/pull/17360#discussion_r3670072213
##########
core/src/main/java/org/apache/iceberg/util/SnapshotUtil.java:
##########
@@ -390,9 +390,12 @@ public static long snapshotIdAsOfTime(Table table, long timestampMillis) {
public static Long nullableSnapshotIdAsOfTime(Table table, long timestampMillis) {
Long snapshotId = null;
+ long bestTimestamp = Long.MIN_VALUE;
for (HistoryEntry logEntry : table.history()) {
- if (logEntry.timestampMillis() <= timestampMillis) {
+ if (logEntry.timestampMillis() <= timestampMillis
+ && logEntry.timestampMillis() > bestTimestamp) {
Review Comment:
Makes sense to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Fix time-travel snapshot lookup to not assume snapshot-log orde… [iceberg]
Posted by "xndai (via GitHub)" <gi...@apache.org>.
xndai commented on code in PR #17360:
URL: https://github.com/apache/iceberg/pull/17360#discussion_r3650599197
##########
core/src/main/java/org/apache/iceberg/util/SnapshotUtil.java:
##########
@@ -390,9 +390,12 @@ public static long snapshotIdAsOfTime(Table table, long timestampMillis) {
public static Long nullableSnapshotIdAsOfTime(Table table, long timestampMillis) {
Long snapshotId = null;
+ long bestTimestamp = Long.MIN_VALUE;
for (HistoryEntry logEntry : table.history()) {
- if (logEntry.timestampMillis() <= timestampMillis) {
+ if (logEntry.timestampMillis() <= timestampMillis
+ && logEntry.timestampMillis() > bestTimestamp) {
Review Comment:
it doesn't really matter. In this implementation the first snapshot appears in the chain would be picked. Since this is something undefined by the spec, I feel it's ok to have such behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Fix time-travel snapshot lookup to not assume snapshot-log orde… [iceberg]
Posted by "xndai (via GitHub)" <gi...@apache.org>.
xndai commented on PR #17360:
URL: https://github.com/apache/iceberg/pull/17360#issuecomment-5079399546
> we are trying to fix this with having monotonic timestamps of snapshot in v4
>
> https://github.com/apache/iceberg/pull/16294/changes
>
> Can we trust these timestamps in the first place as there is no requirement of true time semantics in first place, so i wonder if its worth fixing the old behaviour vs just saying hey v4 is when its gonna get fixed ?
Thanks for letting me know that. I'd say it still worth fixing as we still have v1, v2 and v3 tables out there. Once v4 becomes official, we could add a short circuit break in this loop for v4 as we can assume there are strictly ordered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Fix time-travel snapshot lookup to not assume snapshot-log orde… [iceberg]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY merged PR #17360:
URL: https://github.com/apache/iceberg/pull/17360
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Fix time-travel snapshot lookup to not assume snapshot-log orde… [iceberg]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #17360:
URL: https://github.com/apache/iceberg/pull/17360#discussion_r3648775798
##########
core/src/main/java/org/apache/iceberg/util/SnapshotUtil.java:
##########
@@ -390,9 +390,12 @@ public static long snapshotIdAsOfTime(Table table, long timestampMillis) {
public static Long nullableSnapshotIdAsOfTime(Table table, long timestampMillis) {
Long snapshotId = null;
+ long bestTimestamp = Long.MIN_VALUE;
for (HistoryEntry logEntry : table.history()) {
- if (logEntry.timestampMillis() <= timestampMillis) {
+ if (logEntry.timestampMillis() <= timestampMillis
+ && logEntry.timestampMillis() > bestTimestamp) {
Review Comment:
Could there be snapshot with equal timestamps? Maybe we should go with `logEntry.timestampMillis() >= bestTimestamp`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org