You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rshdhere (via GitHub)" <gi...@apache.org> on 2026/10/15 20:51:03 UTC
[PR] Core: Keep metadata_log_entries snapshot details after expiration [iceberg]
rshdhere opened a new pull request, #18529:
URL: https://github.com/apache/iceberg/pull/18529
The latest_* columns were derived from the current snapshot log, which expiring snapshots trims, so older entries started returning null. Read them from the metadata file each entry points to, only when one of those columns is projected. If that file is gone, fall back to the previous lookup instead of failing the scan.
Fixes #17761
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Keep metadata_log_entries snapshot details after expiration [iceberg]
Posted by "rshdhere (via GitHub)" <gi...@apache.org>.
rshdhere commented on PR #18529:
URL: https://github.com/apache/iceberg/pull/18529#issuecomment-6280163925
Thanks @uros-b, both points make sense.
1. I'll switch the missing-file case to match #16859: on `NotFoundException`, return null and LOG.warn, and let `RuntimeIOException` propagate. I'll drop the `snapshotIdAsOfTime` fallback, so a missing file no longer shows up as a time-derived row.
2. For the tests, `snapshotColumnsNotProjected` will spy on the `FileIO` and verify(io, never()).`newInputFile`(previous.file()), like `doesNotLoadPropertiesWhenNotProjected`. In `snapshotDetailsAfterExpiration`, I'll replace the after.get(1) check with assertions that first has second's snapshot ID, schema ID and sequence number from when it was current.
I'll leave the prefetch for a follow-up shared with #16859, and I won't skip reads at or after the oldest retained snapshot-log timestamp, since that would bring back #17761.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Keep metadata_log_entries snapshot details after expiration [iceberg]
Posted by "rshdhere (via GitHub)" <gi...@apache.org>.
rshdhere commented on PR #18529:
URL: https://github.com/apache/iceberg/pull/18529#issuecomment-6213909026
PTAL @uros-b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Keep metadata_log_entries snapshot details after expiration [iceberg]
Posted by "rshdhere (via GitHub)" <gi...@apache.org>.
rshdhere commented on PR #18529:
URL: https://github.com/apache/iceberg/pull/18529#issuecomment-6280499863
done with it https://github.com/apache/iceberg/pull/18529/commits/792b300251f63953b9fdc1fb065fc7c84a589ebb, now let's wait for the CI to be green and ig will be ready to merge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Keep metadata_log_entries snapshot details after expiration [iceberg]
Posted by "rshdhere (via GitHub)" <gi...@apache.org>.
rshdhere commented on PR #18529:
URL: https://github.com/apache/iceberg/pull/18529#issuecomment-6354564636
PTAL @uros-b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org