You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "technicolorbeat (via GitHub)" <gi...@apache.org> on 2026/08/12 19:41:32 UTC
[I] REST: lazy-load unreferenced snapshots when snapshot-loading-mode=refs [iceberg-go]
technicolorbeat opened a new issue, #1792:
URL: https://github.com/apache/iceberg-go/issues/1792
### Proposed Change
When `snapshot-loading-mode=refs` is set on the REST catalog, `LoadTable` sends `?snapshots=refs` and the server returns only snapshots referenced by a named branch or tag. This reduces response size for tables with long snapshot histories.
However, there is currently no way for a caller to lazily fetch the remaining snapshots if they are needed — for example, to time-travel to a snapshot that is not referenced by any branch or tag. Such a request surfaces as a plain "snapshot not found" error with no signal that the snapshot may exist but was simply not loaded.
The Java `RESTSessionCatalog` handles this via a `snapshotsSupplier` — a lazy supplier that fetches the full snapshot list on demand when a snapshot lookup fails. iceberg-go has no equivalent.
**Proposed change:**
- When a snapshot lookup fails and `snapshot-loading-mode=refs` is in use, the catalog should detect that the metadata may be incomplete and trigger a follow-up `loadTable` call with `?snapshots=all` to obtain the full snapshot list.
- Alternatively, expose a mechanism for callers to explicitly request a full reload.
- The error returned to the caller should distinguish "snapshot absent from partial metadata" from "snapshot genuinely does not exist".
**Spec reference:**
REST Catalog Open API – loadTable:
https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/apache/iceberg/apache-iceberg-1.10.1/open-api/rest-catalog-open-api.yaml#tag/Catalog-API/operation/loadTable
**Related:**
- #1748 — added `snapshot-loading-mode` catalog property
- apache/iceberg-go#1749 — implementation PR; lazy loading deferred as follow-up
**Willingness to contribute**
- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
### Proposal document
_No response_
### Specifications
- [x] Table
- [ ] View
- [x] REST
- [ ] Puffin
- [ ] Encryption
- [ ] Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] REST: lazy-load unreferenced snapshots when snapshot-loading-mode=refs [iceberg-go]
Posted by "technicolorbeat (via GitHub)" <gi...@apache.org>.
technicolorbeat commented on issue #1792:
URL: https://github.com/apache/iceberg-go/issues/1792#issuecomment-5428564403
> [@technicolorbeat](https://github.com/technicolorbeat) [@zeroshade](https://github.com/zeroshade) I'd like to take this if it's still available.
>
> I saw this was deferred from [#1749](https://github.com/apache/iceberg-go/pull/1749) as a follow-up, and the issue notes you'd be willing to implement it. Are you still planning to work on it? If not, I'm happy to take the Java `snapshotsSupplier` path (lazy `?snapshots=all` reload when a lookup misses under `snapshot-loading-mode=refs`).
yes, I'm working on this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] REST: lazy-load unreferenced snapshots when snapshot-loading-mode=refs [iceberg-go]
Posted by "dgvj-work (via GitHub)" <gi...@apache.org>.
dgvj-work commented on issue #1792:
URL: https://github.com/apache/iceberg-go/issues/1792#issuecomment-5374744818
@technicolorbeat @zeroshade I'd like to take this if it's still available.
I saw this was deferred from #1749 as a follow-up, and the issue notes you'd be willing to implement it. Are you still planning to work on it? If not, I'm happy to take the Java `snapshotsSupplier` path (lazy `?snapshots=all` reload when a lookup misses under `snapshot-loading-mode=refs`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org