You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "alexjbuck (via GitHub)" <gi...@apache.org> on 2025/12/16 20:41:29 UTC
[I] Add the ability to transactionally create named references (tags) to a snapshot in the FastAppendAction [iceberg-rust]
alexjbuck opened a new issue, #1939:
URL: https://github.com/apache/iceberg-rust/issues/1939
### Is your feature request related to a problem or challenge?
In the current implementation you cannot simultaneously create a snapshot through the FastAppendAction transaction API _and_ add a tag to the new snapshot reference.
Without the atomic transaction guarantees, it is possible for other processes to add new snapshots
and cleanup old snapshots (including the one _we_ just created) before a tag can be added to the
snapshot, thus protecting it from most automated snapshot expiration policies.
With the atomic guarantees, we can guarantee that either our data was not committed into the table,
or it was _and_ it has a tagged snapshot reference that is protected from expiry.
The FastAppendAction currently creates a new snapshot an then updates the main reference to point to this new snapshot.
There is no set of Transaction actions that supports the SetSnapshotRef update for custom tags.
### Describe the solution you'd like
I would like a method (`.with_tag(tag:&str)`?) on the `FastAppendAction` that allows for this sort of calling pattern:
```rust
let tx = Transaction::new(&table);
let action = tx
.fast_append()
.add_data_files(data_files)
.with_tag("v1.0.0");
let tx = action.apply(tx)?;
let table = tx.commit(&catalog).await?;
```
This extends the fast_append() action which already uses the `SetSnapshotRef`--but specifically for the `main` branch--to be able to also `SetSnapshotRef` for custom tags.
### Willingness to contribute
I can contribute to this feature independently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Add the ability to transactionally create named references (tags) to a snapshot in the FastAppendAction [iceberg-rust]
Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 commented on issue #1939:
URL: https://github.com/apache/iceberg-rust/issues/1939#issuecomment-3690677601
Hi, @alexjbuck I have left comments here: https://github.com/apache/iceberg-rust/pull/1940#pullrequestreview-3609727411
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Add the ability to transactionally create named references (tags) to a snapshot in the FastAppendAction [iceberg-rust]
Posted by "hippalus (via GitHub)" <gi...@apache.org>.
hippalus commented on issue #1939:
URL: https://github.com/apache/iceberg-rust/issues/1939#issuecomment-4794613503
Hi @liurenjie1024 I'd like to pick this up. I'll add a dedicated ManageSnapshotsAction modeled on Java's [ManageSnapshots](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/SnapshotManager.java) / [UpdateSnapshotReferencesOperation](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/UpdateSnapshotReferencesOperation.java), rather than extending FastAppendAction.
First PR will focus on snapshot ref management, create/remove/replace/rename branches and tags, retention setters, and fast-forward. Rollback and cherrypick will come as follow-ups.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Add the ability to transactionally create named references (tags) to a snapshot in the FastAppendAction [iceberg-rust]
Posted by "alexjbuck (via GitHub)" <gi...@apache.org>.
alexjbuck commented on issue #1939:
URL: https://github.com/apache/iceberg-rust/issues/1939#issuecomment-3690681683
Okay! I'll look into that sort of implementation. I'm not familiar with the Java API. Did you ask for the design change so the result is consistent with an existing API implementation i.e. Java's?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Add the ability to transactionally create named references (tags) to a snapshot in the FastAppendAction [iceberg-rust]
Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 commented on issue #1939:
URL: https://github.com/apache/iceberg-rust/issues/1939#issuecomment-3690684155
> Okay! I'll look into that sort of implementation. I'm not familiar with the Java API. Did you ask for the design change so the result is consistent with an existing API implementation i.e. Java's?
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Add the ability to transactionally create named references (tags) to a snapshot in the FastAppendAction [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1939:
URL: https://github.com/apache/iceberg-rust/issues/1939#issuecomment-4784821502
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org