You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "jonathanc-n (via GitHub)" <gi...@apache.org> on 2025/04/02 02:38:02 UTC
[PR] feat: Add basic conflict detection [iceberg-rust]
jonathanc-n opened a new pull request, #1154:
URL: https://github.com/apache/iceberg-rust/pull/1154
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
- Mentioned [here](https://github.com/apache/iceberg-rust/issues/964)
## What changes are included in this PR?
Added basic conflict detection. The current idea is that before commit we can use catalog to refresh the table state before commit to compare with the base metadata.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat: Add `refresh` to get get updated `TableMetadata` [iceberg-rust]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #1154:
URL: https://github.com/apache/iceberg-rust/pull/1154#discussion_r2083093675
##########
crates/iceberg/src/transaction/mod.rs:
##########
@@ -127,12 +127,14 @@ impl<'a> Transaction<'a> {
}
/// Creates a fast append action.
- pub fn fast_append(
+ pub async fn fast_append(
Review Comment:
Makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat: Add `refresh` to get get updated `TableMetadata` [iceberg-rust]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on PR #1154:
URL: https://github.com/apache/iceberg-rust/pull/1154#issuecomment-2787625768
Is there an easy way to be able to create dummy catalog instances and pass them into new table instances for testing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat: Add basic conflict detection [iceberg-rust]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on PR #1154:
URL: https://github.com/apache/iceberg-rust/pull/1154#issuecomment-2771198141
Is this on track with what we want? cc @liurenjie1024 @Fokko @ZENOTME
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat: Add basic conflict detection [iceberg-rust]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #1154:
URL: https://github.com/apache/iceberg-rust/pull/1154#issuecomment-2772487148
@jonathanc-n This is the very first step, and we indeed want this.
I don't think this is the actual conflict detection. When between the `fast_append()` and `commit()` something changes, it will still trigger a conflict. A next step will be to do basic conflict detection, and check if the operations are compatible. For example, such as: https://github.com/apache/iceberg-python/pull/1772, see https://github.com/apache/iceberg-python/issues/1678.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat: Add `refresh` to get get updated `TableMetadata` [iceberg-rust]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n closed pull request #1154: feat: Add `refresh` to get get updated `TableMetadata`
URL: https://github.com/apache/iceberg-rust/pull/1154
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat: Add basic conflict detection [iceberg-rust]
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on code in PR #1154:
URL: https://github.com/apache/iceberg-rust/pull/1154#discussion_r2024859346
##########
crates/iceberg/src/transaction/mod.rs:
##########
@@ -127,12 +127,14 @@ impl<'a> Transaction<'a> {
}
/// Creates a fast append action.
- pub fn fast_append(
+ pub async fn fast_append(
Review Comment:
According to iceberg-java, looks like we don't need to refresh here. The lifecycle of a transaction looks like this:
- create transaction and get current table
- apply action to current table
- apply action to current table
- commit transaction
- refresh and get the latest table, see: https://github.com/apache/iceberg/blob/c661a71091e496393c743ddd879d9e1a0f2747b2/core/src/main/java/org/apache/iceberg/BaseTransaction.java#L503
- reapply all update before and commit to catalog
I think can delay this part after #949. cc @Fokko @liurenjie1024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat: Add basic conflict detection [iceberg-rust]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on PR #1154:
URL: https://github.com/apache/iceberg-rust/pull/1154#issuecomment-2773774727
@Fokko @ZENOTME Yeah will change title, I just wanted to open this to discuss what our plan was for the conflict detection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org