You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ZENOTME (via GitHub)" <gi...@apache.org> on 2025/03/18 11:19:52 UTC
[I] Support RowDeltaAction [iceberg-rust]
ZENOTME opened a new issue, #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104
### Is your feature request related to a problem or challenge?
As #798、#1081, there are requirements for append delete data files (position delete, equality delete). This action is used to support the append of this kind of file.
### Describe the solution you'd like
The path to support:
- [ ] add conflict detection
- [ ] add retry logic #964
- [ ] complete RowDeltaAction
### Willingness to contribute
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support RowDeltaAction [iceberg-rust]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on issue #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2746763897
One thing I would like to implement before the filter manager are the residual predicates mentioned by @Fokko [here](https://github.com/apache/iceberg-rust/pull/377#discussion_r1611209532)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support RowDeltaAction [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-3530269972
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support RowDeltaAction [iceberg-rust]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on issue #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2742239268
I'll take a deeper look into the implementation tomorrow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support RowDeltaAction [iceberg-rust]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on issue #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2746763712
@ZENOTME Sorry for the delay, i was a little busy with family. In the example you showed in the python implementation, the filtering for `OverwriteFiles` and `DeleteFiles` seems to be inherited from the snapshot producer. We could create the `MergingSnapshotManager` or something similar and have it be called when an action that produces a new snapshot. I think we do still need some things before this happens.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support RowDeltaAction [iceberg-rust]
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on issue #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2732833919
cc @Fokko @liurenjie1024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support RowDeltaAction [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on issue #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2888118740
I've created https://github.com/apache/iceberg-rust/issues/1344 to add validate logic, which should be a prerequisite of this issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support RowDeltaAction [iceberg-rust]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on issue #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2742154713
For metadata conflict detection, what is the exact design outline that you are looking to implement?
For the row level detection I can start the implementation the manifest filter manager and manifest merge manager to build towards the merging snapshot producer used in the RowDelta. This can probably be done after delete files are fully implemented
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support RowDeltaAction [iceberg-rust]
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on issue #1104:
URL: https://github.com/apache/iceberg-rust/issues/1104#issuecomment-2742230817
> For metadata conflict detection, what is the exact design outline that you are looking to implement?
conflict detection implementation based on the validation phase. I would like to introduce the validation phase at SnapshotProduce apply(). After introducing it, we can have some specific implementation of kinds of validation.
> For the row level detection I can start the implementation the manifest filter manager and manifest merge manager to build towards the merging snapshot producer used in the RowDelta. This can probably be done after delete files are fully implemented
Thanks @jonathanc-n!
- For manifest merge manager, I think it has been cover in #902.
- For manifest filter manager, maybe we need it. One interesting finding is that pyiceberg don't have the manifest filter manager. In this abstraction, [it filter the delete entry directly](https://github.com/apache/iceberg-python/blob/c06e32034fbd32d5aab9b2ef19ad2d253141479d/pyiceberg/table/update/snapshot.py#L542). I haven't dive into the manifest filter manager, is needed in the future? cc @Fokko @kevinjqliu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org