You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "sungwy (via GitHub)" <gi...@apache.org> on 2024/08/05 16:49:57 UTC
[I] Prevent `add_files` from adding a file that's already referenced by the Iceberg Table [iceberg-python]
sungwy opened a new issue, #998:
URL: https://github.com/apache/iceberg-python/issues/998
### Feature Request / Improvement
Currently `add_files` doesn't have a check to prevent adding an object that's already referenced by the Iceberg Table.
We should include these two checks to prevent bad behaviors of adding an already referenced data file as a new manifest entry.
We could do this by running the following two checks before the file addition:
1. First check that the list of `file_paths` is unique
2. Check that all the files in the `file_paths` aren't referenced by any of the manifests in the current snapshot of the Iceberg Table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Prevent `add_files` from adding a file that's already referenced by the Iceberg Table [iceberg-python]
Posted by "amitgilad3 (via GitHub)" <gi...@apache.org>.
amitgilad3 commented on issue #998:
URL: https://github.com/apache/iceberg-python/issues/998#issuecomment-2269615389
Hey, im new to the pyiceberg but would love to take take a crack at this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Prevent `add_files` from adding a file that's already referenced by the Iceberg Table [iceberg-python]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy closed issue #998: Prevent `add_files` from adding a file that's already referenced by the Iceberg Table
URL: https://github.com/apache/iceberg-python/issues/998
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Prevent `add_files` from adding a file that's already referenced by the Iceberg Table [iceberg-python]
Posted by "sungwy (via GitHub)" <gi...@apache.org>.
sungwy commented on issue #998:
URL: https://github.com/apache/iceberg-python/issues/998#issuecomment-2269917041
Hi @amitgilad3 sounds great! I'll get this assigned to you. Please let me know if you'd like some pointers :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Prevent `add_files` from adding a file that's already referenced by the Iceberg Table [iceberg-python]
Posted by "amitgilad3 (via GitHub)" <gi...@apache.org>.
amitgilad3 commented on issue #998:
URL: https://github.com/apache/iceberg-python/issues/998#issuecomment-2281990423
Hey @sungwy - just created my first pr #1036 , would really appreciate your review and if you have any suggestions or if i choose the wrong place to implement my checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org