You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/01/04 10:01:25 UTC
[GitHub] [iceberg] xloya opened a new pull request #3842: [Core][Rewrite] Fix merge manifests lost sequence number for added files when rewriting using staring sequence number
xloya opened a new pull request #3842:
URL: https://github.com/apache/iceberg/pull/3842
We use the starting-sequence-number configuration when rewriting, then the data written in this rewrite will set the `status` to `ADDED` in the manifest entry, and set the `sequence_number` to the sequence number of the snapshot read before the rewrite. If this rewrite also triggers the `manifest merge` at the end, the sequence number of the new added data file's manifest entry will be lost.
For example:
We rewrite the snapshot with sequence number equals to `1` and enable `use-starting-sequence-number`, which triggers the rewriting of some data files and the merging of manifests. It is expected that the sequence number of the new manifest entries of the new rewritten data files should be `1`. When actually rewriting, because the manifest merge is triggered, the current logic sets the sequence number to `null`, which does not meet the actual expectations.
cc @rdblue @jackye1995 @RussellSpitzer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] xloya commented on pull request #3842: [Core][Rewrite] Fix merge manifests lost sequence number for added files when rewriting using starting sequence number
Posted by GitBox <gi...@apache.org>.
xloya commented on pull request #3842:
URL: https://github.com/apache/iceberg/pull/3842#issuecomment-1005319020
@rdblue Thanks for your review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #3842: [Core][Rewrite] Fix merge manifests lost sequence number for added files when rewriting using staring sequence number
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #3842:
URL: https://github.com/apache/iceberg/pull/3842#issuecomment-1005138709
Good catch, @xloya! Thank you for fixing this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #3842: [Core][Rewrite] Fix merge manifests lost sequence number for added files when rewriting using staring sequence number
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #3842:
URL: https://github.com/apache/iceberg/pull/3842
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org