You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/09/01 23:48:28 UTC
[GitHub] [iceberg] rdblue opened a new pull request, #5691: Spark: Fix stats in rewrite metadata action
rdblue opened a new pull request, #5691:
URL: https://github.com/apache/iceberg/pull/5691
This is an update to #5665. There was discussion on that PR about how to project the partition tuple to the correct final type, and this PR implements the update by passing the desired data file type into `StructDataFile`. I think this is the cleanest way to fix the problem.
Because `StructDataFile` does not implement `StructLike` and is instead passed into a `ManifestWriter` that uses the `DataFile` API, the projection can't happen in `writeManifest`. Inside `StructDataFile`, only the partition tuple's type can differ, so only that field needs to be projected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5691: Spark: Fix stats in rewrite metadata action
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5691:
URL: https://github.com/apache/iceberg/pull/5691#issuecomment-1235656791
Thanks for the review, @Fokko! And nice work finding the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko merged pull request #5691: Spark: Fix stats in rewrite metadata action
Posted by GitBox <gi...@apache.org>.
Fokko merged PR #5691:
URL: https://github.com/apache/iceberg/pull/5691
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5691: Spark: Fix stats in rewrite metadata action
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5691:
URL: https://github.com/apache/iceberg/pull/5691#issuecomment-1234918265
@Fokko, I took a look at #5665 and got it working. I'm opening this so that you have the option to either cherry-pick my changes into your PR or just go ahead and merge this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org