You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/07/04 17:59:04 UTC
[GitHub] [iceberg] bryanck opened a new pull request, #5198: Arrow: Fix for dictionary encoded fixed length binary decimals
bryanck opened a new pull request, #5198:
URL: https://github.com/apache/iceberg/pull/5198
This PR fixes the vectorized reader for decimals that are fixed length binary and dictionary-encoded. Before these decimals would be downcast to 8 byte (long) precision. This only affects Parquet V2, as fixed length binary decimals aren't dictionary-encoded in Parquet V1. The Spark vectorized reader test for Parquet V2 was modified so this is now tested.
Also included is a minor refactor of the big endian padding to share some common logic and make it easier to enhance later.
In addition, the Spark benchmark test for the vectorized reader was enhanced to include both long encoded and fixed width binary encoded decimals.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #5198: Arrow: Fix for dictionary encoded fixed length binary decimals
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #5198:
URL: https://github.com/apache/iceberg/pull/5198#issuecomment-1175602634
This looks good to me. Thanks for fixing it, @bryanck!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #5198: Arrow: Fix for dictionary encoded fixed length binary decimals
Posted by GitBox <gi...@apache.org>.
rdblue merged PR #5198:
URL: https://github.com/apache/iceberg/pull/5198
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org