You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2023/01/11 15:28:08 UTC
[GitHub] [arrow] fatemehp commented on pull request #17877: PARQUET-2225:[C++][Parquet] Allow reading dense with RecordReader
fatemehp commented on PR #17877:
URL: https://github.com/apache/arrow/pull/17877#issuecomment-1378948765
Adding more clarification here.
The change proposed here is about the vector of values that is returned. Currently, we first come up with the location of the null values and then make a vector that has an empty space for the null values (reading spaced). When reading dense, we do not make space for the null values.
For example:
def_levels: [0, 1]
values: [10]
Reading spaced: [null, 10]
Reading dense: [10]
The change here is meaningful for nullable columns. The savings come when we have null values. The issue is that 1) it is inefficient to come up with the exact space of the nulls and move the values around to make space for null values and 2) Some readers may want to indeed read dense and so they have to move the null values out again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org