You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/10/10 14:20:39 UTC

[GitHub] [arrow] nevi-me commented on pull request #8402: ARROW-8426: [Rust] [Parquet] - Add more support for converting Dicts

nevi-me commented on pull request #8402:
URL: https://github.com/apache/arrow/pull/8402#issuecomment-706556497


   Parquet's dictionary encoding is a complexity on its own. My understanding's that after a certain size, the dictionary no longer grows, but the additional values are stored the normal way. I'm still to spend more time on parquet-mr and the format.
   I think the approach of not forcing Arrow dictionaries to have Parquet dictionary encoding is good.
   
   > also only supports Int32 index types in this commit, also removes NULLs
   
   Do you want to work on other index types and supporting primitive Arrow dictionaries? We could keep this PR open for longer; as long as it's not blocking any additional unit of work.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org