[GitHub] [arrow-datafusion] tustvold commented on pull request #5545: Support arbitrary user defined partition column in `ListingTable` (rather than assuming they are always Dictionary encoded)

["tustvold (via GitHub)" <gi...@apache.org>]

tustvold commented on PR #5545:
URL: https://github.com/apache/arrow-datafusion/pull/5545#issuecomment-1587656337

   > provide the set of unique values
   
   FWIW this isn't how dictionaries are implemented today, there are various situations where the dictionary will contain values not referenced by an index, and/or the same value repeated multiple times. As it currently stands primitive dictionaries will almost always be less efficient both from a memory usage and performance standpoint 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org