[GitHub] [arrow-datafusion] crepererum commented on pull request #5545: Support arbitrary user defined partition column in `ListingTable` (rather than assuming they are always Dictionary encoded)

["crepererum (via GitHub)" <gi...@apache.org>]

crepererum commented on PR #5545:
URL: https://github.com/apache/arrow-datafusion/pull/5545#issuecomment-1466503229

   > to something like (to force the API to change) ... `Field`
   
   `Field` contains other information that is irrelevant here like nullability. We could assert on the correct value though (i.e. not nullable). Also this only concerns on of the two API changes, the other one (`PartitionedFile`) is still silently breaking. To avoid that we would need to rename `partition_values` to something else. However I'm not sure if a rename would trigger people to not only follow the rename but also change the types correctly.
   
   > And then add some a method and docs to help the upgrade
   
   Since the user needs to adjust their code anyways, I'm not sure if such a random method helps. It's hard to find and because the uint16 dict type is so arbitrary, I'm not sure it should exist in the long run.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org