You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/06/30 20:12:36 UTC

[GitHub] [arrow-datafusion] alamb commented on issue #6784: The `dict_id` was lost when constructing the logic plan.

alamb commented on issue #6784:
URL: https://github.com/apache/arrow-datafusion/issues/6784#issuecomment-1615148606

   I would love to know what you think about dict_id handling in general -- from what I can see so far it is not well supported in arrow-rs. We have similar problems with `metadata` which can be hung off a schema or a field and gets lost frequently
   
   I am also not 100% clear if dict_id is supposed to (potentially) different per record batch or if it would be the same for the entire plan 
   
   One thing that might be possible is to compare the pointer for the dictionary array to decide if it was the same dictionary rather than trying to keep `dict_id` all the way through the plan 🤔 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org