You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Joris Peeters (Jira)" <ji...@apache.org> on 2021/03/02 16:19:00 UTC

[jira] [Created] (ARROW-11838) [C++] Support reading IPC data with shared dictionaries

Joris Peeters created ARROW-11838:
-------------------------------------

             Summary: [C++] Support reading IPC data with shared dictionaries
                 Key: ARROW-11838
                 URL: https://issues.apache.org/jira/browse/ARROW-11838
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C++
            Reporter: Joris Peeters


The Arrow format supports dictionaries being re-used across multiple columns. At present, neither reading nor writing these is supported from C++ (and thus Python). 

Support for writing these would require extending the interfaces, but reading might be a relatively quick win - and this issue is concerned with the latter only.

The Java implementation supports writing, so can be used to generate test data. 

Once the C++ side is handled, pyarrow should automatically be able to convert such tables to pandas, with appropriate use of categoricals. This should also be verified, and otherwise addressed.

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)