You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2021/07/27 18:52:00 UTC

[jira] [Created] (ARROW-13467) [C++] Support delta dictionaries in the IPC file format

Weston Pace created ARROW-13467:
-----------------------------------

             Summary: [C++] Support delta dictionaries in the IPC file format
                 Key: ARROW-13467
                 URL: https://issues.apache.org/jira/browse/ARROW-13467
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C++
            Reporter: Weston Pace


The spec states:

> In the file format, there is no requirement that dictionary keys should be defined in a
> {{DictionaryBatch}} before they are used in a {{RecordBatch}}, as long as the keys are
> defined somewhere in the file. Further more, it is invalid to have more than one *non-delta*
> dictionary batch per dictionary ID (i.e. dictionary replacement is not supported). Delta
> dictionaries are applied in the order they appear in the file footer.

This seems to suggest that dictionary deltas (but not replacements) are allowed.  However, both reading and writing an IPC file with a dictionary delta fails.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)