You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Weston Pace (Jira)" <ji...@apache.org> on 2021/07/27 18:52:00 UTC
[jira] [Created] (ARROW-13467) [C++] Support delta dictionaries in
the IPC file format
Weston Pace created ARROW-13467:
-----------------------------------
Summary: [C++] Support delta dictionaries in the IPC file format
Key: ARROW-13467
URL: https://issues.apache.org/jira/browse/ARROW-13467
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Weston Pace
The spec states:
> In the file format, there is no requirement that dictionary keys should be defined in a
> {{DictionaryBatch}} before they are used in a {{RecordBatch}}, as long as the keys are
> defined somewhere in the file. Further more, it is invalid to have more than one *non-delta*
> dictionary batch per dictionary ID (i.e. dictionary replacement is not supported). Delta
> dictionaries are applied in the order they appear in the file footer.
This seems to suggest that dictionary deltas (but not replacements) are allowed. However, both reading and writing an IPC file with a dictionary delta fails.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)