You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Micah Kornfield (Jira)" <ji...@apache.org> on 2020/06/29 03:17:00 UTC

[jira] [Created] (ARROW-9264) [C++] Cleanup Parquet Arrow Schema code

Micah Kornfield created ARROW-9264:
--------------------------------------

             Summary: [C++] Cleanup Parquet Arrow Schema code
                 Key: ARROW-9264
                 URL: https://issues.apache.org/jira/browse/ARROW-9264
             Project: Apache Arrow
          Issue Type: Sub-task
          Components: C++
            Reporter: Micah Kornfield
            Assignee: Micah Kornfield


We need a function/class that can take the parquet schema and a proposed arrow schema (potentially retrieved from parquet metadata) and outputs a data structure that contains, all of the information in "SchemaField" and the following additional options:

 

1.  Corresponding Definition level for nullability (wouldn't be populated for non-null arrays).

2.  Correspond Repetition level for lists (wouldn't be populated for for non-lists).

3.  Definition level for "empty lists".  (wouldn't be populated for legacy two level encoded lists).

 

One option is to augment and populate these on the SchemaField.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)