You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "Gang Wu (Jira)" <ji...@apache.org> on 2023/02/21 04:39:00 UTC
[jira] [Updated] (ORC-1376) [C++] Support schema evolution

     [ https://issues.apache.org/jira/browse/ORC-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gang Wu updated ORC-1376:
-------------------------
    Description: 
The Java ORC reader supports schema evolution by comparing readSchema with fileSchema [1] and creates a set of ConvertTreeReaders [2] to do the conversion. The conversion happens when reading columnar data into vector and evaluating expressions for predicate push down. The C++ reader does not support it yet.

The type conversion on the Java side originates from Apache Hive. We should define what conversions are supposed to implement on the C++ side.

[1] https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/impl/SchemaEvolution.java
[2] https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/impl/ConvertTreeReaderFactory.java#L60:14

  was:
The Java ORC reader supports schema evolution by comparing readSchema with fileSchema [1] and creates a set of ConvertTreeReaders [2] to do the conversion.

[1] https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/impl/SchemaEvolution.java
[2] https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/impl/ConvertTreeReaderFactory.java#L60:14

The C++ reader does not support it yet.



> [C++] Support schema evolution
> ------------------------------
>
>                 Key: ORC-1376
>                 URL: https://issues.apache.org/jira/browse/ORC-1376
>             Project: ORC
>          Issue Type: New Feature
>          Components: C++
>            Reporter: Gang Wu
>            Priority: Major
>
> The Java ORC reader supports schema evolution by comparing readSchema with fileSchema [1] and creates a set of ConvertTreeReaders [2] to do the conversion. The conversion happens when reading columnar data into vector and evaluating expressions for predicate push down. The C++ reader does not support it yet.
> The type conversion on the Java side originates from Apache Hive. We should define what conversions are supposed to implement on the C++ side.
> [1] https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/impl/SchemaEvolution.java
> [2] https://github.com/apache/orc/blob/main/java/core/src/java/org/apache/orc/impl/ConvertTreeReaderFactory.java#L60:14



--
This message was sent by Atlassian Jira
(v8.20.10#820010)