You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Csaba Ringhofer (JIRA)" <ji...@apache.org> on 2018/11/26 18:15:00 UTC

[jira] [Commented] (IMPALA-7889) Write new logical types in Parquet

    [ https://issues.apache.org/jira/browse/IMPALA-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699386#comment-16699386 ] 

Csaba Ringhofer commented on IMPALA-7889:
-----------------------------------------

A note about string columns and marking them as UTF8:
IMPALA-2069 added a query option to decide whether to mark String columns as UTF8 or not and left the default as False. The rationale was that setting the default to True could break existing workloads, because some tools may behave differently depending on this information. The plan was to set the default to True in 3.0. but this did not happen.

> Write new logical types in Parquet
> ----------------------------------
>
>                 Key: IMPALA-7889
>                 URL: https://issues.apache.org/jira/browse/IMPALA-7889
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>            Reporter: Csaba Ringhofer
>            Assignee: Csaba Ringhofer
>            Priority: Major
>              Labels: parquet
>
> Parquet added LogicalTypeAnnotation as a new way to express metadata about columns. For types that already exist, the old way (ConvertedType) should be also written to remain compatible with older readers, but new types (like int64 nano) are only added as LogicalTypeAnnotation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org