You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Marta Kuczora (Jira)" <ji...@apache.org> on 2020/01/20 15:56:00 UTC
[jira] [Updated] (HIVE-21215) Read Parquet INT64 timestamp
[ https://issues.apache.org/jira/browse/HIVE-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marta Kuczora updated HIVE-21215:
---------------------------------
Description:
This patch enables Hive to start reading timestamps from Parquet written with the new semantics:
With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
* boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics).
* enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
Upon reading, the semantics of these new timestamps will be determined by their metadata, while the semantics of INT96 timestamps will continue to be deduced from the writer metadata.
This feature will be behind a flag for now.
was:
[WIP]
This patch enables Hive to start reading timestamps from Parquet written with the new semantics:
With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
* boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics).
* enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
Upon reading, the semantics of these new timestamps will be determined by their metadata, while the semantics of INT96 timestamps will continue to be deduced from the writer metadata.
This feature will be behind a flag for now.
> Read Parquet INT64 timestamp
> ----------------------------
>
> Key: HIVE-21215
> URL: https://issues.apache.org/jira/browse/HIVE-21215
> Project: Hive
> Issue Type: New Feature
> Reporter: Karen Coppage
> Assignee: Marta Kuczora
> Priority: Major
>
> This patch enables Hive to start reading timestamps from Parquet written with the new semantics:
> With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
> * boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics).
> * enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
> Upon reading, the semantics of these new timestamps will be determined by their metadata, while the semantics of INT96 timestamps will continue to be deduced from the writer metadata.
> This feature will be behind a flag for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)