You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Karen Coppage (Jira)" <ji...@apache.org> on 2020/02/05 13:36:00 UTC
[jira] [Updated] (HIVE-21216) Write Parquet INT64 timestamp
[ https://issues.apache.org/jira/browse/HIVE-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karen Coppage updated HIVE-21216:
---------------------------------
Description:
This patch enables Hive to start writing int64 timestamps in Parquet.
With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics)
enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
The timestamp will have LocalDateTime semantics (not converted to UTC).
Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL.
Time unit (granularity) will be determined by the user. Default is milliseconds.
was:
[WIP] NO PRECOMMIT TESTS
This patch enables Hive to start writing int64 timestamps in Parquet.
With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics)
enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
The timestamp will have LocalDateTime semantics (not converted to UTC).
Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL.
Time unit (granularity) will be determined by the user. Default is milliseconds.
> Write Parquet INT64 timestamp
> -----------------------------
>
> Key: HIVE-21216
> URL: https://issues.apache.org/jira/browse/HIVE-21216
> Project: Hive
> Issue Type: New Feature
> Components: Hive
> Reporter: Karen Coppage
> Assignee: Karen Coppage
> Priority: Major
> Labels: breaking_change
> Attachments: HIVE-21216.1.patch
>
>
> This patch enables Hive to start writing int64 timestamps in Parquet.
> With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
> boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics)
> enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
> The timestamp will have LocalDateTime semantics (not converted to UTC).
> Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL.
> Time unit (granularity) will be determined by the user. Default is milliseconds.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)