You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (JIRA)" <ji...@apache.org> on 2018/11/15 16:24:00 UTC
[jira] [Updated] (IMPALA-7853) Add support to read int64 NANO
timestamps to the parquet scanner
[ https://issues.apache.org/jira/browse/IMPALA-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Csaba Ringhofer updated IMPALA-7853:
------------------------------------
Description:
PARQUET-1387 added int64 timestamps with nanosecond precision.
As 64 bits are not enough to represent the whole 1400..9999 range of Impala timestamps, so this new new type works with a limited range:
1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
The benefit of the reduced range is that no validation is necessary during scanning, as every possible 64 bit value represents a valid timestamp in Impala. This may mean that this has the potential be the fastest way to store timestamps.
Another way NANO differs from MICRO and MILLI is that NANO can be be only described with new logical types in Parquet, it has no converted type equivalent.
was:
PARQUET-1387 added int64 timestamps with nanosecond precision.
As 64 bits are not enough to represent the whole 1400..9999 range of Impala timestamps, so this new new type works with a limited range:
1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
The benefit of the reduced range is that no validation is necessary during scanning, as every possible 64 bit value represents a valid timestamp in Impala. This may mean that this has the potential be the fastest way to store timestamps.
> Add support to read int64 NANO timestamps to the parquet scanner
> ----------------------------------------------------------------
>
> Key: IMPALA-7853
> URL: https://issues.apache.org/jira/browse/IMPALA-7853
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Major
>
> PARQUET-1387 added int64 timestamps with nanosecond precision.
> As 64 bits are not enough to represent the whole 1400..9999 range of Impala timestamps, so this new new type works with a limited range:
> 1677-09-21 00:12:43.145224192 .. 2262-04-11 23:47:16.854775807 UTC
> The benefit of the reduced range is that no validation is necessary during scanning, as every possible 64 bit value represents a valid timestamp in Impala. This may mean that this has the potential be the fastest way to store timestamps.
> Another way NANO differs from MICRO and MILLI is that NANO can be be only described with new logical types in Parquet, it has no converted type equivalent.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org