You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (JIRA)" <ji...@apache.org> on 2018/08/09 16:53:00 UTC
[jira] [Commented] (IMPALA-5050) Add support to read
TIMESTAMP_MILLIS and TIMESTAMP_MICROS to the parquet scanner
[ https://issues.apache.org/jira/browse/IMPALA-5050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575124#comment-16575124 ]
Csaba Ringhofer commented on IMPALA-5050:
-----------------------------------------
I have done some benchmarks with https://gerrit.cloudera.org/#/c/11057/ :
reading a parquet file with 9999999 timestamps took 0.42 sec with int64 milli/micro, and 0.32 sec with int96 nano (my work desktop machine, release build, quite stable results).
With some changes in sub-second unix time->TimestampValue conversion I could bring down reading milli/micro to 0.32 too.
I plan to do this change in a separate commit, IMPALA-7417 is created to track this.
> Add support to read TIMESTAMP_MILLIS and TIMESTAMP_MICROS to the parquet scanner
> --------------------------------------------------------------------------------
>
> Key: IMPALA-5050
> URL: https://issues.apache.org/jira/browse/IMPALA-5050
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend
> Affects Versions: Impala 2.9.0
> Reporter: Lars Volker
> Assignee: Csaba Ringhofer
> Priority: Major
>
> This requires updating {{parquet.thrift}} to a version that includes the {{TIMESTAMP_MICROS}} logical type.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org