[jira] [Commented] (IMPALA-7472) Consider removing TimestampValue::FromSubsecondUnixTime

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-7472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344720#comment-17344720 ] 

ASF subversion and git services commented on IMPALA-7472:
---------------------------------------------------------

Commit 4697db021411e95165cfd199e1aa3fc848dbfc8b in impala's branch refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4697db0 ]

IMPALA-5121: Fix AVG() on timestamp col with use_local_tz_for_unix_timestamp_conversions

AVG used to contain a back and forth timezone conversion if
use_local_tz_for_unix_timestamp_conversions is true. This could
affect the results if there were values from different DST rules.

Note that AVG on timestamps has other issues besides this, see
IMPALA-7472 for details.

Testing:
- added a regression test

Change-Id: I999099de8e07269b96b75d473f5753be4479cecd
Reviewed-on: http://gerrit.cloudera.org:8080/17412
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Consider removing TimestampValue::FromSubsecondUnixTime
> -------------------------------------------------------
>
>                 Key: IMPALA-7472
>                 URL: https://issues.apache.org/jira/browse/IMPALA-7472
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>            Reporter: Csaba Ringhofer
>            Priority: Minor
>              Labels: timestamp
>
> TimestampValue::FromSubsecondUnixTime converts a double interpreted as unix time in seconds to TimestampValue. Impala uses it in two cases:
> 1: double <-> timestamp casting
> 2: aggregate function AVG on timestamps - it converts timestamp to double, calculates the average on doubles, and converts the average back to timestamp.
> My concern about this is double's varying's precision depending on the distance from 0 (1970-01-01) - subsec timestamps around 2018 are rounded/distorted with several nanosecs, which can add up if a large number of timestamps are averaged.
> I think that decimal (with nanosec precision) could be used for both cases:
> 1. Decimal could used as an intermediate stage - double < - >decimal and decimal < - > timestamp conversions have well defined rounding rules since decimal V2.
> 2. Decimal could be used as aggregation state - this would probably make the AVG slower, but it would be much more precise.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org