[jira] [Created] (IMPALA-7472) Consider removing TimestampValue::FromSubsecondUnixTime

["Csaba Ringhofer (JIRA)" <ji...@apache.org>]

Csaba Ringhofer created IMPALA-7472:
---------------------------------------

             Summary: Consider removing TimestampValue::FromSubsecondUnixTime
                 Key: IMPALA-7472
                 URL: https://issues.apache.org/jira/browse/IMPALA-7472
             Project: IMPALA
          Issue Type: Bug
          Components: Backend
            Reporter: Csaba Ringhofer


TimestampValue::FromSubsecondUnixTime converts a double interpreted as unix time in seconds to TimestampValue. Impala uses it in two cases:
1: double <-> timestamp casting
2: aggregate function AVG on timestamps - it converts timestamp to double, calculates the average on doubles, and converts the average back to timestamp.

My concern about this is double's varying's precision depending on the distance from 0 (1970-01-01) - subsec timestamps around 2018 are rounded/distorted with several nanosecs, which can add up if a large number of timestamps are averaged.

I think that decimal (with nanosec precision) could be used for both cases:
1. Decimal could used as an intermediate stage - double <->decimal and decimal<-> timestamp conversions have well defined rounding rules since decimal V2.
2. Decimal could be used as aggregation state - this would probably make the AVG slower, but it would be much more precise.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org