You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (JIRA)" <ji...@apache.org> on 2019/02/12 03:00:00 UTC
[jira] [Resolved] (SPARK-26740) Statistics for date and timestamp
columns depend on system time zone
[ https://issues.apache.org/jira/browse/SPARK-26740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-26740.
---------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 23662
[https://github.com/apache/spark/pull/23662]
> Statistics for date and timestamp columns depend on system time zone
> --------------------------------------------------------------------
>
> Key: SPARK-26740
> URL: https://issues.apache.org/jira/browse/SPARK-26740
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Maxim Gekk
> Assignee: Maxim Gekk
> Priority: Major
> Fix For: 3.0.0
>
>
> While saving statistics for timestamp/date columns, default time zone is used in conversion of internal type (microseconds or days since epoch) to textual representation. The textual representation doesn't contain time zone. So, when it is converted back to internal types (Long for TimestampType or DateType), the Timestamp.valueOf and Date.valueOf are used in conversions. The methods use current system time zone.
> If system time zone is different while saving and retrieving statistics for timestamp/date columns, restored microseconds/days since epoch will be different.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org