You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Max Gekk (Jira)" <ji...@apache.org> on 2021/04/18 06:05:00 UTC
[jira] [Resolved] (SPARK-35116) The generated data fits the
precision of DayTimeIntervalType in spark
[ https://issues.apache.org/jira/browse/SPARK-35116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Max Gekk resolved SPARK-35116.
------------------------------
Fix Version/s: 3.2.0
Resolution: Fixed
Issue resolved by pull request 32212
[https://github.com/apache/spark/pull/32212]
> The generated data fits the precision of DayTimeIntervalType in spark
> ---------------------------------------------------------------------
>
> Key: SPARK-35116
> URL: https://issues.apache.org/jira/browse/SPARK-35116
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: jiaan.geng
> Assignee: jiaan.geng
> Priority: Major
> Fix For: 3.2.0
>
>
> The precision of java.time.Duration is nanosecond, but when it is used as DayTimeIntervalType in Spark, it is microsecond.
> At present, the DayTimeIntervalType data generated in the implementation of RandomDataGenerator is accurate to nanosecond, which will cause the DayTimeIntervalType to be converted to long, and then back to DayTimeIntervalType to lose the accuracy, which will cause the test to fail. For example: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137390/testReport/org.apache.spark.sql.hive.execution/HashAggregationQueryWithControlledFallbackSuite/udaf_with_all_data_types/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org