[jira] [Commented] (SPARK-35116) The generated data fits the precision of DayTimeIntervalType in spark

["Apache Spark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-35116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324278#comment-17324278 ] 

Apache Spark commented on SPARK-35116:
--------------------------------------

User 'beliefer' has created a pull request for this issue:
https://github.com/apache/spark/pull/32212

> The generated data fits the precision of DayTimeIntervalType in spark
> ---------------------------------------------------------------------
>
>                 Key: SPARK-35116
>                 URL: https://issues.apache.org/jira/browse/SPARK-35116
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 3.2.0
>            Reporter: jiaan.geng
>            Priority: Major
>
> The precision of java.time.Duration is nanosecond, but when it is used as DayTimeIntervalType in Spark, it is microsecond.
> At present, the DayTimeIntervalType data generated in the implementation of RandomDataGenerator is accurate to nanosecond, which will cause the DayTimeIntervalType to be converted to long, and then back to DayTimeIntervalType to lose the accuracy, which will cause the test to fail. For example: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137390/testReport/org.apache.spark.sql.hive.execution/HashAggregationQueryWithControlledFallbackSuite/udaf_with_all_data_types/



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org