[jira] [Commented] (SPARK-35581) Casting special strings to DATE/TIMESTAMP returns inconsistent results

["Apache Spark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-35581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17404707#comment-17404707 ] 

Apache Spark commented on SPARK-35581:
--------------------------------------

User 'MaxGekk' has created a pull request for this issue:
https://github.com/apache/spark/pull/33840

> Casting special strings to DATE/TIMESTAMP returns inconsistent results
> ----------------------------------------------------------------------
>
>                 Key: SPARK-35581
>                 URL: https://issues.apache.org/jira/browse/SPARK-35581
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.2.0
>            Reporter: Max Gekk
>            Assignee: Max Gekk
>            Priority: Major
>             Fix For: 3.2.0
>
>
> When casting the special values "now", "today", "tomorrow", and "yesterday" to DATE/TIMESTAMP, Spark may return inconsistent results.
> Looks like Spark runs the expression on each executor, on every row independently. So the results could differ across executors if they have different system time, and across rows because of the resolution of "now".
> https://github.com/databricks/runtime/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala#L876



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org