[jira] [Created] (SPARK-31984) Make micros rebasing functions via local timestamps pure

["Maxim Gekk (Jira)" <ji...@apache.org>]

Maxim Gekk created SPARK-31984:
----------------------------------

             Summary: Make micros rebasing functions via local timestamps pure
                 Key: SPARK-31984
                 URL: https://issues.apache.org/jira/browse/SPARK-31984
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.0.1, 3.1.0
            Reporter: Maxim Gekk


The functions rebaseGregorianToJulianMicros(zoneId: ZoneId, ...) and rebaseJulianToGregorianMicros(zoneId: ZoneId, ...) accept the zone id as the first parameter but use it only while forming ZonedDateTime and ignore in Java 7 GregorianCalendar. The Calendar instance uses the default JVM time zone internally. This causes the following problems:
# The functions depend on the global state variable. And calling the functions from different threads can return wrong results if the the default JVM time zone is changed during the execution.
# It is impossible to speed up generation of JSON files with diff/switch via parallelisation.
# The functions don't fully use the passed ZoneId.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org