[jira] [Created] (SPARK-26902) Support java.time.Instant as an external type of TimestampType

["Maxim Gekk (JIRA)" <ji...@apache.org>]

Maxim Gekk created SPARK-26902:
----------------------------------

             Summary: Support java.time.Instant as an external type of TimestampType
                 Key: SPARK-26902
                 URL: https://issues.apache.org/jira/browse/SPARK-26902
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.4.0
            Reporter: Maxim Gekk


Currently, Spark supports the java.sql.Date and java.sql.Timestamp types as external types for Catalyst's DateType and TimestampType. It accepts and produces values of such types. Since Java 8, base classes for dates and timestamps are java.time.Instant, java.time.LocalDate/LocalDateTime, and java.time.ZonedDateTime. Need to add new converters from/to Instant.

The Instant type holds epoch seconds (and nanoseconds), and directly reflects to Catalyst's TimestampType.

Main motivations for the changes:
- Smoothly support Java 8 time API
- Avoid inconsistency of calendars used inside Spark 3.0 (Proleptic Gregorian calendar) and inside of java.sql.Timestamp (hybrid calendar - Julian + Gregorian). 
- Make conversion independent from current system timezone.

In case of collecting values of Date/TimestampType, the following SQL config can control types of returned values:
 - spark.sql.catalyst.timestampType with supported values "java.sql.Timestamp" (by default) and "java.time.Instant"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org