[GitHub] MaxGekk opened a new pull request #23913: [SPARK-27008][SQL] Support java.time.LocalDate as an external type of DateType

[GitBox <gi...@apache.org>]

MaxGekk opened a new pull request #23913: [SPARK-27008][SQL] Support java.time.LocalDate as an external type of DateType
URL: https://github.com/apache/spark/pull/23913
 
 
   ## What changes were proposed in this pull request?
   
   In the PR, I propose to add new Catalyst type converter for `DateType`. It should be able to convert `java.time.LocalDate` to/from `DateType`.
   
   Main motivations for the changes:
   - Smoothly support Java 8 time API
   - Avoid inconsistency of calendars used inside of Spark 3.0 (Proleptic Gregorian calendar) and `java.sql.Date` (hybrid calendar - Julian + Gregorian).
   - Make conversion independent from current system timezone.
   
   By default, Spark converts values of `DateType` to `java.sql.Date` instances but the SQL config `spark.sql.catalyst.dateType` can change the behavior. It accepts two values `Date` (default) and `LocalDate`. If the former one is set, Spark returns `java.time.LocalDate` instances for timestamp values.
   
   ## How was this patch tested?
   
   Added new testes to `CatalystTypeConvertersSuite` to check conversion of `DateType` to/from `java.time.LocalDate`, `JavaUDFSuite`/ `UDFSuite` to test usage of `LocalDate` type in Scala/Java UDFs.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org