You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/12/20 22:23:00 UTC
[jira] [Commented] (SPARK-26424) Use java.time API in
timestamp/date functions
[ https://issues.apache.org/jira/browse/SPARK-26424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726270#comment-16726270 ]
ASF GitHub Bot commented on SPARK-26424:
----------------------------------------
MaxGekk opened a new pull request #23358: [SPARK-26424][SQL] Use java.time API in date/timestamp expressions
URL: https://github.com/apache/spark/pull/23358
## What changes were proposed in this pull request?
In the PR, I propose to switch the `DateFormatClass`, `ToUnixTimestamp`, `FromUnixTime`, `UnixTime` on java.time API for parsing/formatting dates and timestamps. The API has been already implemented by the `Timestamp`/`DateFormatter` classes. One of benefit is those classes support parsing timestamps with microsecond precision. Old behaviour can be switched on via SQL config: `spark.sql.legacy.timeParser.enabled` (`false` by default).
## How was this patch tested?
It was tested by existing test suites - `DateFunctionsSuite`, `DateExpressionsSuite`, `JsonSuite`, `CsvSuite`, `SQLQueryTestSuite` as well as PySpark tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Use java.time API in timestamp/date functions
> ----------------------------------------------
>
> Key: SPARK-26424
> URL: https://issues.apache.org/jira/browse/SPARK-26424
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Maxim Gekk
> Priority: Minor
>
> Currently, date/time expressions like DateFormatClass, UnixTimestamp, FromUnixTime use SimpleDateFormat to parse/format dates and timestamps. SimpleDateFormat cannot parse timestamp with microsecond precision. The ticket aims to switch the expression on TimestampFormatter which is able to parse timestamp with microsecond precision.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org