[jira] [Comment Edited] (FLINK-11935) Remove DateTimeUtils pull-in and fix datetime casting problem

["Rong Rong (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16826516#comment-16826516 ] 

Rong Rong edited comment on FLINK-11935 at 4/26/19 2:18 AM:
------------------------------------------------------------

Hi [~yanghua], I understand what you said. I think to summarize all the discussions: Calcite users Julian date, and Flink internal uses Gregorian.

Are you suggest that in Flink we should stick with Gregorian date, and keep that removed piece of do that does the transformation? If yes, I am fine with that as long as Calcite support it. 

I guess the fundamental part of this ticket is to investigate whether we can remove the {{DateTimeUtils}} file copied from Calcite/Avartica. 
If we decided that we cannot do so at this stage, we need to 
1. backport the DateTimeUtils to latest version Flink use (which is 1.18),
2. make changes on top as needed
3. create Calcite/Avatica Jira ticket and marked in the DataTimeUtils file to properly log the issue why we still need the file. I guess you might need to drive a discussion on Calcite to let system switch between Julian and Gregorian. 



was (Author: walterddr):
Hi [~yanghua], I understand what you said. I think to summarize all the discussions: Calcite users Julian date, and Flink internal uses Gregorian.

Are you suggest that in Flink we should stick with Gregorian date, and remain the transformation? If yes, I am fine with that as long as CALCITE support it. 

I guess the fundamental part of this ticket is to investigate whether we can remove the {{DateTimeUtils}} file copied from Calcite/Avartica. 
If we decided that we cannot do so at this stage, we need to 
1. backport the DateTimeUtils to latest version Flink use (which is 1.18),
2. make changes on top as needed
3. create Calcite/Avatica Jira ticket and marked in the DataTimeUtils file to properly log the issue why we still need the file. I guess you might need to drive a discussion on Calcite to let system switch between Julian and Gregorian. 


> Remove DateTimeUtils pull-in and fix datetime casting problem
> -------------------------------------------------------------
>
>                 Key: FLINK-11935
>                 URL: https://issues.apache.org/jira/browse/FLINK-11935
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / API
>            Reporter: Rong Rong
>            Assignee: vinoyang
>            Priority: Major
>
> This {{DateTimeUtils}} was pulled in in FLINK-7235.
> Originally the time operation was not correctly done via the {{ymdToJulian}} function before the date {{1970-01-01}} thus we need the fix. similar to addressing this problem:
> {code:java}
>  Optimized :1017-12-05 22:58:58.998 
>  Expected :1017-11-29 22:58:58.998
>  Actual :1017-12-05 22:58:58.998
> {code}
>  
> However, after pulling in avatica 1.13, I found out that the optimized plans of the time operations are actually correct. it is in fact the casting part that creates problem:
> For example, the following:
> *{{(plus(-12000.months, cast('2017-11-29 22:58:58.998', TIMESTAMP))}}*
> result in a StringTestExpression of:
> *{{CAST(1017-11-29 22:58:58.998):VARCHAR(65536) CHARACTER SET "UTF-16LE" COLLATE "ISO-8859-1$en_US$primary" NOT NULL}}*
> but the testing results are:
> {code:java}
>  Optimized :1017-11-29 22:58:58.998
>  Expected :1017-11-29 22:58:58.998
>  Actual :1017-11-23 22:58:58.998
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)