[jira] [Comment Edited] (FLINK-19757) TimeStampData can cause time inconsistent problem

["Jark Wu (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-19757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218809#comment-17218809 ] 

Jark Wu edited comment on FLINK-19757 at 10/22/20, 6:55 AM:
------------------------------------------------------------

Hi [~zhoujira86], {{TimestampData}} is the internal data structure for both {{TIMESTAMP}} (= Java {{LocalDateTime}}) and {{TIMESTAMP WITH LOCAL TIME ZONE}} (= Java {{Instant}}). It always stores the epoch seconds since {{1970-01-01 00:00:00}}. 

It always safe to convert without time zone if they are they same logical type. But not correct if they are different logical type, e.g.  {{Instant}} => {{TimestampData}} => {{LocalDateTime}} is not correct if no time zone considered. But such conversion only happends in Flink SQL internal, and handled correctly. 

You problem above is not in {{TimestampData}} data structure, but the {{PROCTIME()}}  currently returns {{TIMESTAMP}} type which is not right, it should be {{TIMESTAMP WITH LOCAL TIME ZONE}}.


was (Author: jark):
Hi [~zhoujira86], {{TimestampData}} is the internal data structure for both {{TIMESTAMP}} (= Java {{LocalDateTime}}) and {{TIMESTAMP WITH LOCAL TIME ZONE}} (= Java {{Instant}}). It always stores the epoch seconds since {{1970-01-01 00:00:00}}. 

It always safe to convert without time zone if they are they same logical type. But not correct if they are different logical type, e.g.  {{Instant}} => {{TimestampData}} => {{LocalDateTime}} is not correct if no time zone considered. But such conversion only happends in Flink SQL internal, and handled correctly. 

You problem above is not in {{TimestampData}} data structure, but the {{PROCTIME()}}  currently returns {{TIMESTAMP}} type which is not right, it should be {{}}

 {{LocalDateTime}} => {{TimestampData}} => {{LocalDateTime}} and {{TimestampData}} <=> {{Instant}} without time zone considered. But it it right to convert  {{TIMESTAMP WITH LOCAL TIME ZONE}}.

> TimeStampData can cause time inconsistent problem
> -------------------------------------------------
>
>                 Key: FLINK-19757
>                 URL: https://issues.apache.org/jira/browse/FLINK-19757
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / Runtime
>    Affects Versions: 1.11.1
>            Reporter: xiaogang zhou
>            Priority: Major
>              Labels: pull-request-available
>
> when we check jdk LocalDateTime code，we find that
>  
> {code:java}
> // code placeholder
> public static LocalDateTime ofEpochSecond(long epochSecond, int nanoOfSecond, ZoneOffset offset) {
>     Objects.requireNonNull(offset, "offset");
>     NANO_OF_SECOND.checkValidValue(nanoOfSecond);
>     long localSecond = epochSecond + offset.getTotalSeconds();  // overflow caught later
>     long localEpochDay = Math.floorDiv(localSecond, SECONDS_PER_DAY);
>     int secsOfDay = (int)Math.floorMod(localSecond, SECONDS_PER_DAY);
>     LocalDate date = LocalDate.ofEpochDay(localEpochDay);
>     LocalTime time = LocalTime.ofNanoOfDay(secsOfDay * NANOS_PER_SECOND + nanoOfSecond);
>     return new LocalDateTime(date, time);
> }
> {code}
>  
> offset.getTotalSeconds() they add the offset, but in the TimeStampData
> toLocalDateTime, we don't add a offset.
>  
> I'd like to add a TimeZone.getDefault().getRawOffset() in the 
> toLocalDateTime()
> and minus a TimeZone.getDefault().getRawOffset() in the 
> fromLocalDateTime



--
This message was sent by Atlassian Jira
(v8.3.4#803005)