You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Vitalii Diravka (JIRA)" <ji...@apache.org> on 2018/10/11 09:25:00 UTC

[jira] [Commented] (DRILL-5332) DateVector support uses questionable conversions to a time

    [ https://issues.apache.org/jira/browse/DRILL-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646181#comment-16646181 ] 

Vitalii Diravka commented on DRILL-5332:
----------------------------------------

Actually {{DateVector}} was changed in the DRILL-6242, but not sure that solved the current issue. For now it looks like [this|https://github.com/apache/drill/blob/master/exec/vector/src/main/codegen/templates/FixedValueVectors.java#L537]:
{code}
    @Override
    public LocalDate getObject(int index) {
      return LocalDateTime.ofInstant(Instant.ofEpochMilli(get(index)), ZoneOffset.UTC).toLocalDate();
    }
{code}
It still uses UTC timeZone. 
Possibly it can be return statement could be replaced with 
{code}
return LocalDate.ofEpochDay(get(index));
{code}
[~paul-rogers] Does it make sense?

> DateVector support uses questionable conversions to a time
> ----------------------------------------------------------
>
>                 Key: DRILL-5332
>                 URL: https://issues.apache.org/jira/browse/DRILL-5332
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.9.0
>            Reporter: Paul Rogers
>            Priority: Major
>
> The following code in {{DateVector}} is worrisome:
> {code}
>     @Override
>     public DateTime getObject(int index) {
>         org.joda.time.DateTime date = new org.joda.time.DateTime(get(index), org.joda.time.DateTimeZone.UTC);
>         date = date.withZoneRetainFields(org.joda.time.DateTimeZone.getDefault());
>         return date;
>     }
> {code}
> This code takes a date/time value stored in a value vector, converts it to UTC, then strips the time zone and replaces it with local time. The result it a "timestamp" in Java format (seconds since the epoch), but not really, it really the time since the epoch, as if the epoch had started in the local time zone rather than UTC.
> This is the kind of fun & games that people used to do in Java with the {{Date}}  type before the advent of Joda time (and the migration of Joda into Java 8.)
> It is, in short, very bad practice and nearly impossible to get right.
> Further, converting a pure date (since this is a {{DateVector}}) into a date/time is fraught with peril. A date has no corresponding time. 1 AM on Friday in one time zone might be 11 PM on Thursday in another. Converting from dates to times is very difficult.
> If the {{DateVector}} corresponds to a date, then it should be simple date with no implied time zone and no implied relationship to time. If there is to be a mapping of time, it must be to a {{LocalTime}} (in Joda and Java 8) that has no implied time zone.
> Note that this code directly contradicts the statement in [Drill documentation|http://drill.apache.org/docs/date-time-and-timestamp/]: "Drill stores values in Coordinated Universal Time (UTC)." Actually, even the documentation is questionable: what does it mean to store a date in UTC because of the above issues?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)