[jira] [Comment Edited] (ORC-27) C++ reader does not read dates correctly

["Aliaksei Sandryhaila (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ORC-27?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743629#comment-14743629 ] 

Aliaksei Sandryhaila edited comment on ORC-27 at 9/14/15 3:08 PM:
------------------------------------------------------------------

This is a library implementation issue: when Hive writes ORC files, it uses the Java library that converts dates before 1583/01/01 from Gregorian to Julian calendar. The C++ ORC reader uses a C library, which does not perform the conversion and uses the propleptic Gregorian calendar (that is, treats all dates as Gregorian). Some relevant info:
http://www.timeanddate.com/calendar/julian-gregorian-switch.html
https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar

A suggested resolution: document this discrepancy. If a user reads from an ORC file dates earlier than 1583/01/01, they should be aware that there will be an difference of up to 10 days between what they see in Hive and what the C++ ORC reader outputs. This applies both to DATE and TIMESTAMP data types. 


was (Author: asandryh):
This is a library implementation issue: when Hive writes ORC files, it uses the Java library that converts dates before 1582 from Gregorian to Julian calendar. The C++ ORC reader uses a C library, which does not perform the conversion and uses the propleptic Gregorian calendar (that is, treats all dates as Gregorian). Some relevant info:
http://www.timeanddate.com/calendar/julian-gregorian-switch.html
https://en.wikipedia.org/wiki/Proleptic_Gregorian_calendar

A suggested resolution: document this discrepancy. If a user reads from an ORC file dates earlier than 1583/01/01, they should be aware that there will be an difference of up to 10 days between what they see in Hive and what the C++ ORC reader outputs. This applies both to DATE and TIMESTAMP data types. 

> C++ reader does not read dates correctly
> ----------------------------------------
>
>                 Key: ORC-27
>                 URL: https://issues.apache.org/jira/browse/ORC-27
>             Project: Orc
>          Issue Type: Bug
>            Reporter: Aliaksei Sandryhaila
>            Assignee: Aliaksei Sandryhaila
>
> Reproducer:
> 1) create a Hive table with a date column
> 2) insert date 01-01-01
> 3) get the corresponding orc file from HDFS
> 4) read with the C++ reader
> The reader produces 0-12-30 (off by 2 days)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)