You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "Edgar Rodriguez (Jira)" <ji...@apache.org> on 2020/09/18 23:10:00 UTC
[jira] [Commented] (ORC-645) Timestamp between -1 and 0 seconds
from UNIX epoch changes after write-read
[ https://issues.apache.org/jira/browse/ORC-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198577#comment-17198577 ]
Edgar Rodriguez commented on ORC-645:
-------------------------------------
Seems like this is a duplicate of ORC-342
> Timestamp between -1 and 0 seconds from UNIX epoch changes after write-read
> ---------------------------------------------------------------------------
>
> Key: ORC-645
> URL: https://issues.apache.org/jira/browse/ORC-645
> Project: ORC
> Issue Type: Bug
> Reporter: Devavret Makkar
> Priority: Major
>
> When writing timestamp between -1 seconds and 0 seconds from UNIX epoch like this:
> {code:java}
> TypeDescription schema =
> TypeDescription.fromString("struct<x:timestamp>");
> Writer writer = OrcFile.createWriter(new Path("time-file.orc"),
> OrcFile.writerOptions(new Configuration())
> .setSchema(schema));
> VectorizedRowBatch batch = schema.createRowBatch();
> TimestampColumnVector x = (TimestampColumnVector) batch.cols[0];
> int row = batch.size++;
> // This is supposed to be 1969-12-31 23:59:59.762
> x.set(row, new Timestamp(-238L));
> writer.addRowBatch(batch);
> writer.close();
> {code}
> And reading it back with pyarrow.orc
> {code:python}
> import pyarrow.orc as orc
> pdf = orc.ORCFile("time-file.orc").read().to_pandas()
> print(pdf)
> {code}
> I get:
> {noformat}
> x
> 0 1970-01-01 00:00:00.762000000
> {noformat}
>
> This is probably because of special handling of negative timestamps here:
> [https://github.com/apache/orc/blob/fa9c011e13e8376d2a185bd76af834bd644f4332/java/core/src/java/org/apache/orc/impl/TreeReaderFactory.java#L1221-L1227]
> Millis will not be < 0 in this particular case so it will not be reduced by MILLIS_PER_SECOND.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)