[GitHub] [iceberg] rdblue commented on pull request #1355: Fixed non-greenway time zone, data loss with day partition

[GitBox <gi...@apache.org>]

rdblue commented on pull request #1355:
URL: https://github.com/apache/iceberg/pull/1355#issuecomment-675604146


   @zhangdove, Iceberg does not modify timestamps other than converting between representations (like micros to millis).
   
   Timestamp modification or adjustment to display a value by time zone is done by processing engines, not by Iceberg itself. Iceberg must only store values and return the same values.
   
   Timestamp values are in microseconds and come in two flavors: timestamp is a zoneless date and time represented as microseconds from the Unix epoch in UTC, and timestamptz is an instant in time represented as microseconds from the Unix epoch in UTC.
   
   The report in #1354 uses Spark. Spark only supports timestamptz and it passes values to Iceberg as microseconds from epoch, which Iceberg returns to Spark unmodified. The `day` partition function is not specific to a zone, which is probably the source of the confusion. The daily boundaries used by that function are UTC day boundaries, if I remember correctly.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org