[GitHub] [iceberg] boroknagyz commented on pull request #1981: Fix date and timestamp transforms

[GitBox <gi...@apache.org>]

boroknagyz commented on pull request #1981:
URL: https://github.com/apache/iceberg/pull/1981#issuecomment-757054106


   Thanks for fixing this, I've just run into this issue with Impala. (And this fix is also a prerequisite for issue #2043)
   
   So the plan is that 0.11+ writers will write the correct transformation values IIUC. Does it also mean that pre-0.11 readers will not be able to read the new files correctly when the data is before the epoch?
   
   E.g. new writer writes '1969-12-31 23:05:00' with YEAR partition transform (so ts_year=-1), old reader scans for 'ts > 1969-12-31 23:00:00' (so with old transform the projected predicate is ts_year >=0), and the file will be skipped because the partition data has ts_year=-1.
   
   I just want to clarify this, because with Impala I'm planning to write V1 files with the fixed transforms.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org