[GitHub] [orc] dongjoon-hyun commented on a change in pull request #540: ORC-661. Use LocalDate and day of epoch instead of java's Date for column statistics

[GitBox <gi...@apache.org>]

dongjoon-hyun commented on a change in pull request #540:
URL: https://github.com/apache/orc/pull/540#discussion_r475823344



##########
File path: java/core/src/java/org/apache/orc/impl/RecordReaderImpl.java
##########
@@ -801,12 +805,17 @@ private static Comparable getBaseObjectForComparison(PredicateLeaf.Type type,
           return Boolean.valueOf(obj.toString());
         }
       case DATE:
-        if (obj instanceof Date) {
+        if (obj instanceof LocalDate) {
           return obj;
+        } else if (obj instanceof java.sql.Date) {
+          return ((java.sql.Date) obj).toLocalDate();
+        } else if (obj instanceof Date) {
+          return LocalDateTime.ofInstant(((Date) obj).toInstant(),
+              ZoneOffset.UTC).toLocalDate();

Review comment:
       So, this is the fix for the following?
   ```
   ORC PPD evaluation for Date type uses java.sql.Date for min/max comparison causing incorrect results.
   Date.compareTo uses millis offset which can return incorrect results depending on the timezone. 
   Running the testcase in HIVE-19726 passed in Los Angeles but failed in Paris
   as Date.compareTo return 0 for Los Angeles but returned -1 for Paris.
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org