[GitHub] [hudi] weimingdiit commented on a diff in pull request #8245: [HUDI-5944] Added the ability to fix partitiion missing in hudi synctool

["weimingdiit (via GitHub)" <gi...@apache.org>]

weimingdiit commented on code in PR #8245:
URL: https://github.com/apache/hudi/pull/8245#discussion_r1145638109


##########
pom.xml:
##########
@@ -117,7 +117,7 @@
     <joda.version>2.9.9</joda.version>
     <hadoop.version>2.10.1</hadoop.version>
     <hive.groupid>org.apache.hive</hive.groupid>
-    <hive.version>2.3.1</hive.version>
+    <hive.version>2.3.4</hive.version>

Review Comment:
   The hive version is updated from 2.3.1 to 2.3.4.
   The bug of MetaStoreUtils, 【Long newStat = long.parselong (newPart.getParameters().get(stat));】 
   Perhaps newPart does not contain 'stat', in which case parseLong will report an error.
   
   
   `
   in 2.3.4
     
     static boolean isFastStatsSame(Partition oldPart, Partition newPart) {
       // requires to calculate stats if new and old have different fast stats
       if ((oldPart != null) && (oldPart.getParameters() != null)) {
         for (String stat : StatsSetupConst.fastStats) {
           if (oldPart.getParameters().containsKey(stat)) {
             Long oldStat = Long.parseLong(oldPart.getParameters().get(stat));
             String newStat = newPart.getParameters().get(stat);
             if (newStat == null || !oldStat.equals(Long.parseLong(newStat))) {
               return false;
             }
           } else {
             return false;
           }
         }
         return true;
       }
       return false;
     }
   
   
    
   in 2.3.1
     static boolean isFastStatsSame(Partition oldPart, Partition newPart) {
       // requires to calculate stats if new and old have different fast stats
       if ((oldPart != null) && (oldPart.getParameters() != null)) {
         for (String stat : StatsSetupConst.fastStats) {
           if (oldPart.getParameters().containsKey(stat)) {
             Long oldStat = Long.parseLong(oldPart.getParameters().get(stat));
             Long newStat = Long.parseLong(newPart.getParameters().get(stat));
             if (!oldStat.equals(newStat)) {
               return false;
             }
           } else {
             return false;
           }
         }
         return true;
       }
       return false;
     }
   
   
   
   
   
   
   `



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org