[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1036: [HUDI-353] Add hive style partitioning path

[GitBox <gi...@apache.org>]

bvaradar commented on a change in pull request #1036: [HUDI-353] Add hive style partitioning path
URL: https://github.com/apache/incubator-hudi/pull/1036#discussion_r351090279
 
 

 ##########
 File path: hudi-hive/src/main/java/org/apache/hudi/hive/HiveSyncTool.java
 ##########
 @@ -163,6 +163,11 @@ private void syncSchema(boolean tableExists, boolean isRealTime, MessageType sch
    */
   private void syncPartitions(List<String> writtenPartitionsSince) {
     try {
+      if (cfg.useHiveStylePartitioning) {
+        LOG.info("Sync partitions through MSCK Repair");
+        hoodieHiveClient.syncPartitionsByMSCK();
 
 Review comment:
   @zhedoubushishi : Wouldn't "msck repair" list all partitions in DFS (S3). With commit metadata, we can identify the list of partitions without listing all data partitions. Can we leverage this here ?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services