[GitHub] [incubator-hudi] lamber-ken commented on issue #1105: [WIP] [HUDI-405] Fix sync no hive partition at first time

[GitBox <gi...@apache.org>]

lamber-ken commented on issue #1105: [WIP] [HUDI-405] Fix sync no hive partition at first time
URL: https://github.com/apache/incubator-hudi/pull/1105#issuecomment-569978944
 
 
   > @lamber-ken I still don't fully understand the need for this fix.. Can you please summarize where we are?
   
   Hi @vinothchandar 
   
   As we know, hudi's partition supports `yyyy/mm/dd` form. If the partition data is `yyyy-mm-dd` form in fact data, user needs to  implements `PartitionValueExtractor`.
   
   From the definition of `HIVE_ASSUME_DATE_PARTITION_OPT_KEY`, it means that user needs to set it `true` if they customed the `PartitionValueExtractor`
   
   But, this variable is used incorrectly, it is a logic error. The right usage is `!assumeDatePartitioning`.
   
   
   <br>
   <br>
   
   **The definition of `HIVE_ASSUME_DATE_PARTITION_OPT_KEY`**
   Property: `hoodie.datasource.hive_sync.assume_date_partitioning`, Default: `false` 
   <span style="color:grey">Assume partitioning is `yyyy/mm/dd`</span>
   
   **FSUtils#getAllPartitionPaths**
   ```
   public static List<String> getAllPartitionPaths(FileSystem fs, String basePathStr, boolean assumeDatePartitioning)
       throws IOException {
     if (assumeDatePartitioning) {
       return getAllPartitionFoldersThreeLevelsDown(fs, basePathStr);
     } else {
       return getAllFoldersWithPartitionMetaFile(fs, basePathStr);
     }
   }
   ```
   
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services