[jira] [Created] (HUDI-3880) Check file listing for common partitions when partition list mismatches in the validation

["Ethan Guo (Jira)" <ji...@apache.org>]

Ethan Guo created HUDI-3880:
-------------------------------

             Summary: Check file listing for common partitions when partition list mismatches in the validation
                 Key: HUDI-3880
                 URL: https://issues.apache.org/jira/browse/HUDI-3880
             Project: Apache Hudi
          Issue Type: Improvement
            Reporter: Ethan Guo
            Assignee: Yue Zhang
             Fix For: 0.12.0


{code:java}
org.apache.hudi.exception.HoodieValidationException: Compare Partitions Failed! AllPartitionPathsFromFS : [2021/12/1, 2022/1/24, 2022/1/25, 2022/1/26, 2022/1/27, 2022/1/28, 2022/1/29, 2022/1/30, 2022/1/31, 2022/2/1, 2022/2/2] and allPartitionPathsMeta : [2021/12/1, 2021/12/2, 2022/1/24, 2022/1/25, 2022/1/26, 2022/1/27, 2022/1/28, 2022/1/29, 2022/1/30, 2022/1/31, 2022/2/1, 2022/2/2] {code}
When the partition listing is different between FS and metadata table, we can still check the file listing for common partitions and print the result.  For the partition that only appears in one side, we can check if it is empty or not.  If it is, we can mark it as matching; otherwise, we throw an error for the partition.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)