You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Alexey Kudinkin (Jira)" <ji...@apache.org> on 2022/12/20 19:55:00 UTC

[jira] [Updated] (HUDI-3880) Check file listing for common partitions when partition list mismatches in the validation

     [ https://issues.apache.org/jira/browse/HUDI-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexey Kudinkin updated HUDI-3880:
----------------------------------
    Priority: Major  (was: Blocker)

> Check file listing for common partitions when partition list mismatches in the validation
> -----------------------------------------------------------------------------------------
>
>                 Key: HUDI-3880
>                 URL: https://issues.apache.org/jira/browse/HUDI-3880
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Ethan Guo
>            Assignee: Yue Zhang
>            Priority: Major
>             Fix For: 0.13.0
>
>
> {code:java}
> org.apache.hudi.exception.HoodieValidationException: Compare Partitions Failed! AllPartitionPathsFromFS : [2021/12/1, 2022/1/24, 2022/1/25, 2022/1/26, 2022/1/27, 2022/1/28, 2022/1/29, 2022/1/30, 2022/1/31, 2022/2/1, 2022/2/2] and allPartitionPathsMeta : [2021/12/1, 2021/12/2, 2022/1/24, 2022/1/25, 2022/1/26, 2022/1/27, 2022/1/28, 2022/1/29, 2022/1/30, 2022/1/31, 2022/2/1, 2022/2/2] {code}
> When the partition listing is different between FS and metadata table, we can still check the file listing for common partitions and print the result.  For the partition that only appears in one side, we can check if it is empty or not.  If it is, we can mark it as matching; otherwise, we throw an error for the partition.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)