[jira] [Updated] (HUDI-6039) Fix FS based listing in clean planner

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HUDI-6039:
---------------------------------
    Labels: pull-request-available  (was: )

> Fix FS based listing in clean planner
> -------------------------------------
>
>                 Key: HUDI-6039
>                 URL: https://issues.apache.org/jira/browse/HUDI-6039
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: cleaning, writer-core
>            Reporter: sivabalan narayanan
>            Priority: Major
>              Labels: pull-request-available
>
> we are using FS based listing when full cleaning is invoked w/ clean planner. 
> {code:java}
> /**   * Scan and list all partitions for cleaning.   * @return all partitions paths for the dataset.   */  private List<String> getPartitionPathsForFullCleaning() {    // Go to brute force mode of scanning all partitions    try {      // Because the partition of BaseTableMetadata has been deleted,      // all partition information can only be obtained from FileSystemBackedTableMetadata.      FileSystemBackedTableMetadata fsBackedTableMetadata = new FileSystemBackedTableMetadata(context,          context.getHadoopConf(), config.getBasePath(), config.shouldAssumeDatePartitioning());      return fsBackedTableMetadata.getAllPartitionPaths();    } catch (IOException e) {      return Collections.emptyList();    }  } {code}
>  
> we should be using Metadata based listing if metadata is enabled. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)