You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Sagar Sumit (Jira)" <ji...@apache.org> on 2022/03/29 03:26:00 UTC

[jira] [Created] (HUDI-3737) Make reader path table config aware

Sagar Sumit created HUDI-3737:
---------------------------------

             Summary: Make reader path table config aware
                 Key: HUDI-3737
                 URL: https://issues.apache.org/jira/browse/HUDI-3737
             Project: Apache Hudi
          Issue Type: Task
            Reporter: Sagar Sumit
             Fix For: 0.11.0


Currently `HoodieFileIndex#lookupCandidateFilesInMetadataTable` assumes that metadata indexing is synchronous and depends on fs.exists check. With async indexing, we could have initialized the column_stats index and partition would have been created so fs.exists check will pass.

With async index, table config becomes the source of truth for inflight or available MT partitions for writers. We can use the table config to determine available partitions.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)