You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Danny Chen (Jira)" <ji...@apache.org> on 2024/09/04 03:51:00 UTC

[jira] [Created] (HUDI-8164) Chain multiple indices support in HoodieFileIndex

Danny Chen created HUDI-8164:
--------------------------------

             Summary: Chain multiple indices support in HoodieFileIndex
                 Key: HUDI-8164
                 URL: https://issues.apache.org/jira/browse/HUDI-8164
             Project: Apache Hudi
          Issue Type: Improvement
          Components: reader-core
            Reporter: Danny Chen
            Assignee: Danny Chen
             Fix For: 1.0.0


In HoodieFileIndex, we have 5 indices support: 

 
 * RecordLevelIndexSupport
 * BucketIndexSupport
 * 
SecondaryIndexSupport
 * 
FunctionalIndexSupport
 * BloomFiltersIndexSupport
 * ColumnStatsIndexSupport

 
But the logic would return early when the first index support applied and got pruned files.
A better way is we can chain the multiple indices support all-together: one index support can still be applied based on the pruned files returned by another index support.

Key points in contribution:
1. think through the shared contract/param among these indices support;
2. add a orthoganal test for random index supoprt composition, valid the result set and the indices support are applied effectively.
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)