You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Ethan Guo (Jira)" <ji...@apache.org> on 2023/02/07 22:53:00 UTC

[jira] [Updated] (HUDI-3582) Introduce Secondary Index to Improve HUDI Query Performance

     [ https://issues.apache.org/jira/browse/HUDI-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ethan Guo updated HUDI-3582:
----------------------------
    Fix Version/s: 0.14.0
                       (was: 0.13.0)

> Introduce Secondary Index to Improve HUDI Query Performance
> -----------------------------------------------------------
>
>                 Key: HUDI-3582
>                 URL: https://issues.apache.org/jira/browse/HUDI-3582
>             Project: Apache Hudi
>          Issue Type: New Feature
>          Components: index
>            Reporter: shibei
>            Assignee: shibei
>            Priority: Blocker
>             Fix For: 0.14.0
>
>
> In query processing, we need to scan many data blocks in HUDI table. However, most of them may not match the query predicate even after using column statistic info in the metadata table, row group level or page level statistics in parquet files, etc.
> The total data size of touched blocks determines the query speed, and how to save IO has become the key point to improving query performance. To address this problem, we introduce secondary index to improve hudi query performance.
> In the initial implementation, secondary index based on lucene will be carried out.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)