[jira] [Updated] (HUDI-6708) Support Record Index with the Async Indexer

["Sagar Sumit (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sagar Sumit updated HUDI-6708:
------------------------------
    Description: Record index can be created using the async indexer if there are no inflight commits. With inflight commits, the catch-up task will [throw the UnsupportedException|https://github.com/apache/hudi/blob/616b663c19eb059438dc3a7f80c5c84b39a6ef6f/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java#L380-L382] as we don't have WriteStatus (which is needed to build record index) at that time. Hudi should be able to build the complete record index concurrently with ingestion (except for the limitations that currently exist for concurrent indexing).

> Support Record Index with the Async Indexer
> -------------------------------------------
>
>                 Key: HUDI-6708
>                 URL: https://issues.apache.org/jira/browse/HUDI-6708
>             Project: Apache Hudi
>          Issue Type: Task
>            Reporter: Sagar Sumit
>            Priority: Major
>             Fix For: 1.0.0
>
>
> Record index can be created using the async indexer if there are no inflight commits. With inflight commits, the catch-up task will [throw the UnsupportedException|https://github.com/apache/hudi/blob/616b663c19eb059438dc3a7f80c5c84b39a6ef6f/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/index/RunIndexActionExecutor.java#L380-L382] as we don't have WriteStatus (which is needed to build record index) at that time. Hudi should be able to build the complete record index concurrently with ingestion (except for the limitations that currently exist for concurrent indexing).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)