[jira] [Updated] (HUDI-1624) The state based index should bootstrap from existing base files

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-1624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HUDI-1624:
---------------------------------
    Labels: pull-request-available  (was: )

> The state based index should bootstrap from existing base files
> ---------------------------------------------------------------
>
>                 Key: HUDI-1624
>                 URL: https://issues.apache.org/jira/browse/HUDI-1624
>             Project: Apache Hudi
>          Issue Type: Sub-task
>          Components: Flink Integration
>            Reporter: Danny Chen
>            Priority: Major
>              Labels: pull-request-available
>
> The index should bootstrap from existing base files if there are, in the design, we load all the keys for one partition if we found that the key does not exist in the index for {{processElement}}, if there are many records for this partition, the processing may block and trigger back pressure. When all the records are loaded, we only need to check the state each time a record is tagged.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)