[jira] [Updated] (HUDI-9283) Benchmark RLI flow with a large table to improve performance

["Rajesh Mahindra (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajesh Mahindra updated HUDI-9283:
----------------------------------
    Component/s: index

> Benchmark RLI flow with a large table to improve performance
> ------------------------------------------------------------
>
>                 Key: HUDI-9283
>                 URL: https://issues.apache.org/jira/browse/HUDI-9283
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: core, index
>            Reporter: Rajesh Mahindra
>            Assignee: Rajesh Mahindra
>            Priority: Major
>
> High level context
> Benchmark RLI for tables on an existing table with large number of record keys (~100B). 
> Incrementally ingest about 10GB of data, MoR table, partitioned with ~500 partitions. 
> Use Hfile size of 2GB.
>  * Ensure the bootstrap of RLI works as expected.
>  * Measure the read and write latencies for the RLI index
>  * Find and measure all bottlenecks
>  * Report any issue with the core indexing or RLI or MDT DAGs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)