You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Raymond Xu (Jira)" <ji...@apache.org> on 2022/05/03 12:07:00 UTC

[jira] [Updated] (HUDI-3806) Improve HoodieBloomIndex using bloom_filter and col_stats in MDT

     [ https://issues.apache.org/jira/browse/HUDI-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raymond Xu updated HUDI-3806:
-----------------------------
    Sprint: Hudi-Sprint-Apr-05, Hudi-Sprint-Apr-12  (was: Hudi-Sprint-Apr-05, Hudi-Sprint-Apr-12, 2022/04/25)

> Improve HoodieBloomIndex using bloom_filter and col_stats in MDT
> ----------------------------------------------------------------
>
>                 Key: HUDI-3806
>                 URL: https://issues.apache.org/jira/browse/HUDI-3806
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Ethan Guo
>            Assignee: Ethan Guo
>            Priority: Blocker
>             Fix For: 0.12.0
>
>
> For a Delastreamer job doing bulk inserts of 10GB batches, the job is stuck at the stage when HoodieBloomIndex reads bloom filter from the metadata table, taking more than 2 hours.  When bloom filter is disabled in metadata table, each commit takes 10-20 minutes.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)