You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Raymond Xu (Jira)" <ji...@apache.org> on 2023/03/07 01:30:00 UTC
[jira] [Updated] (HUDI-3806) Improve HoodieBloomIndex using bloom_filter and col_stats in MDT
[ https://issues.apache.org/jira/browse/HUDI-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raymond Xu updated HUDI-3806:
-----------------------------
Component/s: metadata
> Improve HoodieBloomIndex using bloom_filter and col_stats in MDT
> ----------------------------------------------------------------
>
> Key: HUDI-3806
> URL: https://issues.apache.org/jira/browse/HUDI-3806
> Project: Apache Hudi
> Issue Type: Improvement
> Components: metadata
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Fix For: 0.13.0
>
>
> For a Delastreamer job doing bulk inserts of 10GB batches, the job is stuck at the stage when HoodieBloomIndex reads bloom filter from the metadata table, taking more than 2 hours. When bloom filter is disabled in metadata table, each commit takes 10-20 minutes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)