You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/07/07 21:07:00 UTC
[jira] [Commented] (IMPALA-9789) Disable ineffective bloom filters
for Kudu scan
[ https://issues.apache.org/jira/browse/IMPALA-9789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153094#comment-17153094 ]
ASF subversion and git services commented on IMPALA-9789:
---------------------------------------------------------
Commit 9429ca7aa2fe127af11073b2fc09bc8e62d2e410 in impala's branch refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=9429ca7 ]
IMPALA-9789: Disable ineffective bloom filters for Kudu scan
Bump Kudu version to 23f67ae0d. This pulls in the latest version of
Kudu which includes a performance enhancement in Kudu server for bloom
filter. Kudu uses same heuristics as HDFS to disable ineffective bloom
filter. This fixs the performance regression issue for queries like
TPCH-Q9 so that we can set query option ENABLED_RUNTIME_FILTER_TYPES
as ALL to enable bloom filter for Kudu by default.
Testing:
- Ran single_node_perf_run.py with TPCH for Kudu and verified that
there was no regression issue for each query.
- Passed all core tests.
Change-Id: Ib55fa359e2fdb2c03c890f2303ffd9173c3c4af1
Reviewed-on: http://gerrit.cloudera.org:8080/16138
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Disable ineffective bloom filters for Kudu scan
> -----------------------------------------------
>
> Key: IMPALA-9789
> URL: https://issues.apache.org/jira/browse/IMPALA-9789
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
> Fix For: Impala 4.0
>
>
> In bloom-filter benchmark for Kudu, there is performance regression for query TPCH-Q9. In Profile shows that 5 bloom filters are generated by hash join. Some of those filters are not useful for filtering rows. When pushing all bloom filters to Kudu, the bloom filter evaluations add extra cost for Kudu scan, which cause performance regression.
>
> The regression on Q9 looks a lot like https://issues.apache.org/jira/browse/IMPALA-9302, where Q9 regressed a lot with multithreading initially because ineffective filters weren't being disabled. This query is a bit special in that there are many filters pushed to scan 2, and most of them are not useful. Based on our experience there, we need to add a method to disable ineffective filters for Kudu scan.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org