You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/06/01 06:00:00 UTC
[jira] [Commented] (IMPALA-5280) Coalesce chains of OR conditions
to an IN predicate.
[ https://issues.apache.org/jira/browse/IMPALA-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497619#comment-16497619 ]
ASF subversion and git services commented on IMPALA-5280:
---------------------------------------------------------
Commit f26a2d4409f3d3322646b7f4ef441a5b3a0df03e in impala's branch refs/heads/2.x from [~tianyiwang]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=f26a2d4 ]
IMPALA-7090: Limit the size of expr created by EqualityDisjunctsToInRule
Currently EqualityDisjunctsToInRule introduced in IMPALA-5280 might
create an expr with unlimited number of children and fails a query,
which should be avoided. The easy solution is to not apply the
rewrite when the number of children is large.
Change-Id: Ie40c3210271a9e3c7f1b2b869a8c2ec8bacaa72a
Reviewed-on: http://gerrit.cloudera.org:8080/10528
Reviewed-by: Tianyi Wang <tw...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Coalesce chains of OR conditions to an IN predicate.
> ----------------------------------------------------
>
> Key: IMPALA-5280
> URL: https://issues.apache.org/jira/browse/IMPALA-5280
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 2.8.0
> Reporter: Alexander Behm
> Assignee: sandeep akinapelli
> Priority: Major
> Labels: newbie, perfomance
> Fix For: Impala 2.10.0
>
> Attachments: same_query_profile_on_CDH5.12.txt
>
>
> Would be nice to implement an ExprRewriteRule that coalesces multiple compatible OR conditions to an IN predicate, e.g.:
> {code}
> (c=1) OR (c=2) OR (c=3) OR (c=4) ...
> ->
> c IN (1, 2, 3, 4...)
> {code}
> Long chains of OR are generally unwieldy, and transforming them to IN has the following benefits:
> * IN predicates with long value lists are evaluated with an O(log n) lookup in the BE
> * It is easier to extract min/max values from an IN predicate for Parquet min/max filtering
> * The IN predicate may be faster to codegen than a deep binary tree or ORs
> Note that this new rule complements existing rules to yield interesting improvements, e.g.:
> {code}
> (c1=1 AND c2='a') OR (c1=2 AND c2='a') OR (c1=3 AND c2='a')
> ->
> c2='a' AND c1 IN (1, 2, 3)
> {code}
> I've attached a relevant query profile from one of Mostafa's experiments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org