You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/04/23 08:41:00 UTC
[jira] [Commented] (IMPALA-11536) Invalid push down predicates in outer join simplification
[ https://issues.apache.org/jira/browse/IMPALA-11536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715408#comment-17715408 ]
ASF subversion and git services commented on IMPALA-11536:
----------------------------------------------------------
Commit 4f1d8d4d39bda9a1d91db0fc2e57fe4634ecadd0 in impala's branch refs/heads/master from hexianqing
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4f1d8d4d3 ]
IMPALA-11536: fix invalid predicates propagate for outer join simplification
When set ENABLE_OUTER_JOIN_TO_INNER_TRANSFORMATION = true, the planner
will simplify outer joins if the WHERE clause contains at least one
null rejecting condition and then remove the outer-joined tuple id
from the map of GlobalState#outerJoinedTupleIds.
However, there may be false removals for right join simplification or
full join simplification. This may lead to incorrect results since it
is incorrect to propagate a non null-rejecting predicate into a plan
subtree that is on the nullable side of an outer join.
GlobalState#outerJoinedTupleIds indicates whether a table is on the
nullable side of an outer join.
E.g.
SELECT COUNT(*)
FROM functional.nullrows t1
FULL JOIN functional.nullrows t2 ON t1.id = t2.id
FULL JOIN functional.nullrows t3 ON coalesce(t1.id, t2.id) = t3.id
WHERE t1.group_str = 'a'
AND coalesce(t2.group_str, 'f') = 'f'
The predicate coalesce(t2.group_str, 'f') = 'f' will propagate into t2
if we remove t2 from GlobalState#outerJoinedTupleIds.
Testing:
- Add new plan tests in outer-to-inner-joins.test
- Add new query tests to verify the correctness on transformation
Change-Id: I6565c5bff0d2f24f30118ba47a2583383e83fff7
Reviewed-on: http://gerrit.cloudera.org:8080/19116
Reviewed-by: Qifan Chen <qf...@hotmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Invalid push down predicates in outer join simplification
> ---------------------------------------------------------
>
> Key: IMPALA-11536
> URL: https://issues.apache.org/jira/browse/IMPALA-11536
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.0.0, Impala 4.1.0
> Reporter: Xianqing He
> Assignee: Xianqing He
> Priority: Major
> Attachments: image-2022-08-25-14-47-51-966.png
>
>
> When set ENABLE_OUTER_JOIN_TO_INNER_TRANSFORMATION = true;
> It may invalid push down the predicate that is not null rejecting in outer join simplification.
> e.g.
> SELECT COALESCE(jointbl.test_id, testtbl.id, dimtbl.id) AS id, test_zip,testtbl.zip
> FROM functional.jointbl
> FULL OUTER JOIN
> functional.testtbl
> ON jointbl.test_id = testtbl.id
> FULL OUTER JOIN
> functional.dimtbl
> ON coalesce(jointbl.test_id, testtbl.id) = dimtbl.id
> WHERE
> `jointbl`.`test_zip` = 94611 and coalesce(`testtbl`.`zip`, 0) = 0;
>
> !image-2022-08-25-14-47-51-966.png!
> We can't push down the predicate 'coalesce(testtbl.zip, 0) = 0' to ScanNode since it is not null rejecting
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org