[jira] [Commented] (IMPALA-11008) Invalid to propagate inferred predicates into the nullable side of an outer join

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-11008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517281#comment-17517281 ] 

ASF subversion and git services commented on IMPALA-11008:
----------------------------------------------------------

Commit abfa5a72b6a6046037f27ad02d19b31f7b31c4e5 in impala's branch refs/heads/master from xqhe
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=abfa5a72b ]

IMPALA-11008: fix incorrect to propagate inferred predicates

It is incorrect to propagate predicates inferred from equi-join
conjuncts into a plan subtree that is on the nullable side of an
outer join if the predicate is not null-filtering for the nullable
side.

For example:
SELECT *
FROM (
  SELECT id IS NOT NULL
    AND col IS NULL AS a
  FROM (
    SELECT A.id, B.col
    FROM A
      LEFT JOIN B ON A.id = B.id
  ) t
) t
WHERE a = 1
Before this patch the inferred predicate '(B.id is not null and
B.col is null) = 1' is evaluated at the scanner of B. This is
incorrect since the predicate '(A.id is not null and B.col is null)
= 1' is not null-filtering for B.
To generate the inferred predicate we substitue the non-outer-join
slots first and use 'isNullableConjunct' to do a more strict check
on the conjunct before the final substitution.

Tests:
  - Add plan tests in predicate-propagation.test
  - Add new query tests to verify the correctness of inferred
    predicates propagation
  - Ran the full set of verifications in Impala Public Jenkins

Change-Id: I9e64230f6d0c2b9ef1560186ceba349a5920ccdf
Reviewed-on: http://gerrit.cloudera.org:8080/18234
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Invalid to propagate inferred predicates into the nullable side of an outer join
> --------------------------------------------------------------------------------
>
>                 Key: IMPALA-11008
>                 URL: https://issues.apache.org/jira/browse/IMPALA-11008
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>    Affects Versions: Impala 3.1.0, Impala 3.2.0, Impala 4.0.0, Impala 3.4.0
>            Reporter: Xianqing He
>            Assignee: Xianqing He
>            Priority: Major
>              Labels: correctness
>
> {code:java}
> Query: explain select * from (
>  select case when id is not null and some_nulls is null then true else null end a from (select straight_join t1.id, t2.some_nulls from functional.nullrows t1 left join functional.nullrows t2 on t1.id=t2.id) t)t where a is not null
> Max Per-Host Resource Reservation: Memory=1.95MB Threads=3
> Per-Host Resource Estimates: Memory=66MB
> Codegen disabled by planner
> ""
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [LEFT OUTER JOIN]
> |  hash predicates: t1.id = t2.id
> |  other predicates: CASE WHEN t1.id IS NOT NULL AND t2.some_nulls IS NULL THEN TRUE ELSE NULL END IS NOT NULL
> |  row-size=39B cardinality=26
> |
> |--01:SCAN HDFS [functional.nullrows t2]
> |     partitions=1/1 files=1 size=541B
> |     predicates: CASE WHEN t2.id IS NOT NULL AND t2.some_nulls IS NULL THEN TRUE ELSE NULL END IS NOT NULL
> |     row-size=26B cardinality=3
> |
> 00:SCAN HDFS [functional.nullrows t1]
>    partitions=1/1 files=1 size=541B
>    row-size=13B cardinality=26 {code}
> It  is incorrect to propagate this inferred predicates into a plan subtree that is on the nullable side of an outer join.
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org