[jira] [Commented] (IMPALA-9694) IllegalStateException when inlineView has AggregationNode and different alias on the same column

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259291#comment-17259291 ] 

ASF subversion and git services commented on IMPALA-9694:
---------------------------------------------------------

Commit 49680559b0da843fbb6ff949d52c9d43f98364b1 in impala's branch refs/heads/master from Aman Sinha
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4968055 ]

IMPALA-10182: Don't add inferred identity predicates to SELECT node

For an inferred equality predicates of type c1 = c2 if both sides
are referring to the same underlying tuple and slot, it is an identity
predicate which should not be evaluated by the SELECT node since it
will incorrectly eliminate NULL rows. This patch fixes the behavior.

Testing:
 - Added planner tests with base table and with outer join
 - Added runtime tests with base table and with outer join
 - Added planner test for IMPALA-9694 (same root cause)
 - Ran PlannerTest .. no other plans changed

Change-Id: I924044f582652dbc50085851cc639f3dee1cd1f4
Reviewed-on: http://gerrit.cloudera.org:8080/16917
Reviewed-by: Aman Sinha <am...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> IllegalStateException when inlineView has AggregationNode and different alias on the same column
> ------------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-9694
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9694
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>            Reporter: Quanlong Huang
>            Assignee: Aman Sinha
>            Priority: Critical
>
> The following query fails with IllegalStateException:
> {code:sql}
> > create table t1(ts TIMESTAMP, id BIGINT);
> > select count(*) from (
>     select lead(ts) over (partition by id order by ts),
>            id foo,
>            id bar
>     from t1
>   ) v;
> ERROR: IllegalStateException: Illegal reference to non-materialized slot: tid=0 sid=1
> {code}
> The query works without the PartitionBy clause:
> {code:sql}
> select count(*) from (
>   select lead(ts) over (order by ts),
>          id foo,
>          id bar
>   from t1
> ) v;
> {code}
> Or avoid different alias on the same column:
> {code:sql}
> select count(*) from (
>   select lead(ts) over (partition by id order by ts),
>          id foo
>   from t1
> ) v;
> {code}
> *Clues*
>  Enabling TRACE level logging of the coordinator and rerun the failed query, the illegal predicate "v.foo = v.bar" shows up:
> {code:java}
> I0426 20:10:10.194453  1068 Analyzer.java:2143] 4241809b91bf54de:d5365a1700000000] Created inferred predicate: BinaryPredicate{op==, SlotRef{label=v.foo, type=BIGINT, id=4} SlotRef{label=v.bar, type=BIGINT, id=5}, isInferred=true}
> I0426 20:10:10.194509  1068 Analyzer.java:2574] 4241809b91bf54de:d5365a1700000000] Assigned BinaryPredicate{op==, SlotRef{label=v.foo, type=BIGINT, id=4} SlotRef{label=v.bar, type=BIGINT, id=5}, isInferred=true}
> {code}
> It's finally migrated into the inlineView and references to "t1.id" which is non-materialized. The planner generates this predicate since there are two auxiliary predicates "v.foo = t1.id", "v.bar = t1.id". These two auxiliary predicates only occur when the AggregationNode exists. We need to look deeper into this bug.
> *Workaround*
>  Replace the second occurrence of the reference column with an identity expression, e.g. change the above-failed query to:
> {code:sql}
> select count(*) from (
>   select lead(ts) over (partition by id order by ts),
>          id foo,
>          id - 1 + 1 as bar
>   from t1
> ) v;
> {code}
> By replacing the second occurrence of "id" to "id- 1 + 1", the planner can't detect that the equivalence between "v.foo" and "v.bar". So the AggregationNode won't have the redundant predicate "v.foo = v.bar" and won't have illegal reference to "t1.id".



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org