You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Luis E Martinez-Poblete (JIRA)" <ji...@apache.org> on 2018/12/11 19:04:00 UTC
[jira] [Updated] (IMPALA-7957) UNION ALL query returns incorrect
results
[ https://issues.apache.org/jira/browse/IMPALA-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luis E Martinez-Poblete updated IMPALA-7957:
--------------------------------------------
Description:
Synopsis:
=========
UNION ALL query returns incorrect results
Problem:
========
Customer reported a UNION ALL query returning incorrect results. The UNION ALL query has 2 legs, but Impala is only returning information from one leg.
Issue can be reproduced in the latest version of Impala. Below is the reproduction case:
{noformat}
create table mytest_t (c1 timestamp, c2 timestamp, c3 int, c4 int);
insert into mytest_t values (now(), ADDDATE (now(),1), 1,1);
insert into mytest_t values (now(), ADDDATE (now(),1), 2,2);
insert into mytest_t values (now(), ADDDATE (now(),1), 3,3);
SELECT t.c1
FROM
(SELECT c1, c2
FROM mytest_t) t
LEFT JOIN
(SELECT c1, c2
FROM mytest_t
WHERE c2 = c1) t2 ON (t.c2 = t2.c2)
UNION ALL
VALUES (NULL)
{noformat}
The above query produces the following execution plan:
{noformat}
+------------------------------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=34.02MB Threads=5 |
| Per-Host Resource Estimates: Memory=2.06GB |
| WARNING: The following tables are missing relevant table and/or column statistics. |
| default.mytest_t |
| |
| PLAN-ROOT SINK |
| | |
| 06:EXCHANGE [UNPARTITIONED] |
| | |
| 00:UNION |
| | constant-operands=1 |
| | |
| 04:SELECT |
| | predicates: default.mytest_t.c1 = default.mytest_t.c2 |
| | |
| 03:HASH JOIN [LEFT OUTER JOIN, BROADCAST] |
| | hash predicates: c2 = c2 |
| | |
| |--05:EXCHANGE [BROADCAST] |
| | | |
| | 02:SCAN HDFS [default.mytest_t] |
| | partitions=1/1 files=3 size=192B |
| | predicates: c2 = c1 |
| | |
| 01:SCAN HDFS [default.mytest_t] |
| partitions=1/1 files=3 size=192B |
+------------------------------------------------------------------------------------+
{noformat}
The issue is in operator 4:
{noformat}
| 04:SELECT |
| | predicates: default.mytest_t.c1 = default.mytest_t.c2 |
{noformat}
It's definitely a bug with predicate placement - that c1 = c2 predicate shouldn't be evaluated outside the right branch of the LEFT JOIN.
Thanks,
Luis Martinez.
was:
Synopsis:
=========
UNION ALL query returns incorrect results
Problem:
========
Customer reported a UNION ALL query returning incorrect results. The UNION ALL query has 2 legs, but Impala is only returning information from one leg.
Issue can be reproduced in the latest version of Impala. Below is the reproduction case:
create table mytest_t (c1 timestamp, c2 timestamp, c3 int, c4 int);
insert into mytest_t values (now(), ADDDATE (now(),1), 1,1);
insert into mytest_t values (now(), ADDDATE (now(),1), 2,2);
insert into mytest_t values (now(), ADDDATE (now(),1), 3,3);
SELECT t.c1
FROM
(SELECT c1, c2
FROM mytest_t) t
LEFT JOIN
(SELECT c1, c2
FROM mytest_t
WHERE c2 = c1) t2 ON (t.c2 = t2.c2)
UNION ALL
VALUES (NULL)
The above query produces the following execution plan:
+------------------------------------------------------------------------------------+
| Explain String |
+------------------------------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=34.02MB Threads=5 |
| Per-Host Resource Estimates: Memory=2.06GB |
| WARNING: The following tables are missing relevant table and/or column statistics. |
| default.mytest_t |
| |
| PLAN-ROOT SINK |
| | |
| 06:EXCHANGE [UNPARTITIONED] |
| | |
| 00:UNION |
| | constant-operands=1 |
| | |
| 04:SELECT |
| | predicates: default.mytest_t.c1 = default.mytest_t.c2 |
| | |
| 03:HASH JOIN [LEFT OUTER JOIN, BROADCAST] |
| | hash predicates: c2 = c2 |
| | |
| |--05:EXCHANGE [BROADCAST] |
| | | |
| | 02:SCAN HDFS [default.mytest_t] |
| | partitions=1/1 files=3 size=192B |
| | predicates: c2 = c1 |
| | |
| 01:SCAN HDFS [default.mytest_t] |
| partitions=1/1 files=3 size=192B |
+------------------------------------------------------------------------------------+
The issue is in operator 4:
| 04:SELECT |
| | predicates: default.mytest_t.c1 = default.mytest_t.c2 |
It's definitely a bug with predicate placement - that c1 = c2 predicate shouldn't be evaluated outside the right branch of the LEFT JOIN.
Thanks,
Luis Martinez.
> UNION ALL query returns incorrect results
> -----------------------------------------
>
> Key: IMPALA-7957
> URL: https://issues.apache.org/jira/browse/IMPALA-7957
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 2.12.0
> Reporter: Luis E Martinez-Poblete
> Priority: Major
> Labels: correctness
>
> Synopsis:
> =========
> UNION ALL query returns incorrect results
> Problem:
> ========
> Customer reported a UNION ALL query returning incorrect results. The UNION ALL query has 2 legs, but Impala is only returning information from one leg.
> Issue can be reproduced in the latest version of Impala. Below is the reproduction case:
> {noformat}
> create table mytest_t (c1 timestamp, c2 timestamp, c3 int, c4 int);
> insert into mytest_t values (now(), ADDDATE (now(),1), 1,1);
> insert into mytest_t values (now(), ADDDATE (now(),1), 2,2);
> insert into mytest_t values (now(), ADDDATE (now(),1), 3,3);
> SELECT t.c1
> FROM
> (SELECT c1, c2
> FROM mytest_t) t
> LEFT JOIN
> (SELECT c1, c2
> FROM mytest_t
> WHERE c2 = c1) t2 ON (t.c2 = t2.c2)
> UNION ALL
> VALUES (NULL)
> {noformat}
> The above query produces the following execution plan:
> {noformat}
> +------------------------------------------------------------------------------------+
> | Explain String |
> +------------------------------------------------------------------------------------+
> | Max Per-Host Resource Reservation: Memory=34.02MB Threads=5 |
> | Per-Host Resource Estimates: Memory=2.06GB |
> | WARNING: The following tables are missing relevant table and/or column statistics. |
> | default.mytest_t |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 06:EXCHANGE [UNPARTITIONED] |
> | | |
> | 00:UNION |
> | | constant-operands=1 |
> | | |
> | 04:SELECT |
> | | predicates: default.mytest_t.c1 = default.mytest_t.c2 |
> | | |
> | 03:HASH JOIN [LEFT OUTER JOIN, BROADCAST] |
> | | hash predicates: c2 = c2 |
> | | |
> | |--05:EXCHANGE [BROADCAST] |
> | | | |
> | | 02:SCAN HDFS [default.mytest_t] |
> | | partitions=1/1 files=3 size=192B |
> | | predicates: c2 = c1 |
> | | |
> | 01:SCAN HDFS [default.mytest_t] |
> | partitions=1/1 files=3 size=192B |
> +------------------------------------------------------------------------------------+
> {noformat}
> The issue is in operator 4:
> {noformat}
> | 04:SELECT |
> | | predicates: default.mytest_t.c1 = default.mytest_t.c2 |
> {noformat}
> It's definitely a bug with predicate placement - that c1 = c2 predicate shouldn't be evaluated outside the right branch of the LEFT JOIN.
> Thanks,
> Luis Martinez.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org