You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Yongzhi Chen (Jira)" <ji...@apache.org> on 2021/06/21 09:20:00 UTC
[jira] [Created] (IMPALA-10758) S3PlannerTest.testNestedCollections
fails because of mismatch plan
Yongzhi Chen created IMPALA-10758:
-------------------------------------
Summary: S3PlannerTest.testNestedCollections fails because of mismatch plan
Key: IMPALA-10758
URL: https://issues.apache.org/jira/browse/IMPALA-10758
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Yongzhi Chen
S3PlannerTest.testNestedCollections fails in impala-asf-master-core-s3 with following error:
Error Message
Section PLAN of query:
select 1
from tpch_nested_parquet.region.r_nations t1
inner join tpch_nested_parquet.customer t2 on t2.c_nationkey = t1.pos
inner join tpch_nested_parquet.region t3 on t3.r_comment = t2.c_address
left join t2.c_orders t4
inner join tpch_nested_parquet.region t5 on t5.r_regionkey = t2.c_custkey
left join t4.item.o_lineitems t6 on t6.item.l_returnflag = t4.item.o_orderpriority
Actual does not match expected result:
PLAN-ROOT SINK
|
14:SUBPLAN
| row-size=183B cardinality=1
|
|--12:SUBPLAN
| | row-size=183B cardinality=1
| |
| |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN]
| | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority
| | | row-size=183B cardinality=10
| | |
| | |--08:SINGULAR ROW SRC
| | | row-size=171B cardinality=1
| | |
| | 09:UNNEST [t4.item.o_lineitems t6]
| | row-size=0B cardinality=10
| |
| 11:NESTED LOOP JOIN [RIGHT OUTER JOIN]
| | row-size=171B cardinality=1
| |
| |--06:SINGULAR ROW SRC
| | row-size=147B cardinality=1
| |
| 07:UNNEST [t2.c_orders t4]
| row-size=0B cardinality=10
|
13:HASH JOIN [INNER JOIN]
| hash predicates: t1.pos = t2.c_nationkey
| runtime filters: RF000 <- t2.c_nationkey, RF001 <- t2.c_nationkey
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| row-size=147B cardinality=1
|
|--05:HASH JOIN [INNER JOIN]
| | hash predicates: t3.r_comment = t2.c_address
| | runtime filters: RF002 <- t2.c_address
| | row-size=139B cardinality=1
| |
| |--04:HASH JOIN [INNER JOIN]
| | | hash predicates: t2.c_custkey = t5.r_regionkey
| | | runtime filters: RF004 <- t5.r_regionkey
| | | row-size=61B cardinality=5
| | |
| | |--03:SCAN S3 [tpch_nested_parquet.region t5]
| | | S3 partitions=1/1 files=1 size=3.59KB
| | | row-size=2B cardinality=5
| | |
| | 01:SCAN S3 [tpch_nested_parquet.customer t2]
| | S3 partitions=1/1 files=4 size=289.06MB
| | runtime filters: RF004 -> t2.c_custkey
| | row-size=59B cardinality=150.00K
| |
| 02:SCAN S3 [tpch_nested_parquet.region t3]
| S3 partitions=1/1 files=1 size=3.59KB
| runtime filters: RF002 -> t3.r_comment
| row-size=78B cardinality=5
|
00:SCAN S3 [tpch_nested_parquet.region.r_nations t1]
S3 partitions=1/1 files=1 size=3.59KB
runtime filters: RF001 -> t1.pos, RF000 -> t1.pos
row-size=8B cardinality=50
Expected:
PLAN-ROOT SINK
|
14:SUBPLAN
| row-size=183B cardinality=1
|
|--12:SUBPLAN
| | row-size=183B cardinality=1
| |
| |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN]
| | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority
| | | row-size=183B cardinality=10
| | |
| | |--08:SINGULAR ROW SRC
| | | row-size=171B cardinality=1
| | |
| | 09:UNNEST [t4.item.o_lineitems t6]
| | row-size=0B cardinality=10
| |
| 11:NESTED LOOP JOIN [RIGHT OUTER JOIN]
| | row-size=171B cardinality=1
| |
| |--06:SINGULAR ROW SRC
| | row-size=147B cardinality=1
| |
| 07:UNNEST [t2.c_orders t4]
| row-size=0B cardinality=10
|
13:HASH JOIN [INNER JOIN]
| hash predicates: t1.pos = t2.c_nationkey
| runtime filters: RF000 <- t2.c_nationkey
| row-size=147B cardinality=1
|
|--05:HASH JOIN [INNER JOIN]
| | hash predicates: t3.r_comment = t2.c_address
| | runtime filters: RF002 <- t2.c_address
| | row-size=139B cardinality=1
| |
| |--04:HASH JOIN [INNER JOIN]
| | | hash predicates: t2.c_custkey = t5.r_regionkey
| | | runtime filters: RF004 <- t5.r_regionkey
| | | row-size=61B cardinality=5
| | |
| | |--03:SCAN HDFS [tpch_nested_parquet.region t5]
| | | HDFS partitions=1/1 files=1 size=3.59KB
| | | row-size=2B cardinality=5
| | |
| | 01:SCAN HDFS [tpch_nested_parquet.customer t2]
| | HDFS partitions=1/1 files=4 size=289.02MB
| | runtime filters: RF004 -> t2.c_custkey
| | row-size=59B cardinality=150.00K
| |
| 02:SCAN HDFS [tpch_nested_parquet.region t3]
| HDFS partitions=1/1 files=1 size=3.59KB
| runtime filters: RF002 -> t3.r_comment
| row-size=78B cardinality=5
|
00:SCAN HDFS [tpch_nested_parquet.region.r_nations t1]
HDFS partitions=1/1 files=1 size=3.59KB
runtime filters: RF000 -> t1.pos
row-size=8B cardinality=50
Verbose plan:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
Per-Host Resources: mem-estimate=500.81MB mem-reservation=28.84MB thread-reservation=5 runtime-filters-memory=3.00MB
PLAN-ROOT SINK
| output exprs: CAST(1 AS TINYINT)
| mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0
|
14:SUBPLAN
| mem-estimate=0B mem-reservation=0B thread-reservation=0
| tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=1
| in pipelines: 00(GETNEXT)
|
|--12:SUBPLAN
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
| | tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=1
| | in pipelines: 00(GETNEXT)
| |
| |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN]
| | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority
| | | mem-estimate=172B mem-reservation=0B thread-reservation=0
| | | tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=10
| | | in pipelines: 00(GETNEXT)
| | |
| | |--08:SINGULAR ROW SRC
| | | parent-subplan=12
| | | mem-estimate=0B mem-reservation=0B thread-reservation=0
| | | tuple-ids=3N,0,2,1,4 row-size=171B cardinality=1
| | | in pipelines: 00(GETNEXT)
| | |
| | 09:UNNEST [t4.item.o_lineitems t6]
| | parent-subplan=12
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
| | tuple-ids=5 row-size=0B cardinality=10
| | in pipelines: 00(GETNEXT)
| |
| 11:NESTED LOOP JOIN [RIGHT OUTER JOIN]
| | mem-estimate=148B mem-reservation=0B thread-reservation=0
| | tuple-ids=3N,0,2,1,4 row-size=171B cardinality=1
| | in pipelines: 00(GETNEXT)
| |
| |--06:SINGULAR ROW SRC
| | parent-subplan=14
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
| | tuple-ids=0,2,1,4 row-size=147B cardinality=1
| | in pipelines: 00(GETNEXT)
| |
| 07:UNNEST [t2.c_orders t4]
| parent-subplan=14
| mem-estimate=0B mem-reservation=0B thread-reservation=0
| tuple-ids=3 row-size=0B cardinality=10
| in pipelines: 00(GETNEXT)
|
13:HASH JOIN [INNER JOIN]
| hash predicates: t1.pos = t2.c_nationkey
| fk/pk conjuncts: assumed fk/pk
| runtime filters: RF000[bloom] <- t2.c_nationkey, RF001[min_max] <- t2.c_nationkey
| mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| tuple-ids=0,2,1,4 row-size=147B cardinality=1
| in pipelines: 00(GETNEXT), 02(OPEN)
|
|--05:HASH JOIN [INNER JOIN]
| | hash predicates: t3.r_comment = t2.c_address
| | fk/pk conjuncts: t3.r_comment = t2.c_address
| | runtime filters: RF002[bloom] <- t2.c_address
| | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | tuple-ids=2,1,4 row-size=139B cardinality=1
| | in pipelines: 02(GETNEXT), 01(OPEN)
| |
| |--04:HASH JOIN [INNER JOIN]
| | | hash predicates: t2.c_custkey = t5.r_regionkey
| | | fk/pk conjuncts: t2.c_custkey = t5.r_regionkey
| | | runtime filters: RF004[bloom] <- t5.r_regionkey
| | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
| | | tuple-ids=1,4 row-size=61B cardinality=5
| | | in pipelines: 01(GETNEXT), 03(OPEN)
| | |
| | |--03:SCAN S3 [tpch_nested_parquet.region t5]
| | | S3 partitions=1/1 files=1 size=3.59KB
| | | stored statistics:
| | | table: rows=5 size=3.59KB
| | | columns: all
| | | extrapolated-rows=disabled max-scan-range-rows=5
| | | file formats: [PARQUET]
| | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
| | | tuple-ids=4 row-size=2B cardinality=5
| | | in pipelines: 03(GETNEXT)
| | |
| | 01:SCAN S3 [tpch_nested_parquet.customer t2]
| | S3 partitions=1/1 files=4 size=289.06MB
| | runtime filters: RF004[bloom] -> t2.c_custkey
| | stored statistics:
| | table: rows=150.00K size=289.06MB
| | columns missing stats: c_orders
| | extrapolated-rows=disabled max-scan-range-rows=50.28K
| | file formats: [PARQUET]
| | mem-estimate=440.00MB mem-reservation=16.00MB thread-reservation=1
| | tuple-ids=1 row-size=59B cardinality=150.00K
| | in pipelines: 01(GETNEXT)
| |
| 02:SCAN S3 [tpch_nested_parquet.region t3]
| S3 partitions=1/1 files=1 size=3.59KB
| runtime filters: RF002[bloom] -> t3.r_comment
| stored statistics:
| table: rows=5 size=3.59KB
| columns: all
| extrapolated-rows=disabled max-scan-range-rows=5
| file formats: [PARQUET]
| mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
| tuple-ids=2 row-size=78B cardinality=5
| in pipelines: 02(GETNEXT)
|
00:SCAN S3 [tpch_nested_parquet.region.r_nations t1]
S3 partitions=1/1 files=1 size=3.59KB
runtime filters: RF001[min_max] -> t1.pos, RF000[bloom] -> t1.pos
stored statistics:
table: rows=5 size=3.59KB
columns: all
extrapolated-rows=disabled max-scan-range-rows=5
file formats: [PARQUET]
mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
tuple-ids=0 row-size=8B cardinality=50
in pipelines: 00(GETNEXT)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)