You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Qifan Chen (Jira)" <ji...@apache.org> on 2022/10/13 16:54:00 UTC
[jira] [Closed] (IMPALA-10758) S3PlannerTest.testNestedCollections fails because of mismatch plan
[ https://issues.apache.org/jira/browse/IMPALA-10758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Qifan Chen closed IMPALA-10758.
-------------------------------
Resolution: Not A Bug
Verified that the plan difference does not show up in recent core s3 tests. The test passes.
> S3PlannerTest.testNestedCollections fails because of mismatch plan
> ------------------------------------------------------------------
>
> Key: IMPALA-10758
> URL: https://issues.apache.org/jira/browse/IMPALA-10758
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Yongzhi Chen
> Assignee: Qifan Chen
> Priority: Critical
>
> S3PlannerTest.testNestedCollections fails in impala-asf-master-core-s3 with following error:
> {noformat}
> Error Message
> Section PLAN of query:
> select 1
> from tpch_nested_parquet.region.r_nations t1
> inner join tpch_nested_parquet.customer t2 on t2.c_nationkey = t1.pos
> inner join tpch_nested_parquet.region t3 on t3.r_comment = t2.c_address
> left join t2.c_orders t4
> inner join tpch_nested_parquet.region t5 on t5.r_regionkey = t2.c_custkey
> left join t4.item.o_lineitems t6 on t6.item.l_returnflag = t4.item.o_orderpriority
> Actual does not match expected result:
> PLAN-ROOT SINK
> |
> 14:SUBPLAN
> | row-size=183B cardinality=1
> |
> |--12:SUBPLAN
> | | row-size=183B cardinality=1
> | |
> | |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN]
> | | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority
> | | | row-size=183B cardinality=10
> | | |
> | | |--08:SINGULAR ROW SRC
> | | | row-size=171B cardinality=1
> | | |
> | | 09:UNNEST [t4.item.o_lineitems t6]
> | | row-size=0B cardinality=10
> | |
> | 11:NESTED LOOP JOIN [RIGHT OUTER JOIN]
> | | row-size=171B cardinality=1
> | |
> | |--06:SINGULAR ROW SRC
> | | row-size=147B cardinality=1
> | |
> | 07:UNNEST [t2.c_orders t4]
> | row-size=0B cardinality=10
> |
> 13:HASH JOIN [INNER JOIN]
> | hash predicates: t1.pos = t2.c_nationkey
> | runtime filters: RF000 <- t2.c_nationkey, RF001 <- t2.c_nationkey
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> | row-size=147B cardinality=1
> |
> |--05:HASH JOIN [INNER JOIN]
> | | hash predicates: t3.r_comment = t2.c_address
> | | runtime filters: RF002 <- t2.c_address
> | | row-size=139B cardinality=1
> | |
> | |--04:HASH JOIN [INNER JOIN]
> | | | hash predicates: t2.c_custkey = t5.r_regionkey
> | | | runtime filters: RF004 <- t5.r_regionkey
> | | | row-size=61B cardinality=5
> | | |
> | | |--03:SCAN S3 [tpch_nested_parquet.region t5]
> | | | S3 partitions=1/1 files=1 size=3.59KB
> | | | row-size=2B cardinality=5
> | | |
> | | 01:SCAN S3 [tpch_nested_parquet.customer t2]
> | | S3 partitions=1/1 files=4 size=289.06MB
> | | runtime filters: RF004 -> t2.c_custkey
> | | row-size=59B cardinality=150.00K
> | |
> | 02:SCAN S3 [tpch_nested_parquet.region t3]
> | S3 partitions=1/1 files=1 size=3.59KB
> | runtime filters: RF002 -> t3.r_comment
> | row-size=78B cardinality=5
> |
> 00:SCAN S3 [tpch_nested_parquet.region.r_nations t1]
> S3 partitions=1/1 files=1 size=3.59KB
> runtime filters: RF001 -> t1.pos, RF000 -> t1.pos
> row-size=8B cardinality=50
> Expected:
> PLAN-ROOT SINK
> |
> 14:SUBPLAN
> | row-size=183B cardinality=1
> |
> |--12:SUBPLAN
> | | row-size=183B cardinality=1
> | |
> | |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN]
> | | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority
> | | | row-size=183B cardinality=10
> | | |
> | | |--08:SINGULAR ROW SRC
> | | | row-size=171B cardinality=1
> | | |
> | | 09:UNNEST [t4.item.o_lineitems t6]
> | | row-size=0B cardinality=10
> | |
> | 11:NESTED LOOP JOIN [RIGHT OUTER JOIN]
> | | row-size=171B cardinality=1
> | |
> | |--06:SINGULAR ROW SRC
> | | row-size=147B cardinality=1
> | |
> | 07:UNNEST [t2.c_orders t4]
> | row-size=0B cardinality=10
> |
> 13:HASH JOIN [INNER JOIN]
> | hash predicates: t1.pos = t2.c_nationkey
> | runtime filters: RF000 <- t2.c_nationkey
> | row-size=147B cardinality=1
> |
> |--05:HASH JOIN [INNER JOIN]
> | | hash predicates: t3.r_comment = t2.c_address
> | | runtime filters: RF002 <- t2.c_address
> | | row-size=139B cardinality=1
> | |
> | |--04:HASH JOIN [INNER JOIN]
> | | | hash predicates: t2.c_custkey = t5.r_regionkey
> | | | runtime filters: RF004 <- t5.r_regionkey
> | | | row-size=61B cardinality=5
> | | |
> | | |--03:SCAN HDFS [tpch_nested_parquet.region t5]
> | | | HDFS partitions=1/1 files=1 size=3.59KB
> | | | row-size=2B cardinality=5
> | | |
> | | 01:SCAN HDFS [tpch_nested_parquet.customer t2]
> | | HDFS partitions=1/1 files=4 size=289.02MB
> | | runtime filters: RF004 -> t2.c_custkey
> | | row-size=59B cardinality=150.00K
> | |
> | 02:SCAN HDFS [tpch_nested_parquet.region t3]
> | HDFS partitions=1/1 files=1 size=3.59KB
> | runtime filters: RF002 -> t3.r_comment
> | row-size=78B cardinality=5
> |
> 00:SCAN HDFS [tpch_nested_parquet.region.r_nations t1]
> HDFS partitions=1/1 files=1 size=3.59KB
> runtime filters: RF000 -> t1.pos
> row-size=8B cardinality=50
> Verbose plan:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> Per-Host Resources: mem-estimate=500.81MB mem-reservation=28.84MB thread-reservation=5 runtime-filters-memory=3.00MB
> PLAN-ROOT SINK
> | output exprs: CAST(1 AS TINYINT)
> | mem-estimate=4.00MB mem-reservation=4.00MB spill-buffer=2.00MB thread-reservation=0
> |
> 14:SUBPLAN
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=1
> | in pipelines: 00(GETNEXT)
> |
> |--12:SUBPLAN
> | | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | | tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=1
> | | in pipelines: 00(GETNEXT)
> | |
> | |--10:NESTED LOOP JOIN [RIGHT OUTER JOIN]
> | | | join predicates: t6.item.l_returnflag = t4.item.o_orderpriority
> | | | mem-estimate=172B mem-reservation=0B thread-reservation=0
> | | | tuple-ids=5N,3N,0,2,1,4 row-size=183B cardinality=10
> | | | in pipelines: 00(GETNEXT)
> | | |
> | | |--08:SINGULAR ROW SRC
> | | | parent-subplan=12
> | | | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | | | tuple-ids=3N,0,2,1,4 row-size=171B cardinality=1
> | | | in pipelines: 00(GETNEXT)
> | | |
> | | 09:UNNEST [t4.item.o_lineitems t6]
> | | parent-subplan=12
> | | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | | tuple-ids=5 row-size=0B cardinality=10
> | | in pipelines: 00(GETNEXT)
> | |
> | 11:NESTED LOOP JOIN [RIGHT OUTER JOIN]
> | | mem-estimate=148B mem-reservation=0B thread-reservation=0
> | | tuple-ids=3N,0,2,1,4 row-size=171B cardinality=1
> | | in pipelines: 00(GETNEXT)
> | |
> | |--06:SINGULAR ROW SRC
> | | parent-subplan=14
> | | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | | tuple-ids=0,2,1,4 row-size=147B cardinality=1
> | | in pipelines: 00(GETNEXT)
> | |
> | 07:UNNEST [t2.c_orders t4]
> | parent-subplan=14
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=3 row-size=0B cardinality=10
> | in pipelines: 00(GETNEXT)
> |
> 13:HASH JOIN [INNER JOIN]
> | hash predicates: t1.pos = t2.c_nationkey
> | fk/pk conjuncts: assumed fk/pk
> | runtime filters: RF000[bloom] <- t2.c_nationkey, RF001[min_max] <- t2.c_nationkey
> | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
> | tuple-ids=0,2,1,4 row-size=147B cardinality=1
> | in pipelines: 00(GETNEXT), 02(OPEN)
> |
> |--05:HASH JOIN [INNER JOIN]
> | | hash predicates: t3.r_comment = t2.c_address
> | | fk/pk conjuncts: t3.r_comment = t2.c_address
> | | runtime filters: RF002[bloom] <- t2.c_address
> | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
> | | tuple-ids=2,1,4 row-size=139B cardinality=1
> | | in pipelines: 02(GETNEXT), 01(OPEN)
> | |
> | |--04:HASH JOIN [INNER JOIN]
> | | | hash predicates: t2.c_custkey = t5.r_regionkey
> | | | fk/pk conjuncts: t2.c_custkey = t5.r_regionkey
> | | | runtime filters: RF004[bloom] <- t5.r_regionkey
> | | | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB thread-reservation=0
> | | | tuple-ids=1,4 row-size=61B cardinality=5
> | | | in pipelines: 01(GETNEXT), 03(OPEN)
> | | |
> | | |--03:SCAN S3 [tpch_nested_parquet.region t5]
> | | | S3 partitions=1/1 files=1 size=3.59KB
> | | | stored statistics:
> | | | table: rows=5 size=3.59KB
> | | | columns: all
> | | | extrapolated-rows=disabled max-scan-range-rows=5
> | | | file formats: [PARQUET]
> | | | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
> | | | tuple-ids=4 row-size=2B cardinality=5
> | | | in pipelines: 03(GETNEXT)
> | | |
> | | 01:SCAN S3 [tpch_nested_parquet.customer t2]
> | | S3 partitions=1/1 files=4 size=289.06MB
> | | runtime filters: RF004[bloom] -> t2.c_custkey
> | | stored statistics:
> | | table: rows=150.00K size=289.06MB
> | | columns missing stats: c_orders
> | | extrapolated-rows=disabled max-scan-range-rows=50.28K
> | | file formats: [PARQUET]
> | | mem-estimate=440.00MB mem-reservation=16.00MB thread-reservation=1
> | | tuple-ids=1 row-size=59B cardinality=150.00K
> | | in pipelines: 01(GETNEXT)
> | |
> | 02:SCAN S3 [tpch_nested_parquet.region t3]
> | S3 partitions=1/1 files=1 size=3.59KB
> | runtime filters: RF002[bloom] -> t3.r_comment
> | stored statistics:
> | table: rows=5 size=3.59KB
> | columns: all
> | extrapolated-rows=disabled max-scan-range-rows=5
> | file formats: [PARQUET]
> | mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
> | tuple-ids=2 row-size=78B cardinality=5
> | in pipelines: 02(GETNEXT)
> |
> 00:SCAN S3 [tpch_nested_parquet.region.r_nations t1]
> S3 partitions=1/1 files=1 size=3.59KB
> runtime filters: RF001[min_max] -> t1.pos, RF000[bloom] -> t1.pos
> stored statistics:
> table: rows=5 size=3.59KB
> columns: all
> extrapolated-rows=disabled max-scan-range-rows=5
> file formats: [PARQUET]
> mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
> tuple-ids=0 row-size=8B cardinality=50
> in pipelines: 00(GETNEXT)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)