You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/06/01 22:49:00 UTC
[jira] [Created] (IMPALA-9814) Analytic planner can
under-parallelise with mt_dop
Tim Armstrong created IMPALA-9814:
-------------------------------------
Summary: Analytic planner can under-parallelise with mt_dop
Key: IMPALA-9814
URL: https://issues.apache.org/jira/browse/IMPALA-9814
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Tim Armstrong
Assignee: Tim Armstrong
This is a TODO-MT in AnalyticPlanner.java. Here is an example of a query that is under-parallelised
{noformat}
set mt_dop=3;
explain select count(*) over (partition by ss_addr_sk, ss_store_sk), count(*) over (partition by ss_sold_date_sk, ss_st
ore_sk)
from tpcds_parquet.store_sales;
+---------------------------------------------------------------------------+
| Explain String |
+---------------------------------------------------------------------------+
| Max Per-Host Resource Reservation: Memory=61.50MB Threads=7 |
| Per-Host Resource Estimates: Memory=148MB |
| |
| PLAN-ROOT SINK |
| | |
| 06:EXCHANGE [UNPARTITIONED] |
| | |
| 04:ANALYTIC |
| | functions: count(*) |
| | partition by: ss_sold_date_sk, ss_store_sk |
| | row-size=28B cardinality=2.88M |
| | |
| 03:SORT |
| | order by: ss_sold_date_sk ASC NULLS FIRST, ss_store_sk ASC NULLS FIRST |
| | row-size=20B cardinality=2.88M |
| | |
| 02:ANALYTIC |
| | functions: count(*) |
| | partition by: ss_addr_sk, ss_store_sk |
| | row-size=20B cardinality=2.88M |
| | |
| 01:SORT |
| | order by: ss_addr_sk ASC NULLS FIRST, ss_store_sk ASC NULLS FIRST |
| | row-size=12B cardinality=2.88M |
| | |
| 05:EXCHANGE [HASH(ss_store_sk)] |
| | |
| 00:SCAN HDFS [tpcds_parquet.store_sales] |
| HDFS partitions=1824/1824 files=1824 size=196.96MB |
| row-size=12B cardinality=2.88M |
+---------------------------------------------------------------------------+
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org