You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/06/03 18:13:00 UTC
[jira] [Commented] (IMPALA-9814) Analytic planner can
under-parallelise with mt_dop
[ https://issues.apache.org/jira/browse/IMPALA-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125200#comment-17125200 ]
ASF subversion and git services commented on IMPALA-9814:
---------------------------------------------------------
Commit 69b0b15f59169520cdffdbc31f90a5a04aea79e0 in impala's branch refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=69b0b15 ]
IMPALA-9000: clean up misc TODO-MT comments
Filed IMPALA-9812, IMPALA-9811 for misc cleanup tasks and
IMPALA-9814 for the planner estimation issue in AnalyticPlanner.
Change-Id: Ieeffebf6883427ca88640d4fe57159078516ba11
Reviewed-on: http://gerrit.cloudera.org:8080/16023
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Analytic planner can under-parallelise with mt_dop
> --------------------------------------------------
>
> Key: IMPALA-9814
> URL: https://issues.apache.org/jira/browse/IMPALA-9814
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: multithreading
>
> This is a TODO-MT in AnalyticPlanner.java. Here is an example of a query that is under-parallelised
> {noformat}
>
> set mt_dop=3;
> explain select count(*) over (partition by ss_addr_sk, ss_store_sk), count(*) over (partition by ss_sold_date_sk, ss_st
> ore_sk)
> from tpcds_parquet.store_sales;
>
> +---------------------------------------------------------------------------+
> | Explain String |
> +---------------------------------------------------------------------------+
> | Max Per-Host Resource Reservation: Memory=61.50MB Threads=7 |
> | Per-Host Resource Estimates: Memory=148MB |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 06:EXCHANGE [UNPARTITIONED] |
> | | |
> | 04:ANALYTIC |
> | | functions: count(*) |
> | | partition by: ss_sold_date_sk, ss_store_sk |
> | | row-size=28B cardinality=2.88M |
> | | |
> | 03:SORT |
> | | order by: ss_sold_date_sk ASC NULLS FIRST, ss_store_sk ASC NULLS FIRST |
> | | row-size=20B cardinality=2.88M |
> | | |
> | 02:ANALYTIC |
> | | functions: count(*) |
> | | partition by: ss_addr_sk, ss_store_sk |
> | | row-size=20B cardinality=2.88M |
> | | |
> | 01:SORT |
> | | order by: ss_addr_sk ASC NULLS FIRST, ss_store_sk ASC NULLS FIRST |
> | | row-size=12B cardinality=2.88M |
> | | |
> | 05:EXCHANGE [HASH(ss_store_sk)] |
> | | |
> | 00:SCAN HDFS [tpcds_parquet.store_sales] |
> | HDFS partitions=1824/1824 files=1824 size=196.96MB |
> | row-size=12B cardinality=2.88M |
> +---------------------------------------------------------------------------+
>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org