You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/04/02 15:43:00 UTC
[jira] [Commented] (IMPALA-12032) PROCESSING_COST_MIN_THREADS fail to cap minimum parallelism
[ https://issues.apache.org/jira/browse/IMPALA-12032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707681#comment-17707681 ]
ASF subversion and git services commented on IMPALA-12032:
----------------------------------------------------------
Commit 4e1751534b45d84b5c290b13a90d82c5ba15ed4a in impala's branch refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4e1751534 ]
IMPALA-12032: Fix min parallelism bug in PlanFragment
When COMPUTE_PROCESSING_COST=true, PROCESSING_COST_MIN_THREADS control
the minimum parallelism of query fragment per executor. However,
PlanFragment.adjustToMaxParallelism() did not take account of it during
adjustment.
This patch fix that bug by capping per-node fragment parallelism to
PROCESSING_COST_MIN_THREADS if cost based parallelism comes up with
number that is lower.
Testing:
- Set PROCESSING_COST_MIN_THREADS in PlannerTest.testProcessingCost.
- Pass test_executor_groups.py.
- Add test cases in query-options-test.cc.
Change-Id: I6e58d5d54f60818c003f488b1681b8660552f1e9
Reviewed-on: http://gerrit.cloudera.org:8080/19663
Reviewed-by: Kurt Deschler <kd...@cloudera.com>
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> PROCESSING_COST_MIN_THREADS fail to cap minimum parallelism
> -----------------------------------------------------------
>
> Key: IMPALA-12032
> URL: https://issues.apache.org/jira/browse/IMPALA-12032
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.2.0
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
> Fix For: Impala 4.2.0
>
>
> PROCESSING_COST_MIN_THREADS suppose to be a lower bound of per node parallelism when CPU costing algorithm adjust fragment parallelism across executor group set. But PlanFragment.adjustToMaxParallelism() does not take account of it during adjustment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org