You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/06/13 07:07:00 UTC
[jira] [Commented] (IMPALA-11200) Redundant additions to ExecOption field in query profile of grouping aggregator node when inside a subplan
[ https://issues.apache.org/jira/browse/IMPALA-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731919#comment-17731919 ]
ASF subversion and git services commented on IMPALA-11200:
----------------------------------------------------------
Commit 47309d14ca6bd274dd72674e12092f6dd3e034f3 in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=47309d14c ]
IMPALA-12204: Fix redundant codegen info added in subplan profiles
The SUBPLAN node will open its right child node many times in its
GetNext(), depending on how many rows generated from its left child. The
right child of a SUBPLAN node is a subtree of operators. They should not
add codegen info into profile in their Open() method since it will be
invoked repeatedly.
Currently, DataSink and UnionNode have such an issue. This patch fixes
them by adding the codegen info to profile in Close() instead of Open(),
just like what we did in IMPALA-11200.
Tests:
- Add e2e tests
Change-Id: I99a0a842df63a03c61024e2b77d5118ca63a2b2d
Reviewed-on: http://gerrit.cloudera.org:8080/20037
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
> Redundant additions to ExecOption field in query profile of grouping aggregator node when inside a subplan
> -----------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-11200
> URL: https://issues.apache.org/jira/browse/IMPALA-11200
> Project: IMPALA
> Issue Type: Bug
> Reporter: Bikramjeet Vig
> Assignee: Csaba Ringhofer
> Priority: Major
> Fix For: Impala 4.1.0
>
>
> There is an excessive addition of "Codegen Enabled" text to "ExecOption" field in the query profile when a grouping agg is a part of a subplan that is created to unnest a complex type.
> I was able to reproduce this using one of the queries we used for end to end testing:
> {noformat}
> use tpch_nested_parquet;
> select c_custkey, v.* from customer c,
> (select o_orderpriority, count(o_orderkey) c, sum(o_totalprice) s,
> avg(o_totalprice) a, max(o_orderstatus) mx,
> min(o_orderdate) mn
> from c.c_orders
> group by o_orderpriority) v
> where c_custkey < 4;
> {noformat}
> From the query profile:
> {noformat}
> AGGREGATION_NODE (id=4):
> - InactiveTotalTime: 0.000ns
> - PeakMemoryUsage: 36.04 MB (37794944)
> - RowsReturned: 0 (0)
> - RowsReturnedRate: 0
> - TotalTime: 1.571ms
> GroupingAggregator 0:
> ExecOption: Codegen Enabled, Codegen Enabled, Codegen Enabled <== THIS PART!
> - BuildTime: 68.253us
> - GetResultsTime: 72.634us
> {noformat}
> The reason this happens is because "Codegen Enabled" is added to the ExecOption everytime the agg node is Opened as a result of the subplan being called again and again to unnest.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org