[ https://issues.apache.org/jira/browse/IMPALA-11914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705720#comment-17705720 ]
ASF subversion and git services commented on IMPALA-11914:
----------------------------------------------------------
Commit d85d34fc31d77cd9147a58f34f05f514af35180d in impala's branch refs/heads/branch-4.1.2 from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d85d34fc3 ]
IMPALA-11914: Fix broken verbose explain on MT_DOP > 0
When running with MT_DOP>0, EXPLAIN_LEVEL=VERBOSE will produce broken
explain string on fragment consuming input from join build fragment. It
will continue printing the explain plan for that child fragment, the
join build fragment, rather than skipping it. This patch fix the issue
by skipping such children fragment.
Testing:
- Add PlannerTests.testExplainVerboseMtDop
Backport Note for 4.1.2:
Resovled trivial conflicts in PlannerTest.java
Change-Id: Iad082074933204daaba0c675abb34c144e12c3f7
Reviewed-on: http://gerrit.cloudera.org:8080/19491
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> EXPLAIN_LEVEL=VERBOSE is broken by join build fragment (MT_DOP>0)
> -----------------------------------------------------------------
>
> Key: IMPALA-11914
> URL: https://issues.apache.org/jira/browse/IMPALA-11914
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.2.0
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Minor
> Labels: ramp-up
> Fix For: Impala 4.3.0
>
> Attachments: explain-verbose-tpcds-q12.txt
>
>
> When running with MT_DOP>0, EXPLAIN_LEVEL=VERBOSE will produce broken explain string on fragment consuming input from join build fragment.
> {code:java}
> | F00:PLAN FRAGMENT [RANDOM] hosts=2 instances=2 |
> | Per-Host Shared Resources: mem-estimate=2.00MB mem-reservation=2.00MB thread-reservation=0 runtime-filters-memory=2.00MB |
> | Per-Instance Resources: mem-estimate=67.58MB mem-reservation=42.00MB thread-reservation=1 |
> | DATASTREAM SINK [FRAGMENT=F03, EXCHANGE=11, HASH(i_class)] |
> | | mem-estimate=1.58MB mem-reservation=0B thread-reservation=0 |
> | 05:AGGREGATE [STREAMING] |
> | | output: sum(ws_ext_sales_price) |
> | | group by: i_item_id, i_item_desc, i_category, i_class, i_current_price |
> | | mem-estimate=34.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0 |
> | | tuple-ids=3 row-size=198B cardinality=215.82K |
> | | in pipelines: 00(GETNEXT) |
> | | |
> | 04:HASH JOIN [INNER JOIN, BROADCAST] |
> | | hash-table-id=00 |
> | | hash predicates: ws_sold_date_sk = d_date_sk |
> | | fk/pk conjuncts: ws_sold_date_sk = d_date_sk |
> | | mem-estimate=0B mem-reservation=0B spill-buffer=64.00KB thread-reservation=0 |
> | | tuple-ids=0,1,2 row-size=232B cardinality=215.82K |
> | | in pipelines: 00(GETNEXT), 02(OPEN) |
> | | |
> | F05:PLAN FRAGMENT [RANDOM] hosts=2 instances=2 |
> | Per-Instance Resources: mem-estimate=8.96MB mem-reservation=8.75MB thread-reservation=1 runtime-filters-memory=1.00MB |
> | JOIN BUILD |
> | | join-table-id=00 plan-id=01 cohort-id=01 |
> | | build expressions: d_date_sk |
> | | runtime filters: RF000[bloom] <- d_date_sk |
> | | mem-estimate=7.75MB mem-reservation=7.75MB spill-buffer=64.00KB thread-reservation=0 |
> | | |
> | 10:EXCHANGE [BROADCAST] |
> | mem-estimate=215.48KB mem-reservation=0B thread-reservation=0 |
> | tuple-ids=2 row-size=26B cardinality=7.30K |
> | in pipelines: 02(GETNEXT) |
> | | |
> | 03:HASH JOIN [INNER JOIN, BROADCAST] |
> | | hash-table-id=01 |
> | | hash predicates: ws_item_sk = i_item_sk |
> | | fk/pk conjuncts: ws_item_sk = i_item_sk |
> | | mem-estimate=0B mem-reservation=0B spill-buffer=128.00KB thread-reservation=0 |
> | | tuple-ids=0,1 row-size=206B cardinality=215.82K |
> | | in pipelines: 00(GETNEXT), 01(OPEN) |
> | | |
> | F06:PLAN FRAGMENT [RANDOM] hosts=2 instances=2 |
> | Per-Instance Resources: mem-estimate=13.67MB mem-reservation=12.50MB thread-reservation=1 runtime-filters-memory=1.00MB |
> | JOIN BUILD |
> | | join-table-id=01 plan-id=02 cohort-id=01 |
> | | build expressions: i_item_sk |
> | | runtime filters: RF002[bloom] <- i_item_sk |
> | | mem-estimate=11.50MB mem-reservation=11.50MB spill-buffer=128.00KB thread-reservation=0 |
> | | |
> | 09:EXCHANGE [BROADCAST] |
> | mem-estimate=1.17MB mem-reservation=0B thread-reservation=0 |
> | tuple-ids=1 row-size=190B cardinality=5.40K |
> | in pipelines: 01(GETNEXT) |
> | | |
> | 00:SCAN HDFS [tpcds_parquet.web_sales, RANDOM] |
> | HDFS partitions=1/1 files=2 size=45.09MB |
> | runtime filters: RF000[bloom] -> ws_sold_date_sk, RF002[bloom] -> ws_item_sk |
> | stored statistics: |
> | table: rows=719.38K size=45.09MB |
> | columns: all |
> | extrapolated-rows=disabled max-scan-range-rows=644.77K |
> | file formats: [PARQUET] |
> | mem-estimate=32.00MB mem-reservation=8.00MB thread-reservation=0 |
> | tuple-ids=0 row-size=16B cardinality=719.38K |
> | in pipelines: 00(GETNEXT) | {code}
> F00 should not print F05 and F06 within it. Attached is the full explain output.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org