You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/06/11 20:21:00 UTC
[jira] [Resolved] (IMPALA-3021) #Rows in exchange may be > #Rows in
scans
[ https://issues.apache.org/jira/browse/IMPALA-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-3021.
-----------------------------------
Resolution: Cannot Reproduce
> #Rows in exchange may be > #Rows in scans
> -----------------------------------------
>
> Key: IMPALA-3021
> URL: https://issues.apache.org/jira/browse/IMPALA-3021
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.5.0
> Reporter: casey
> Priority: Minor
>
> A TPC-DS query was hung on the physical stress cluster (it was the only query running though) and I happened to look at the summary and noticed that the #Rows in the exchanges were larger than the #Rows in the scans.
> Ex: the "29:EXCHANGE" near the bottom.
> {noformat}
> Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
> ---------------------------------------------------------------------------------------------------------------------------------
> 27:SORT 0 0.000ns 0.000ns 0 1 0 -1.00 B
> 26:NESTED LOOP JOIN 0 0.000ns 0.000ns 0 1 0 -1.00 B CROSS JOIN, BROADCAST
> |--45:EXCHANGE 0 0.000ns 0.000ns 0 1 0 -1.00 B UNPARTITIONED
> | 44:AGGREGATE 1 188.164ms 188.164ms 0 1 4.00 KB -1.00 B FINALIZE
> | 43:EXCHANGE 1 0.000ns 0.000ns 0 1 0 -1.00 B UNPARTITIONED
> | 25:AGGREGATE 7 147.499ms 181.624ms 0 1 4.00 KB 10.00 MB
> | 24:HASH JOIN 7 52m19s 1h1m 0 431.88K 2.06 MB 5.25 MB INNER JOIN, BROADCAST
> | |--42:EXCHANGE 7 197.206us 267.914us 9.22K 416.67K 0 0 BROADCAST
> | | 18:SCAN HDFS 0 0.000ns 0.000ns 0 416.67K 0 32.00 MB tpcds_300_decimal_parquet.c...
> | 23:HASH JOIN 7 52m19s 1h1m 0 2.27M 2.07 MB 11.99 MB INNER JOIN, PARTITIONED
> | |--41:EXCHANGE 7 134.263us 164.507us 32.05K 5.00M 0 0 HASH(c_customer_sk)
> | | 17:SCAN HDFS 0 0.000ns 0.000ns 0 5.00M 0 96.00 MB tpcds_300_decimal_parquet.c...
> | 40:EXCHANGE 7 0.000ns 0.000ns 0 2.27M 0 0 HASH(ss_customer_sk)
> | 22:HASH JOIN 7 123.264ms 158.649ms 0 2.27M 2.05 MB 2.59 KB INNER JOIN, BROADCAST
> | |--39:EXCHANGE 7 17.200us 24.923us 456 201 0 0 BROADCAST
> | | 15:SCAN HDFS 1 3.241ms 3.241ms 456 201 152.00 KB 32.00 MB tpcds_300_decimal_parquet.s...
> | 21:HASH JOIN 7 52m20s 1h1m 0 4.54M 2.06 MB 770.57 KB INNER JOIN, BROADCAST
> | |--38:EXCHANGE 7 22.364us 30.269us 1.02K 24.00K 0 0 BROADCAST
> | | 19:SCAN HDFS 0 0.000ns 0.000ns 0 24.00K 0 80.00 MB tpcds_300_decimal_parquet.item
> | 20:HASH JOIN 7 2.853ms 3.979ms 0 49.92M 8.00 KB 1.86 KB INNER JOIN, BROADCAST
> | |--37:EXCHANGE 7 0.000ns 0.000ns 0 108 0 0 BROADCAST
> | | 16:SCAN HDFS 0 0.000ns 0.000ns 0 108 0 48.00 MB tpcds_300_decimal_parquet.d...
> | 14:SCAN HDFS 7 1.072ms 1.420ms 0 864.00M 12.00 KB 128.00 MB tpcds_300_decimal_parquet.s...
> 36:AGGREGATE 0 0.000ns 0.000ns 0 1 0 -1.00 B FINALIZE
> 35:EXCHANGE 0 0.000ns 0.000ns 0 1 0 -1.00 B UNPARTITIONED
> 13:AGGREGATE 7 143.098ms 177.610ms 0 1 4.00 KB 10.00 MB
> 12:HASH JOIN 7 52m20s 1h1m 0 349.57K 2.07 MB 5.25 MB INNER JOIN, BROADCAST
> |--34:EXCHANGE 7 159.458us 253.220us 9.22K 416.67K 0 0 BROADCAST
> | 05:SCAN HDFS 0 0.000ns 0.000ns 0 416.67K 0 32.00 MB tpcds_300_decimal_parquet.c...
> 11:HASH JOIN 7 52m20s 1h1m 0 1.84M 2.08 MB 11.99 MB INNER JOIN, PARTITIONED
> |--33:EXCHANGE 7 107.544us 164.232us 32.05K 5.00M 0 0 HASH(c_customer_sk)
> | 04:SCAN HDFS 0 0.000ns 0.000ns 0 5.00M 0 96.00 MB tpcds_300_decimal_parquet.c...
> 32:EXCHANGE 7 0.000ns 0.000ns 0 1.84M 0 0 HASH(ss_customer_sk)
> 10:HASH JOIN 7 130.205ms 163.486ms 0 1.84M 2.08 MB 68.59 KB INNER JOIN, BROADCAST
> |--31:EXCHANGE 7 19.664us 31.319us 664 1.08K 0 0 BROADCAST
> | 02:SCAN HDFS 1 5.303ms 5.303ms 664 1.08K 257.47 KB 64.00 MB tpcds_300_decimal_parquet.p...
> 09:HASH JOIN 7 3.546ms 4.910ms 0 2.27M 2.05 MB 2.59 KB INNER JOIN, BROADCAST
> |--30:EXCHANGE 7 17.286us 26.280us 456 201 0 0 BROADCAST
> | 01:SCAN HDFS 1 2.815ms 2.815ms 456 201 152.00 KB 32.00 MB tpcds_300_decimal_parquet.s...
> 08:HASH JOIN 7 52m20s 1h1m 0 4.54M 2.06 MB 770.57 KB INNER JOIN, BROADCAST
> |--29:EXCHANGE 7 25.826us 31.966us 1.02K 24.00K 0 0 BROADCAST
> | 06:SCAN HDFS 0 0.000ns 0.000ns 0 24.00K 0 80.00 MB tpcds_300_decimal_parquet.item
> 07:HASH JOIN 7 2.943ms 4.007ms 0 49.92M 8.00 KB 1.86 KB INNER JOIN, BROADCAST
> |--28:EXCHANGE 7 0.000ns 0.000ns 0 108 0 0 BROADCAST
> | 03:SCAN HDFS 0 0.000ns 0.000ns 0 108 0 48.00 MB tpcds_300_decimal_parquet.d...
> 00:SCAN HDFS 7 1.073ms 1.501ms 0 864.00M 28.00 KB 160.00 MB tpcds_300_decimal_parquet.s...
> {noformat}
> I don't think this is supposed to happen?
> Query:
> {noformat}
> select promotions,total,cast(promotions as decimal(15,4))/cast(total as decimal(15,4))*100
> from
> (select sum(ss_ext_sales_price) promotions
> from store_sales
> ,store
> ,promotion
> ,date_dim
> ,customer
> ,customer_address
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_store_sk = s_store_sk
> and ss_promo_sk = p_promo_sk
> and ss_customer_sk= c_customer_sk
> and ca_address_sk = c_current_addr_sk
> and ss_item_sk = i_item_sk
> and ca_gmt_offset = -5
> and i_category = 'Books'
> and (p_channel_dmail = 'Y' or p_channel_email = 'Y' or p_channel_tv = 'Y')
> and s_gmt_offset = -5
> and d_year = 2000
> and d_moy = 11) promotional_sales,
> (select sum(ss_ext_sales_price) total
> from store_sales
> ,store
> ,date_dim
> ,customer
> ,customer_address
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_store_sk = s_store_sk
> and ss_customer_sk= c_customer_sk
> and ca_address_sk = c_current_addr_sk
> and ss_item_sk = i_item_sk
> and ca_gmt_offset = -5
> and i_category = 'Books'
> and s_gmt_offset = -5
> and d_year = 2000
> and d_moy = 11) all_sales
> order by promotions, total
> {noformat}
> Version:
> impalad version 2.5.0-cdh5-INTERNAL RELEASE (build 53cc3a3d3ac00e0e6b8db273dc7558cec89279ad)
> Built on Wed, 17 Feb 2016 09:09:14 PST
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org