You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by se...@apache.org on 2018/07/18 18:52:06 UTC
[10/48] hive git commit: HIVE-20090 : Extend creation of semijoin
reduction filters to be able to discover new opportunities (Jesus Camacho
Rodriguez via Deepak Jaiswal)
http://git-wip-us.apache.org/repos/asf/hive/blob/ab9e954d/ql/src/test/results/clientpositive/perf/tez/query59.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query59.q.out b/ql/src/test/results/clientpositive/perf/tez/query59.q.out
index 6b2dcc3..29cf136 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query59.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query59.q.out
@@ -109,51 +109,51 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_229]
- Limit [LIM_228] (rows=100 width=88)
+ File Output Operator [FS_235]
+ Limit [LIM_234] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_227] (rows=421657640 width=88)
+ Select Operator [SEL_233] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
<-Reducer 6 [SIMPLE_EDGE]
SHUFFLE [RS_60]
Select Operator [SEL_59] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
- Merge Join Operator [MERGEJOIN_180] (rows=421657640 width=88)
+ Merge Join Operator [MERGEJOIN_186] (rows=421657640 width=88)
Conds:RS_56._col2, _col1=RS_57._col1, (_col0 - 52)(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col12","_col13","_col14","_col15","_col16","_col17"]
<-Reducer 13 [SIMPLE_EDGE]
SHUFFLE [RS_57]
PartitionCols:_col1, (_col0 - 52)
Select Operator [SEL_55] (rows=383325119 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- Merge Join Operator [MERGEJOIN_179] (rows=383325119 width=88)
- Conds:RS_52._col1=RS_216._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6","_col7","_col11"]
+ Merge Join Operator [MERGEJOIN_185] (rows=383325119 width=88)
+ Conds:RS_52._col1=RS_222._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6","_col7","_col11"]
<-Map 19 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_216]
+ PARTITION_ONLY_SHUFFLE [RS_222]
PartitionCols:_col0
- Select Operator [SEL_215] (rows=1704 width=1910)
+ Select Operator [SEL_221] (rows=1704 width=1910)
Output:["_col0","_col1"]
- Filter Operator [FIL_214] (rows=1704 width=1910)
+ Filter Operator [FIL_220] (rows=1704 width=1910)
predicate:(s_store_id is not null and s_store_sk is not null)
TableScan [TS_46] (rows=1704 width=1910)
default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id"]
<-Reducer 12 [SIMPLE_EDGE]
SHUFFLE [RS_52]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_178] (rows=348477374 width=88)
- Conds:RS_226._col0=RS_211._col1(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
+ Merge Join Operator [MERGEJOIN_184] (rows=348477374 width=88)
+ Conds:RS_232._col0=RS_217._col1(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
<-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_211]
+ SHUFFLE [RS_217]
PartitionCols:_col1
- Select Operator [SEL_209] (rows=8116 width=1119)
+ Select Operator [SEL_215] (rows=8116 width=1119)
Output:["_col1"]
- Filter Operator [FIL_207] (rows=8116 width=1119)
+ Filter Operator [FIL_213] (rows=8116 width=1119)
predicate:(d_month_seq BETWEEN 1197 AND 1208 and d_week_seq is not null)
TableScan [TS_15] (rows=73049 width=1119)
default@date_dim,d,Tbl:COMPLETE,Col:NONE,Output:["d_month_seq","d_week_seq"]
<-Reducer 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_226]
+ SHUFFLE [RS_232]
PartitionCols:_col0
- Group By Operator [GBY_225] (rows=316797606 width=88)
+ Group By Operator [GBY_231] (rows=316797606 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)"],keys:KEY._col0, KEY._col1
<-Reducer 10 [SIMPLE_EDGE]
SHUFFLE [RS_40]
@@ -162,81 +162,81 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"],aggregations:["sum(_col2)","sum(_col3)","sum(_col5)","sum(_col6)","sum(_col7)","sum(_col8)"],keys:_col0, _col1
Select Operator [SEL_37] (rows=633595212 width=88)
Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col7","_col8"]
- Merge Join Operator [MERGEJOIN_177] (rows=633595212 width=88)
- Conds:RS_224._col0=RS_185._col0(Inner),Output:["_col1","_col2","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_183] (rows=633595212 width=88)
+ Conds:RS_230._col0=RS_191._col0(Inner),Output:["_col1","_col2","_col4","_col5"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_185]
+ PARTITION_ONLY_SHUFFLE [RS_191]
PartitionCols:_col0
- Select Operator [SEL_182] (rows=73049 width=1119)
+ Select Operator [SEL_188] (rows=73049 width=1119)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_181] (rows=73049 width=1119)
+ Filter Operator [FIL_187] (rows=73049 width=1119)
predicate:(d_date_sk is not null and d_week_seq is not null)
TableScan [TS_3] (rows=73049 width=1119)
default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_week_seq","d_day_name"]
<-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_224]
+ SHUFFLE [RS_230]
PartitionCols:_col0
- Select Operator [SEL_223] (rows=575995635 width=88)
+ Select Operator [SEL_229] (rows=575995635 width=88)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_222] (rows=575995635 width=88)
+ Filter Operator [FIL_228] (rows=575995635 width=88)
predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_35_date_dim_d_date_sk_min) AND DynamicValue(RS_35_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_35_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_53_store_s_store_sk_min) AND DynamicValue(RS_53_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_53_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
TableScan [TS_28] (rows=575995635 width=88)
default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_store_sk","ss_sales_price"]
<-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_213]
- Group By Operator [GBY_212] (rows=1 width=12)
+ BROADCAST [RS_219]
+ Group By Operator [GBY_218] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_190]
- Group By Operator [GBY_188] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_196]
+ Group By Operator [GBY_194] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_186] (rows=73049 width=1119)
+ Select Operator [SEL_192] (rows=73049 width=1119)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_182]
+ Please refer to the previous Select Operator [SEL_188]
<-Reducer 20 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_221]
- Group By Operator [GBY_220] (rows=1 width=12)
+ BROADCAST [RS_227]
+ Group By Operator [GBY_226] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 19 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_219]
- Group By Operator [GBY_218] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_225]
+ Group By Operator [GBY_224] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_217] (rows=1704 width=1910)
+ Select Operator [SEL_223] (rows=1704 width=1910)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_215]
+ Please refer to the previous Select Operator [SEL_221]
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_56]
PartitionCols:_col2, _col1
Select Operator [SEL_27] (rows=383325119 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
- Merge Join Operator [MERGEJOIN_176] (rows=383325119 width=88)
- Conds:RS_24._col1=RS_195._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col12","_col13"]
+ Merge Join Operator [MERGEJOIN_182] (rows=383325119 width=88)
+ Conds:RS_24._col1=RS_201._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col12","_col13"]
<-Map 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_195]
+ SHUFFLE [RS_201]
PartitionCols:_col0
- Select Operator [SEL_194] (rows=1704 width=1910)
+ Select Operator [SEL_200] (rows=1704 width=1910)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_193] (rows=1704 width=1910)
+ Filter Operator [FIL_199] (rows=1704 width=1910)
predicate:(s_store_id is not null and s_store_sk is not null)
TableScan [TS_18] (rows=1704 width=1910)
default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id","s_store_name"]
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_24]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_175] (rows=348477374 width=88)
- Conds:RS_205._col0=RS_210._col1(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
+ Merge Join Operator [MERGEJOIN_181] (rows=348477374 width=88)
+ Conds:RS_211._col0=RS_216._col1(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
<-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_210]
+ SHUFFLE [RS_216]
PartitionCols:_col1
- Select Operator [SEL_208] (rows=8116 width=1119)
+ Select Operator [SEL_214] (rows=8116 width=1119)
Output:["_col1"]
- Filter Operator [FIL_206] (rows=8116 width=1119)
+ Filter Operator [FIL_212] (rows=8116 width=1119)
predicate:(d_month_seq BETWEEN 1185 AND 1196 and d_week_seq is not null)
Please refer to the previous TableScan [TS_15]
<-Reducer 3 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_205]
+ SHUFFLE [RS_211]
PartitionCols:_col0
- Group By Operator [GBY_204] (rows=316797606 width=88)
+ Group By Operator [GBY_210] (rows=316797606 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)"],keys:KEY._col0, KEY._col1
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_12]
@@ -245,41 +245,41 @@ Stage-0
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)","sum(_col5)","sum(_col6)","sum(_col7)","sum(_col8)"],keys:_col0, _col1
Select Operator [SEL_9] (rows=633595212 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- Merge Join Operator [MERGEJOIN_174] (rows=633595212 width=88)
- Conds:RS_203._col0=RS_183._col0(Inner),Output:["_col1","_col2","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_180] (rows=633595212 width=88)
+ Conds:RS_209._col0=RS_189._col0(Inner),Output:["_col1","_col2","_col4","_col5"]
<-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_183]
+ PARTITION_ONLY_SHUFFLE [RS_189]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_182]
+ Please refer to the previous Select Operator [SEL_188]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_203]
+ SHUFFLE [RS_209]
PartitionCols:_col0
- Select Operator [SEL_202] (rows=575995635 width=88)
+ Select Operator [SEL_208] (rows=575995635 width=88)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_201] (rows=575995635 width=88)
+ Filter Operator [FIL_207] (rows=575995635 width=88)
predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_7_date_dim_d_date_sk_min) AND DynamicValue(RS_7_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_7_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_25_store_s_store_sk_min) AND DynamicValue(RS_25_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_25_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
TableScan [TS_0] (rows=575995635 width=88)
default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_store_sk","ss_sales_price"]
<-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_200]
- Group By Operator [GBY_199] (rows=1 width=12)
+ BROADCAST [RS_206]
+ Group By Operator [GBY_205] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 16 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_198]
- Group By Operator [GBY_197] (rows=1 width=12)
+ SHUFFLE [RS_204]
+ Group By Operator [GBY_203] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_196] (rows=1704 width=1910)
+ Select Operator [SEL_202] (rows=1704 width=1910)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_194]
+ Please refer to the previous Select Operator [SEL_200]
<-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_192]
- Group By Operator [GBY_191] (rows=1 width=12)
+ BROADCAST [RS_198]
+ Group By Operator [GBY_197] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_189]
- Group By Operator [GBY_187] (rows=1 width=12)
+ PARTITION_ONLY_SHUFFLE [RS_195]
+ Group By Operator [GBY_193] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_184] (rows=73049 width=1119)
+ Select Operator [SEL_190] (rows=73049 width=1119)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_182]
+ Please refer to the previous Select Operator [SEL_188]