You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by jc...@apache.org on 2018/07/19 20:59:39 UTC
[01/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Repository: hive
Updated Branches:
refs/heads/master e867d1c69 -> 851c8aba8
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query82.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query82.q.out b/ql/src/test/results/clientpositive/perf/tez/query82.q.out
index f1765e7..bb5a9e9 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query82.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query82.q.out
@@ -46,83 +46,85 @@ Stage-0
limit:100
Stage-1
Reducer 4 vectorized
- File Output Operator [FS_96]
- Limit [LIM_95] (rows=100 width=88)
+ File Output Operator [FS_97]
+ Limit [LIM_96] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_94] (rows=633595212 width=88)
+ Select Operator [SEL_95] (rows=633595212 width=88)
Output:["_col0","_col1","_col2"]
<-Reducer 3 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_93]
- Group By Operator [GBY_92] (rows=633595212 width=88)
+ SHUFFLE [RS_94]
+ Group By Operator [GBY_93] (rows=633595212 width=88)
Output:["_col0","_col1","_col2"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_22]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_21] (rows=1267190424 width=88)
Output:["_col0","_col1","_col2"],keys:_col2, _col3, _col4
- Merge Join Operator [MERGEJOIN_72] (rows=1267190424 width=88)
- Conds:RS_91._col0=RS_75._col0(Inner),RS_75._col0=RS_18._col1(Inner),Output:["_col2","_col3","_col4"]
- <-Map 5 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_75]
- PartitionCols:_col0
- Select Operator [SEL_74] (rows=25666 width=1436)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_73] (rows=25666 width=1436)
- predicate:((i_manufact_id) IN (437, 129, 727, 663) and i_current_price BETWEEN 30 AND 60 and i_item_sk is not null)
- TableScan [TS_3] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc","i_current_price","i_manufact_id"]
- <-Reducer 8 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_18]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_71] (rows=4593600 width=15)
- Conds:RS_83._col0=RS_86._col0(Inner),Output:["_col1"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_86]
- PartitionCols:_col0
- Select Operator [SEL_85] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_84] (rows=8116 width=1119)
- predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'2002-05-30 00:00:00' AND TIMESTAMP'2002-07-29 00:00:00' and d_date_sk is not null)
- TableScan [TS_9] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
- <-Map 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_83]
- PartitionCols:_col0
- Select Operator [SEL_82] (rows=4176000 width=15)
- Output:["_col0","_col1"]
- Filter Operator [FIL_81] (rows=4176000 width=15)
- predicate:(inv_date_sk is not null and inv_item_sk is not null and inv_quantity_on_hand BETWEEN 100 AND 500)
- TableScan [TS_6] (rows=37584000 width=15)
- default@inventory,inventory,Tbl:COMPLETE,Col:NONE,Output:["inv_date_sk","inv_item_sk","inv_quantity_on_hand"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_91]
- PartitionCols:_col0
- Select Operator [SEL_90] (rows=575995635 width=88)
- Output:["_col0"]
- Filter Operator [FIL_89] (rows=575995635 width=88)
- predicate:((ss_item_sk BETWEEN DynamicValue(RS_17_item_i_item_sk_min) AND DynamicValue(RS_17_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_17_item_i_item_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_18_inventory_inv_item_sk_min) AND DynamicValue(RS_18_inventory_inv_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_18_inventory_inv_item_sk_bloom_filter))) and ss_item_sk is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_item_sk"]
- <-Reducer 6 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_80]
- Group By Operator [GBY_79] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 5 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_78]
- Group By Operator [GBY_77] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_76] (rows=25666 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_74]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_88]
- Group By Operator [GBY_87] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=4593600)"]
- <-Reducer 8 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_50]
- Group By Operator [GBY_49] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=4593600)"]
- Select Operator [SEL_48] (rows=4593600 width=15)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_71]
+ Top N Key Operator [TNK_43] (rows=1267190424 width=88)
+ keys:_col2, _col3, _col4,sort order:+++,top n:100
+ Merge Join Operator [MERGEJOIN_73] (rows=1267190424 width=88)
+ Conds:RS_92._col0=RS_76._col0(Inner),RS_76._col0=RS_18._col1(Inner),Output:["_col2","_col3","_col4"]
+ <-Map 5 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_76]
+ PartitionCols:_col0
+ Select Operator [SEL_75] (rows=25666 width=1436)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_74] (rows=25666 width=1436)
+ predicate:((i_manufact_id) IN (437, 129, 727, 663) and i_current_price BETWEEN 30 AND 60 and i_item_sk is not null)
+ TableScan [TS_3] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc","i_current_price","i_manufact_id"]
+ <-Reducer 8 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_18]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_72] (rows=4593600 width=15)
+ Conds:RS_84._col0=RS_87._col0(Inner),Output:["_col1"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_87]
+ PartitionCols:_col0
+ Select Operator [SEL_86] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_85] (rows=8116 width=1119)
+ predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'2002-05-30 00:00:00' AND TIMESTAMP'2002-07-29 00:00:00' and d_date_sk is not null)
+ TableScan [TS_9] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
+ <-Map 7 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_84]
+ PartitionCols:_col0
+ Select Operator [SEL_83] (rows=4176000 width=15)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_82] (rows=4176000 width=15)
+ predicate:(inv_date_sk is not null and inv_item_sk is not null and inv_quantity_on_hand BETWEEN 100 AND 500)
+ TableScan [TS_6] (rows=37584000 width=15)
+ default@inventory,inventory,Tbl:COMPLETE,Col:NONE,Output:["inv_date_sk","inv_item_sk","inv_quantity_on_hand"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_92]
+ PartitionCols:_col0
+ Select Operator [SEL_91] (rows=575995635 width=88)
+ Output:["_col0"]
+ Filter Operator [FIL_90] (rows=575995635 width=88)
+ predicate:((ss_item_sk BETWEEN DynamicValue(RS_17_item_i_item_sk_min) AND DynamicValue(RS_17_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_17_item_i_item_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_18_inventory_inv_item_sk_min) AND DynamicValue(RS_18_inventory_inv_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_18_inventory_inv_item_sk_bloom_filter))) and ss_item_sk is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_item_sk"]
+ <-Reducer 6 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_81]
+ Group By Operator [GBY_80] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 5 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_79]
+ Group By Operator [GBY_78] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_77] (rows=25666 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_75]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_89]
+ Group By Operator [GBY_88] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=4593600)"]
+ <-Reducer 8 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_51]
+ Group By Operator [GBY_50] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=4593600)"]
+ Select Operator [SEL_49] (rows=4593600 width=15)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_72]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query99.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query99.q.out b/ql/src/test/results/clientpositive/perf/tez/query99.q.out
index b0f6a45..456fd8c 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query99.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query99.q.out
@@ -86,128 +86,130 @@ Stage-0
limit:-1
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_141]
- Limit [LIM_140] (rows=100 width=135)
+ File Output Operator [FS_142]
+ Limit [LIM_141] (rows=100 width=135)
Number of rows:100
- Select Operator [SEL_139] (rows=210822976 width=135)
+ Select Operator [SEL_140] (rows=210822976 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_138]
- Select Operator [SEL_137] (rows=210822976 width=135)
+ SHUFFLE [RS_139]
+ Select Operator [SEL_138] (rows=210822976 width=135)
Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- Group By Operator [GBY_136] (rows=210822976 width=135)
+ Group By Operator [GBY_137] (rows=210822976 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_30]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_29] (rows=421645953 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"],aggregations:["sum(_col3)","sum(_col4)","sum(_col5)","sum(_col6)","sum(_col7)"],keys:_col0, _col1, _col2
- Select Operator [SEL_27] (rows=421645953 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- Merge Join Operator [MERGEJOIN_100] (rows=421645953 width=135)
- Conds:RS_24._col3=RS_127._col0(Inner),Output:["_col0","_col1","_col8","_col10","_col12"]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_127]
- PartitionCols:_col0
- Select Operator [SEL_126] (rows=1 width=0)
- Output:["_col0","_col1"]
- Filter Operator [FIL_125] (rows=1 width=0)
- predicate:sm_ship_mode_sk is not null
- TableScan [TS_12] (rows=1 width=0)
- default@ship_mode,ship_mode,Tbl:PARTIAL,Col:NONE,Output:["sm_ship_mode_sk","sm_type"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_24]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_99] (rows=383314495 width=135)
- Conds:RS_21._col4=RS_119._col0(Inner),Output:["_col0","_col1","_col3","_col8","_col10"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_119]
- PartitionCols:_col0
- Select Operator [SEL_118] (rows=27 width=1029)
- Output:["_col0","_col1"]
- Filter Operator [FIL_117] (rows=27 width=1029)
- predicate:w_warehouse_sk is not null
- TableScan [TS_9] (rows=27 width=1029)
- default@warehouse,warehouse,Tbl:COMPLETE,Col:NONE,Output:["w_warehouse_sk","w_warehouse_name"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col4
- Merge Join Operator [MERGEJOIN_98] (rows=348467716 width=135)
- Conds:RS_18._col2=RS_111._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col8"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_111]
- PartitionCols:_col0
- Select Operator [SEL_110] (rows=60 width=2045)
- Output:["_col0","_col1"]
- Filter Operator [FIL_109] (rows=60 width=2045)
- predicate:cc_call_center_sk is not null
- TableScan [TS_6] (rows=60 width=2045)
- default@call_center,call_center,Tbl:COMPLETE,Col:NONE,Output:["cc_call_center_sk","cc_name"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_97] (rows=316788826 width=135)
- Conds:RS_135._col1=RS_103._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_103]
- PartitionCols:_col0
- Select Operator [SEL_102] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_101] (rows=8116 width=1119)
- predicate:(d_date_sk is not null and d_month_seq BETWEEN 1212 AND 1223)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_month_seq"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_135]
- PartitionCols:_col1
- Select Operator [SEL_134] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_133] (rows=287989836 width=135)
- predicate:((cs_call_center_sk BETWEEN DynamicValue(RS_19_call_center_cc_call_center_sk_min) AND DynamicValue(RS_19_call_center_cc_call_center_sk_max) and in_bloom_filter(cs_call_center_sk, DynamicValue(RS_19_call_center_cc_call_center_sk_bloom_filter))) and (cs_ship_date_sk BETWEEN DynamicValue(RS_16_date_dim_d_date_sk_min) AND DynamicValue(RS_16_date_dim_d_date_sk_max) and in_bloom_filter(cs_ship_date_sk, DynamicValue(RS_16_date_dim_d_date_sk_bloom_filter))) and (cs_ship_mode_sk BETWEEN DynamicValue(RS_25_ship_mode_sm_ship_mode_sk_min) AND DynamicValue(RS_25_ship_mode_sm_ship_mode_sk_max) and in_bloom_filter(cs_ship_mode_sk, DynamicValue(RS_25_ship_mode_sm_ship_mode_sk_bloom_filter))) and (cs_warehouse_sk BETWEEN DynamicValue(RS_22_warehouse_w_warehouse_sk_min) AND DynamicValue(RS_22_warehouse_w_warehouse_sk_max) and in_bloom_filter(cs_warehouse_sk, DynamicValue(RS_22_warehouse_w_warehouse_sk_bloom_filter))) and cs_call_center_sk is not n
ull and cs_ship_date_sk is not null and cs_ship_mode_sk is not null and cs_warehouse_sk is not null)
- TableScan [TS_0] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_ship_date_sk","cs_call_center_sk","cs_ship_mode_sk","cs_warehouse_sk"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_116]
- Group By Operator [GBY_115] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_114]
- Group By Operator [GBY_113] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_112] (rows=60 width=2045)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_110]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_124]
- Group By Operator [GBY_123] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_122]
- Group By Operator [GBY_121] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_120] (rows=27 width=1029)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_118]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_132]
- Group By Operator [GBY_131] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 14 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_130]
- Group By Operator [GBY_129] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_128] (rows=1 width=0)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_126]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_108]
- Group By Operator [GBY_107] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_106]
- Group By Operator [GBY_105] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_104] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_102]
+ Top N Key Operator [TNK_57] (rows=421645953 width=135)
+ keys:_col0, _col1, _col2,sort order:+++,top n:100
+ Select Operator [SEL_27] (rows=421645953 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
+ Merge Join Operator [MERGEJOIN_101] (rows=421645953 width=135)
+ Conds:RS_24._col3=RS_128._col0(Inner),Output:["_col0","_col1","_col8","_col10","_col12"]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_128]
+ PartitionCols:_col0
+ Select Operator [SEL_127] (rows=1 width=0)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_126] (rows=1 width=0)
+ predicate:sm_ship_mode_sk is not null
+ TableScan [TS_12] (rows=1 width=0)
+ default@ship_mode,ship_mode,Tbl:PARTIAL,Col:NONE,Output:["sm_ship_mode_sk","sm_type"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_24]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_100] (rows=383314495 width=135)
+ Conds:RS_21._col4=RS_120._col0(Inner),Output:["_col0","_col1","_col3","_col8","_col10"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_120]
+ PartitionCols:_col0
+ Select Operator [SEL_119] (rows=27 width=1029)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_118] (rows=27 width=1029)
+ predicate:w_warehouse_sk is not null
+ TableScan [TS_9] (rows=27 width=1029)
+ default@warehouse,warehouse,Tbl:COMPLETE,Col:NONE,Output:["w_warehouse_sk","w_warehouse_name"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col4
+ Merge Join Operator [MERGEJOIN_99] (rows=348467716 width=135)
+ Conds:RS_18._col2=RS_112._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col8"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_112]
+ PartitionCols:_col0
+ Select Operator [SEL_111] (rows=60 width=2045)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_110] (rows=60 width=2045)
+ predicate:cc_call_center_sk is not null
+ TableScan [TS_6] (rows=60 width=2045)
+ default@call_center,call_center,Tbl:COMPLETE,Col:NONE,Output:["cc_call_center_sk","cc_name"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_98] (rows=316788826 width=135)
+ Conds:RS_136._col1=RS_104._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_104]
+ PartitionCols:_col0
+ Select Operator [SEL_103] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_102] (rows=8116 width=1119)
+ predicate:(d_date_sk is not null and d_month_seq BETWEEN 1212 AND 1223)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_month_seq"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_136]
+ PartitionCols:_col1
+ Select Operator [SEL_135] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_134] (rows=287989836 width=135)
+ predicate:((cs_call_center_sk BETWEEN DynamicValue(RS_19_call_center_cc_call_center_sk_min) AND DynamicValue(RS_19_call_center_cc_call_center_sk_max) and in_bloom_filter(cs_call_center_sk, DynamicValue(RS_19_call_center_cc_call_center_sk_bloom_filter))) and (cs_ship_date_sk BETWEEN DynamicValue(RS_16_date_dim_d_date_sk_min) AND DynamicValue(RS_16_date_dim_d_date_sk_max) and in_bloom_filter(cs_ship_date_sk, DynamicValue(RS_16_date_dim_d_date_sk_bloom_filter))) and (cs_ship_mode_sk BETWEEN DynamicValue(RS_25_ship_mode_sm_ship_mode_sk_min) AND DynamicValue(RS_25_ship_mode_sm_ship_mode_sk_max) and in_bloom_filter(cs_ship_mode_sk, DynamicValue(RS_25_ship_mode_sm_ship_mode_sk_bloom_filter))) and (cs_warehouse_sk BETWEEN DynamicValue(RS_22_warehouse_w_warehouse_sk_min) AND DynamicValue(RS_22_warehouse_w_warehouse_sk_max) and in_bloom_filter(cs_warehouse_sk, DynamicValue(RS_22_warehouse_w_warehouse_sk_bloom_filter))) and cs_call_center_sk is not
null and cs_ship_date_sk is not null and cs_ship_mode_sk is not null and cs_warehouse_sk is not null)
+ TableScan [TS_0] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_ship_date_sk","cs_call_center_sk","cs_ship_mode_sk","cs_warehouse_sk"]
+ <-Reducer 11 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_117]
+ Group By Operator [GBY_116] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_115]
+ Group By Operator [GBY_114] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_113] (rows=60 width=2045)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_111]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_125]
+ Group By Operator [GBY_124] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_123]
+ Group By Operator [GBY_122] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_121] (rows=27 width=1029)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_119]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_133]
+ Group By Operator [GBY_132] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 14 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_131]
+ Group By Operator [GBY_130] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_129] (rows=1 width=0)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_127]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_109]
+ Group By Operator [GBY_108] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_107]
+ Group By Operator [GBY_106] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_105] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_103]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/tez/topnkey.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/topnkey.q.out b/ql/src/test/results/clientpositive/tez/topnkey.q.out
new file mode 100644
index 0000000..66b9191
--- /dev/null
+++ b/ql/src/test/results/clientpositive/tez/topnkey.q.out
@@ -0,0 +1,162 @@
+PREHOOK: query: EXPLAIN
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: EXPLAIN
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:5
+ Stage-1
+ Reducer 3
+ File Output Operator [FS_10]
+ Limit [LIM_9] (rows=5 width=95)
+ Number of rows:5
+ Select Operator [SEL_8] (rows=250 width=95)
+ Output:["_col0","_col1"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_7]
+ Group By Operator [GBY_5] (rows=250 width=95)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_4]
+ PartitionCols:_col0
+ Group By Operator [GBY_3] (rows=250 width=95)
+ Output:["_col0","_col1"],aggregations:["sum(_col1)"],keys:_col0
+ Top N Key Operator [TNK_11] (rows=500 width=178)
+ keys:_col0,sort order:+,top n:5
+ Select Operator [SEL_1] (rows=500 width=178)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+
+PREHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: hdfs://### HDFS PATH ###
+POSTHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: hdfs://### HDFS PATH ###
+0 0
+10 10
+100 200
+103 206
+104 208
+PREHOOK: query: EXPLAIN
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: EXPLAIN
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:5
+ Stage-1
+ Reducer 3
+ File Output Operator [FS_9]
+ Limit [LIM_8] (rows=5 width=87)
+ Number of rows:5
+ Select Operator [SEL_7] (rows=250 width=87)
+ Output:["_col0"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_6]
+ Group By Operator [GBY_4] (rows=250 width=87)
+ Output:["_col0"],keys:KEY._col0
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_3]
+ PartitionCols:_col0
+ Group By Operator [GBY_2] (rows=250 width=87)
+ Output:["_col0"],keys:key
+ Top N Key Operator [TNK_10] (rows=500 width=87)
+ keys:key,sort order:+,top n:5
+ Select Operator [SEL_1] (rows=500 width=87)
+ Output:["key"]
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+
+PREHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: hdfs://### HDFS PATH ###
+POSTHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: hdfs://### HDFS PATH ###
+0
+10
+100
+103
+104
+PREHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:5
+ Stage-1
+ Reducer 3
+ File Output Operator [FS_13]
+ Limit [LIM_12] (rows=5 width=178)
+ Number of rows:5
+ Select Operator [SEL_11] (rows=791 width=178)
+ Output:["_col0","_col1"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_10]
+ Select Operator [SEL_9] (rows=791 width=178)
+ Output:["_col0","_col1"]
+ Merge Join Operator [MERGEJOIN_28] (rows=791 width=178)
+ Conds:RS_6._col0=RS_7._col0(Inner),Output:["_col0","_col2"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=500 width=87)
+ Output:["_col0"]
+ Filter Operator [FIL_16] (rows=500 width=87)
+ predicate:key is not null
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src1,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ <-Map 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=500 width=178)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_17] (rows=500 width=178)
+ predicate:key is not null
+ TableScan [TS_3] (rows=500 width=178)
+ default@src,src2,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+
+PREHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: hdfs://### HDFS PATH ###
+POSTHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: hdfs://### HDFS PATH ###
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/tez/vector_topnkey.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/vector_topnkey.q.out b/ql/src/test/results/clientpositive/tez/vector_topnkey.q.out
new file mode 100644
index 0000000..d6f7cc2
--- /dev/null
+++ b/ql/src/test/results/clientpositive/tez/vector_topnkey.q.out
@@ -0,0 +1,162 @@
+PREHOOK: query: explain vectorization detail
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:5
+ Stage-1
+ Reducer 3 vectorized
+ File Output Operator [FS_20]
+ Limit [LIM_19] (rows=5 width=95)
+ Number of rows:5
+ Select Operator [SEL_18] (rows=250 width=95)
+ Output:["_col0","_col1"]
+ <-Reducer 2 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_17]
+ Group By Operator [GBY_16] (rows=250 width=95)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_15]
+ PartitionCols:_col0
+ Group By Operator [GBY_14] (rows=250 width=95)
+ Output:["_col0","_col1"],aggregations:["sum(_col1)"],keys:_col0
+ Top N Key Operator [TNK_13] (rows=500 width=178)
+ keys:_col0,sort order:+,top n:5
+ Select Operator [SEL_12] (rows=500 width=178)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+
+PREHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: hdfs://### HDFS PATH ###
+POSTHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: hdfs://### HDFS PATH ###
+0 0
+10 10
+100 200
+103 206
+104 208
+PREHOOK: query: explain vectorization detail
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:5
+ Stage-1
+ Reducer 3 vectorized
+ File Output Operator [FS_19]
+ Limit [LIM_18] (rows=5 width=87)
+ Number of rows:5
+ Select Operator [SEL_17] (rows=250 width=87)
+ Output:["_col0"]
+ <-Reducer 2 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_16]
+ Group By Operator [GBY_15] (rows=250 width=87)
+ Output:["_col0"],keys:KEY._col0
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_14]
+ PartitionCols:_col0
+ Group By Operator [GBY_13] (rows=250 width=87)
+ Output:["_col0"],keys:key
+ Top N Key Operator [TNK_12] (rows=500 width=87)
+ keys:key,sort order:+,top n:5
+ Select Operator [SEL_11] (rows=500 width=87)
+ Output:["key"]
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+
+PREHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: hdfs://### HDFS PATH ###
+POSTHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: hdfs://### HDFS PATH ###
+0
+10
+100
+103
+104
+PREHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:5
+ Stage-1
+ Reducer 3 vectorized
+ File Output Operator [FS_37]
+ Limit [LIM_36] (rows=5 width=178)
+ Number of rows:5
+ Select Operator [SEL_35] (rows=791 width=178)
+ Output:["_col0","_col1"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_10]
+ Select Operator [SEL_9] (rows=791 width=178)
+ Output:["_col0","_col1"]
+ Merge Join Operator [MERGEJOIN_28] (rows=791 width=178)
+ Conds:RS_31._col0=RS_34._col0(Inner),Output:["_col0","_col2"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_31]
+ PartitionCols:_col0
+ Select Operator [SEL_30] (rows=500 width=87)
+ Output:["_col0"]
+ Filter Operator [FIL_29] (rows=500 width=87)
+ predicate:key is not null
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src1,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ <-Map 4 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_34]
+ PartitionCols:_col0
+ Select Operator [SEL_33] (rows=500 width=178)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_32] (rows=500 width=178)
+ predicate:key is not null
+ TableScan [TS_3] (rows=500 width=178)
+ default@src,src2,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+
+PREHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: hdfs://### HDFS PATH ###
+POSTHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: hdfs://### HDFS PATH ###
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/topnkey.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/topnkey.q.out b/ql/src/test/results/clientpositive/topnkey.q.out
new file mode 100644
index 0000000..31f3a70
--- /dev/null
+++ b/ql/src/test/results/clientpositive/topnkey.q.out
@@ -0,0 +1,301 @@
+PREHOOK: query: EXPLAIN
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: EXPLAIN
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+
+STAGE PLANS:
+ Stage: Stage-1
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string), UDFToInteger(substr(value, 5)) (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: sum(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: sum(VALUE._col0)
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+
+ Stage: Stage-2
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 475 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 475 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0 0
+10 10
+100 200
+103 206
+104 208
+PREHOOK: query: EXPLAIN
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: EXPLAIN
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+
+STAGE PLANS:
+ Stage: Stage-1
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: key
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+
+ Stage: Stage-2
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 435 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 435 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0
+10
+100
+103
+104
+PREHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+PLAN VECTORIZATION:
+ enabled: false
+ enabledConditionsNotMet: [hive.vectorized.execution.enabled IS false]
+
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+
+STAGE PLANS:
+ Stage: Stage-1
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ alias: src1
+ filterExpr: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ TableScan
+ alias: src2
+ filterExpr: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: string)
+ Reduce Operator Tree:
+ Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col2
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: string), _col2 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+
+ Stage: Stage-2
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: string)
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/vector_topnkey.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/vector_topnkey.q.out b/ql/src/test/results/clientpositive/vector_topnkey.q.out
new file mode 100644
index 0000000..ed829e2
--- /dev/null
+++ b/ql/src/test/results/clientpositive/vector_topnkey.q.out
@@ -0,0 +1,480 @@
+PREHOOK: query: explain vectorization detail
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+PLAN VECTORIZATION:
+ enabled: true
+ enabledConditionsMet: [hive.vectorized.execution.enabled IS true]
+
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+
+STAGE PLANS:
+ Stage: Stage-1
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:key:string, 1:value:string, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
+ Select Operator
+ expressions: key (type: string), UDFToInteger(substr(value, 5)) (type: int)
+ outputColumnNames: _col0, _col1
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0, 4]
+ selectExpressions: CastStringToLong(col 3:string)(children: StringSubstrColStart(col 1:string, start 4) -> 3:string) -> 4:int
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: sum(_col1)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 4:int) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkOperator
+ native: false
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ nativeConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
+ Execution mode: vectorized
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: [DECIMAL_64]
+ featureSupportInUse: [DECIMAL_64]
+ inputFileFormats: org.apache.hadoop.mapred.TextInputFormat
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ includeColumns: [0, 1]
+ dataColumns: key:string, value:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: [string, bigint]
+ Reduce Vectorization:
+ enabled: false
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true
+ enableConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: sum(VALUE._col0)
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+
+ Stage: Stage-2
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:_col0:string, 1:_col1:bigint]
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkOperator
+ native: false
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ nativeConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
+ Execution mode: vectorized
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: []
+ featureSupportInUse: []
+ inputFileFormats: org.apache.hadoop.mapred.SequenceFileInputFormat
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ includeColumns: [0, 1]
+ dataColumns: _col0:string, _col1:bigint
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Vectorization:
+ enabled: false
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true
+ enableConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 475 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 475 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0 0
+10 10
+100 200
+103 206
+104 208
+PREHOOK: query: explain vectorization detail
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+PLAN VECTORIZATION:
+ enabled: true
+ enabledConditionsMet: [hive.vectorized.execution.enabled IS true]
+
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+
+STAGE PLANS:
+ Stage: Stage-1
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:key:string, 1:value:string, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: key
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0]
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ Group By Vectorization:
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: []
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkOperator
+ native: false
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ nativeConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ Execution mode: vectorized
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: [DECIMAL_64]
+ featureSupportInUse: [DECIMAL_64]
+ inputFileFormats: org.apache.hadoop.mapred.TextInputFormat
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ includeColumns: [0]
+ dataColumns: key:string, value:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Vectorization:
+ enabled: false
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true
+ enableConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+
+ Stage: Stage-2
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:_col0:string]
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkOperator
+ native: false
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ nativeConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ Execution mode: vectorized
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: []
+ featureSupportInUse: []
+ inputFileFormats: org.apache.hadoop.mapred.SequenceFileInputFormat
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 1
+ includeColumns: [0]
+ dataColumns: _col0:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Vectorization:
+ enabled: false
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true
+ enableConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 435 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 435 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0
+10
+100
+103
+104
+PREHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+PLAN VECTORIZATION:
+ enabled: true
+ enabledConditionsMet: [hive.vectorized.execution.enabled IS true]
+
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+
+STAGE PLANS:
+ Stage: Stage-1
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ alias: src1
+ filterExpr: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ TableScan
+ alias: src2
+ filterExpr: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: string)
+ Map Vectorization:
+ enabled: false
+ enabledConditionsNotMet: Vectorized map work only works with 1 TableScanOperator IS false
+ Reduce Vectorization:
+ enabled: false
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true
+ enableConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Reduce Operator Tree:
+ Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col2
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: string), _col2 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+
+ Stage: Stage-2
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:_col0:string, 1:_col1:string]
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkOperator
+ native: false
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ nativeConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: string)
+ Execution mode: vectorized
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: []
+ featureSupportInUse: []
+ inputFileFormats: org.apache.hadoop.mapred.SequenceFileInputFormat
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ includeColumns: [0, 1]
+ dataColumns: _col0:string, _col1:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Vectorization:
+ enabled: false
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true
+ enableConditionsNotMet: hive.execution.engine mr IN [tez, spark] IS false
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
----------------------------------------------------------------------
diff --git a/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java b/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
index 9393fb8..044fd16 100644
--- a/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
+++ b/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java
@@ -941,6 +941,25 @@ public final class ObjectInspectorUtils {
return 0;
}
+ public static int compare(Object[] o1, ObjectInspector[] oi1, Object[] o2,
+ ObjectInspector[] oi2, boolean[] columnSortOrderIsDesc) {
+ assert (o1.length == oi1.length);
+ assert (o2.length == oi2.length);
+ assert (o1.length == o2.length);
+
+ for (int i = 0; i < o1.length; i++) {
+ int r = compare(o1[i], oi1[i], o2[i], oi2[i]);
+ if (r != 0) {
+ if (columnSortOrderIsDesc[i]) {
+ return r;
+ } else {
+ return -r;
+ }
+ }
+ }
+ return 0;
+ }
+
/**
* Whether comparison is supported for this type.
* Currently all types that references any map are not comparable.
[06/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query66.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query66.q.out b/ql/src/test/results/clientpositive/perf/tez/query66.q.out
index 5dec2f8..432dd7e 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query66.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query66.q.out
@@ -471,240 +471,244 @@ Stage-0
limit:-1
Stage-1
Reducer 9 vectorized
- File Output Operator [FS_274]
- Select Operator [SEL_273] (rows=100 width=135)
+ File Output Operator [FS_278]
+ Select Operator [SEL_277] (rows=100 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41","_col42","_col43"]
- Limit [LIM_272] (rows=100 width=135)
+ Limit [LIM_276] (rows=100 width=135)
Number of rows:100
- Select Operator [SEL_271] (rows=158120068 width=135)
+ Select Operator [SEL_275] (rows=158120068 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41"]
<-Reducer 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_270]
- Group By Operator [GBY_269] (rows=158120068 width=135)
+ SHUFFLE [RS_274]
+ Group By Operator [GBY_273] (rows=158120068 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)","sum(VALUE._col7)","sum(VALUE._col8)","sum(VALUE._col9)","sum(VALUE._col10)","sum(VALUE._col11)","sum(VALUE._col12)","sum(VALUE._col13)","sum(VALUE._col14)","sum(VALUE._col15)","sum(VALUE._col16)","sum(VALUE._col17)","sum(VALUE._col18)","sum(VALUE._col19)","sum(VALUE._col20)","sum(VALUE._col21)","sum(VALUE._col22)","sum(VALUE._col23)","sum(VALUE._col24)","sum(VALUE._col25)","sum(VALUE._col26)","sum(VALUE._col27)","sum(VALUE._col28)","sum(VALUE._col29)
","sum(VALUE._col30)","sum(VALUE._col31)","sum(VALUE._col32)","sum(VALUE._col33)","sum(VALUE._col34)","sum(VALUE._col35)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
<-Union 7 [SIMPLE_EDGE]
<-Reducer 16 [CONTAINS] vectorized
- Reduce Output Operator [RS_289]
+ Reduce Output Operator [RS_294]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_288] (rows=316240137 width=135)
+ Group By Operator [GBY_293] (rows=316240137 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41"],aggregations:["sum(_col6)","sum(_col7)","sum(_col8)","sum(_col9)","sum(_col10)","sum(_col11)","sum(_col12)","sum(_col13)","sum(_col14)","sum(_col15)","sum(_col16)","sum(_col17)","sum(_col18)","sum(_col19)","sum(_col20)","sum(_col21)","sum(_col22)","sum(_col23)","sum(_col24)","sum(_col25)","sum(_col26)","sum(_col27)","sum(_col28)","sum(_col29)","sum(_col30)","sum(_col31)","sum(_col32)","sum(_col33)","sum(_col34)","sum(_col35)","sum(_col36)","sum(_col37)","sum(_col38)","sum(_col39)","sum(_col40)","sum(_col41)"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_287] (rows=316240137 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41"]
- Group By Operator [GBY_286] (rows=210822976 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)","sum(VALUE._col7)","sum(VALUE._col8)","sum(VALUE._col9)","sum(VALUE._col10)","sum(VALUE._col11)","sum(VALUE._col12)","sum(VALUE._col13)","sum(VALUE._col14)","sum(VALUE._col15)","sum(VALUE._col16)","sum(VALUE._col17)","sum(VALUE._col18)","sum(VALUE._col19)","sum(VALUE._col20)","sum(VALUE._col21)","sum(VALUE._col22)","sum(VALUE._col23)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_63]
- PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_62] (rows=421645953 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"],aggregations:["sum(_col6)","sum(_col7)","sum(_col8)","sum(_col9)","sum(_col10)","sum(_col11)","sum(_col12)","sum(_col13)","sum(_col14)","sum(_col15)","sum(_col16)","sum(_col17)","sum(_col18)","sum(_col19)","sum(_col20)","sum(_col21)","sum(_col22)","sum(_col23)","sum(_col24)","sum(_col25)","sum(_col26)","sum(_col27)","sum(_col28)","sum(_col29)"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_60] (rows=421645953 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"]
- Merge Join Operator [MERGEJOIN_203] (rows=421645953 width=135)
- Conds:RS_57._col3=RS_254._col0(Inner),Output:["_col4","_col5","_col6","_col11","_col15","_col16","_col17","_col18","_col19","_col20"]
- <-Map 24 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_254]
- PartitionCols:_col0
- Select Operator [SEL_251] (rows=27 width=1029)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_250] (rows=27 width=1029)
- predicate:w_warehouse_sk is not null
- TableScan [TS_12] (rows=27 width=1029)
- default@warehouse,warehouse,Tbl:COMPLETE,Col:NONE,Output:["w_warehouse_sk","w_warehouse_name","w_warehouse_sq_ft","w_city","w_county","w_state","w_country"]
- <-Reducer 14 [SIMPLE_EDGE]
- SHUFFLE [RS_57]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_202] (rows=383314495 width=135)
- Conds:RS_54._col2=RS_242._col0(Inner),Output:["_col3","_col4","_col5","_col6","_col11"]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_242]
- PartitionCols:_col0
- Select Operator [SEL_239] (rows=1 width=0)
- Output:["_col0"]
- Filter Operator [FIL_238] (rows=1 width=0)
- predicate:((sm_carrier) IN ('DIAMOND', 'AIRBORNE') and sm_ship_mode_sk is not null)
- TableScan [TS_9] (rows=1 width=0)
- default@ship_mode,ship_mode,Tbl:PARTIAL,Col:NONE,Output:["sm_ship_mode_sk","sm_carrier"]
- <-Reducer 13 [SIMPLE_EDGE]
- SHUFFLE [RS_54]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_201] (rows=348467716 width=135)
- Conds:RS_51._col0=RS_230._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6","_col11"]
- <-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_230]
- PartitionCols:_col0
- Select Operator [SEL_227] (rows=36524 width=1119)
- Output:["_col0","_col2"]
- Filter Operator [FIL_226] (rows=36524 width=1119)
- predicate:((d_year = 2002) and d_date_sk is not null)
- TableScan [TS_6] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
- <-Reducer 12 [SIMPLE_EDGE]
- SHUFFLE [RS_51]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_200] (rows=316788826 width=135)
- Conds:RS_285._col1=RS_218._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_218]
- PartitionCols:_col0
- Select Operator [SEL_215] (rows=9600 width=471)
- Output:["_col0"]
- Filter Operator [FIL_214] (rows=9600 width=471)
- predicate:(t_time BETWEEN 49530 AND 78330 and t_time_sk is not null)
- TableScan [TS_3] (rows=86400 width=471)
- default@time_dim,time_dim,Tbl:COMPLETE,Col:NONE,Output:["t_time_sk","t_time"]
- <-Map 27 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_285]
- PartitionCols:_col1
- Select Operator [SEL_284] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_283] (rows=287989836 width=135)
- predicate:((cs_ship_mode_sk BETWEEN DynamicValue(RS_55_ship_mode_sm_ship_mode_sk_min) AND DynamicValue(RS_55_ship_mode_sm_ship_mode_sk_max) and in_bloom_filter(cs_ship_mode_sk, DynamicValue(RS_55_ship_mode_sm_ship_mode_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_52_date_dim_d_date_sk_min) AND DynamicValue(RS_52_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_52_date_dim_d_date_sk_bloom_filter))) and (cs_sold_time_sk BETWEEN DynamicValue(RS_49_time_dim_t_time_sk_min) AND DynamicValue(RS_49_time_dim_t_time_sk_max) and in_bloom_filter(cs_sold_time_sk, DynamicValue(RS_49_time_dim_t_time_sk_bloom_filter))) and (cs_warehouse_sk BETWEEN DynamicValue(RS_58_warehouse_w_warehouse_sk_min) AND DynamicValue(RS_58_warehouse_w_warehouse_sk_max) and in_bloom_filter(cs_warehouse_sk, DynamicValue(RS_58_warehouse_w_warehouse_sk_bloom_filter))) and cs_ship_mode_sk is not null and cs_sold_date_sk is no
t null and cs_sold_time_sk is not null and cs_warehouse_sk is not null)
- TableScan [TS_33] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_sold_time_sk","cs_ship_mode_sk","cs_warehouse_sk","cs_quantity","cs_ext_sales_price","cs_net_paid_inc_ship_tax"]
- <-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_276]
- Group By Operator [GBY_275] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_223]
- Group By Operator [GBY_221] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_219] (rows=9600 width=471)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_215]
- <-Reducer 20 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_278]
- Group By Operator [GBY_277] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 18 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_235]
- Group By Operator [GBY_233] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_231] (rows=36524 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_227]
- <-Reducer 23 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_280]
- Group By Operator [GBY_279] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_247]
- Group By Operator [GBY_245] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_243] (rows=1 width=0)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_239]
- <-Reducer 26 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_282]
- Group By Operator [GBY_281] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 24 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_259]
- Group By Operator [GBY_257] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_255] (rows=27 width=1029)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_251]
+ Top N Key Operator [TNK_292] (rows=316240137 width=135)
+ keys:_col0, _col1, _col2, _col3, _col4, _col5,sort order:++++++,top n:100
+ Select Operator [SEL_291] (rows=316240137 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41"]
+ Group By Operator [GBY_290] (rows=210822976 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)","sum(VALUE._col7)","sum(VALUE._col8)","sum(VALUE._col9)","sum(VALUE._col10)","sum(VALUE._col11)","sum(VALUE._col12)","sum(VALUE._col13)","sum(VALUE._col14)","sum(VALUE._col15)","sum(VALUE._col16)","sum(VALUE._col17)","sum(VALUE._col18)","sum(VALUE._col19)","sum(VALUE._col20)","sum(VALUE._col21)","sum(VALUE._col22)","sum(VALUE._col23)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
+ <-Reducer 15 [SIMPLE_EDGE]
+ SHUFFLE [RS_63]
+ PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
+ Group By Operator [GBY_62] (rows=421645953 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"],aggregations:["sum(_col6)","sum(_col7)","sum(_col8)","sum(_col9)","sum(_col10)","sum(_col11)","sum(_col12)","sum(_col13)","sum(_col14)","sum(_col15)","sum(_col16)","sum(_col17)","sum(_col18)","sum(_col19)","sum(_col20)","sum(_col21)","sum(_col22)","sum(_col23)","sum(_col24)","sum(_col25)","sum(_col26)","sum(_col27)","sum(_col28)","sum(_col29)"],keys:_col0, _col1, _col2, _col3, _col4, _col5
+ Select Operator [SEL_60] (rows=421645953 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"]
+ Merge Join Operator [MERGEJOIN_204] (rows=421645953 width=135)
+ Conds:RS_57._col3=RS_257._col0(Inner),Output:["_col4","_col5","_col6","_col11","_col15","_col16","_col17","_col18","_col19","_col20"]
+ <-Map 24 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_257]
+ PartitionCols:_col0
+ Select Operator [SEL_254] (rows=27 width=1029)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
+ Filter Operator [FIL_253] (rows=27 width=1029)
+ predicate:w_warehouse_sk is not null
+ TableScan [TS_12] (rows=27 width=1029)
+ default@warehouse,warehouse,Tbl:COMPLETE,Col:NONE,Output:["w_warehouse_sk","w_warehouse_name","w_warehouse_sq_ft","w_city","w_county","w_state","w_country"]
+ <-Reducer 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_57]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_203] (rows=383314495 width=135)
+ Conds:RS_54._col2=RS_245._col0(Inner),Output:["_col3","_col4","_col5","_col6","_col11"]
+ <-Map 21 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_245]
+ PartitionCols:_col0
+ Select Operator [SEL_242] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_241] (rows=1 width=0)
+ predicate:((sm_carrier) IN ('DIAMOND', 'AIRBORNE') and sm_ship_mode_sk is not null)
+ TableScan [TS_9] (rows=1 width=0)
+ default@ship_mode,ship_mode,Tbl:PARTIAL,Col:NONE,Output:["sm_ship_mode_sk","sm_carrier"]
+ <-Reducer 13 [SIMPLE_EDGE]
+ SHUFFLE [RS_54]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_202] (rows=348467716 width=135)
+ Conds:RS_51._col0=RS_233._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6","_col11"]
+ <-Map 18 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_233]
+ PartitionCols:_col0
+ Select Operator [SEL_230] (rows=36524 width=1119)
+ Output:["_col0","_col2"]
+ Filter Operator [FIL_229] (rows=36524 width=1119)
+ predicate:((d_year = 2002) and d_date_sk is not null)
+ TableScan [TS_6] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_51]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_201] (rows=316788826 width=135)
+ Conds:RS_289._col1=RS_221._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_221]
+ PartitionCols:_col0
+ Select Operator [SEL_218] (rows=9600 width=471)
+ Output:["_col0"]
+ Filter Operator [FIL_217] (rows=9600 width=471)
+ predicate:(t_time BETWEEN 49530 AND 78330 and t_time_sk is not null)
+ TableScan [TS_3] (rows=86400 width=471)
+ default@time_dim,time_dim,Tbl:COMPLETE,Col:NONE,Output:["t_time_sk","t_time"]
+ <-Map 27 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_289]
+ PartitionCols:_col1
+ Select Operator [SEL_288] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
+ Filter Operator [FIL_287] (rows=287989836 width=135)
+ predicate:((cs_ship_mode_sk BETWEEN DynamicValue(RS_55_ship_mode_sm_ship_mode_sk_min) AND DynamicValue(RS_55_ship_mode_sm_ship_mode_sk_max) and in_bloom_filter(cs_ship_mode_sk, DynamicValue(RS_55_ship_mode_sm_ship_mode_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_52_date_dim_d_date_sk_min) AND DynamicValue(RS_52_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_52_date_dim_d_date_sk_bloom_filter))) and (cs_sold_time_sk BETWEEN DynamicValue(RS_49_time_dim_t_time_sk_min) AND DynamicValue(RS_49_time_dim_t_time_sk_max) and in_bloom_filter(cs_sold_time_sk, DynamicValue(RS_49_time_dim_t_time_sk_bloom_filter))) and (cs_warehouse_sk BETWEEN DynamicValue(RS_58_warehouse_w_warehouse_sk_min) AND DynamicValue(RS_58_warehouse_w_warehouse_sk_max) and in_bloom_filter(cs_warehouse_sk, DynamicValue(RS_58_warehouse_w_warehouse_sk_bloom_filter))) and cs_ship_mode_sk is not null and cs_sold_date_sk is
not null and cs_sold_time_sk is not null and cs_warehouse_sk is not null)
+ TableScan [TS_33] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_sold_time_sk","cs_ship_mode_sk","cs_warehouse_sk","cs_quantity","cs_ext_sales_price","cs_net_paid_inc_ship_tax"]
+ <-Reducer 17 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_280]
+ Group By Operator [GBY_279] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_226]
+ Group By Operator [GBY_224] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_222] (rows=9600 width=471)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_218]
+ <-Reducer 20 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_282]
+ Group By Operator [GBY_281] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 18 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_238]
+ Group By Operator [GBY_236] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_234] (rows=36524 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_230]
+ <-Reducer 23 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_284]
+ Group By Operator [GBY_283] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_250]
+ Group By Operator [GBY_248] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_246] (rows=1 width=0)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_242]
+ <-Reducer 26 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_286]
+ Group By Operator [GBY_285] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 24 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_262]
+ Group By Operator [GBY_260] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_258] (rows=27 width=1029)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_254]
<-Reducer 6 [CONTAINS] vectorized
- Reduce Output Operator [RS_268]
+ Reduce Output Operator [RS_272]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_267] (rows=316240137 width=135)
+ Group By Operator [GBY_271] (rows=316240137 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41"],aggregations:["sum(_col6)","sum(_col7)","sum(_col8)","sum(_col9)","sum(_col10)","sum(_col11)","sum(_col12)","sum(_col13)","sum(_col14)","sum(_col15)","sum(_col16)","sum(_col17)","sum(_col18)","sum(_col19)","sum(_col20)","sum(_col21)","sum(_col22)","sum(_col23)","sum(_col24)","sum(_col25)","sum(_col26)","sum(_col27)","sum(_col28)","sum(_col29)","sum(_col30)","sum(_col31)","sum(_col32)","sum(_col33)","sum(_col34)","sum(_col35)","sum(_col36)","sum(_col37)","sum(_col38)","sum(_col39)","sum(_col40)","sum(_col41)"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_266] (rows=316240137 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41"]
- Group By Operator [GBY_265] (rows=105417161 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)","sum(VALUE._col7)","sum(VALUE._col8)","sum(VALUE._col9)","sum(VALUE._col10)","sum(VALUE._col11)","sum(VALUE._col12)","sum(VALUE._col13)","sum(VALUE._col14)","sum(VALUE._col15)","sum(VALUE._col16)","sum(VALUE._col17)","sum(VALUE._col18)","sum(VALUE._col19)","sum(VALUE._col20)","sum(VALUE._col21)","sum(VALUE._col22)","sum(VALUE._col23)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
- <-Reducer 5 [SIMPLE_EDGE]
- SHUFFLE [RS_30]
- PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
- Group By Operator [GBY_29] (rows=210834322 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"],aggregations:["sum(_col6)","sum(_col7)","sum(_col8)","sum(_col9)","sum(_col10)","sum(_col11)","sum(_col12)","sum(_col13)","sum(_col14)","sum(_col15)","sum(_col16)","sum(_col17)","sum(_col18)","sum(_col19)","sum(_col20)","sum(_col21)","sum(_col22)","sum(_col23)","sum(_col24)","sum(_col25)","sum(_col26)","sum(_col27)","sum(_col28)","sum(_col29)"],keys:_col0, _col1, _col2, _col3, _col4, _col5
- Select Operator [SEL_27] (rows=210834322 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"]
- Merge Join Operator [MERGEJOIN_199] (rows=210834322 width=135)
- Conds:RS_24._col3=RS_252._col0(Inner),Output:["_col4","_col5","_col6","_col11","_col15","_col16","_col17","_col18","_col19","_col20"]
- <-Map 24 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_252]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_251]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_24]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_198] (rows=191667562 width=135)
- Conds:RS_21._col2=RS_240._col0(Inner),Output:["_col3","_col4","_col5","_col6","_col11"]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_240]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_239]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_197] (rows=174243235 width=135)
- Conds:RS_18._col0=RS_228._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6","_col11"]
- <-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_228]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_227]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_196] (rows=158402938 width=135)
- Conds:RS_264._col1=RS_216._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_216]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_215]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_264]
- PartitionCols:_col1
- Select Operator [SEL_263] (rows=144002668 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_262] (rows=144002668 width=135)
- predicate:((ws_ship_mode_sk BETWEEN DynamicValue(RS_22_ship_mode_sm_ship_mode_sk_min) AND DynamicValue(RS_22_ship_mode_sm_ship_mode_sk_max) and in_bloom_filter(ws_ship_mode_sk, DynamicValue(RS_22_ship_mode_sm_ship_mode_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and (ws_sold_time_sk BETWEEN DynamicValue(RS_16_time_dim_t_time_sk_min) AND DynamicValue(RS_16_time_dim_t_time_sk_max) and in_bloom_filter(ws_sold_time_sk, DynamicValue(RS_16_time_dim_t_time_sk_bloom_filter))) and (ws_warehouse_sk BETWEEN DynamicValue(RS_25_warehouse_w_warehouse_sk_min) AND DynamicValue(RS_25_warehouse_w_warehouse_sk_max) and in_bloom_filter(ws_warehouse_sk, DynamicValue(RS_25_warehouse_w_warehouse_sk_bloom_filter))) and ws_ship_mode_sk is not null and ws_sold_date_sk is no
t null and ws_sold_time_sk is not null and ws_warehouse_sk is not null)
- TableScan [TS_0] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_sold_time_sk","ws_ship_mode_sk","ws_warehouse_sk","ws_quantity","ws_sales_price","ws_net_paid_inc_tax"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_225]
- Group By Operator [GBY_224] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_222]
- Group By Operator [GBY_220] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_217] (rows=9600 width=471)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_215]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_237]
- Group By Operator [GBY_236] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 18 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_234]
- Group By Operator [GBY_232] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_229] (rows=36524 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_227]
- <-Reducer 22 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_249]
- Group By Operator [GBY_248] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_246]
- Group By Operator [GBY_244] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_241] (rows=1 width=0)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_239]
- <-Reducer 25 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_261]
- Group By Operator [GBY_260] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 24 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_258]
- Group By Operator [GBY_256] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_253] (rows=27 width=1029)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_251]
+ Top N Key Operator [TNK_270] (rows=316240137 width=135)
+ keys:_col0, _col1, _col2, _col3, _col4, _col5,sort order:++++++,top n:100
+ Select Operator [SEL_269] (rows=316240137 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29","_col30","_col31","_col32","_col33","_col34","_col35","_col36","_col37","_col38","_col39","_col40","_col41"]
+ Group By Operator [GBY_268] (rows=105417161 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)","sum(VALUE._col7)","sum(VALUE._col8)","sum(VALUE._col9)","sum(VALUE._col10)","sum(VALUE._col11)","sum(VALUE._col12)","sum(VALUE._col13)","sum(VALUE._col14)","sum(VALUE._col15)","sum(VALUE._col16)","sum(VALUE._col17)","sum(VALUE._col18)","sum(VALUE._col19)","sum(VALUE._col20)","sum(VALUE._col21)","sum(VALUE._col22)","sum(VALUE._col23)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
+ <-Reducer 5 [SIMPLE_EDGE]
+ SHUFFLE [RS_30]
+ PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
+ Group By Operator [GBY_29] (rows=210834322 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"],aggregations:["sum(_col6)","sum(_col7)","sum(_col8)","sum(_col9)","sum(_col10)","sum(_col11)","sum(_col12)","sum(_col13)","sum(_col14)","sum(_col15)","sum(_col16)","sum(_col17)","sum(_col18)","sum(_col19)","sum(_col20)","sum(_col21)","sum(_col22)","sum(_col23)","sum(_col24)","sum(_col25)","sum(_col26)","sum(_col27)","sum(_col28)","sum(_col29)"],keys:_col0, _col1, _col2, _col3, _col4, _col5
+ Select Operator [SEL_27] (rows=210834322 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23","_col24","_col25","_col26","_col27","_col28","_col29"]
+ Merge Join Operator [MERGEJOIN_200] (rows=210834322 width=135)
+ Conds:RS_24._col3=RS_255._col0(Inner),Output:["_col4","_col5","_col6","_col11","_col15","_col16","_col17","_col18","_col19","_col20"]
+ <-Map 24 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_255]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_254]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_24]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_199] (rows=191667562 width=135)
+ Conds:RS_21._col2=RS_243._col0(Inner),Output:["_col3","_col4","_col5","_col6","_col11"]
+ <-Map 21 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_243]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_242]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_198] (rows=174243235 width=135)
+ Conds:RS_18._col0=RS_231._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6","_col11"]
+ <-Map 18 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_231]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_230]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_197] (rows=158402938 width=135)
+ Conds:RS_267._col1=RS_219._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_219]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_218]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_267]
+ PartitionCols:_col1
+ Select Operator [SEL_266] (rows=144002668 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
+ Filter Operator [FIL_265] (rows=144002668 width=135)
+ predicate:((ws_ship_mode_sk BETWEEN DynamicValue(RS_22_ship_mode_sm_ship_mode_sk_min) AND DynamicValue(RS_22_ship_mode_sm_ship_mode_sk_max) and in_bloom_filter(ws_ship_mode_sk, DynamicValue(RS_22_ship_mode_sm_ship_mode_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and (ws_sold_time_sk BETWEEN DynamicValue(RS_16_time_dim_t_time_sk_min) AND DynamicValue(RS_16_time_dim_t_time_sk_max) and in_bloom_filter(ws_sold_time_sk, DynamicValue(RS_16_time_dim_t_time_sk_bloom_filter))) and (ws_warehouse_sk BETWEEN DynamicValue(RS_25_warehouse_w_warehouse_sk_min) AND DynamicValue(RS_25_warehouse_w_warehouse_sk_max) and in_bloom_filter(ws_warehouse_sk, DynamicValue(RS_25_warehouse_w_warehouse_sk_bloom_filter))) and ws_ship_mode_sk is not null and ws_sold_date_sk is
not null and ws_sold_time_sk is not null and ws_warehouse_sk is not null)
+ TableScan [TS_0] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_sold_time_sk","ws_ship_mode_sk","ws_warehouse_sk","ws_quantity","ws_sales_price","ws_net_paid_inc_tax"]
+ <-Reducer 11 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_228]
+ Group By Operator [GBY_227] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_225]
+ Group By Operator [GBY_223] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_220] (rows=9600 width=471)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_218]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_240]
+ Group By Operator [GBY_239] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 18 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_237]
+ Group By Operator [GBY_235] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_232] (rows=36524 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_230]
+ <-Reducer 22 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_252]
+ Group By Operator [GBY_251] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_249]
+ Group By Operator [GBY_247] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_244] (rows=1 width=0)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_242]
+ <-Reducer 25 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_264]
+ Group By Operator [GBY_263] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 24 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_261]
+ Group By Operator [GBY_259] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_256] (rows=27 width=1029)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_254]
[11/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query29.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query29.q.out b/ql/src/test/results/clientpositive/perf/tez/query29.q.out
index 791ddb6..9bfcdfa 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query29.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query29.q.out
@@ -117,200 +117,202 @@ Stage-0
limit:100
Stage-1
Reducer 5 vectorized
- File Output Operator [FS_259]
- Limit [LIM_258] (rows=100 width=88)
+ File Output Operator [FS_260]
+ Limit [LIM_259] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_257] (rows=463823414 width=88)
+ Select Operator [SEL_258] (rows=463823414 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
<-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_256]
- Group By Operator [GBY_255] (rows=463823414 width=88)
+ SHUFFLE [RS_257]
+ Group By Operator [GBY_256] (rows=463823414 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_49]
PartitionCols:_col0, _col1, _col2, _col3
Group By Operator [GBY_48] (rows=927646829 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col14)","sum(_col22)","sum(_col3)"],keys:_col7, _col8, _col27, _col28
- Merge Join Operator [MERGEJOIN_204] (rows=927646829 width=88)
- Conds:RS_44._col1, _col2=RS_45._col14, _col13(Inner),Output:["_col3","_col7","_col8","_col14","_col22","_col27","_col28"]
- <-Reducer 12 [SIMPLE_EDGE]
- SHUFFLE [RS_45]
- PartitionCols:_col14, _col13
- Select Operator [SEL_40] (rows=843315281 width=88)
- Output:["_col1","_col2","_col8","_col13","_col14","_col16","_col21","_col22"]
- Merge Join Operator [MERGEJOIN_203] (rows=843315281 width=88)
- Conds:RS_37._col3=RS_246._col0(Inner),Output:["_col5","_col10","_col11","_col13","_col18","_col19","_col21","_col22"]
- <-Map 22 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_246]
+ Top N Key Operator [TNK_93] (rows=927646829 width=88)
+ keys:_col7, _col8, _col27, _col28,sort order:++++,top n:100
+ Merge Join Operator [MERGEJOIN_205] (rows=927646829 width=88)
+ Conds:RS_44._col1, _col2=RS_45._col14, _col13(Inner),Output:["_col3","_col7","_col8","_col14","_col22","_col27","_col28"]
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_45]
+ PartitionCols:_col14, _col13
+ Select Operator [SEL_40] (rows=843315281 width=88)
+ Output:["_col1","_col2","_col8","_col13","_col14","_col16","_col21","_col22"]
+ Merge Join Operator [MERGEJOIN_204] (rows=843315281 width=88)
+ Conds:RS_37._col3=RS_247._col0(Inner),Output:["_col5","_col10","_col11","_col13","_col18","_col19","_col21","_col22"]
+ <-Map 22 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_247]
+ PartitionCols:_col0
+ Select Operator [SEL_246] (rows=1704 width=1910)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_245] (rows=1704 width=1910)
+ predicate:s_store_sk is not null
+ TableScan [TS_25] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id","s_store_name"]
+ <-Reducer 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_37]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_203] (rows=766650239 width=88)
+ Conds:RS_34._col1=RS_239._col0(Inner),Output:["_col3","_col5","_col10","_col11","_col13","_col18","_col19"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_239]
+ PartitionCols:_col0
+ Select Operator [SEL_238] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_237] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_22] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc"]
+ <-Reducer 10 [SIMPLE_EDGE]
+ SHUFFLE [RS_34]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_202] (rows=696954748 width=88)
+ Conds:RS_31._col1, _col2, _col4=RS_32._col1, _col2, _col3(Inner),Output:["_col1","_col3","_col5","_col10","_col11","_col13"]
+ <-Reducer 15 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_32]
+ PartitionCols:_col1, _col2, _col3
+ Merge Join Operator [MERGEJOIN_201] (rows=63350266 width=77)
+ Conds:RS_230._col0=RS_223._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ <-Map 13 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_223]
+ PartitionCols:_col0
+ Select Operator [SEL_220] (rows=4058 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_218] (rows=4058 width=1119)
+ predicate:((d_year = 1999) and d_date_sk is not null and d_moy BETWEEN 4 AND 7)
+ TableScan [TS_9] (rows=73049 width=1119)
+ default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Map 19 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_230]
+ PartitionCols:_col0
+ Select Operator [SEL_229] (rows=57591150 width=77)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_228] (rows=57591150 width=77)
+ predicate:(sr_customer_sk is not null and sr_item_sk is not null and sr_returned_date_sk is not null and sr_ticket_number is not null)
+ TableScan [TS_12] (rows=57591150 width=77)
+ default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_return_quantity"]
+ <-Reducer 9 [SIMPLE_EDGE]
+ SHUFFLE [RS_31]
+ PartitionCols:_col1, _col2, _col4
+ Merge Join Operator [MERGEJOIN_200] (rows=633595212 width=88)
+ Conds:RS_255._col0=RS_221._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
+ <-Map 13 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_221]
+ PartitionCols:_col0
+ Select Operator [SEL_219] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_217] (rows=18262 width=1119)
+ predicate:((d_moy = 4) and (d_year = 1999) and d_date_sk is not null)
+ Please refer to the previous TableScan [TS_9]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_255]
+ PartitionCols:_col0
+ Select Operator [SEL_254] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_253] (rows=575995635 width=88)
+ predicate:((ss_customer_sk BETWEEN DynamicValue(RS_32_store_returns_sr_customer_sk_min) AND DynamicValue(RS_32_store_returns_sr_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_32_store_returns_sr_customer_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_32_store_returns_sr_item_sk_min) AND DynamicValue(RS_32_store_returns_sr_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_32_store_returns_sr_item_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_35_item_i_item_sk_min) AND DynamicValue(RS_35_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_35_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_29_d1_d_date_sk_min) AND DynamicValue(RS_29_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_29_d1_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_38_store_s_store_sk_min) AND DynamicValue(RS_38_store_s_store_sk_
max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_38_store_s_store_sk_bloom_filter))) and (ss_ticket_number BETWEEN DynamicValue(RS_32_store_returns_sr_ticket_number_min) AND DynamicValue(RS_32_store_returns_sr_ticket_number_max) and in_bloom_filter(ss_ticket_number, DynamicValue(RS_32_store_returns_sr_ticket_number_bloom_filter))) and ss_customer_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null and ss_ticket_number is not null)
+ TableScan [TS_6] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_quantity"]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_227]
+ Group By Operator [GBY_226] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_225]
+ Group By Operator [GBY_224] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_222] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_219]
+ <-Reducer 16 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_232]
+ Group By Operator [GBY_231] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_121]
+ Group By Operator [GBY_120] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_119] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_201]
+ <-Reducer 17 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_234]
+ Group By Operator [GBY_233] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_126]
+ Group By Operator [GBY_125] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_124] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_201]
+ <-Reducer 18 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_236]
+ Group By Operator [GBY_235] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_131]
+ Group By Operator [GBY_130] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_129] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_201]
+ <-Reducer 21 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_244]
+ Group By Operator [GBY_243] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_242]
+ Group By Operator [GBY_241] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_240] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_238]
+ <-Reducer 23 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_252]
+ Group By Operator [GBY_251] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 22 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_250]
+ Group By Operator [GBY_249] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_248] (rows=1704 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_246]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_44]
+ PartitionCols:_col1, _col2
+ Merge Join Operator [MERGEJOIN_199] (rows=316788826 width=135)
+ Conds:RS_216._col0=RS_208._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 6 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_208]
PartitionCols:_col0
- Select Operator [SEL_245] (rows=1704 width=1910)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_244] (rows=1704 width=1910)
- predicate:s_store_sk is not null
- TableScan [TS_25] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id","s_store_name"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_37]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_202] (rows=766650239 width=88)
- Conds:RS_34._col1=RS_238._col0(Inner),Output:["_col3","_col5","_col10","_col11","_col13","_col18","_col19"]
- <-Map 20 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_238]
- PartitionCols:_col0
- Select Operator [SEL_237] (rows=462000 width=1436)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_236] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- TableScan [TS_22] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc"]
- <-Reducer 10 [SIMPLE_EDGE]
- SHUFFLE [RS_34]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_201] (rows=696954748 width=88)
- Conds:RS_31._col1, _col2, _col4=RS_32._col1, _col2, _col3(Inner),Output:["_col1","_col3","_col5","_col10","_col11","_col13"]
- <-Reducer 15 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_32]
- PartitionCols:_col1, _col2, _col3
- Merge Join Operator [MERGEJOIN_200] (rows=63350266 width=77)
- Conds:RS_229._col0=RS_222._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_222]
- PartitionCols:_col0
- Select Operator [SEL_219] (rows=4058 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_217] (rows=4058 width=1119)
- predicate:((d_year = 1999) and d_date_sk is not null and d_moy BETWEEN 4 AND 7)
- TableScan [TS_9] (rows=73049 width=1119)
- default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
- <-Map 19 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_229]
- PartitionCols:_col0
- Select Operator [SEL_228] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_227] (rows=57591150 width=77)
- predicate:(sr_customer_sk is not null and sr_item_sk is not null and sr_returned_date_sk is not null and sr_ticket_number is not null)
- TableScan [TS_12] (rows=57591150 width=77)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_return_quantity"]
- <-Reducer 9 [SIMPLE_EDGE]
- SHUFFLE [RS_31]
- PartitionCols:_col1, _col2, _col4
- Merge Join Operator [MERGEJOIN_199] (rows=633595212 width=88)
- Conds:RS_254._col0=RS_220._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_220]
- PartitionCols:_col0
- Select Operator [SEL_218] (rows=18262 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_216] (rows=18262 width=1119)
- predicate:((d_moy = 4) and (d_year = 1999) and d_date_sk is not null)
- Please refer to the previous TableScan [TS_9]
- <-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_254]
- PartitionCols:_col0
- Select Operator [SEL_253] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_252] (rows=575995635 width=88)
- predicate:((ss_customer_sk BETWEEN DynamicValue(RS_32_store_returns_sr_customer_sk_min) AND DynamicValue(RS_32_store_returns_sr_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_32_store_returns_sr_customer_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_32_store_returns_sr_item_sk_min) AND DynamicValue(RS_32_store_returns_sr_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_32_store_returns_sr_item_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_35_item_i_item_sk_min) AND DynamicValue(RS_35_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_35_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_29_d1_d_date_sk_min) AND DynamicValue(RS_29_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_29_d1_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_38_store_s_store_sk_min) AND DynamicValue(RS_38_store_s_store_sk_ma
x) and in_bloom_filter(ss_store_sk, DynamicValue(RS_38_store_s_store_sk_bloom_filter))) and (ss_ticket_number BETWEEN DynamicValue(RS_32_store_returns_sr_ticket_number_min) AND DynamicValue(RS_32_store_returns_sr_ticket_number_max) and in_bloom_filter(ss_ticket_number, DynamicValue(RS_32_store_returns_sr_ticket_number_bloom_filter))) and ss_customer_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null and ss_ticket_number is not null)
- TableScan [TS_6] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_quantity"]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_226]
- Group By Operator [GBY_225] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_224]
- Group By Operator [GBY_223] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_221] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_218]
- <-Reducer 16 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_231]
- Group By Operator [GBY_230] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_120]
- Group By Operator [GBY_119] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_118] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_200]
- <-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_233]
- Group By Operator [GBY_232] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_125]
- Group By Operator [GBY_124] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_123] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_200]
- <-Reducer 18 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_235]
- Group By Operator [GBY_234] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 15 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_130]
- Group By Operator [GBY_129] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_128] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_200]
- <-Reducer 21 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_243]
- Group By Operator [GBY_242] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_241]
- Group By Operator [GBY_240] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_239] (rows=462000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_237]
- <-Reducer 23 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_251]
- Group By Operator [GBY_250] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 22 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_249]
- Group By Operator [GBY_248] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_247] (rows=1704 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_245]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_44]
- PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_198] (rows=316788826 width=135)
- Conds:RS_215._col0=RS_207._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 6 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_207]
- PartitionCols:_col0
- Select Operator [SEL_206] (rows=36525 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_205] (rows=36525 width=1119)
- predicate:((d_year) IN (1999, 2000, 2001) and d_date_sk is not null)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,d3,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_215]
- PartitionCols:_col0
- Select Operator [SEL_214] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_213] (rows=287989836 width=135)
- predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_42_d3_d_date_sk_min) AND DynamicValue(RS_42_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_42_d3_d_date_sk_bloom_filter))) and cs_bill_customer_sk is not null and cs_item_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_0] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_quantity"]
- <-Reducer 7 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_212]
- Group By Operator [GBY_211] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 6 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_210]
- Group By Operator [GBY_209] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_208] (rows=36525 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_206]
+ Select Operator [SEL_207] (rows=36525 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_206] (rows=36525 width=1119)
+ predicate:((d_year) IN (1999, 2000, 2001) and d_date_sk is not null)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,d3,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_216]
+ PartitionCols:_col0
+ Select Operator [SEL_215] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_214] (rows=287989836 width=135)
+ predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_42_d3_d_date_sk_min) AND DynamicValue(RS_42_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_42_d3_d_date_sk_bloom_filter))) and cs_bill_customer_sk is not null and cs_item_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_0] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_quantity"]
+ <-Reducer 7 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_213]
+ Group By Operator [GBY_212] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 6 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_211]
+ Group By Operator [GBY_210] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_209] (rows=36525 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_207]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query35.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query35.q.out b/ql/src/test/results/clientpositive/perf/tez/query35.q.out
index 4fe3856..4ad92c2 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query35.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query35.q.out
@@ -136,188 +136,190 @@ Stage-0
limit:-1
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_225]
- Limit [LIM_224] (rows=100 width=88)
+ File Output Operator [FS_226]
+ Limit [LIM_225] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_223] (rows=1045432122 width=88)
+ Select Operator [SEL_224] (rows=1045432122 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_222]
- Select Operator [SEL_221] (rows=1045432122 width=88)
+ SHUFFLE [RS_223]
+ Select Operator [SEL_222] (rows=1045432122 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col9","_col10","_col11","_col12","_col14","_col15","_col16","_col17"]
- Group By Operator [GBY_220] (rows=1045432122 width=88)
+ Group By Operator [GBY_221] (rows=1045432122 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"],aggregations:["count(VALUE._col0)","sum(VALUE._col1)","count(VALUE._col2)","max(VALUE._col3)","sum(VALUE._col4)","count(VALUE._col5)","max(VALUE._col6)","sum(VALUE._col7)","count(VALUE._col8)","max(VALUE._col9)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_63]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5
Group By Operator [GBY_62] (rows=2090864244 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"],aggregations:["count()","sum(_col8)","count(_col8)","max(_col8)","sum(_col9)","count(_col9)","max(_col9)","sum(_col10)","count(_col10)","max(_col10)"],keys:_col4, _col6, _col7, _col8, _col9, _col10
- Select Operator [SEL_61] (rows=2090864244 width=88)
- Output:["_col4","_col6","_col7","_col8","_col9","_col10"]
- Filter Operator [FIL_60] (rows=2090864244 width=88)
- predicate:(_col12 is not null or _col14 is not null)
- Merge Join Operator [MERGEJOIN_173] (rows=2090864244 width=88)
- Conds:RS_55._col0=RS_56._col0(Left Semi),RS_55._col0=RS_211._col0(Left Outer),RS_55._col0=RS_219._col0(Left Outer),Output:["_col4","_col6","_col7","_col8","_col9","_col10","_col12","_col14"]
- <-Reducer 3 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_55]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_169] (rows=96800003 width=860)
- Conds:RS_50._col1=RS_182._col0(Inner),Output:["_col0","_col4","_col6","_col7","_col8","_col9","_col10"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_182]
- PartitionCols:_col0
- Select Operator [SEL_181] (rows=1861800 width=385)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_180] (rows=1861800 width=385)
- predicate:cd_demo_sk is not null
- TableScan [TS_6] (rows=1861800 width=385)
- default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_dep_count","cd_dep_employed_count","cd_dep_college_count"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_50]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_168] (rows=88000001 width=860)
- Conds:RS_176._col2=RS_179._col0(Inner),Output:["_col0","_col1","_col4"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_176]
- PartitionCols:_col2
- Select Operator [SEL_175] (rows=80000000 width=860)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_174] (rows=80000000 width=860)
- predicate:(c_current_addr_sk is not null and c_current_cdemo_sk is not null and c_customer_sk is not null)
- TableScan [TS_0] (rows=80000000 width=860)
- default@customer,c,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_cdemo_sk","c_current_addr_sk"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_179]
- PartitionCols:_col0
- Select Operator [SEL_178] (rows=40000000 width=1014)
- Output:["_col0","_col1"]
- Filter Operator [FIL_177] (rows=40000000 width=1014)
- predicate:ca_address_sk is not null
- TableScan [TS_3] (rows=40000000 width=1014)
- default@customer_address,ca,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_56]
- PartitionCols:_col0
- Group By Operator [GBY_54] (rows=633595212 width=88)
- Output:["_col0"],keys:_col0
- Select Operator [SEL_18] (rows=633595212 width=88)
- Output:["_col0"]
- Merge Join Operator [MERGEJOIN_170] (rows=633595212 width=88)
- Conds:RS_203._col0=RS_185._col0(Inner),Output:["_col1"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_185]
+ Top N Key Operator [TNK_104] (rows=2090864244 width=88)
+ keys:_col4, _col6, _col7, _col8, _col9, _col10,sort order:++++++,top n:100
+ Select Operator [SEL_61] (rows=2090864244 width=88)
+ Output:["_col4","_col6","_col7","_col8","_col9","_col10"]
+ Filter Operator [FIL_60] (rows=2090864244 width=88)
+ predicate:(_col12 is not null or _col14 is not null)
+ Merge Join Operator [MERGEJOIN_174] (rows=2090864244 width=88)
+ Conds:RS_55._col0=RS_56._col0(Left Semi),RS_55._col0=RS_212._col0(Left Outer),RS_55._col0=RS_220._col0(Left Outer),Output:["_col4","_col6","_col7","_col8","_col9","_col10","_col12","_col14"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_55]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_170] (rows=96800003 width=860)
+ Conds:RS_50._col1=RS_183._col0(Inner),Output:["_col0","_col4","_col6","_col7","_col8","_col9","_col10"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_183]
+ PartitionCols:_col0
+ Select Operator [SEL_182] (rows=1861800 width=385)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_181] (rows=1861800 width=385)
+ predicate:cd_demo_sk is not null
+ TableScan [TS_6] (rows=1861800 width=385)
+ default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_dep_count","cd_dep_employed_count","cd_dep_college_count"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_50]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_169] (rows=88000001 width=860)
+ Conds:RS_177._col2=RS_180._col0(Inner),Output:["_col0","_col1","_col4"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_177]
+ PartitionCols:_col2
+ Select Operator [SEL_176] (rows=80000000 width=860)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_175] (rows=80000000 width=860)
+ predicate:(c_current_addr_sk is not null and c_current_cdemo_sk is not null and c_customer_sk is not null)
+ TableScan [TS_0] (rows=80000000 width=860)
+ default@customer,c,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_cdemo_sk","c_current_addr_sk"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_180]
+ PartitionCols:_col0
+ Select Operator [SEL_179] (rows=40000000 width=1014)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_178] (rows=40000000 width=1014)
+ predicate:ca_address_sk is not null
+ TableScan [TS_3] (rows=40000000 width=1014)
+ default@customer_address,ca,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
+ <-Reducer 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_56]
+ PartitionCols:_col0
+ Group By Operator [GBY_54] (rows=633595212 width=88)
+ Output:["_col0"],keys:_col0
+ Select Operator [SEL_18] (rows=633595212 width=88)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_171] (rows=633595212 width=88)
+ Conds:RS_204._col0=RS_186._col0(Inner),Output:["_col1"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_186]
+ PartitionCols:_col0
+ Select Operator [SEL_185] (rows=12174 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_184] (rows=12174 width=1119)
+ predicate:((d_qoy < 4) and (d_year = 1999) and d_date_sk is not null)
+ TableScan [TS_12] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_204]
+ PartitionCols:_col0
+ Select Operator [SEL_203] (rows=575995635 width=88)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_202] (rows=575995635 width=88)
+ predicate:((ss_customer_sk BETWEEN DynamicValue(RS_55_c_c_customer_sk_min) AND DynamicValue(RS_55_c_c_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_55_c_c_customer_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_16_date_dim_d_date_sk_min) AND DynamicValue(RS_16_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_16_date_dim_d_date_sk_bloom_filter))) and ss_customer_sk is not null and ss_sold_date_sk is not null)
+ TableScan [TS_9] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_customer_sk"]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_199]
+ Group By Operator [GBY_198] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_195]
+ Group By Operator [GBY_192] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_187] (rows=12174 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_185]
+ <-Reducer 7 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_201]
+ Group By Operator [GBY_200] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=96800000)"]
+ <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_137]
+ Group By Operator [GBY_136] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=96800000)"]
+ Select Operator [SEL_135] (rows=96800003 width=860)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_170]
+ <-Reducer 15 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_212]
+ PartitionCols:_col0
+ Select Operator [SEL_211] (rows=79201469 width=135)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_210] (rows=79201469 width=135)
+ Output:["_col0"],keys:KEY._col0
+ <-Reducer 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_30]
PartitionCols:_col0
- Select Operator [SEL_184] (rows=12174 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_183] (rows=12174 width=1119)
- predicate:((d_qoy < 4) and (d_year = 1999) and d_date_sk is not null)
- TableScan [TS_12] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_203]
+ Group By Operator [GBY_29] (rows=158402938 width=135)
+ Output:["_col0"],keys:_col1
+ Merge Join Operator [MERGEJOIN_172] (rows=158402938 width=135)
+ Conds:RS_209._col0=RS_188._col0(Inner),Output:["_col1"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_188]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_185]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_209]
+ PartitionCols:_col0
+ Select Operator [SEL_208] (rows=144002668 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_207] (rows=144002668 width=135)
+ predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_26_date_dim_d_date_sk_min) AND DynamicValue(RS_26_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_26_date_dim_d_date_sk_bloom_filter))) and ws_bill_customer_sk is not null and ws_sold_date_sk is not null)
+ TableScan [TS_19] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_bill_customer_sk"]
+ <-Reducer 16 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_206]
+ Group By Operator [GBY_205] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_196]
+ Group By Operator [GBY_193] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_189] (rows=12174 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_185]
+ <-Reducer 18 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_220]
+ PartitionCols:_col0
+ Select Operator [SEL_219] (rows=158394413 width=135)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_218] (rows=158394413 width=135)
+ Output:["_col0"],keys:KEY._col0
+ <-Reducer 17 [SIMPLE_EDGE]
+ SHUFFLE [RS_44]
PartitionCols:_col0
- Select Operator [SEL_202] (rows=575995635 width=88)
- Output:["_col0","_col1"]
- Filter Operator [FIL_201] (rows=575995635 width=88)
- predicate:((ss_customer_sk BETWEEN DynamicValue(RS_55_c_c_customer_sk_min) AND DynamicValue(RS_55_c_c_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_55_c_c_customer_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_16_date_dim_d_date_sk_min) AND DynamicValue(RS_16_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_16_date_dim_d_date_sk_bloom_filter))) and ss_customer_sk is not null and ss_sold_date_sk is not null)
- TableScan [TS_9] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_customer_sk"]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_198]
- Group By Operator [GBY_197] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_194]
- Group By Operator [GBY_191] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_186] (rows=12174 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_184]
- <-Reducer 7 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_200]
- Group By Operator [GBY_199] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=96800000)"]
- <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_136]
- Group By Operator [GBY_135] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=96800000)"]
- Select Operator [SEL_134] (rows=96800003 width=860)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_169]
- <-Reducer 15 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_211]
- PartitionCols:_col0
- Select Operator [SEL_210] (rows=79201469 width=135)
- Output:["_col0","_col1"]
- Group By Operator [GBY_209] (rows=79201469 width=135)
- Output:["_col0"],keys:KEY._col0
- <-Reducer 14 [SIMPLE_EDGE]
- SHUFFLE [RS_30]
- PartitionCols:_col0
- Group By Operator [GBY_29] (rows=158402938 width=135)
- Output:["_col0"],keys:_col1
- Merge Join Operator [MERGEJOIN_171] (rows=158402938 width=135)
- Conds:RS_208._col0=RS_187._col0(Inner),Output:["_col1"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_187]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_184]
- <-Map 20 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_208]
- PartitionCols:_col0
- Select Operator [SEL_207] (rows=144002668 width=135)
- Output:["_col0","_col1"]
- Filter Operator [FIL_206] (rows=144002668 width=135)
- predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_26_date_dim_d_date_sk_min) AND DynamicValue(RS_26_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_26_date_dim_d_date_sk_bloom_filter))) and ws_bill_customer_sk is not null and ws_sold_date_sk is not null)
- TableScan [TS_19] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_bill_customer_sk"]
- <-Reducer 16 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_205]
- Group By Operator [GBY_204] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_195]
- Group By Operator [GBY_192] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_188] (rows=12174 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_184]
- <-Reducer 18 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_219]
- PartitionCols:_col0
- Select Operator [SEL_218] (rows=158394413 width=135)
- Output:["_col0","_col1"]
- Group By Operator [GBY_217] (rows=158394413 width=135)
- Output:["_col0"],keys:KEY._col0
- <-Reducer 17 [SIMPLE_EDGE]
- SHUFFLE [RS_44]
- PartitionCols:_col0
- Group By Operator [GBY_43] (rows=316788826 width=135)
- Output:["_col0"],keys:_col1
- Merge Join Operator [MERGEJOIN_172] (rows=316788826 width=135)
- Conds:RS_216._col0=RS_189._col0(Inner),Output:["_col1"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_189]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_184]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_216]
- PartitionCols:_col0
- Select Operator [SEL_215] (rows=287989836 width=135)
- Output:["_col0","_col1"]
- Filter Operator [FIL_214] (rows=287989836 width=135)
- predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_40_date_dim_d_date_sk_min) AND DynamicValue(RS_40_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_40_date_dim_d_date_sk_bloom_filter))) and cs_ship_customer_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_33] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_ship_customer_sk"]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_213]
- Group By Operator [GBY_212] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_196]
- Group By Operator [GBY_193] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_190] (rows=12174 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_184]
+ Group By Operator [GBY_43] (rows=316788826 width=135)
+ Output:["_col0"],keys:_col1
+ Merge Join Operator [MERGEJOIN_173] (rows=316788826 width=135)
+ Conds:RS_217._col0=RS_190._col0(Inner),Output:["_col1"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_190]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_185]
+ <-Map 21 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_217]
+ PartitionCols:_col0
+ Select Operator [SEL_216] (rows=287989836 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_215] (rows=287989836 width=135)
+ predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_40_date_dim_d_date_sk_min) AND DynamicValue(RS_40_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_40_date_dim_d_date_sk_bloom_filter))) and cs_ship_customer_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_33] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_ship_customer_sk"]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_214]
+ Group By Operator [GBY_213] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_197]
+ Group By Operator [GBY_194] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_191] (rows=12174 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_185]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query37.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query37.q.out b/ql/src/test/results/clientpositive/perf/tez/query37.q.out
index 9db8538..2b3ae52 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query37.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query37.q.out
@@ -46,83 +46,85 @@ Stage-0
limit:100
Stage-1
Reducer 4 vectorized
- File Output Operator [FS_96]
- Limit [LIM_95] (rows=100 width=135)
+ File Output Operator [FS_97]
+ Limit [LIM_96] (rows=100 width=135)
Number of rows:100
- Select Operator [SEL_94] (rows=316788826 width=135)
+ Select Operator [SEL_95] (rows=316788826 width=135)
Output:["_col0","_col1","_col2"]
<-Reducer 3 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_93]
- Group By Operator [GBY_92] (rows=316788826 width=135)
+ SHUFFLE [RS_94]
+ Group By Operator [GBY_93] (rows=316788826 width=135)
Output:["_col0","_col1","_col2"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_22]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_21] (rows=633577652 width=135)
Output:["_col0","_col1","_col2"],keys:_col2, _col3, _col4
- Merge Join Operator [MERGEJOIN_72] (rows=633577652 width=135)
- Conds:RS_91._col0=RS_75._col0(Inner),RS_75._col0=RS_18._col1(Inner),Output:["_col2","_col3","_col4"]
- <-Map 5 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_75]
- PartitionCols:_col0
- Select Operator [SEL_74] (rows=25666 width=1436)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_73] (rows=25666 width=1436)
- predicate:((i_manufact_id) IN (678, 964, 918, 849) and i_current_price BETWEEN 22 AND 52 and i_item_sk is not null)
- TableScan [TS_3] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc","i_current_price","i_manufact_id"]
- <-Reducer 8 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_18]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_71] (rows=4593600 width=15)
- Conds:RS_83._col0=RS_86._col0(Inner),Output:["_col1"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_86]
- PartitionCols:_col0
- Select Operator [SEL_85] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_84] (rows=8116 width=1119)
- predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'2001-06-02 00:00:00' AND TIMESTAMP'2001-08-01 00:00:00' and d_date_sk is not null)
- TableScan [TS_9] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
- <-Map 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_83]
- PartitionCols:_col0
- Select Operator [SEL_82] (rows=4176000 width=15)
- Output:["_col0","_col1"]
- Filter Operator [FIL_81] (rows=4176000 width=15)
- predicate:(inv_date_sk is not null and inv_item_sk is not null and inv_quantity_on_hand BETWEEN 100 AND 500)
- TableScan [TS_6] (rows=37584000 width=15)
- default@inventory,inventory,Tbl:COMPLETE,Col:NONE,Output:["inv_date_sk","inv_item_sk","inv_quantity_on_hand"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_91]
- PartitionCols:_col0
- Select Operator [SEL_90] (rows=287989836 width=135)
- Output:["_col0"]
- Filter Operator [FIL_89] (rows=287989836 width=135)
- predicate:((cs_item_sk BETWEEN DynamicValue(RS_17_item_i_item_sk_min) AND DynamicValue(RS_17_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_17_item_i_item_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_18_inventory_inv_item_sk_min) AND DynamicValue(RS_18_inventory_inv_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_18_inventory_inv_item_sk_bloom_filter))) and cs_item_sk is not null)
- TableScan [TS_0] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_item_sk"]
- <-Reducer 6 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_80]
- Group By Operator [GBY_79] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 5 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_78]
- Group By Operator [GBY_77] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_76] (rows=25666 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_74]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_88]
- Group By Operator [GBY_87] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=4593600)"]
- <-Reducer 8 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_50]
- Group By Operator [GBY_49] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=4593600)"]
- Select Operator [SEL_48] (rows=4593600 width=15)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_71]
+ Top N Key Operator [TNK_43] (rows=633577652 width=135)
+ keys:_col2, _col3, _col4,sort order:+++,top n:100
+ Merge Join Operator [MERGEJOIN_73] (rows=633577652 width=135)
+ Conds:RS_92._col0=RS_76._col0(Inner),RS_76._col0=RS_18._col1(Inner),Output:["_col2","_col3","_col4"]
+ <-Map 5 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_76]
+ PartitionCols:_col0
+ Select Operator [SEL_75] (rows=25666 width=1436)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_74] (rows=25666 width=1436)
+ predicate:((i_manufact_id) IN (678, 964, 918, 849) and i_current_price BETWEEN 22 AND 52 and i_item_sk is not null)
+ TableScan [TS_3] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc","i_current_price","i_manufact_id"]
+ <-Reducer 8 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_18]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_72] (rows=4593600 width=15)
+ Conds:RS_84._col0=RS_87._col0(Inner),Output:["_col1"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_87]
+ PartitionCols:_col0
+ Select Operator [SEL_86] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_85] (rows=8116 width=1119)
+ predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'2001-06-02 00:00:00' AND TIMESTAMP'2001-08-01 00:00:00' and d_date_sk is not null)
+ TableScan [TS_9] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
+ <-Map 7 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_84]
+ PartitionCols:_col0
+ Select Operator [SEL_83] (rows=4176000 width=15)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_82] (rows=4176000 width=15)
+ predicate:(inv_date_sk is not null and inv_item_sk is not null and inv_quantity_on_hand BETWEEN 100 AND 500)
+ TableScan [TS_6] (rows=37584000 width=15)
+ default@inventory,inventory,Tbl:COMPLETE,Col:NONE,Output:["inv_date_sk","inv_item_sk","inv_quantity_on_hand"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_92]
+ PartitionCols:_col0
+ Select Operator [SEL_91] (rows=287989836 width=135)
+ Output:["_col0"]
+ Filter Operator [FIL_90] (rows=287989836 width=135)
+ predicate:((cs_item_sk BETWEEN DynamicValue(RS_17_item_i_item_sk_min) AND DynamicValue(RS_17_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_17_item_i_item_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_18_inventory_inv_item_sk_min) AND DynamicValue(RS_18_inventory_inv_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_18_inventory_inv_item_sk_bloom_filter))) and cs_item_sk is not null)
+ TableScan [TS_0] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_item_sk"]
+ <-Reducer 6 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_81]
+ Group By Operator [GBY_80] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 5 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_79]
+ Group By Operator [GBY_78] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_77] (rows=25666 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_75]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_89]
+ Group By Operator [GBY_88] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=4593600)"]
+ <-Reducer 8 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_51]
+ Group By Operator [GBY_50] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=4593600)"]
+ Select Operator [SEL_49] (rows=4593600 width=15)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_72]
[10/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query40.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query40.q.out b/ql/src/test/results/clientpositive/perf/tez/query40.q.out
index 9920ad3..a3b6c03 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query40.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query40.q.out
@@ -71,115 +71,117 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_135]
- Limit [LIM_134] (rows=100 width=135)
+ File Output Operator [FS_136]
+ Limit [LIM_135] (rows=100 width=135)
Number of rows:100
- Select Operator [SEL_133] (rows=210822976 width=135)
+ Select Operator [SEL_134] (rows=210822976 width=135)
Output:["_col0","_col1","_col2","_col3"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_132]
- Group By Operator [GBY_131] (rows=210822976 width=135)
+ SHUFFLE [RS_133]
+ Group By Operator [GBY_132] (rows=210822976 width=135)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0, KEY._col1
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_30]
PartitionCols:_col0, _col1
Group By Operator [GBY_29] (rows=421645953 width=135)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col0, _col1
- Select Operator [SEL_27] (rows=421645953 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_100] (rows=421645953 width=135)
- Conds:RS_24._col1=RS_119._col0(Inner),Output:["_col4","_col7","_col9","_col11","_col14"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_119]
- PartitionCols:_col0
- Select Operator [SEL_118] (rows=27 width=1029)
- Output:["_col0","_col1"]
- Filter Operator [FIL_117] (rows=27 width=1029)
- predicate:w_warehouse_sk is not null
- TableScan [TS_12] (rows=27 width=1029)
- default@warehouse,warehouse,Tbl:COMPLETE,Col:NONE,Output:["w_warehouse_sk","w_state"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_24]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_99] (rows=383314495 width=135)
- Conds:RS_21._col2=RS_111._col0(Inner),Output:["_col1","_col4","_col7","_col9","_col11"]
- <-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_111]
- PartitionCols:_col0
- Select Operator [SEL_110] (rows=51333 width=1436)
- Output:["_col0","_col1"]
- Filter Operator [FIL_109] (rows=51333 width=1436)
- predicate:(i_current_price BETWEEN 0.99 AND 1.49 and i_item_sk is not null)
- TableScan [TS_9] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_current_price"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_98] (rows=348467716 width=135)
- Conds:RS_18._col0=RS_103._col0(Inner),Output:["_col1","_col2","_col4","_col7","_col9"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_103]
- PartitionCols:_col0
- Select Operator [SEL_102] (rows=8116 width=1119)
- Output:["_col0","_col1"]
- Filter Operator [FIL_101] (rows=8116 width=1119)
- predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-03-09 00:00:00' AND TIMESTAMP'1998-05-08 00:00:00' and d_date_sk is not null)
- TableScan [TS_6] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_97] (rows=316788826 width=135)
- Conds:RS_127._col2, _col3=RS_130._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col4","_col7"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_127]
- PartitionCols:_col2, _col3
- Select Operator [SEL_126] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_125] (rows=287989836 width=135)
- predicate:((cs_item_sk BETWEEN DynamicValue(RS_22_item_i_item_sk_min) AND DynamicValue(RS_22_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_22_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and (cs_warehouse_sk BETWEEN DynamicValue(RS_25_warehouse_w_warehouse_sk_min) AND DynamicValue(RS_25_warehouse_w_warehouse_sk_max) and in_bloom_filter(cs_warehouse_sk, DynamicValue(RS_25_warehouse_w_warehouse_sk_bloom_filter))) and cs_item_sk is not null and cs_sold_date_sk is not null and cs_warehouse_sk is not null)
- TableScan [TS_0] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_warehouse_sk","cs_item_sk","cs_order_number","cs_sales_price"]
- <-Reducer 10 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_108]
- Group By Operator [GBY_107] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_106]
- Group By Operator [GBY_105] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_104] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_102]
- <-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_116]
- Group By Operator [GBY_115] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_114]
- Group By Operator [GBY_113] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_112] (rows=51333 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_110]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_124]
- Group By Operator [GBY_123] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_122]
- Group By Operator [GBY_121] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_120] (rows=27 width=1029)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_118]
- <-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_130]
- PartitionCols:_col0, _col1
- Select Operator [SEL_129] (rows=28798881 width=106)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_128] (rows=28798881 width=106)
- predicate:cr_item_sk is not null
- TableScan [TS_3] (rows=28798881 width=106)
- default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_item_sk","cr_order_number","cr_refunded_cash"]
+ Top N Key Operator [TNK_55] (rows=421645953 width=135)
+ keys:_col0, _col1,sort order:++,top n:100
+ Select Operator [SEL_27] (rows=421645953 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_101] (rows=421645953 width=135)
+ Conds:RS_24._col1=RS_120._col0(Inner),Output:["_col4","_col7","_col9","_col11","_col14"]
+ <-Map 13 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_120]
+ PartitionCols:_col0
+ Select Operator [SEL_119] (rows=27 width=1029)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_118] (rows=27 width=1029)
+ predicate:w_warehouse_sk is not null
+ TableScan [TS_12] (rows=27 width=1029)
+ default@warehouse,warehouse,Tbl:COMPLETE,Col:NONE,Output:["w_warehouse_sk","w_state"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_24]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_100] (rows=383314495 width=135)
+ Conds:RS_21._col2=RS_112._col0(Inner),Output:["_col1","_col4","_col7","_col9","_col11"]
+ <-Map 11 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_112]
+ PartitionCols:_col0
+ Select Operator [SEL_111] (rows=51333 width=1436)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_110] (rows=51333 width=1436)
+ predicate:(i_current_price BETWEEN 0.99 AND 1.49 and i_item_sk is not null)
+ TableScan [TS_9] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_current_price"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_99] (rows=348467716 width=135)
+ Conds:RS_18._col0=RS_104._col0(Inner),Output:["_col1","_col2","_col4","_col7","_col9"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_104]
+ PartitionCols:_col0
+ Select Operator [SEL_103] (rows=8116 width=1119)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_102] (rows=8116 width=1119)
+ predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-03-09 00:00:00' AND TIMESTAMP'1998-05-08 00:00:00' and d_date_sk is not null)
+ TableScan [TS_6] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_98] (rows=316788826 width=135)
+ Conds:RS_128._col2, _col3=RS_131._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col4","_col7"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_128]
+ PartitionCols:_col2, _col3
+ Select Operator [SEL_127] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_126] (rows=287989836 width=135)
+ predicate:((cs_item_sk BETWEEN DynamicValue(RS_22_item_i_item_sk_min) AND DynamicValue(RS_22_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_22_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and (cs_warehouse_sk BETWEEN DynamicValue(RS_25_warehouse_w_warehouse_sk_min) AND DynamicValue(RS_25_warehouse_w_warehouse_sk_max) and in_bloom_filter(cs_warehouse_sk, DynamicValue(RS_25_warehouse_w_warehouse_sk_bloom_filter))) and cs_item_sk is not null and cs_sold_date_sk is not null and cs_warehouse_sk is not null)
+ TableScan [TS_0] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_warehouse_sk","cs_item_sk","cs_order_number","cs_sales_price"]
+ <-Reducer 10 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_109]
+ Group By Operator [GBY_108] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_107]
+ Group By Operator [GBY_106] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_105] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_103]
+ <-Reducer 12 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_117]
+ Group By Operator [GBY_116] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_115]
+ Group By Operator [GBY_114] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_113] (rows=51333 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_111]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_125]
+ Group By Operator [GBY_124] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_123]
+ Group By Operator [GBY_122] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_121] (rows=27 width=1029)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_119]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_131]
+ PartitionCols:_col0, _col1
+ Select Operator [SEL_130] (rows=28798881 width=106)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_129] (rows=28798881 width=106)
+ predicate:cr_item_sk is not null
+ TableScan [TS_3] (rows=28798881 width=106)
+ default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_item_sk","cr_order_number","cr_refunded_cash"]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query43.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query43.q.out b/ql/src/test/results/clientpositive/perf/tez/query43.q.out
index 394f728..afa3363 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query43.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query43.q.out
@@ -50,76 +50,78 @@ Stage-0
limit:100
Stage-1
Reducer 5 vectorized
- File Output Operator [FS_78]
- Limit [LIM_77] (rows=100 width=88)
+ File Output Operator [FS_79]
+ Limit [LIM_78] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_76] (rows=348477374 width=88)
+ Select Operator [SEL_77] (rows=348477374 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
<-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_75]
- Group By Operator [GBY_74] (rows=348477374 width=88)
+ SHUFFLE [RS_76]
+ Group By Operator [GBY_75] (rows=348477374 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)"],keys:KEY._col0, KEY._col1
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_18]
PartitionCols:_col0, _col1
Group By Operator [GBY_17] (rows=696954748 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)","sum(_col5)","sum(_col6)","sum(_col7)","sum(_col8)"],keys:_col0, _col1
- Select Operator [SEL_15] (rows=696954748 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- Merge Join Operator [MERGEJOIN_54] (rows=696954748 width=88)
- Conds:RS_12._col1=RS_65._col0(Inner),Output:["_col2","_col5","_col7","_col8"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_65]
- PartitionCols:_col0
- Select Operator [SEL_64] (rows=852 width=1910)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_63] (rows=852 width=1910)
- predicate:((s_gmt_offset = -6) and s_store_sk is not null)
- TableScan [TS_6] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id","s_store_name","s_gmt_offset"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_12]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_53] (rows=633595212 width=88)
- Conds:RS_73._col0=RS_57._col0(Inner),Output:["_col1","_col2","_col5"]
- <-Map 6 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_57]
- PartitionCols:_col0
- Select Operator [SEL_56] (rows=36524 width=1119)
- Output:["_col0","_col2"]
- Filter Operator [FIL_55] (rows=36524 width=1119)
- predicate:((d_year = 1998) and d_date_sk is not null)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_day_name"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_73]
- PartitionCols:_col0
- Select Operator [SEL_72] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_71] (rows=575995635 width=88)
- predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_10_date_dim_d_date_sk_min) AND DynamicValue(RS_10_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_10_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_13_store_s_store_sk_min) AND DynamicValue(RS_13_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_13_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_store_sk","ss_sales_price"]
- <-Reducer 7 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_62]
- Group By Operator [GBY_61] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 6 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_60]
- Group By Operator [GBY_59] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_58] (rows=36524 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_56]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_70]
- Group By Operator [GBY_69] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_68]
- Group By Operator [GBY_67] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_66] (rows=852 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_64]
+ Top N Key Operator [TNK_33] (rows=696954748 width=88)
+ keys:_col0, _col1,sort order:++,top n:100
+ Select Operator [SEL_15] (rows=696954748 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
+ Merge Join Operator [MERGEJOIN_55] (rows=696954748 width=88)
+ Conds:RS_12._col1=RS_66._col0(Inner),Output:["_col2","_col5","_col7","_col8"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_66]
+ PartitionCols:_col0
+ Select Operator [SEL_65] (rows=852 width=1910)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_64] (rows=852 width=1910)
+ predicate:((s_gmt_offset = -6) and s_store_sk is not null)
+ TableScan [TS_6] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id","s_store_name","s_gmt_offset"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_12]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_54] (rows=633595212 width=88)
+ Conds:RS_74._col0=RS_58._col0(Inner),Output:["_col1","_col2","_col5"]
+ <-Map 6 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_58]
+ PartitionCols:_col0
+ Select Operator [SEL_57] (rows=36524 width=1119)
+ Output:["_col0","_col2"]
+ Filter Operator [FIL_56] (rows=36524 width=1119)
+ predicate:((d_year = 1998) and d_date_sk is not null)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_day_name"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_74]
+ PartitionCols:_col0
+ Select Operator [SEL_73] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_72] (rows=575995635 width=88)
+ predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_10_date_dim_d_date_sk_min) AND DynamicValue(RS_10_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_10_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_13_store_s_store_sk_min) AND DynamicValue(RS_13_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_13_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_store_sk","ss_sales_price"]
+ <-Reducer 7 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_63]
+ Group By Operator [GBY_62] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 6 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_61]
+ Group By Operator [GBY_60] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_59] (rows=36524 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_57]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_71]
+ Group By Operator [GBY_70] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_69]
+ Group By Operator [GBY_68] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_67] (rows=852 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_65]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query45.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query45.q.out b/ql/src/test/results/clientpositive/perf/tez/query45.q.out
index c9d820b..edb047d 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query45.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query45.q.out
@@ -1,4 +1,4 @@
-Warning: Shuffle Join MERGEJOIN[132][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[133][tables = [$hdt$_0, $hdt$_1, $hdt$_2, $hdt$_3]] in Stage 'Reducer 4' is a cross product
PREHOOK: query: explain
select ca_zip, ca_county, sum(ws_sales_price)
from web_sales, customer, customer_address, date_dim, item
@@ -59,147 +59,149 @@ Stage-0
limit:100
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_172]
- Limit [LIM_171] (rows=100 width=152)
+ File Output Operator [FS_173]
+ Limit [LIM_172] (rows=100 width=152)
Number of rows:100
- Select Operator [SEL_170] (rows=95833781 width=152)
+ Select Operator [SEL_171] (rows=95833781 width=152)
Output:["_col0","_col1","_col2"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_169]
- Group By Operator [GBY_168] (rows=95833781 width=152)
+ SHUFFLE [RS_170]
+ Group By Operator [GBY_169] (rows=95833781 width=152)
Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_53]
PartitionCols:_col0, _col1
Group By Operator [GBY_52] (rows=191667562 width=152)
Output:["_col0","_col1","_col2"],aggregations:["sum(_col3)"],keys:_col8, _col7
- Select Operator [SEL_51] (rows=191667562 width=152)
- Output:["_col3","_col7","_col8"]
- Filter Operator [FIL_50] (rows=191667562 width=152)
- predicate:((substr(_col8, 1, 5)) IN ('85669', '86197', '88274', '83405', '86475', '85392', '85460', '80348', '81792') or CASE WHEN ((_col14 = 0L)) THEN (false) WHEN (_col17 is not null) THEN (true) WHEN (_col13 is null) THEN (null) WHEN ((_col15 < _col14)) THEN (null) ELSE (false) END)
- Select Operator [SEL_49] (rows=191667562 width=152)
- Output:["_col3","_col7","_col8","_col13","_col14","_col15","_col17"]
- Merge Join Operator [MERGEJOIN_132] (rows=191667562 width=152)
- Conds:(Inner),Output:["_col3","_col4","_col6","_col8","_col12","_col16","_col17"]
- <-Reducer 13 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_167]
- Group By Operator [GBY_166] (rows=1 width=16)
- Output:["_col0","_col1"],aggregations:["count(VALUE._col0)","count(VALUE._col1)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_149]
- Group By Operator [GBY_147] (rows=1 width=16)
- Output:["_col0","_col1"],aggregations:["count()","count(i_item_id)"]
- Select Operator [SEL_144] (rows=231000 width=1436)
- Output:["i_item_id"]
- Filter Operator [FIL_141] (rows=231000 width=1436)
- predicate:(i_item_sk) IN (2, 3, 5, 7, 11, 13, 17, 19, 23, 29)
- TableScan [TS_6] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
- <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_46]
- Merge Join Operator [MERGEJOIN_131] (rows=191667562 width=135)
- Conds:RS_43._col0=RS_44._col6(Inner),Output:["_col3","_col4","_col6","_col8","_col12"]
- <-Reducer 10 [SIMPLE_EDGE]
- SHUFFLE [RS_44]
- PartitionCols:_col6
- Merge Join Operator [MERGEJOIN_130] (rows=174243235 width=135)
- Conds:RS_29._col0=RS_30._col1(Inner),Output:["_col1","_col3","_col6","_col7"]
- <-Reducer 9 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_29]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_128] (rows=508200 width=1436)
- Conds:RS_145._col1=RS_152._col0(Left Outer),Output:["_col0","_col1","_col3"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_145]
- PartitionCols:_col1
- Select Operator [SEL_142] (rows=462000 width=1436)
- Output:["_col0","_col1"]
- Filter Operator [FIL_139] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- Please refer to the previous TableScan [TS_6]
- <-Reducer 12 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_152]
- PartitionCols:_col0
- Select Operator [SEL_151] (rows=115500 width=1436)
- Output:["_col0","_col1"]
- Group By Operator [GBY_150] (rows=115500 width=1436)
- Output:["_col0"],keys:KEY._col0
- <-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_148]
- PartitionCols:_col0
- Group By Operator [GBY_146] (rows=231000 width=1436)
- Output:["_col0"],keys:i_item_id
- Select Operator [SEL_143] (rows=231000 width=1436)
- Output:["i_item_id"]
- Filter Operator [FIL_140] (rows=231000 width=1436)
- predicate:(i_item_sk) IN (2, 3, 5, 7, 11, 13, 17, 19, 23, 29)
- Please refer to the previous TableScan [TS_6]
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_30]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_129] (rows=158402938 width=135)
- Conds:RS_165._col0=RS_155._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 16 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_155]
- PartitionCols:_col0
- Select Operator [SEL_154] (rows=18262 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_153] (rows=18262 width=1119)
- predicate:((d_qoy = 2) and (d_year = 2000) and d_date_sk is not null)
- TableScan [TS_19] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_165]
- PartitionCols:_col0
- Select Operator [SEL_164] (rows=144002668 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_163] (rows=144002668 width=135)
- predicate:((ws_item_sk BETWEEN DynamicValue(RS_29_item_i_item_sk_min) AND DynamicValue(RS_29_item_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_29_item_i_item_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_23_date_dim_d_date_sk_min) AND DynamicValue(RS_23_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_23_date_dim_d_date_sk_bloom_filter))) and ws_bill_customer_sk is not null and ws_item_sk is not null and ws_sold_date_sk is not null)
- TableScan [TS_16] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_bill_customer_sk","ws_sales_price"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_162]
- Group By Operator [GBY_161] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Reducer 9 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_114]
- Group By Operator [GBY_113] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_112] (rows=508200 width=1436)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_128]
- <-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_160]
- Group By Operator [GBY_159] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 16 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_158]
- Group By Operator [GBY_157] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_156] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_154]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_43]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_127] (rows=88000001 width=860)
- Conds:RS_135._col1=RS_138._col0(Inner),Output:["_col0","_col3","_col4"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_135]
- PartitionCols:_col1
- Select Operator [SEL_134] (rows=80000000 width=860)
- Output:["_col0","_col1"]
- Filter Operator [FIL_133] (rows=80000000 width=860)
- predicate:(c_current_addr_sk is not null and c_customer_sk is not null)
- TableScan [TS_0] (rows=80000000 width=860)
- default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_addr_sk"]
- <-Map 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_138]
- PartitionCols:_col0
- Select Operator [SEL_137] (rows=40000000 width=1014)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_136] (rows=40000000 width=1014)
- predicate:ca_address_sk is not null
- TableScan [TS_3] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_county","ca_zip"]
+ Top N Key Operator [TNK_82] (rows=191667562 width=152)
+ keys:_col8, _col7,sort order:++,top n:100
+ Select Operator [SEL_51] (rows=191667562 width=152)
+ Output:["_col3","_col7","_col8"]
+ Filter Operator [FIL_50] (rows=191667562 width=152)
+ predicate:((substr(_col8, 1, 5)) IN ('85669', '86197', '88274', '83405', '86475', '85392', '85460', '80348', '81792') or CASE WHEN ((_col14 = 0L)) THEN (false) WHEN (_col17 is not null) THEN (true) WHEN (_col13 is null) THEN (null) WHEN ((_col15 < _col14)) THEN (null) ELSE (false) END)
+ Select Operator [SEL_49] (rows=191667562 width=152)
+ Output:["_col3","_col7","_col8","_col13","_col14","_col15","_col17"]
+ Merge Join Operator [MERGEJOIN_133] (rows=191667562 width=152)
+ Conds:(Inner),Output:["_col3","_col4","_col6","_col8","_col12","_col16","_col17"]
+ <-Reducer 13 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_168]
+ Group By Operator [GBY_167] (rows=1 width=16)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_150]
+ Group By Operator [GBY_148] (rows=1 width=16)
+ Output:["_col0","_col1"],aggregations:["count()","count(i_item_id)"]
+ Select Operator [SEL_145] (rows=231000 width=1436)
+ Output:["i_item_id"]
+ Filter Operator [FIL_142] (rows=231000 width=1436)
+ predicate:(i_item_sk) IN (2, 3, 5, 7, 11, 13, 17, 19, 23, 29)
+ TableScan [TS_6] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
+ <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_46]
+ Merge Join Operator [MERGEJOIN_132] (rows=191667562 width=135)
+ Conds:RS_43._col0=RS_44._col6(Inner),Output:["_col3","_col4","_col6","_col8","_col12"]
+ <-Reducer 10 [SIMPLE_EDGE]
+ SHUFFLE [RS_44]
+ PartitionCols:_col6
+ Merge Join Operator [MERGEJOIN_131] (rows=174243235 width=135)
+ Conds:RS_29._col0=RS_30._col1(Inner),Output:["_col1","_col3","_col6","_col7"]
+ <-Reducer 9 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_29]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_129] (rows=508200 width=1436)
+ Conds:RS_146._col1=RS_153._col0(Left Outer),Output:["_col0","_col1","_col3"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_146]
+ PartitionCols:_col1
+ Select Operator [SEL_143] (rows=462000 width=1436)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_140] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ Please refer to the previous TableScan [TS_6]
+ <-Reducer 12 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_153]
+ PartitionCols:_col0
+ Select Operator [SEL_152] (rows=115500 width=1436)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_151] (rows=115500 width=1436)
+ Output:["_col0"],keys:KEY._col0
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_149]
+ PartitionCols:_col0
+ Group By Operator [GBY_147] (rows=231000 width=1436)
+ Output:["_col0"],keys:i_item_id
+ Select Operator [SEL_144] (rows=231000 width=1436)
+ Output:["i_item_id"]
+ Filter Operator [FIL_141] (rows=231000 width=1436)
+ predicate:(i_item_sk) IN (2, 3, 5, 7, 11, 13, 17, 19, 23, 29)
+ Please refer to the previous TableScan [TS_6]
+ <-Reducer 15 [SIMPLE_EDGE]
+ SHUFFLE [RS_30]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_130] (rows=158402938 width=135)
+ Conds:RS_166._col0=RS_156._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 16 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_156]
+ PartitionCols:_col0
+ Select Operator [SEL_155] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_154] (rows=18262 width=1119)
+ predicate:((d_qoy = 2) and (d_year = 2000) and d_date_sk is not null)
+ TableScan [TS_19] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_166]
+ PartitionCols:_col0
+ Select Operator [SEL_165] (rows=144002668 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_164] (rows=144002668 width=135)
+ predicate:((ws_item_sk BETWEEN DynamicValue(RS_29_item_i_item_sk_min) AND DynamicValue(RS_29_item_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_29_item_i_item_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_23_date_dim_d_date_sk_min) AND DynamicValue(RS_23_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_23_date_dim_d_date_sk_bloom_filter))) and ws_bill_customer_sk is not null and ws_item_sk is not null and ws_sold_date_sk is not null)
+ TableScan [TS_16] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_bill_customer_sk","ws_sales_price"]
+ <-Reducer 11 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_163]
+ Group By Operator [GBY_162] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Reducer 9 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_115]
+ Group By Operator [GBY_114] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_113] (rows=508200 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_129]
+ <-Reducer 17 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_161]
+ Group By Operator [GBY_160] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 16 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_159]
+ Group By Operator [GBY_158] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_157] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_155]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_43]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_128] (rows=88000001 width=860)
+ Conds:RS_136._col1=RS_139._col0(Inner),Output:["_col0","_col3","_col4"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_136]
+ PartitionCols:_col1
+ Select Operator [SEL_135] (rows=80000000 width=860)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_134] (rows=80000000 width=860)
+ predicate:(c_current_addr_sk is not null and c_customer_sk is not null)
+ TableScan [TS_0] (rows=80000000 width=860)
+ default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_addr_sk"]
+ <-Map 7 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_139]
+ PartitionCols:_col0
+ Select Operator [SEL_138] (rows=40000000 width=1014)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_137] (rows=40000000 width=1014)
+ predicate:ca_address_sk is not null
+ TableScan [TS_3] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_county","ca_zip"]
[02/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query80.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query80.q.out b/ql/src/test/results/clientpositive/perf/tez/query80.q.out
index ee8cdd8..e02f45f 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query80.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query80.q.out
@@ -232,394 +232,400 @@ Stage-0
limit:100
Stage-1
Reducer 10 vectorized
- File Output Operator [FS_460]
- Limit [LIM_459] (rows=100 width=108)
+ File Output Operator [FS_465]
+ Limit [LIM_464] (rows=100 width=108)
Number of rows:100
- Select Operator [SEL_458] (rows=1217531358 width=108)
+ Select Operator [SEL_463] (rows=1217531358 width=108)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_457]
- Select Operator [SEL_456] (rows=1217531358 width=108)
+ SHUFFLE [RS_462]
+ Select Operator [SEL_461] (rows=1217531358 width=108)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_455] (rows=1217531358 width=108)
+ Group By Operator [GBY_460] (rows=1217531358 width=108)
Output:["_col0","_col1","_col3","_col4","_col5"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 8 [SIMPLE_EDGE]
<-Reducer 18 [CONTAINS] vectorized
- Reduce Output Operator [RS_484]
+ Reduce Output Operator [RS_490]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_483] (rows=2435062716 width=108)
+ Group By Operator [GBY_489] (rows=2435062716 width=108)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_482] (rows=231905279 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_481] (rows=231905279 width=135)
- Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
- <-Reducer 17 [SIMPLE_EDGE]
- SHUFFLE [RS_75]
- PartitionCols:_col0
- Group By Operator [GBY_74] (rows=463810558 width=135)
- Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
- Select Operator [SEL_72] (rows=463810558 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_368] (rows=463810558 width=135)
- Conds:RS_69._col1=RS_469._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col18"]
- <-Map 39 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_469]
- PartitionCols:_col0
- Select Operator [SEL_468] (rows=46000 width=460)
- Output:["_col0","_col1"]
- Filter Operator [FIL_467] (rows=46000 width=460)
- predicate:cp_catalog_page_sk is not null
- TableScan [TS_54] (rows=46000 width=460)
- default@catalog_page,catalog_page,Tbl:COMPLETE,Col:NONE,Output:["cp_catalog_page_sk","cp_catalog_page_id"]
- <-Reducer 16 [SIMPLE_EDGE]
- SHUFFLE [RS_69]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_367] (rows=421645953 width=135)
- Conds:RS_66._col3=RS_425._col0(Inner),Output:["_col1","_col5","_col6","_col9","_col10"]
- <-Map 30 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_425]
- PartitionCols:_col0
- Select Operator [SEL_422] (rows=1150 width=1179)
- Output:["_col0"]
- Filter Operator [FIL_421] (rows=1150 width=1179)
- predicate:((p_channel_tv = 'N') and p_promo_sk is not null)
- TableScan [TS_12] (rows=2300 width=1179)
- default@promotion,promotion,Tbl:COMPLETE,Col:NONE,Output:["p_promo_sk","p_channel_tv"]
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_66]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_366] (rows=383314495 width=135)
- Conds:RS_63._col2=RS_409._col0(Inner),Output:["_col1","_col3","_col5","_col6","_col9","_col10"]
- <-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_409]
- PartitionCols:_col0
- Select Operator [SEL_406] (rows=154000 width=1436)
- Output:["_col0"]
- Filter Operator [FIL_405] (rows=154000 width=1436)
- predicate:((i_current_price > 50) and i_item_sk is not null)
- TableScan [TS_9] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_current_price"]
- <-Reducer 14 [SIMPLE_EDGE]
- SHUFFLE [RS_63]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_365] (rows=348467716 width=135)
- Conds:RS_60._col0=RS_393._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_393]
- PartitionCols:_col0
- Select Operator [SEL_390] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_389] (rows=8116 width=1119)
- predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-08-04 00:00:00' AND TIMESTAMP'1998-09-03 00:00:00' and d_date_sk is not null)
- TableScan [TS_6] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
- <-Reducer 37 [SIMPLE_EDGE]
- SHUFFLE [RS_60]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_364] (rows=316788826 width=135)
- Conds:RS_477._col2, _col4=RS_480._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
- <-Map 36 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_477]
- PartitionCols:_col2, _col4
- Select Operator [SEL_476] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_475] (rows=287989836 width=135)
- predicate:((cs_catalog_page_sk BETWEEN DynamicValue(RS_70_catalog_page_cp_catalog_page_sk_min) AND DynamicValue(RS_70_catalog_page_cp_catalog_page_sk_max) and in_bloom_filter(cs_catalog_page_sk, DynamicValue(RS_70_catalog_page_cp_catalog_page_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_64_item_i_item_sk_min) AND DynamicValue(RS_64_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_64_item_i_item_sk_bloom_filter))) and (cs_promo_sk BETWEEN DynamicValue(RS_67_promotion_p_promo_sk_min) AND DynamicValue(RS_67_promotion_p_promo_sk_max) and in_bloom_filter(cs_promo_sk, DynamicValue(RS_67_promotion_p_promo_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_61_date_dim_d_date_sk_min) AND DynamicValue(RS_61_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_61_date_dim_d_date_sk_bloom_filter))) and cs_catalog_page_sk is not null and cs_item_sk is not null and cs_pr
omo_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_39] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_catalog_page_sk","cs_item_sk","cs_promo_sk","cs_order_number","cs_ext_sales_price","cs_net_profit"]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_462]
- Group By Operator [GBY_461] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_401]
- Group By Operator [GBY_398] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_394] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_390]
- <-Reducer 28 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_464]
- Group By Operator [GBY_463] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 26 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_417]
- Group By Operator [GBY_414] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_410] (rows=154000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_406]
- <-Reducer 32 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_466]
- Group By Operator [GBY_465] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 30 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_433]
- Group By Operator [GBY_430] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_426] (rows=1150 width=1179)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_422]
- <-Reducer 40 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_474]
- Group By Operator [GBY_473] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 39 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_472]
- Group By Operator [GBY_471] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_470] (rows=46000 width=460)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_468]
- <-Map 38 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_480]
- PartitionCols:_col0, _col1
- Select Operator [SEL_479] (rows=28798881 width=106)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_478] (rows=28798881 width=106)
- predicate:cr_item_sk is not null
- TableScan [TS_42] (rows=28798881 width=106)
- default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_item_sk","cr_order_number","cr_return_amount","cr_net_loss"]
+ Top N Key Operator [TNK_488] (rows=811687572 width=108)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_487] (rows=231905279 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Group By Operator [GBY_486] (rows=231905279 width=135)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
+ <-Reducer 17 [SIMPLE_EDGE]
+ SHUFFLE [RS_75]
+ PartitionCols:_col0
+ Group By Operator [GBY_74] (rows=463810558 width=135)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
+ Select Operator [SEL_72] (rows=463810558 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_369] (rows=463810558 width=135)
+ Conds:RS_69._col1=RS_474._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col18"]
+ <-Map 39 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_474]
+ PartitionCols:_col0
+ Select Operator [SEL_473] (rows=46000 width=460)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_472] (rows=46000 width=460)
+ predicate:cp_catalog_page_sk is not null
+ TableScan [TS_54] (rows=46000 width=460)
+ default@catalog_page,catalog_page,Tbl:COMPLETE,Col:NONE,Output:["cp_catalog_page_sk","cp_catalog_page_id"]
+ <-Reducer 16 [SIMPLE_EDGE]
+ SHUFFLE [RS_69]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_368] (rows=421645953 width=135)
+ Conds:RS_66._col3=RS_429._col0(Inner),Output:["_col1","_col5","_col6","_col9","_col10"]
+ <-Map 30 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_429]
+ PartitionCols:_col0
+ Select Operator [SEL_426] (rows=1150 width=1179)
+ Output:["_col0"]
+ Filter Operator [FIL_425] (rows=1150 width=1179)
+ predicate:((p_channel_tv = 'N') and p_promo_sk is not null)
+ TableScan [TS_12] (rows=2300 width=1179)
+ default@promotion,promotion,Tbl:COMPLETE,Col:NONE,Output:["p_promo_sk","p_channel_tv"]
+ <-Reducer 15 [SIMPLE_EDGE]
+ SHUFFLE [RS_66]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_367] (rows=383314495 width=135)
+ Conds:RS_63._col2=RS_413._col0(Inner),Output:["_col1","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 26 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_413]
+ PartitionCols:_col0
+ Select Operator [SEL_410] (rows=154000 width=1436)
+ Output:["_col0"]
+ Filter Operator [FIL_409] (rows=154000 width=1436)
+ predicate:((i_current_price > 50) and i_item_sk is not null)
+ TableScan [TS_9] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_current_price"]
+ <-Reducer 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_63]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_366] (rows=348467716 width=135)
+ Conds:RS_60._col0=RS_397._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_397]
+ PartitionCols:_col0
+ Select Operator [SEL_394] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_393] (rows=8116 width=1119)
+ predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-08-04 00:00:00' AND TIMESTAMP'1998-09-03 00:00:00' and d_date_sk is not null)
+ TableScan [TS_6] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
+ <-Reducer 37 [SIMPLE_EDGE]
+ SHUFFLE [RS_60]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_365] (rows=316788826 width=135)
+ Conds:RS_482._col2, _col4=RS_485._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 36 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_482]
+ PartitionCols:_col2, _col4
+ Select Operator [SEL_481] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
+ Filter Operator [FIL_480] (rows=287989836 width=135)
+ predicate:((cs_catalog_page_sk BETWEEN DynamicValue(RS_70_catalog_page_cp_catalog_page_sk_min) AND DynamicValue(RS_70_catalog_page_cp_catalog_page_sk_max) and in_bloom_filter(cs_catalog_page_sk, DynamicValue(RS_70_catalog_page_cp_catalog_page_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_64_item_i_item_sk_min) AND DynamicValue(RS_64_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_64_item_i_item_sk_bloom_filter))) and (cs_promo_sk BETWEEN DynamicValue(RS_67_promotion_p_promo_sk_min) AND DynamicValue(RS_67_promotion_p_promo_sk_max) and in_bloom_filter(cs_promo_sk, DynamicValue(RS_67_promotion_p_promo_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_61_date_dim_d_date_sk_min) AND DynamicValue(RS_61_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_61_date_dim_d_date_sk_bloom_filter))) and cs_catalog_page_sk is not null and cs_item_sk is not null and cs_
promo_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_39] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_catalog_page_sk","cs_item_sk","cs_promo_sk","cs_order_number","cs_ext_sales_price","cs_net_profit"]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_467]
+ Group By Operator [GBY_466] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_405]
+ Group By Operator [GBY_402] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_398] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_394]
+ <-Reducer 28 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_469]
+ Group By Operator [GBY_468] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 26 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_421]
+ Group By Operator [GBY_418] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_414] (rows=154000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_410]
+ <-Reducer 32 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_471]
+ Group By Operator [GBY_470] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 30 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_437]
+ Group By Operator [GBY_434] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_430] (rows=1150 width=1179)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_426]
+ <-Reducer 40 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_479]
+ Group By Operator [GBY_478] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 39 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_477]
+ Group By Operator [GBY_476] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_475] (rows=46000 width=460)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_473]
+ <-Map 38 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_485]
+ PartitionCols:_col0, _col1
+ Select Operator [SEL_484] (rows=28798881 width=106)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_483] (rows=28798881 width=106)
+ predicate:cr_item_sk is not null
+ TableScan [TS_42] (rows=28798881 width=106)
+ default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_item_sk","cr_order_number","cr_return_amount","cr_net_loss"]
<-Reducer 24 [CONTAINS] vectorized
- Reduce Output Operator [RS_508]
+ Reduce Output Operator [RS_515]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_507] (rows=2435062716 width=108)
+ Group By Operator [GBY_514] (rows=2435062716 width=108)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_506] (rows=115958879 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_505] (rows=115958879 width=135)
- Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
- <-Reducer 23 [SIMPLE_EDGE]
- SHUFFLE [RS_115]
- PartitionCols:_col0
- Group By Operator [GBY_114] (rows=231917759 width=135)
- Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
- Select Operator [SEL_112] (rows=231917759 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_373] (rows=231917759 width=135)
- Conds:RS_109._col2=RS_493._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col18"]
- <-Map 44 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_493]
- PartitionCols:_col0
- Select Operator [SEL_492] (rows=84 width=1850)
- Output:["_col0","_col1"]
- Filter Operator [FIL_491] (rows=84 width=1850)
- predicate:web_site_sk is not null
- TableScan [TS_94] (rows=84 width=1850)
- default@web_site,web_site,Tbl:COMPLETE,Col:NONE,Output:["web_site_sk","web_site_id"]
- <-Reducer 22 [SIMPLE_EDGE]
- SHUFFLE [RS_109]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_372] (rows=210834322 width=135)
- Conds:RS_106._col3=RS_427._col0(Inner),Output:["_col2","_col5","_col6","_col9","_col10"]
- <-Map 30 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_427]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_422]
- <-Reducer 21 [SIMPLE_EDGE]
- SHUFFLE [RS_106]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_371] (rows=191667562 width=135)
- Conds:RS_103._col1=RS_411._col0(Inner),Output:["_col2","_col3","_col5","_col6","_col9","_col10"]
- <-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_411]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_406]
- <-Reducer 20 [SIMPLE_EDGE]
- SHUFFLE [RS_103]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_370] (rows=174243235 width=135)
- Conds:RS_100._col0=RS_395._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_395]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_390]
- <-Reducer 42 [SIMPLE_EDGE]
- SHUFFLE [RS_100]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_369] (rows=158402938 width=135)
- Conds:RS_501._col1, _col4=RS_504._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
- <-Map 41 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_501]
- PartitionCols:_col1, _col4
- Select Operator [SEL_500] (rows=144002668 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_499] (rows=144002668 width=135)
- predicate:((ws_item_sk BETWEEN DynamicValue(RS_104_item_i_item_sk_min) AND DynamicValue(RS_104_item_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_104_item_i_item_sk_bloom_filter))) and (ws_promo_sk BETWEEN DynamicValue(RS_107_promotion_p_promo_sk_min) AND DynamicValue(RS_107_promotion_p_promo_sk_max) and in_bloom_filter(ws_promo_sk, DynamicValue(RS_107_promotion_p_promo_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_101_date_dim_d_date_sk_min) AND DynamicValue(RS_101_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_101_date_dim_d_date_sk_bloom_filter))) and (ws_web_site_sk BETWEEN DynamicValue(RS_110_web_site_web_site_sk_min) AND DynamicValue(RS_110_web_site_web_site_sk_max) and in_bloom_filter(ws_web_site_sk, DynamicValue(RS_110_web_site_web_site_sk_bloom_filter))) and ws_item_sk is not null and ws_promo_sk is not null and ws_sold_date_sk is not null and ws_web_si
te_sk is not null)
- TableScan [TS_79] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_web_site_sk","ws_promo_sk","ws_order_number","ws_ext_sales_price","ws_net_profit"]
- <-Reducer 25 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_486]
- Group By Operator [GBY_485] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_402]
- Group By Operator [GBY_399] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_396] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_390]
- <-Reducer 29 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_488]
- Group By Operator [GBY_487] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 26 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_418]
- Group By Operator [GBY_415] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_412] (rows=154000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_406]
- <-Reducer 33 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_490]
- Group By Operator [GBY_489] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 30 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_434]
- Group By Operator [GBY_431] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_428] (rows=1150 width=1179)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_422]
- <-Reducer 45 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_498]
- Group By Operator [GBY_497] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 44 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_496]
- Group By Operator [GBY_495] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_494] (rows=84 width=1850)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_492]
- <-Map 43 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_504]
- PartitionCols:_col0, _col1
- Select Operator [SEL_503] (rows=14398467 width=92)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_502] (rows=14398467 width=92)
- predicate:wr_item_sk is not null
- TableScan [TS_82] (rows=14398467 width=92)
- default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_item_sk","wr_order_number","wr_return_amt","wr_net_loss"]
+ Top N Key Operator [TNK_513] (rows=811687572 width=108)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_512] (rows=115958879 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Group By Operator [GBY_511] (rows=115958879 width=135)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
+ <-Reducer 23 [SIMPLE_EDGE]
+ SHUFFLE [RS_115]
+ PartitionCols:_col0
+ Group By Operator [GBY_114] (rows=231917759 width=135)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
+ Select Operator [SEL_112] (rows=231917759 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_374] (rows=231917759 width=135)
+ Conds:RS_109._col2=RS_499._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col18"]
+ <-Map 44 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_499]
+ PartitionCols:_col0
+ Select Operator [SEL_498] (rows=84 width=1850)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_497] (rows=84 width=1850)
+ predicate:web_site_sk is not null
+ TableScan [TS_94] (rows=84 width=1850)
+ default@web_site,web_site,Tbl:COMPLETE,Col:NONE,Output:["web_site_sk","web_site_id"]
+ <-Reducer 22 [SIMPLE_EDGE]
+ SHUFFLE [RS_109]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_373] (rows=210834322 width=135)
+ Conds:RS_106._col3=RS_431._col0(Inner),Output:["_col2","_col5","_col6","_col9","_col10"]
+ <-Map 30 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_431]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_426]
+ <-Reducer 21 [SIMPLE_EDGE]
+ SHUFFLE [RS_106]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_372] (rows=191667562 width=135)
+ Conds:RS_103._col1=RS_415._col0(Inner),Output:["_col2","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 26 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_415]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_410]
+ <-Reducer 20 [SIMPLE_EDGE]
+ SHUFFLE [RS_103]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_371] (rows=174243235 width=135)
+ Conds:RS_100._col0=RS_399._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_399]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_394]
+ <-Reducer 42 [SIMPLE_EDGE]
+ SHUFFLE [RS_100]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_370] (rows=158402938 width=135)
+ Conds:RS_507._col1, _col4=RS_510._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 41 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_507]
+ PartitionCols:_col1, _col4
+ Select Operator [SEL_506] (rows=144002668 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
+ Filter Operator [FIL_505] (rows=144002668 width=135)
+ predicate:((ws_item_sk BETWEEN DynamicValue(RS_104_item_i_item_sk_min) AND DynamicValue(RS_104_item_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_104_item_i_item_sk_bloom_filter))) and (ws_promo_sk BETWEEN DynamicValue(RS_107_promotion_p_promo_sk_min) AND DynamicValue(RS_107_promotion_p_promo_sk_max) and in_bloom_filter(ws_promo_sk, DynamicValue(RS_107_promotion_p_promo_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_101_date_dim_d_date_sk_min) AND DynamicValue(RS_101_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_101_date_dim_d_date_sk_bloom_filter))) and (ws_web_site_sk BETWEEN DynamicValue(RS_110_web_site_web_site_sk_min) AND DynamicValue(RS_110_web_site_web_site_sk_max) and in_bloom_filter(ws_web_site_sk, DynamicValue(RS_110_web_site_web_site_sk_bloom_filter))) and ws_item_sk is not null and ws_promo_sk is not null and ws_sold_date_sk is not null and ws_web_
site_sk is not null)
+ TableScan [TS_79] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_web_site_sk","ws_promo_sk","ws_order_number","ws_ext_sales_price","ws_net_profit"]
+ <-Reducer 25 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_492]
+ Group By Operator [GBY_491] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_406]
+ Group By Operator [GBY_403] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_400] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_394]
+ <-Reducer 29 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_494]
+ Group By Operator [GBY_493] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 26 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_422]
+ Group By Operator [GBY_419] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_416] (rows=154000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_410]
+ <-Reducer 33 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_496]
+ Group By Operator [GBY_495] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 30 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_438]
+ Group By Operator [GBY_435] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_432] (rows=1150 width=1179)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_426]
+ <-Reducer 45 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_504]
+ Group By Operator [GBY_503] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 44 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_502]
+ Group By Operator [GBY_501] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_500] (rows=84 width=1850)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_498]
+ <-Map 43 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_510]
+ PartitionCols:_col0, _col1
+ Select Operator [SEL_509] (rows=14398467 width=92)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_508] (rows=14398467 width=92)
+ predicate:wr_item_sk is not null
+ TableScan [TS_82] (rows=14398467 width=92)
+ default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_item_sk","wr_order_number","wr_return_amt","wr_net_loss"]
<-Reducer 7 [CONTAINS] vectorized
- Reduce Output Operator [RS_454]
+ Reduce Output Operator [RS_459]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_453] (rows=2435062716 width=108)
+ Group By Operator [GBY_458] (rows=2435062716 width=108)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_452] (rows=463823414 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_451] (rows=463823414 width=88)
- Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
- <-Reducer 6 [SIMPLE_EDGE]
- SHUFFLE [RS_36]
- PartitionCols:_col0
- Group By Operator [GBY_35] (rows=927646829 width=88)
- Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
- Select Operator [SEL_33] (rows=927646829 width=88)
- Output:["_col0","_col1","_col2","_col3"]
- Merge Join Operator [MERGEJOIN_363] (rows=927646829 width=88)
- Conds:RS_30._col2=RS_439._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col18"]
- <-Map 34 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_439]
- PartitionCols:_col0
- Select Operator [SEL_438] (rows=1704 width=1910)
- Output:["_col0","_col1"]
- Filter Operator [FIL_437] (rows=1704 width=1910)
- predicate:s_store_sk is not null
- TableScan [TS_15] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id"]
- <-Reducer 5 [SIMPLE_EDGE]
- SHUFFLE [RS_30]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_362] (rows=843315281 width=88)
- Conds:RS_27._col3=RS_423._col0(Inner),Output:["_col2","_col5","_col6","_col9","_col10"]
- <-Map 30 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_423]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_422]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_27]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_361] (rows=766650239 width=88)
- Conds:RS_24._col1=RS_407._col0(Inner),Output:["_col2","_col3","_col5","_col6","_col9","_col10"]
- <-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_407]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_406]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_24]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_360] (rows=696954748 width=88)
- Conds:RS_21._col0=RS_391._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_391]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_390]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_359] (rows=633595212 width=88)
- Conds:RS_447._col1, _col4=RS_450._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_447]
- PartitionCols:_col1, _col4
- Select Operator [SEL_446] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Filter Operator [FIL_445] (rows=575995635 width=88)
- predicate:((ss_item_sk BETWEEN DynamicValue(RS_25_item_i_item_sk_min) AND DynamicValue(RS_25_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_25_item_i_item_sk_bloom_filter))) and (ss_promo_sk BETWEEN DynamicValue(RS_28_promotion_p_promo_sk_min) AND DynamicValue(RS_28_promotion_p_promo_sk_max) and in_bloom_filter(ss_promo_sk, DynamicValue(RS_28_promotion_p_promo_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_22_date_dim_d_date_sk_min) AND DynamicValue(RS_22_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_22_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_31_store_s_store_sk_min) AND DynamicValue(RS_31_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_31_store_s_store_sk_bloom_filter))) and ss_item_sk is not null and ss_promo_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_store_sk","ss_promo_sk","ss_ticket_number","ss_ext_sales_price","ss_net_profit"]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_404]
- Group By Operator [GBY_403] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_400]
- Group By Operator [GBY_397] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_392] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_390]
- <-Reducer 27 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_420]
- Group By Operator [GBY_419] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 26 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_416]
- Group By Operator [GBY_413] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_408] (rows=154000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_406]
- <-Reducer 31 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_436]
- Group By Operator [GBY_435] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 30 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_432]
- Group By Operator [GBY_429] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_424] (rows=1150 width=1179)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_422]
- <-Reducer 35 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_444]
- Group By Operator [GBY_443] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 34 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_442]
- Group By Operator [GBY_441] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_440] (rows=1704 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_438]
- <-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_450]
- PartitionCols:_col0, _col1
- Select Operator [SEL_449] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_448] (rows=57591150 width=77)
- predicate:sr_item_sk is not null
- TableScan [TS_3] (rows=57591150 width=77)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_item_sk","sr_ticket_number","sr_return_amt","sr_net_loss"]
+ Top N Key Operator [TNK_457] (rows=811687572 width=108)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_456] (rows=463823414 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Group By Operator [GBY_455] (rows=463823414 width=88)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0
+ <-Reducer 6 [SIMPLE_EDGE]
+ SHUFFLE [RS_36]
+ PartitionCols:_col0
+ Group By Operator [GBY_35] (rows=927646829 width=88)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)"],keys:_col0
+ Select Operator [SEL_33] (rows=927646829 width=88)
+ Output:["_col0","_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_364] (rows=927646829 width=88)
+ Conds:RS_30._col2=RS_443._col0(Inner),Output:["_col5","_col6","_col9","_col10","_col18"]
+ <-Map 34 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_443]
+ PartitionCols:_col0
+ Select Operator [SEL_442] (rows=1704 width=1910)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_441] (rows=1704 width=1910)
+ predicate:s_store_sk is not null
+ TableScan [TS_15] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id"]
+ <-Reducer 5 [SIMPLE_EDGE]
+ SHUFFLE [RS_30]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_363] (rows=843315281 width=88)
+ Conds:RS_27._col3=RS_427._col0(Inner),Output:["_col2","_col5","_col6","_col9","_col10"]
+ <-Map 30 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_427]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_426]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_27]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_362] (rows=766650239 width=88)
+ Conds:RS_24._col1=RS_411._col0(Inner),Output:["_col2","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 26 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_411]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_410]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_24]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_361] (rows=696954748 width=88)
+ Conds:RS_21._col0=RS_395._col0(Inner),Output:["_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_395]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_394]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_360] (rows=633595212 width=88)
+ Conds:RS_451._col1, _col4=RS_454._col0, _col1(Left Outer),Output:["_col0","_col1","_col2","_col3","_col5","_col6","_col9","_col10"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_451]
+ PartitionCols:_col1, _col4
+ Select Operator [SEL_450] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
+ Filter Operator [FIL_449] (rows=575995635 width=88)
+ predicate:((ss_item_sk BETWEEN DynamicValue(RS_25_item_i_item_sk_min) AND DynamicValue(RS_25_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_25_item_i_item_sk_bloom_filter))) and (ss_promo_sk BETWEEN DynamicValue(RS_28_promotion_p_promo_sk_min) AND DynamicValue(RS_28_promotion_p_promo_sk_max) and in_bloom_filter(ss_promo_sk, DynamicValue(RS_28_promotion_p_promo_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_22_date_dim_d_date_sk_min) AND DynamicValue(RS_22_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_22_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_31_store_s_store_sk_min) AND DynamicValue(RS_31_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_31_store_s_store_sk_bloom_filter))) and ss_item_sk is not null and ss_promo_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_store_sk","ss_promo_sk","ss_ticket_number","ss_ext_sales_price","ss_net_profit"]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_408]
+ Group By Operator [GBY_407] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_404]
+ Group By Operator [GBY_401] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_396] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_394]
+ <-Reducer 27 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_424]
+ Group By Operator [GBY_423] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 26 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_420]
+ Group By Operator [GBY_417] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_412] (rows=154000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_410]
+ <-Reducer 31 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_440]
+ Group By Operator [GBY_439] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 30 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_436]
+ Group By Operator [GBY_433] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_428] (rows=1150 width=1179)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_426]
+ <-Reducer 35 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_448]
+ Group By Operator [GBY_447] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 34 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_446]
+ Group By Operator [GBY_445] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_444] (rows=1704 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_442]
+ <-Map 11 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_454]
+ PartitionCols:_col0, _col1
+ Select Operator [SEL_453] (rows=57591150 width=77)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_452] (rows=57591150 width=77)
+ predicate:sr_item_sk is not null
+ TableScan [TS_3] (rows=57591150 width=77)
+ default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_item_sk","sr_ticket_number","sr_return_amt","sr_net_loss"]
[03/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query8.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query8.q.out b/ql/src/test/results/clientpositive/perf/tez/query8.q.out
index 26c7d8b..ee20e61 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query8.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query8.q.out
@@ -234,150 +234,152 @@ Stage-0
limit:100
Stage-1
Reducer 5 vectorized
- File Output Operator [FS_150]
- Limit [LIM_149] (rows=100 width=88)
+ File Output Operator [FS_151]
+ Limit [LIM_150] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_148] (rows=348477374 width=88)
+ Select Operator [SEL_149] (rows=348477374 width=88)
Output:["_col0","_col1"]
<-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_147]
- Group By Operator [GBY_146] (rows=348477374 width=88)
+ SHUFFLE [RS_148]
+ Group By Operator [GBY_147] (rows=348477374 width=88)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_57]
PartitionCols:_col0
Group By Operator [GBY_56] (rows=696954748 width=88)
Output:["_col0","_col1"],aggregations:["sum(_col2)"],keys:_col8
- Merge Join Operator [MERGEJOIN_117] (rows=696954748 width=88)
- Conds:RS_52._col1=RS_53._col1(Inner),Output:["_col2","_col8"]
- <-Reducer 12 [SIMPLE_EDGE]
- SHUFFLE [RS_53]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_116] (rows=1874 width=1911)
- Conds:RS_137.substr(_col0, 1, 2)=RS_140.substr(_col2, 1, 2)(Inner),Output:["_col1","_col2"]
- <-Map 19 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_140]
- PartitionCols:substr(_col2, 1, 2)
- Select Operator [SEL_139] (rows=1704 width=1910)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_138] (rows=1704 width=1910)
- predicate:(s_store_sk is not null and substr(s_zip, 1, 2) is not null)
- TableScan [TS_42] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_name","s_zip"]
- <-Reducer 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_137]
- PartitionCols:substr(_col0, 1, 2)
- Select Operator [SEL_136] (rows=1 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_135] (rows=1 width=1014)
- predicate:(_col1 = 2L)
- Group By Operator [GBY_134] (rows=6833333 width=1014)
- Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
- <-Union 10 [SIMPLE_EDGE]
- <-Reducer 17 [CONTAINS] vectorized
- Reduce Output Operator [RS_171]
- PartitionCols:_col0
- Group By Operator [GBY_170] (rows=13666666 width=1014)
- Output:["_col0","_col1"],aggregations:["count(_col1)"],keys:_col0
- Group By Operator [GBY_169] (rows=3666666 width=1014)
- Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
- <-Reducer 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_168]
- PartitionCols:_col0
- Group By Operator [GBY_167] (rows=7333333 width=1014)
- Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
- Select Operator [SEL_166] (rows=7333333 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_165] (rows=7333333 width=1014)
- predicate:(_col1 > 10L)
- Group By Operator [GBY_164] (rows=22000000 width=1014)
- Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_25]
- PartitionCols:_col0
- Group By Operator [GBY_24] (rows=44000000 width=1014)
- Output:["_col0","_col1"],aggregations:["count()"],keys:_col1
- Merge Join Operator [MERGEJOIN_115] (rows=44000000 width=1014)
- Conds:RS_160._col0=RS_163._col0(Inner),Output:["_col1"]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_160]
- PartitionCols:_col0
- Select Operator [SEL_159] (rows=40000000 width=1014)
- Output:["_col0","_col1"]
- Filter Operator [FIL_158] (rows=40000000 width=1014)
- predicate:(ca_address_sk is not null and substr(substr(ca_zip, 1, 5), 1, 2) is not null)
- TableScan [TS_14] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_zip"]
- <-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_163]
- PartitionCols:_col0
- Select Operator [SEL_162] (rows=40000000 width=860)
- Output:["_col0"]
- Filter Operator [FIL_161] (rows=40000000 width=860)
- predicate:((c_preferred_cust_flag = 'Y') and c_current_addr_sk is not null)
- TableScan [TS_17] (rows=80000000 width=860)
- default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_current_addr_sk","c_preferred_cust_flag"]
- <-Reducer 9 [CONTAINS] vectorized
- Reduce Output Operator [RS_157]
- PartitionCols:_col0
- Group By Operator [GBY_156] (rows=13666666 width=1014)
- Output:["_col0","_col1"],aggregations:["count(_col1)"],keys:_col0
- Group By Operator [GBY_155] (rows=10000000 width=1014)
- Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
- <-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_154]
- PartitionCols:_col0
- Group By Operator [GBY_153] (rows=20000000 width=1014)
- Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
- Select Operator [SEL_152] (rows=20000000 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_151] (rows=20000000 width=1014)
- predicate:((substr(ca_zip, 1, 5)) IN ('89436', '30868', '65085', '22977', '83927', '77557', '58429', '40697', '80614', '10502', '32779', '91137', '61265', '98294', '17921', '18427', '21203', '59362', '87291', '84093', '21505', '17184', '10866', '67898', '25797', '28055', '18377', '80332', '74535', '21757', '29742', '90885', '29898', '17819', '40811', '25990', '47513', '89531', '91068', '10391', '18846', '99223', '82637', '41368', '83658', '86199', '81625', '26696', '89338', '88425', '32200', '81427', '19053', '77471', '36610', '99823', '43276', '41249', '48584', '83550', '82276', '18842', '78890', '14090', '38123', '40936', '34425', '19850', '43286', '80072', '79188', '54191', '11395', '50497', '84861', '90733', '21068', '57666', '37119', '25004', '57835', '70067', '62878', '95806', '19303', '18840', '19124', '29785', '16737', '16022', '49613', '89977', '68310', '60069', '98360', '48649', '39050', '41793', '25002', '27413', '39736',
'47208', '16515', '94808', '57648', '15009', '80015', '42961', '63982', '21744', '71853', '81087', '67468', '34175', '64008', '20261', '11201', '51799', '48043', '45645', '61163', '48375', '36447', '57042', '21218', '41100', '89951', '22745', '35851', '83326', '61125', '78298', '80752', '49858', '52940', '96976', '63792', '11376', '53582', '18717', '90226', '50530', '94203', '99447', '27670', '96577', '57856', '56372', '16165', '23427', '54561', '28806', '44439', '22926', '30123', '61451', '92397', '56979', '92309', '70873', '13355', '21801', '46346', '37562', '56458', '28286', '47306', '99555', '69399', '26234', '47546', '49661', '88601', '35943', '39936', '25632', '24611', '44166', '56648', '30379', '59785', '11110', '14329', '93815', '52226', '71381', '13842', '25612', '63294', '14664', '21077', '82626', '18799', '60915', '81020', '56447', '76619', '11433', '13414', '42548', '92713', '70467', '30884', '47484', '16072', '38936', '13036', '88376', '45539', '35901', '19506', '65690'
, '73957', '71850', '49231', '14276', '20005', '18384', '76615', '11635', '38177', '55607', '41369', '95447', '58581', '58149', '91946', '33790', '76232', '75692', '95464', '22246', '51061', '56692', '53121', '77209', '15482', '10688', '14868', '45907', '73520', '72666', '25734', '17959', '24677', '66446', '94627', '53535', '15560', '41967', '69297', '11929', '59403', '33283', '52232', '57350', '43933', '40921', '36635', '10827', '71286', '19736', '80619', '25251', '95042', '15526', '36496', '55854', '49124', '81980', '35375', '49157', '63512', '28944', '14946', '36503', '54010', '18767', '23969', '43905', '66979', '33113', '21286', '58471', '59080', '13395', '79144', '70373', '67031', '38360', '26705', '50906', '52406', '26066', '73146', '15884', '31897', '30045', '61068', '45550', '92454', '13376', '14354', '19770', '22928', '97790', '50723', '46081', '30202', '14410', '20223', '88500', '67298', '13261', '14172', '81410', '93578', '83583', '46047', '94167', '82564', '21156', '1579
9', '86709', '37931', '74703', '83103', '23054', '70470', '72008', '49247', '91911', '69998', '20961', '70070', '63197', '54853', '88191', '91830', '49521', '19454', '81450', '89091', '62378', '25683', '61869', '51744', '36580', '85778', '36871', '48121', '28810', '83712', '45486', '67393', '26935', '42393', '20132', '55349', '86057', '21309', '80218', '10094', '11357', '48819', '39734', '40758', '30432', '21204', '29467', '30214', '61024', '55307', '74621', '11622', '68908', '33032', '52868', '99194', '99900', '84936', '69036', '99149', '45013', '32895', '59004', '32322', '14933', '32936', '33562', '72550', '27385', '58049', '58200', '16808', '21360', '32961', '18586', '79307', '15492') and substr(substr(ca_zip, 1, 5), 1, 2) is not null)
- TableScan [TS_6] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_zip"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_52]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_114] (rows=633595212 width=88)
- Conds:RS_145._col0=RS_128._col0(Inner),Output:["_col1","_col2"]
- <-Map 6 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_128]
- PartitionCols:_col0
- Select Operator [SEL_127] (rows=18262 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_126] (rows=18262 width=1119)
- predicate:((d_qoy = 1) and (d_year = 2002) and d_date_sk is not null)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_145]
- PartitionCols:_col0
- Select Operator [SEL_144] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_143] (rows=575995635 width=88)
- predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_50_date_dim_d_date_sk_min) AND DynamicValue(RS_50_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_50_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_53_store_s_store_sk_min) AND DynamicValue(RS_53_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_53_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_store_sk","ss_net_profit"]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_142]
- Group By Operator [GBY_141] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Reducer 12 [CUSTOM_SIMPLE_EDGE]
- SHUFFLE [RS_91]
- Group By Operator [GBY_90] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_89] (rows=1874 width=1911)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_116]
- <-Reducer 7 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_133]
- Group By Operator [GBY_132] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 6 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_131]
- Group By Operator [GBY_130] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_129] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_127]
+ Top N Key Operator [TNK_84] (rows=696954748 width=88)
+ keys:_col8,sort order:+,top n:100
+ Merge Join Operator [MERGEJOIN_118] (rows=696954748 width=88)
+ Conds:RS_52._col1=RS_53._col1(Inner),Output:["_col2","_col8"]
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_53]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_117] (rows=1874 width=1911)
+ Conds:RS_138.substr(_col0, 1, 2)=RS_141.substr(_col2, 1, 2)(Inner),Output:["_col1","_col2"]
+ <-Map 19 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_141]
+ PartitionCols:substr(_col2, 1, 2)
+ Select Operator [SEL_140] (rows=1704 width=1910)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_139] (rows=1704 width=1910)
+ predicate:(s_store_sk is not null and substr(s_zip, 1, 2) is not null)
+ TableScan [TS_42] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_name","s_zip"]
+ <-Reducer 11 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_138]
+ PartitionCols:substr(_col0, 1, 2)
+ Select Operator [SEL_137] (rows=1 width=1014)
+ Output:["_col0"]
+ Filter Operator [FIL_136] (rows=1 width=1014)
+ predicate:(_col1 = 2L)
+ Group By Operator [GBY_135] (rows=6833333 width=1014)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
+ <-Union 10 [SIMPLE_EDGE]
+ <-Reducer 17 [CONTAINS] vectorized
+ Reduce Output Operator [RS_172]
+ PartitionCols:_col0
+ Group By Operator [GBY_171] (rows=13666666 width=1014)
+ Output:["_col0","_col1"],aggregations:["count(_col1)"],keys:_col0
+ Group By Operator [GBY_170] (rows=3666666 width=1014)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 16 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_169]
+ PartitionCols:_col0
+ Group By Operator [GBY_168] (rows=7333333 width=1014)
+ Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
+ Select Operator [SEL_167] (rows=7333333 width=1014)
+ Output:["_col0"]
+ Filter Operator [FIL_166] (rows=7333333 width=1014)
+ predicate:(_col1 > 10L)
+ Group By Operator [GBY_165] (rows=22000000 width=1014)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 15 [SIMPLE_EDGE]
+ SHUFFLE [RS_25]
+ PartitionCols:_col0
+ Group By Operator [GBY_24] (rows=44000000 width=1014)
+ Output:["_col0","_col1"],aggregations:["count()"],keys:_col1
+ Merge Join Operator [MERGEJOIN_116] (rows=44000000 width=1014)
+ Conds:RS_161._col0=RS_164._col0(Inner),Output:["_col1"]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_161]
+ PartitionCols:_col0
+ Select Operator [SEL_160] (rows=40000000 width=1014)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_159] (rows=40000000 width=1014)
+ predicate:(ca_address_sk is not null and substr(substr(ca_zip, 1, 5), 1, 2) is not null)
+ TableScan [TS_14] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_zip"]
+ <-Map 18 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_164]
+ PartitionCols:_col0
+ Select Operator [SEL_163] (rows=40000000 width=860)
+ Output:["_col0"]
+ Filter Operator [FIL_162] (rows=40000000 width=860)
+ predicate:((c_preferred_cust_flag = 'Y') and c_current_addr_sk is not null)
+ TableScan [TS_17] (rows=80000000 width=860)
+ default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_current_addr_sk","c_preferred_cust_flag"]
+ <-Reducer 9 [CONTAINS] vectorized
+ Reduce Output Operator [RS_158]
+ PartitionCols:_col0
+ Group By Operator [GBY_157] (rows=13666666 width=1014)
+ Output:["_col0","_col1"],aggregations:["count(_col1)"],keys:_col0
+ Group By Operator [GBY_156] (rows=10000000 width=1014)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_155]
+ PartitionCols:_col0
+ Group By Operator [GBY_154] (rows=20000000 width=1014)
+ Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
+ Select Operator [SEL_153] (rows=20000000 width=1014)
+ Output:["_col0"]
+ Filter Operator [FIL_152] (rows=20000000 width=1014)
+ predicate:((substr(ca_zip, 1, 5)) IN ('89436', '30868', '65085', '22977', '83927', '77557', '58429', '40697', '80614', '10502', '32779', '91137', '61265', '98294', '17921', '18427', '21203', '59362', '87291', '84093', '21505', '17184', '10866', '67898', '25797', '28055', '18377', '80332', '74535', '21757', '29742', '90885', '29898', '17819', '40811', '25990', '47513', '89531', '91068', '10391', '18846', '99223', '82637', '41368', '83658', '86199', '81625', '26696', '89338', '88425', '32200', '81427', '19053', '77471', '36610', '99823', '43276', '41249', '48584', '83550', '82276', '18842', '78890', '14090', '38123', '40936', '34425', '19850', '43286', '80072', '79188', '54191', '11395', '50497', '84861', '90733', '21068', '57666', '37119', '25004', '57835', '70067', '62878', '95806', '19303', '18840', '19124', '29785', '16737', '16022', '49613', '89977', '68310', '60069', '98360', '48649', '39050', '41793', '25002', '27413', '39736'
, '47208', '16515', '94808', '57648', '15009', '80015', '42961', '63982', '21744', '71853', '81087', '67468', '34175', '64008', '20261', '11201', '51799', '48043', '45645', '61163', '48375', '36447', '57042', '21218', '41100', '89951', '22745', '35851', '83326', '61125', '78298', '80752', '49858', '52940', '96976', '63792', '11376', '53582', '18717', '90226', '50530', '94203', '99447', '27670', '96577', '57856', '56372', '16165', '23427', '54561', '28806', '44439', '22926', '30123', '61451', '92397', '56979', '92309', '70873', '13355', '21801', '46346', '37562', '56458', '28286', '47306', '99555', '69399', '26234', '47546', '49661', '88601', '35943', '39936', '25632', '24611', '44166', '56648', '30379', '59785', '11110', '14329', '93815', '52226', '71381', '13842', '25612', '63294', '14664', '21077', '82626', '18799', '60915', '81020', '56447', '76619', '11433', '13414', '42548', '92713', '70467', '30884', '47484', '16072', '38936', '13036', '88376', '45539', '35901', '19506', '6569
0', '73957', '71850', '49231', '14276', '20005', '18384', '76615', '11635', '38177', '55607', '41369', '95447', '58581', '58149', '91946', '33790', '76232', '75692', '95464', '22246', '51061', '56692', '53121', '77209', '15482', '10688', '14868', '45907', '73520', '72666', '25734', '17959', '24677', '66446', '94627', '53535', '15560', '41967', '69297', '11929', '59403', '33283', '52232', '57350', '43933', '40921', '36635', '10827', '71286', '19736', '80619', '25251', '95042', '15526', '36496', '55854', '49124', '81980', '35375', '49157', '63512', '28944', '14946', '36503', '54010', '18767', '23969', '43905', '66979', '33113', '21286', '58471', '59080', '13395', '79144', '70373', '67031', '38360', '26705', '50906', '52406', '26066', '73146', '15884', '31897', '30045', '61068', '45550', '92454', '13376', '14354', '19770', '22928', '97790', '50723', '46081', '30202', '14410', '20223', '88500', '67298', '13261', '14172', '81410', '93578', '83583', '46047', '94167', '82564', '21156', '15
799', '86709', '37931', '74703', '83103', '23054', '70470', '72008', '49247', '91911', '69998', '20961', '70070', '63197', '54853', '88191', '91830', '49521', '19454', '81450', '89091', '62378', '25683', '61869', '51744', '36580', '85778', '36871', '48121', '28810', '83712', '45486', '67393', '26935', '42393', '20132', '55349', '86057', '21309', '80218', '10094', '11357', '48819', '39734', '40758', '30432', '21204', '29467', '30214', '61024', '55307', '74621', '11622', '68908', '33032', '52868', '99194', '99900', '84936', '69036', '99149', '45013', '32895', '59004', '32322', '14933', '32936', '33562', '72550', '27385', '58049', '58200', '16808', '21360', '32961', '18586', '79307', '15492') and substr(substr(ca_zip, 1, 5), 1, 2) is not null)
+ TableScan [TS_6] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_zip"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_52]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_115] (rows=633595212 width=88)
+ Conds:RS_146._col0=RS_129._col0(Inner),Output:["_col1","_col2"]
+ <-Map 6 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_129]
+ PartitionCols:_col0
+ Select Operator [SEL_128] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_127] (rows=18262 width=1119)
+ predicate:((d_qoy = 1) and (d_year = 2002) and d_date_sk is not null)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_146]
+ PartitionCols:_col0
+ Select Operator [SEL_145] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_144] (rows=575995635 width=88)
+ predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_50_date_dim_d_date_sk_min) AND DynamicValue(RS_50_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_50_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_53_store_s_store_sk_min) AND DynamicValue(RS_53_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_53_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_store_sk","ss_net_profit"]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_143]
+ Group By Operator [GBY_142] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Reducer 12 [CUSTOM_SIMPLE_EDGE]
+ SHUFFLE [RS_92]
+ Group By Operator [GBY_91] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_90] (rows=1874 width=1911)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_117]
+ <-Reducer 7 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_134]
+ Group By Operator [GBY_133] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 6 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_132]
+ Group By Operator [GBY_131] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_130] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_128]
[14/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query14.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query14.q.out b/ql/src/test/results/clientpositive/perf/tez/query14.q.out
index 6d996b5..b9efa45 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query14.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query14.q.out
@@ -1,6 +1,6 @@
-Warning: Shuffle Join MERGEJOIN[1454][tables = [$hdt$_1, $hdt$_2, $hdt$_0]] in Stage 'Reducer 5' is a cross product
-Warning: Shuffle Join MERGEJOIN[1466][tables = [$hdt$_1, $hdt$_2, $hdt$_0]] in Stage 'Reducer 12' is a cross product
-Warning: Shuffle Join MERGEJOIN[1478][tables = [$hdt$_2, $hdt$_3, $hdt$_1]] in Stage 'Reducer 16' is a cross product
+Warning: Shuffle Join MERGEJOIN[1455][tables = [$hdt$_1, $hdt$_2, $hdt$_0]] in Stage 'Reducer 5' is a cross product
+Warning: Shuffle Join MERGEJOIN[1468][tables = [$hdt$_1, $hdt$_2, $hdt$_0]] in Stage 'Reducer 12' is a cross product
+Warning: Shuffle Join MERGEJOIN[1481][tables = [$hdt$_2, $hdt$_3, $hdt$_1]] in Stage 'Reducer 16' is a cross product
PREHOOK: query: explain
with cross_items as
(select i_item_sk ss_item_sk
@@ -314,1135 +314,1141 @@ Stage-0
limit:100
Stage-1
Reducer 8 vectorized
- File Output Operator [FS_1739]
- Limit [LIM_1738] (rows=100 width=237)
+ File Output Operator [FS_1743]
+ Limit [LIM_1742] (rows=100 width=237)
Number of rows:100
- Select Operator [SEL_1737] (rows=1016388080 width=237)
+ Select Operator [SEL_1741] (rows=1016388080 width=237)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1736]
- Select Operator [SEL_1735] (rows=1016388080 width=237)
+ SHUFFLE [RS_1740]
+ Select Operator [SEL_1739] (rows=1016388080 width=237)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Group By Operator [GBY_1734] (rows=1016388080 width=237)
+ Group By Operator [GBY_1738] (rows=1016388080 width=237)
Output:["_col0","_col1","_col2","_col3","_col5","_col6"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
<-Union 6 [SIMPLE_EDGE]
<-Reducer 12 [CONTAINS]
- Reduce Output Operator [RS_1471]
+ Reduce Output Operator [RS_1474]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_1470] (rows=2032776160 width=237)
+ Group By Operator [GBY_1473] (rows=2032776160 width=237)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col4)","sum(_col5)"],keys:_col0, _col1, _col2, _col3, 0L
- Select Operator [SEL_1468] (rows=116155905 width=264)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_1467] (rows=116155905 width=264)
- predicate:(_col5 > _col1)
- Merge Join Operator [MERGEJOIN_1466] (rows=348467716 width=264)
- Conds:(Inner),(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col6"]
- <-Reducer 11 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1745]
- Select Operator [SEL_1744] (rows=1 width=8)
- Filter Operator [FIL_1743] (rows=1 width=8)
- predicate:(sq_count_check(_col0) <= 1)
- Group By Operator [GBY_1742] (rows=1 width=8)
- Output:["_col0"],aggregations:["count()"]
- Select Operator [SEL_1741] (rows=1 width=8)
- Group By Operator [GBY_1740] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(VALUE._col0)"]
- <-Union 10 [CUSTOM_SIMPLE_EDGE]
- <-Reducer 19 [CONTAINS]
- Reduce Output Operator [RS_1495]
- Group By Operator [GBY_1494] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(_col0)"]
- Select Operator [SEL_1493] (rows=1108786976 width=108)
- Output:["_col0"]
- Select Operator [SEL_1491] (rows=316788826 width=135)
+ Top N Key Operator [TNK_1472] (rows=406555232 width=237)
+ keys:_col0, _col1, _col2, _col3, 0L,sort order:+++++,top n:100
+ Select Operator [SEL_1470] (rows=116155905 width=264)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_1469] (rows=116155905 width=264)
+ predicate:(_col5 > _col1)
+ Merge Join Operator [MERGEJOIN_1468] (rows=348467716 width=264)
+ Conds:(Inner),(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col6"]
+ <-Reducer 11 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1749]
+ Select Operator [SEL_1748] (rows=1 width=8)
+ Filter Operator [FIL_1747] (rows=1 width=8)
+ predicate:(sq_count_check(_col0) <= 1)
+ Group By Operator [GBY_1746] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count()"]
+ Select Operator [SEL_1745] (rows=1 width=8)
+ Group By Operator [GBY_1744] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(VALUE._col0)"]
+ <-Union 10 [CUSTOM_SIMPLE_EDGE]
+ <-Reducer 19 [CONTAINS]
+ Reduce Output Operator [RS_1499]
+ Group By Operator [GBY_1498] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(_col0)"]
+ Select Operator [SEL_1497] (rows=1108786976 width=108)
Output:["_col0"]
- Merge Join Operator [MERGEJOIN_1490] (rows=316788826 width=135)
- Conds:RS_1817._col0=RS_1804._col0(Inner),Output:["_col1"]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1804]
- PartitionCols:_col0
- Select Operator [SEL_1799] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_1798] (rows=8116 width=1119)
- predicate:(d_date_sk is not null and d_year BETWEEN 1998 AND 2000)
- TableScan [TS_13] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Map 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1817]
- PartitionCols:_col0
- Select Operator [SEL_1815] (rows=287989836 width=135)
- Output:["_col0","_col1"]
- Filter Operator [FIL_1814] (rows=287989836 width=135)
- predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_17_date_dim_d_date_sk_min) AND DynamicValue(RS_17_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_17_date_dim_d_date_sk_bloom_filter))) and cs_sold_date_sk is not null)
- TableScan [TS_10] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_quantity"]
- <-Reducer 22 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1813]
- Group By Operator [GBY_1812] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1810]
- Group By Operator [GBY_1808] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1801] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1799]
- <-Reducer 35 [CONTAINS]
- Reduce Output Operator [RS_1531]
- Group By Operator [GBY_1530] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(_col0)"]
- Select Operator [SEL_1529] (rows=1108786976 width=108)
- Output:["_col0"]
- Select Operator [SEL_1527] (rows=158402938 width=135)
+ Select Operator [SEL_1495] (rows=316788826 width=135)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_1494] (rows=316788826 width=135)
+ Conds:RS_1821._col0=RS_1808._col0(Inner),Output:["_col1"]
+ <-Map 21 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1808]
+ PartitionCols:_col0
+ Select Operator [SEL_1803] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_1802] (rows=8116 width=1119)
+ predicate:(d_date_sk is not null and d_year BETWEEN 1998 AND 2000)
+ TableScan [TS_13] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Map 17 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1821]
+ PartitionCols:_col0
+ Select Operator [SEL_1819] (rows=287989836 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_1818] (rows=287989836 width=135)
+ predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_17_date_dim_d_date_sk_min) AND DynamicValue(RS_17_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_17_date_dim_d_date_sk_bloom_filter))) and cs_sold_date_sk is not null)
+ TableScan [TS_10] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_quantity"]
+ <-Reducer 22 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1817]
+ Group By Operator [GBY_1816] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1814]
+ Group By Operator [GBY_1812] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1805] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1803]
+ <-Reducer 35 [CONTAINS]
+ Reduce Output Operator [RS_1535]
+ Group By Operator [GBY_1534] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(_col0)"]
+ Select Operator [SEL_1533] (rows=1108786976 width=108)
Output:["_col0"]
- Merge Join Operator [MERGEJOIN_1526] (rows=158402938 width=135)
- Conds:RS_1845._col0=RS_1832._col0(Inner),Output:["_col1"]
- <-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1832]
- PartitionCols:_col0
- Select Operator [SEL_1827] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_1826] (rows=8116 width=1119)
- predicate:(d_date_sk is not null and d_year BETWEEN 1998 AND 2000)
- TableScan [TS_24] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Map 33 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1845]
- PartitionCols:_col0
- Select Operator [SEL_1843] (rows=144002668 width=135)
- Output:["_col0","_col1"]
- Filter Operator [FIL_1842] (rows=144002668 width=135)
- predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_28_date_dim_d_date_sk_min) AND DynamicValue(RS_28_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_28_date_dim_d_date_sk_bloom_filter))) and ws_sold_date_sk is not null)
- TableScan [TS_21] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_quantity"]
- <-Reducer 38 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1841]
- Group By Operator [GBY_1840] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 37 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1838]
- Group By Operator [GBY_1836] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1829] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1827]
- <-Reducer 9 [CONTAINS]
- Reduce Output Operator [RS_1465]
- Group By Operator [GBY_1464] (rows=1 width=8)
- Output:["_col0"],aggregations:["count(_col0)"]
- Select Operator [SEL_1463] (rows=1108786976 width=108)
- Output:["_col0"]
- Select Operator [SEL_1461] (rows=633595212 width=88)
+ Select Operator [SEL_1531] (rows=158402938 width=135)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_1530] (rows=158402938 width=135)
+ Conds:RS_1849._col0=RS_1836._col0(Inner),Output:["_col1"]
+ <-Map 37 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1836]
+ PartitionCols:_col0
+ Select Operator [SEL_1831] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_1830] (rows=8116 width=1119)
+ predicate:(d_date_sk is not null and d_year BETWEEN 1998 AND 2000)
+ TableScan [TS_24] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Map 33 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1849]
+ PartitionCols:_col0
+ Select Operator [SEL_1847] (rows=144002668 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_1846] (rows=144002668 width=135)
+ predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_28_date_dim_d_date_sk_min) AND DynamicValue(RS_28_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_28_date_dim_d_date_sk_bloom_filter))) and ws_sold_date_sk is not null)
+ TableScan [TS_21] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_quantity"]
+ <-Reducer 38 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1845]
+ Group By Operator [GBY_1844] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 37 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1842]
+ Group By Operator [GBY_1840] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1833] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1831]
+ <-Reducer 9 [CONTAINS]
+ Reduce Output Operator [RS_1467]
+ Group By Operator [GBY_1466] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(_col0)"]
+ Select Operator [SEL_1465] (rows=1108786976 width=108)
Output:["_col0"]
- Merge Join Operator [MERGEJOIN_1460] (rows=633595212 width=88)
- Conds:RS_1641._col0=RS_1622._col0(Inner),Output:["_col1"]
- <-Map 102 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1622]
- PartitionCols:_col0
- Select Operator [SEL_1611] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_1610] (rows=8116 width=1119)
- predicate:(d_date_sk is not null and d_year BETWEEN 1999 AND 2001)
- TableScan [TS_97] (rows=73049 width=1119)
- default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1641]
- PartitionCols:_col0
- Select Operator [SEL_1639] (rows=575995635 width=88)
- Output:["_col0","_col1"]
- Filter Operator [FIL_1638] (rows=575995635 width=88)
- predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_7_date_dim_d_date_sk_min) AND DynamicValue(RS_7_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_7_date_dim_d_date_sk_bloom_filter))) and ss_sold_date_sk is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
- <-Reducer 108 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1637]
- Group By Operator [GBY_1636] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 102 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1634]
- Group By Operator [GBY_1629] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1619] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1611]
- <-Reducer 29 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_1748]
- Select Operator [SEL_1747] (rows=1 width=120)
- Output:["_col0"]
- Group By Operator [GBY_1746] (rows=1 width=120)
- Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
- <-Union 28 [CUSTOM_SIMPLE_EDGE]
- <-Reducer 27 [CONTAINS]
- Reduce Output Operator [RS_1513]
- Group By Operator [GBY_1512] (rows=1 width=120)
- Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1511] (rows=1108786976 width=108)
- Output:["_col0"]
- Select Operator [SEL_1509] (rows=316788826 width=135)
- Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1508] (rows=316788826 width=135)
- Conds:RS_1824._col0=RS_1805._col0(Inner),Output:["_col1","_col2"]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1805]
+ Select Operator [SEL_1463] (rows=633595212 width=88)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_1462] (rows=633595212 width=88)
+ Conds:RS_1645._col0=RS_1626._col0(Inner),Output:["_col1"]
+ <-Map 102 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1626]
+ PartitionCols:_col0
+ Select Operator [SEL_1615] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_1614] (rows=8116 width=1119)
+ predicate:(d_date_sk is not null and d_year BETWEEN 1999 AND 2001)
+ TableScan [TS_97] (rows=73049 width=1119)
+ default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1645]
+ PartitionCols:_col0
+ Select Operator [SEL_1643] (rows=575995635 width=88)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_1642] (rows=575995635 width=88)
+ predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_7_date_dim_d_date_sk_min) AND DynamicValue(RS_7_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_7_date_dim_d_date_sk_bloom_filter))) and ss_sold_date_sk is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity"]
+ <-Reducer 108 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1641]
+ Group By Operator [GBY_1640] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 102 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1638]
+ Group By Operator [GBY_1633] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1623] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Reducer 29 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1752]
+ Select Operator [SEL_1751] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_1750] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Union 28 [CUSTOM_SIMPLE_EDGE]
+ <-Reducer 27 [CONTAINS]
+ Reduce Output Operator [RS_1517]
+ Group By Operator [GBY_1516] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
+ Select Operator [SEL_1515] (rows=1108786976 width=108)
+ Output:["_col0"]
+ Select Operator [SEL_1513] (rows=316788826 width=135)
+ Output:["_col0","_col1"]
+ Merge Join Operator [MERGEJOIN_1512] (rows=316788826 width=135)
+ Conds:RS_1828._col0=RS_1809._col0(Inner),Output:["_col1","_col2"]
+ <-Map 21 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1809]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_1803]
+ <-Map 47 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1828]
+ PartitionCols:_col0
+ Select Operator [SEL_1826] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_1825] (rows=287989836 width=135)
+ predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_62_date_dim_d_date_sk_min) AND DynamicValue(RS_62_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_62_date_dim_d_date_sk_bloom_filter))) and cs_sold_date_sk is not null)
+ TableScan [TS_55] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_quantity","cs_list_price"]
+ <-Reducer 26 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1824]
+ Group By Operator [GBY_1823] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1815]
+ Group By Operator [GBY_1813] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1807] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1803]
+ <-Reducer 41 [CONTAINS]
+ Reduce Output Operator [RS_1553]
+ Group By Operator [GBY_1552] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
+ Select Operator [SEL_1551] (rows=1108786976 width=108)
+ Output:["_col0"]
+ Select Operator [SEL_1549] (rows=158402938 width=135)
+ Output:["_col0","_col1"]
+ Merge Join Operator [MERGEJOIN_1548] (rows=158402938 width=135)
+ Conds:RS_1856._col0=RS_1837._col0(Inner),Output:["_col1","_col2"]
+ <-Map 37 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1837]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_1831]
+ <-Map 48 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1856]
+ PartitionCols:_col0
+ Select Operator [SEL_1854] (rows=144002668 width=135)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_1853] (rows=144002668 width=135)
+ predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_73_date_dim_d_date_sk_min) AND DynamicValue(RS_73_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_73_date_dim_d_date_sk_bloom_filter))) and ws_sold_date_sk is not null)
+ TableScan [TS_66] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_quantity","ws_list_price"]
+ <-Reducer 40 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1852]
+ Group By Operator [GBY_1851] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 37 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1843]
+ Group By Operator [GBY_1841] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1835] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1831]
+ <-Reducer 45 [CONTAINS]
+ Reduce Output Operator [RS_1571]
+ Group By Operator [GBY_1570] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
+ Select Operator [SEL_1569] (rows=1108786976 width=108)
+ Output:["_col0"]
+ Select Operator [SEL_1567] (rows=633595212 width=88)
+ Output:["_col0","_col1"]
+ Merge Join Operator [MERGEJOIN_1566] (rows=633595212 width=88)
+ Conds:RS_1863._col0=RS_1627._col0(Inner),Output:["_col1","_col2"]
+ <-Map 102 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1627]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Map 43 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1863]
+ PartitionCols:_col0
+ Select Operator [SEL_1861] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_1860] (rows=575995635 width=88)
+ predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_52_date_dim_d_date_sk_min) AND DynamicValue(RS_52_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_52_date_dim_d_date_sk_bloom_filter))) and ss_sold_date_sk is not null)
+ TableScan [TS_45] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity","ss_list_price"]
+ <-Reducer 109 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1859]
+ Group By Operator [GBY_1858] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 102 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1639]
+ Group By Operator [GBY_1634] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1625] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Reducer 57 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1772]
+ Group By Operator [GBY_1771] (rows=348467716 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Reducer 56 [SIMPLE_EDGE]
+ SHUFFLE [RS_369]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_368] (rows=696935432 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col3)","count()"],keys:_col0, _col1, _col2
+ Select Operator [SEL_366] (rows=696935432 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Merge Join Operator [MERGEJOIN_1430] (rows=696935432 width=135)
+ Conds:RS_362._col1=RS_1703._col0(Inner),RS_362._col1=RS_1762._col0(Inner),Output:["_col2","_col3","_col8","_col9","_col10"]
+ <-Map 63 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1703]
+ PartitionCols:_col0
+ Select Operator [SEL_1687] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_1678] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_91] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_brand_id","i_class_id","i_category_id"]
+ <-Reducer 80 [ONE_TO_ONE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1762]
+ PartitionCols:_col0
+ Group By Operator [GBY_1761] (rows=254100 width=1436)
+ Output:["_col0"],keys:KEY._col0
+ <-Reducer 79 [SIMPLE_EDGE]
+ SHUFFLE [RS_356]
PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1799]
- <-Map 47 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1824]
+ Group By Operator [GBY_355] (rows=508200 width=1436)
+ Output:["_col0"],keys:_col0
+ Merge Join Operator [MERGEJOIN_1429] (rows=508200 width=1436)
+ Conds:RS_1699._col1, _col2, _col3=RS_1760._col0, _col1, _col2(Inner),Output:["_col0"]
+ <-Map 63 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1699]
+ PartitionCols:_col1, _col2, _col3
+ Select Operator [SEL_1683] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_1674] (rows=462000 width=1436)
+ predicate:(i_brand_id is not null and i_category_id is not null and i_class_id is not null and i_item_sk is not null)
+ Please refer to the previous TableScan [TS_91]
+ <-Reducer 85 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_1760]
+ PartitionCols:_col0, _col1, _col2
+ Select Operator [SEL_1759] (rows=1 width=108)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_1758] (rows=1 width=108)
+ predicate:(_col3 = 3L)
+ Group By Operator [GBY_1757] (rows=304916424 width=108)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Union 84 [SIMPLE_EDGE]
+ <-Reducer 83 [CONTAINS] vectorized
+ Reduce Output Operator [RS_1897]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_1896] (rows=609832849 width=108)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
+ Group By Operator [GBY_1895] (rows=348477374 width=88)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Reducer 82 [SIMPLE_EDGE]
+ SHUFFLE [RS_300]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_299] (rows=696954748 width=88)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col5, _col6, _col7
+ Merge Join Operator [MERGEJOIN_1424] (rows=696954748 width=88)
+ Conds:RS_295._col1=RS_1700._col0(Inner),Output:["_col5","_col6","_col7"]
+ <-Map 63 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1700]
+ PartitionCols:_col0
+ Select Operator [SEL_1684] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_1675] (rows=462000 width=1436)
+ predicate:(i_brand_id is not null and i_category_id is not null and i_class_id is not null and i_item_sk is not null)
+ Please refer to the previous TableScan [TS_91]
+ <-Reducer 101 [SIMPLE_EDGE]
+ SHUFFLE [RS_295]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_1408] (rows=633595212 width=88)
+ Conds:RS_1871._col0=RS_1616._col0(Inner),Output:["_col1"]
+ <-Map 102 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1616]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Map 100 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1871]
+ PartitionCols:_col0
+ Select Operator [SEL_1870] (rows=575995635 width=88)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_1869] (rows=575995635 width=88)
+ predicate:((ss_item_sk BETWEEN DynamicValue(RS_107_iss_i_item_sk_min) AND DynamicValue(RS_107_iss_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_107_iss_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_104_d1_d_date_sk_min) AND DynamicValue(RS_104_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_104_d1_d_date_sk_bloom_filter))) and ss_item_sk is not null and ss_sold_date_sk is not null)
+ TableScan [TS_94] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk"]
+ <-Reducer 103 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1866]
+ Group By Operator [GBY_1865] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 102 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1635]
+ Group By Operator [GBY_1630] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1617] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Reducer 71 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1868]
+ Group By Operator [GBY_1867] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 63 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1714]
+ Group By Operator [GBY_1708] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1692] (rows=462000 width=1436)
+ Output:["_col0"]
+ Select Operator [SEL_1681] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_1672] (rows=462000 width=1436)
+ predicate:(i_brand_id is not null and i_category_id is not null and i_class_id is not null and i_item_sk is not null)
+ Please refer to the previous TableScan [TS_91]
+ <-Reducer 93 [CONTAINS] vectorized
+ Reduce Output Operator [RS_1903]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_1902] (rows=609832849 width=108)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
+ Group By Operator [GBY_1901] (rows=174233858 width=135)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Reducer 92 [SIMPLE_EDGE]
+ SHUFFLE [RS_320]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_319] (rows=348467716 width=135)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col5, _col6, _col7
+ Merge Join Operator [MERGEJOIN_1426] (rows=348467716 width=135)
+ Conds:RS_315._col1=RS_1701._col0(Inner),Output:["_col5","_col6","_col7"]
+ <-Map 63 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1701]
+ PartitionCols:_col0
+ Select Operator [SEL_1685] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_1676] (rows=462000 width=1436)
+ predicate:(i_brand_id is not null and i_category_id is not null and i_class_id is not null and i_item_sk is not null)
+ Please refer to the previous TableScan [TS_91]
+ <-Reducer 104 [SIMPLE_EDGE]
+ SHUFFLE [RS_315]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_1410] (rows=316788826 width=135)
+ Conds:RS_1881._col0=RS_1618._col0(Inner),Output:["_col1"]
+ <-Map 102 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1618]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Map 110 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1881]
+ PartitionCols:_col0
+ Select Operator [SEL_1880] (rows=287989836 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_1879] (rows=287989836 width=135)
+ predicate:((cs_item_sk BETWEEN DynamicValue(RS_127_ics_i_item_sk_min) AND DynamicValue(RS_127_ics_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_127_ics_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_124_d2_d_date_sk_min) AND DynamicValue(RS_124_d2_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_124_d2_d_date_sk_bloom_filter))) and cs_item_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_114] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_item_sk"]
+ <-Reducer 105 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1876]
+ Group By Operator [GBY_1875] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 102 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1636]
+ Group By Operator [GBY_1631] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1619] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Reducer 74 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1878]
+ Group By Operator [GBY_1877] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 63 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1715]
+ Group By Operator [GBY_1709] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1694] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1681]
+ <-Reducer 96 [CONTAINS] vectorized
+ Reduce Output Operator [RS_1909]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_1908] (rows=609832849 width=108)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count(_col3)"],keys:_col0, _col1, _col2
+ Group By Operator [GBY_1907] (rows=87121617 width=135)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Reducer 95 [SIMPLE_EDGE]
+ SHUFFLE [RS_341]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_340] (rows=174243235 width=135)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["count()"],keys:_col5, _col6, _col7
+ Merge Join Operator [MERGEJOIN_1428] (rows=174243235 width=135)
+ Conds:RS_336._col1=RS_1702._col0(Inner),Output:["_col5","_col6","_col7"]
+ <-Map 63 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1702]
+ PartitionCols:_col0
+ Select Operator [SEL_1686] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_1677] (rows=462000 width=1436)
+ predicate:(i_brand_id is not null and i_category_id is not null and i_class_id is not null and i_item_sk is not null)
+ Please refer to the previous TableScan [TS_91]
+ <-Reducer 106 [SIMPLE_EDGE]
+ SHUFFLE [RS_336]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_1412] (rows=158402938 width=135)
+ Conds:RS_1891._col0=RS_1620._col0(Inner),Output:["_col1"]
+ <-Map 102 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1620]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Map 111 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1891]
+ PartitionCols:_col0
+ Select Operator [SEL_1890] (rows=144002668 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_1889] (rows=144002668 width=135)
+ predicate:((ws_item_sk BETWEEN DynamicValue(RS_148_iws_i_item_sk_min) AND DynamicValue(RS_148_iws_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_148_iws_i_item_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_145_d3_d_date_sk_min) AND DynamicValue(RS_145_d3_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_145_d3_d_date_sk_bloom_filter))) and ws_item_sk is not null and ws_sold_date_sk is not null)
+ TableScan [TS_135] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk"]
+ <-Reducer 107 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1886]
+ Group By Operator [GBY_1885] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 102 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1637]
+ Group By Operator [GBY_1632] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1621] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1615]
+ <-Reducer 77 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1888]
+ Group By Operator [GBY_1887] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 63 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1716]
+ Group By Operator [GBY_1710] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_1696] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_1681]
+ <-Reducer 55 [SIMPLE_EDGE]
+ SHUFFLE [RS_362]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_1422] (rows=316788826 width=135)
+ Conds:RS_1770._col0=RS_1660._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 53 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1660]
PartitionCols:_col0
- Select Operator [SEL_1822] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1821] (rows=287989836 width=135)
- predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_62_date_dim_d_date_sk_min) AND DynamicValue(RS_62_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_62_date_dim_d_date_sk_bloom_filter))) and cs_sold_date_sk is not null)
- TableScan [TS_55] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_quantity","cs_list_price"]
- <-Reducer 26 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1820]
- Group By Operator [GBY_1819] (rows=1 width=12)
+ Select Operator [SEL_1657] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_1656] (rows=18262 width=1119)
+ predicate:((d_moy = 11) and (d_year = 2000) and d_date_sk is not null)
+ TableScan [TS_85] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Map 112 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_1770]
+ PartitionCols:_col0
+ Select Operator [SEL_1769] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_1768] (rows=287989836 width=135)
+ predicate:((cs_item_sk BETWEEN DynamicValue(RS_363_item_i_item_sk_min) AND DynamicValue(RS_363_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_363_item_i_item_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_364_item_i_item_sk_min) AND DynamicValue(RS_364_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_364_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_360_date_dim_d_date_sk_min) AND DynamicValue(RS_360_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_360_date_dim_d_date_sk_bloom_filter))) and cs_item_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_271] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_item_sk","cs_quantity","cs_list_price"]
+ <-Reducer 58 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1754]
+ Group By Operator [GBY_1753] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1811]
- Group By Operator [GBY_1809] (rows=1 width=12)
+ <-Map 53 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1668]
+ Group By Operator [GBY_1665] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1803] (rows=8116 width=1119)
+ Select Operator [SEL_1661] (rows=18262 width=1119)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1799]
- <-Reducer 41 [CONTAINS]
- Reduce Output Operator [RS_1549]
- Group By Operator [GBY_1548] (rows=1 width=120)
- Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1547] (rows=1108786976 width=108)
- Output:["_col0"]
- Select Operator [SEL_1545] (rows=158402938 width=135)
- Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1544] (rows=158402938 width=135)
- Conds:RS_1852._col0=RS_1833._col0(Inner),Output:["_col1","_col2"]
- <-Map 37 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1833]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1827]
- <-Map 48 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1852]
- PartitionCols:_col0
- Select Operator [SEL_1850] (rows=144002668 width=135)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1849] (rows=144002668 width=135)
- predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_73_date_dim_d_date_sk_min) AND DynamicValue(RS_73_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_73_date_dim_d_date_sk_bloom_filter))) and ws_sold_date_sk is not null)
- TableScan [TS_66] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_quantity","ws_list_price"]
- <-Reducer 40 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1848]
- Group By Operator [GBY_1847] (rows=1 width=12)
+ Please refer to the previous Select Operator [SEL_1657]
+ <-Reducer 81 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_1767]
+ Group By Operator [GBY_1766] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 37 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1839]
- Group By Operator [GBY_1837] (rows=1 width=12)
+ <-Reducer 80 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_1765]
+ Group By Operator [GBY_1764] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_1831] (rows=8116 width=1119)
+ Select Operator [SEL_1763] (rows=254100 width=1436)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_1827]
- <-Reducer 45 [CONTAINS]
- Reduce Output Operator [RS_1567]
- Group By Operator [GBY_1566] (rows=1 width=120)
- Output:["_col0","_col1"],aggregations:["sum(_col0)","count(_col0)"]
- Select Operator [SEL_1565] (rows=1108786976 width=108)
- Output:["_col0"]
- Select Operator [SEL_1563] (rows=633595212 width=88)
- Output:["_col0","_col1"]
- Merge Join Operator [MERGEJOIN_1562] (rows=633595212 width=88)
- Conds:RS_1859._col0=RS_1623._col0(Inner),Output:["_col1","_col2"]
- <-Map 102 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1623]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_1611]
- <-Map 43 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_1859]
- PartitionCols:_col0
- Select Operator [SEL_1857] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_1856] (rows=575995635 width=88)
- predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_52_date_dim_d_date_sk_min) AND DynamicValue(RS_52_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_52_date_dim_d_date_sk_bloom_filter))) and ss_sold_date_sk is not null)
- TableScan [TS_45] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_quantity","ss_list_price"]
- <-Reducer 109 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_1855]
- Group By Operator [GBY_1854] (rows=1 width=12)
+ Please refer
<TRUNCATED>
[09/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query49.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query49.q.out b/ql/src/test/results/clientpositive/perf/tez/query49.q.out
index 434ed16..6f642ef 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query49.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query49.q.out
@@ -283,253 +283,257 @@ Stage-0
limit:100
Stage-1
Reducer 11 vectorized
- File Output Operator [FS_306]
- Limit [LIM_305] (rows=100 width=101)
+ File Output Operator [FS_310]
+ Limit [LIM_309] (rows=100 width=101)
Number of rows:100
- Select Operator [SEL_304] (rows=5915494 width=101)
+ Select Operator [SEL_308] (rows=5915494 width=101)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_303]
- Select Operator [SEL_302] (rows=5915494 width=101)
+ SHUFFLE [RS_307]
+ Select Operator [SEL_306] (rows=5915494 width=101)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_301] (rows=5915494 width=101)
+ Group By Operator [GBY_305] (rows=5915494 width=101)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
<-Union 9 [SIMPLE_EDGE]
<-Reducer 24 [CONTAINS] vectorized
- Reduce Output Operator [RS_346]
+ Reduce Output Operator [RS_351]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_345] (rows=11830988 width=101)
+ Group By Operator [GBY_350] (rows=11830988 width=101)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
- Select Operator [SEL_344] (rows=8604378 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_343] (rows=8604378 width=88)
- predicate:((_col0 <= 10) or (rank_window_1 <= 10))
- PTF Operator [PTF_342] (rows=12906568 width=88)
- Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
- Select Operator [SEL_341] (rows=12906568 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- <-Reducer 23 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_340]
- PartitionCols:0
- Select Operator [SEL_339] (rows=12906568 width=88)
- Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
- PTF Operator [PTF_338] (rows=12906568 width=88)
- Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
- Select Operator [SEL_337] (rows=12906568 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- <-Reducer 22 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_336]
- PartitionCols:0
- Group By Operator [GBY_335] (rows=12906568 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
- <-Reducer 21 [SIMPLE_EDGE]
- SHUFFLE [RS_89]
- PartitionCols:_col0
- Group By Operator [GBY_88] (rows=25813137 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
- Select Operator [SEL_86] (rows=25813137 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_236] (rows=25813137 width=88)
- Conds:RS_83._col1, _col2=RS_334._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col11","_col12"]
- <-Map 30 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_334]
- PartitionCols:_col0, _col1
- Select Operator [SEL_333] (rows=19197050 width=77)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_332] (rows=19197050 width=77)
- predicate:((sr_return_amt > 10000) and sr_item_sk is not null and sr_ticket_number is not null)
- TableScan [TS_77] (rows=57591150 width=77)
- default@store_returns,sr,Tbl:COMPLETE,Col:NONE,Output:["sr_item_sk","sr_ticket_number","sr_return_quantity","sr_return_amt"]
- <-Reducer 20 [SIMPLE_EDGE]
- SHUFFLE [RS_83]
- PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_235] (rows=23466488 width=88)
- Conds:RS_331._col0=RS_269._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_269]
- PartitionCols:_col0
- Select Operator [SEL_264] (rows=18262 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_263] (rows=18262 width=1119)
- predicate:((d_moy = 12) and (d_year = 2000) and d_date_sk is not null)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
- <-Map 29 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_331]
- PartitionCols:_col0
- Select Operator [SEL_330] (rows=21333171 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_329] (rows=21333171 width=88)
- predicate:((ss_net_paid > 0) and (ss_net_profit > 1) and (ss_quantity > 0) and (ss_sold_date_sk BETWEEN DynamicValue(RS_81_date_dim_d_date_sk_min) AND DynamicValue(RS_81_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_81_date_dim_d_date_sk_bloom_filter))) and ss_item_sk is not null and ss_sold_date_sk is not null and ss_ticket_number is not null)
- TableScan [TS_71] (rows=575995635 width=88)
- default@store_sales,sts,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_ticket_number","ss_quantity","ss_net_paid","ss_net_profit"]
- <-Reducer 25 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_328]
- Group By Operator [GBY_327] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_276]
- Group By Operator [GBY_273] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_270] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_264]
+ Top N Key Operator [TNK_349] (rows=11830988 width=101)
+ keys:_col0, _col3, _col4, _col1, _col2,sort order:+++++,top n:100
+ Select Operator [SEL_348] (rows=8604378 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_347] (rows=8604378 width=88)
+ predicate:((_col0 <= 10) or (rank_window_1 <= 10))
+ PTF Operator [PTF_346] (rows=12906568 width=88)
+ Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
+ Select Operator [SEL_345] (rows=12906568 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ <-Reducer 23 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_344]
+ PartitionCols:0
+ Select Operator [SEL_343] (rows=12906568 width=88)
+ Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
+ PTF Operator [PTF_342] (rows=12906568 width=88)
+ Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
+ Select Operator [SEL_341] (rows=12906568 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ <-Reducer 22 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_340]
+ PartitionCols:0
+ Group By Operator [GBY_339] (rows=12906568 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
+ <-Reducer 21 [SIMPLE_EDGE]
+ SHUFFLE [RS_89]
+ PartitionCols:_col0
+ Group By Operator [GBY_88] (rows=25813137 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
+ Select Operator [SEL_86] (rows=25813137 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_237] (rows=25813137 width=88)
+ Conds:RS_83._col1, _col2=RS_338._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col11","_col12"]
+ <-Map 30 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_338]
+ PartitionCols:_col0, _col1
+ Select Operator [SEL_337] (rows=19197050 width=77)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_336] (rows=19197050 width=77)
+ predicate:((sr_return_amt > 10000) and sr_item_sk is not null and sr_ticket_number is not null)
+ TableScan [TS_77] (rows=57591150 width=77)
+ default@store_returns,sr,Tbl:COMPLETE,Col:NONE,Output:["sr_item_sk","sr_ticket_number","sr_return_quantity","sr_return_amt"]
+ <-Reducer 20 [SIMPLE_EDGE]
+ SHUFFLE [RS_83]
+ PartitionCols:_col1, _col2
+ Merge Join Operator [MERGEJOIN_236] (rows=23466488 width=88)
+ Conds:RS_335._col0=RS_272._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_272]
+ PartitionCols:_col0
+ Select Operator [SEL_267] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_266] (rows=18262 width=1119)
+ predicate:((d_moy = 12) and (d_year = 2000) and d_date_sk is not null)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Map 29 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_335]
+ PartitionCols:_col0
+ Select Operator [SEL_334] (rows=21333171 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_333] (rows=21333171 width=88)
+ predicate:((ss_net_paid > 0) and (ss_net_profit > 1) and (ss_quantity > 0) and (ss_sold_date_sk BETWEEN DynamicValue(RS_81_date_dim_d_date_sk_min) AND DynamicValue(RS_81_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_81_date_dim_d_date_sk_bloom_filter))) and ss_item_sk is not null and ss_sold_date_sk is not null and ss_ticket_number is not null)
+ TableScan [TS_71] (rows=575995635 width=88)
+ default@store_sales,sts,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_ticket_number","ss_quantity","ss_net_paid","ss_net_profit"]
+ <-Reducer 25 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_332]
+ Group By Operator [GBY_331] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_279]
+ Group By Operator [GBY_276] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_273] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_267]
<-Reducer 8 [CONTAINS] vectorized
- Reduce Output Operator [RS_300]
+ Reduce Output Operator [RS_304]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_299] (rows=11830988 width=101)
+ Group By Operator [GBY_303] (rows=11830988 width=101)
Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
- Select Operator [SEL_298] (rows=3226610 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_297] (rows=3226610 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
- <-Union 7 [SIMPLE_EDGE]
- <-Reducer 18 [CONTAINS] vectorized
- Reduce Output Operator [RS_326]
- PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_325] (rows=6453220 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
- Select Operator [SEL_324] (rows=4302070 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_323] (rows=4302070 width=135)
- predicate:((_col0 <= 10) or (rank_window_1 <= 10))
- PTF Operator [PTF_322] (rows=6453105 width=135)
- Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
- Select Operator [SEL_321] (rows=6453105 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- <-Reducer 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_320]
- PartitionCols:0
- Select Operator [SEL_319] (rows=6453105 width=135)
- Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
- PTF Operator [PTF_318] (rows=6453105 width=135)
- Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
- Select Operator [SEL_317] (rows=6453105 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- <-Reducer 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_316]
- PartitionCols:0
- Group By Operator [GBY_315] (rows=6453105 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_50]
- PartitionCols:_col0
- Group By Operator [GBY_49] (rows=12906211 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
- Select Operator [SEL_47] (rows=12906211 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_234] (rows=12906211 width=135)
- Conds:RS_44._col1, _col2=RS_314._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col11","_col12"]
- <-Map 28 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_314]
- PartitionCols:_col0, _col1
- Select Operator [SEL_313] (rows=9599627 width=106)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_312] (rows=9599627 width=106)
- predicate:((cr_return_amount > 10000) and cr_item_sk is not null and cr_order_number is not null)
- TableScan [TS_38] (rows=28798881 width=106)
- default@catalog_returns,cr,Tbl:COMPLETE,Col:NONE,Output:["cr_item_sk","cr_order_number","cr_return_quantity","cr_return_amount"]
- <-Reducer 14 [SIMPLE_EDGE]
- SHUFFLE [RS_44]
- PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_233] (rows=11732919 width=135)
- Conds:RS_311._col0=RS_267._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_267]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_264]
- <-Map 27 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_311]
- PartitionCols:_col0
- Select Operator [SEL_310] (rows=10666290 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_309] (rows=10666290 width=135)
- predicate:((cs_net_paid > 0) and (cs_net_profit > 1) and (cs_quantity > 0) and (cs_sold_date_sk BETWEEN DynamicValue(RS_42_date_dim_d_date_sk_min) AND DynamicValue(RS_42_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_42_date_dim_d_date_sk_bloom_filter))) and cs_item_sk is not null and cs_order_number is not null and cs_sold_date_sk is not null)
- TableScan [TS_32] (rows=287989836 width=135)
- default@catalog_sales,cs,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_item_sk","cs_order_number","cs_quantity","cs_net_paid","cs_net_profit"]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_308]
- Group By Operator [GBY_307] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_275]
- Group By Operator [GBY_272] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_268] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_264]
- <-Reducer 6 [CONTAINS] vectorized
- Reduce Output Operator [RS_296]
- PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_295] (rows=6453220 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
- Select Operator [SEL_294] (rows=2151150 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_293] (rows=2151150 width=135)
- predicate:((_col0 <= 10) or (rank_window_1 <= 10))
- PTF Operator [PTF_292] (rows=3226726 width=135)
- Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
- Select Operator [SEL_291] (rows=3226726 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- <-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_290]
- PartitionCols:0
- Select Operator [SEL_289] (rows=3226726 width=135)
- Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
- PTF Operator [PTF_288] (rows=3226726 width=135)
- Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
- Select Operator [SEL_287] (rows=3226726 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- <-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_286]
- PartitionCols:0
- Group By Operator [GBY_285] (rows=3226726 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
- PartitionCols:_col0
- Group By Operator [GBY_17] (rows=6453452 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
- Select Operator [SEL_15] (rows=6453452 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_232] (rows=6453452 width=135)
- Conds:RS_12._col1, _col2=RS_284._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col11","_col12"]
- <-Map 26 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_284]
- PartitionCols:_col0, _col1
- Select Operator [SEL_283] (rows=4799489 width=92)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_282] (rows=4799489 width=92)
- predicate:((wr_return_amt > 10000) and wr_item_sk is not null and wr_order_number is not null)
- TableScan [TS_6] (rows=14398467 width=92)
- default@web_returns,wr,Tbl:COMPLETE,Col:NONE,Output:["wr_item_sk","wr_order_number","wr_return_quantity","wr_return_amt"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_12]
- PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_231] (rows=5866775 width=135)
- Conds:RS_281._col0=RS_265._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_265]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_264]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_281]
- PartitionCols:_col0
- Select Operator [SEL_280] (rows=5333432 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_279] (rows=5333432 width=135)
- predicate:((ws_net_paid > 0) and (ws_net_profit > 1) and (ws_quantity > 0) and (ws_sold_date_sk BETWEEN DynamicValue(RS_10_date_dim_d_date_sk_min) AND DynamicValue(RS_10_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_10_date_dim_d_date_sk_bloom_filter))) and ws_item_sk is not null and ws_order_number is not null and ws_sold_date_sk is not null)
- TableScan [TS_0] (rows=144002668 width=135)
- default@web_sales,ws,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_order_number","ws_quantity","ws_net_paid","ws_net_profit"]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_278]
- Group By Operator [GBY_277] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_274]
- Group By Operator [GBY_271] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_266] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_264]
+ Top N Key Operator [TNK_302] (rows=11830988 width=101)
+ keys:_col0, _col3, _col4, _col1, _col2,sort order:+++++,top n:100
+ Select Operator [SEL_301] (rows=3226610 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Group By Operator [GBY_300] (rows=3226610 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
+ <-Union 7 [SIMPLE_EDGE]
+ <-Reducer 18 [CONTAINS] vectorized
+ Reduce Output Operator [RS_330]
+ PartitionCols:_col0, _col1, _col2, _col3, _col4
+ Group By Operator [GBY_329] (rows=6453220 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
+ Select Operator [SEL_328] (rows=4302070 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_327] (rows=4302070 width=135)
+ predicate:((_col0 <= 10) or (rank_window_1 <= 10))
+ PTF Operator [PTF_326] (rows=6453105 width=135)
+ Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
+ Select Operator [SEL_325] (rows=6453105 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ <-Reducer 17 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_324]
+ PartitionCols:0
+ Select Operator [SEL_323] (rows=6453105 width=135)
+ Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
+ PTF Operator [PTF_322] (rows=6453105 width=135)
+ Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
+ Select Operator [SEL_321] (rows=6453105 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ <-Reducer 16 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_320]
+ PartitionCols:0
+ Group By Operator [GBY_319] (rows=6453105 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
+ <-Reducer 15 [SIMPLE_EDGE]
+ SHUFFLE [RS_50]
+ PartitionCols:_col0
+ Group By Operator [GBY_49] (rows=12906211 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
+ Select Operator [SEL_47] (rows=12906211 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_235] (rows=12906211 width=135)
+ Conds:RS_44._col1, _col2=RS_318._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col11","_col12"]
+ <-Map 28 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_318]
+ PartitionCols:_col0, _col1
+ Select Operator [SEL_317] (rows=9599627 width=106)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_316] (rows=9599627 width=106)
+ predicate:((cr_return_amount > 10000) and cr_item_sk is not null and cr_order_number is not null)
+ TableScan [TS_38] (rows=28798881 width=106)
+ default@catalog_returns,cr,Tbl:COMPLETE,Col:NONE,Output:["cr_item_sk","cr_order_number","cr_return_quantity","cr_return_amount"]
+ <-Reducer 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_44]
+ PartitionCols:_col1, _col2
+ Merge Join Operator [MERGEJOIN_234] (rows=11732919 width=135)
+ Conds:RS_315._col0=RS_270._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_270]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_267]
+ <-Map 27 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_315]
+ PartitionCols:_col0
+ Select Operator [SEL_314] (rows=10666290 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_313] (rows=10666290 width=135)
+ predicate:((cs_net_paid > 0) and (cs_net_profit > 1) and (cs_quantity > 0) and (cs_sold_date_sk BETWEEN DynamicValue(RS_42_date_dim_d_date_sk_min) AND DynamicValue(RS_42_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_42_date_dim_d_date_sk_bloom_filter))) and cs_item_sk is not null and cs_order_number is not null and cs_sold_date_sk is not null)
+ TableScan [TS_32] (rows=287989836 width=135)
+ default@catalog_sales,cs,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_item_sk","cs_order_number","cs_quantity","cs_net_paid","cs_net_profit"]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_312]
+ Group By Operator [GBY_311] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_278]
+ Group By Operator [GBY_275] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_271] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_267]
+ <-Reducer 6 [CONTAINS] vectorized
+ Reduce Output Operator [RS_299]
+ PartitionCols:_col0, _col1, _col2, _col3, _col4
+ Group By Operator [GBY_298] (rows=6453220 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],keys:_col0, _col3, _col4, _col1, _col2
+ Select Operator [SEL_297] (rows=2151150 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_296] (rows=2151150 width=135)
+ predicate:((_col0 <= 10) or (rank_window_1 <= 10))
+ PTF Operator [PTF_295] (rows=3226726 width=135)
+ Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col4 AS decimal(15,4)) / CAST( _col5 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
+ Select Operator [SEL_294] (rows=3226726 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ <-Reducer 5 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_293]
+ PartitionCols:0
+ Select Operator [SEL_292] (rows=3226726 width=135)
+ Output:["rank_window_0","_col0","_col1","_col2","_col3","_col4"]
+ PTF Operator [PTF_291] (rows=3226726 width=135)
+ Function definitions:[{},{"name:":"windowingtablefunction","order by:":"(CAST( _col1 AS decimal(15,4)) / CAST( _col2 AS decimal(15,4))) ASC NULLS FIRST","partition by:":"0"}]
+ Select Operator [SEL_290] (rows=3226726 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ <-Reducer 4 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_289]
+ PartitionCols:0
+ Group By Operator [GBY_288] (rows=3226726 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col0
+ Group By Operator [GBY_17] (rows=6453452 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col1)","sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0
+ Select Operator [SEL_15] (rows=6453452 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_233] (rows=6453452 width=135)
+ Conds:RS_12._col1, _col2=RS_287._col0, _col1(Inner),Output:["_col1","_col3","_col4","_col11","_col12"]
+ <-Map 26 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_287]
+ PartitionCols:_col0, _col1
+ Select Operator [SEL_286] (rows=4799489 width=92)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_285] (rows=4799489 width=92)
+ predicate:((wr_return_amt > 10000) and wr_item_sk is not null and wr_order_number is not null)
+ TableScan [TS_6] (rows=14398467 width=92)
+ default@web_returns,wr,Tbl:COMPLETE,Col:NONE,Output:["wr_item_sk","wr_order_number","wr_return_quantity","wr_return_amt"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_12]
+ PartitionCols:_col1, _col2
+ Merge Join Operator [MERGEJOIN_232] (rows=5866775 width=135)
+ Conds:RS_284._col0=RS_268._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_268]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_267]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_284]
+ PartitionCols:_col0
+ Select Operator [SEL_283] (rows=5333432 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_282] (rows=5333432 width=135)
+ predicate:((ws_net_paid > 0) and (ws_net_profit > 1) and (ws_quantity > 0) and (ws_sold_date_sk BETWEEN DynamicValue(RS_10_date_dim_d_date_sk_min) AND DynamicValue(RS_10_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_10_date_dim_d_date_sk_bloom_filter))) and ws_item_sk is not null and ws_order_number is not null and ws_sold_date_sk is not null)
+ TableScan [TS_0] (rows=144002668 width=135)
+ default@web_sales,ws,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_order_number","ws_quantity","ws_net_paid","ws_net_profit"]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_281]
+ Group By Operator [GBY_280] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_277]
+ Group By Operator [GBY_274] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_269] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_267]
[13/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query15.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query15.q.out b/ql/src/test/results/clientpositive/perf/tez/query15.q.out
index b593a2e..3c7ae66 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query15.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query15.q.out
@@ -52,81 +52,83 @@ Stage-0
limit:100
Stage-1
Reducer 5 vectorized
- File Output Operator [FS_98]
- Limit [LIM_97] (rows=100 width=135)
+ File Output Operator [FS_99]
+ Limit [LIM_98] (rows=100 width=135)
Number of rows:100
- Select Operator [SEL_96] (rows=174233858 width=135)
+ Select Operator [SEL_97] (rows=174233858 width=135)
Output:["_col0","_col1"]
<-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_95]
- Group By Operator [GBY_94] (rows=174233858 width=135)
+ SHUFFLE [RS_96]
+ Group By Operator [GBY_95] (rows=174233858 width=135)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_25]
PartitionCols:_col0
Group By Operator [GBY_24] (rows=348467716 width=135)
Output:["_col0","_col1"],aggregations:["sum(_col7)"],keys:_col4
- Select Operator [SEL_23] (rows=348467716 width=135)
- Output:["_col4","_col7"]
- Filter Operator [FIL_22] (rows=348467716 width=135)
- predicate:((_col3) IN ('CA', 'WA', 'GA') or (_col7 > 500) or (substr(_col4, 1, 5)) IN ('85669', '86197', '88274', '83405', '86475', '85392', '85460', '80348', '81792'))
- Merge Join Operator [MERGEJOIN_76] (rows=348467716 width=135)
- Conds:RS_19._col0=RS_20._col1(Inner),Output:["_col3","_col4","_col7"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_19]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_74] (rows=88000001 width=860)
- Conds:RS_79._col1=RS_82._col0(Inner),Output:["_col0","_col3","_col4"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_79]
- PartitionCols:_col1
- Select Operator [SEL_78] (rows=80000000 width=860)
- Output:["_col0","_col1"]
- Filter Operator [FIL_77] (rows=80000000 width=860)
- predicate:(c_current_addr_sk is not null and c_customer_sk is not null)
- TableScan [TS_0] (rows=80000000 width=860)
- default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_addr_sk"]
- <-Map 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_82]
- PartitionCols:_col0
- Select Operator [SEL_81] (rows=40000000 width=1014)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_80] (rows=40000000 width=1014)
- predicate:ca_address_sk is not null
- TableScan [TS_3] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state","ca_zip"]
- <-Reducer 8 [SIMPLE_EDGE]
- SHUFFLE [RS_20]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_75] (rows=316788826 width=135)
- Conds:RS_93._col0=RS_85._col0(Inner),Output:["_col1","_col2"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_85]
- PartitionCols:_col0
- Select Operator [SEL_84] (rows=18262 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_83] (rows=18262 width=1119)
- predicate:((d_qoy = 2) and (d_year = 2000) and d_date_sk is not null)
- TableScan [TS_9] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
- <-Map 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_93]
- PartitionCols:_col0
- Select Operator [SEL_92] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_91] (rows=287989836 width=135)
- predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_13_date_dim_d_date_sk_min) AND DynamicValue(RS_13_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_13_date_dim_d_date_sk_bloom_filter))) and cs_bill_customer_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_6] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_sales_price"]
- <-Reducer 10 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_90]
- Group By Operator [GBY_89] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_88]
- Group By Operator [GBY_87] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_86] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_84]
+ Top N Key Operator [TNK_44] (rows=348467716 width=135)
+ keys:_col4,sort order:+,top n:100
+ Select Operator [SEL_23] (rows=348467716 width=135)
+ Output:["_col4","_col7"]
+ Filter Operator [FIL_22] (rows=348467716 width=135)
+ predicate:((_col3) IN ('CA', 'WA', 'GA') or (_col7 > 500) or (substr(_col4, 1, 5)) IN ('85669', '86197', '88274', '83405', '86475', '85392', '85460', '80348', '81792'))
+ Merge Join Operator [MERGEJOIN_77] (rows=348467716 width=135)
+ Conds:RS_19._col0=RS_20._col1(Inner),Output:["_col3","_col4","_col7"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_19]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_75] (rows=88000001 width=860)
+ Conds:RS_80._col1=RS_83._col0(Inner),Output:["_col0","_col3","_col4"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_80]
+ PartitionCols:_col1
+ Select Operator [SEL_79] (rows=80000000 width=860)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_78] (rows=80000000 width=860)
+ predicate:(c_current_addr_sk is not null and c_customer_sk is not null)
+ TableScan [TS_0] (rows=80000000 width=860)
+ default@customer,customer,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_addr_sk"]
+ <-Map 6 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_83]
+ PartitionCols:_col0
+ Select Operator [SEL_82] (rows=40000000 width=1014)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_81] (rows=40000000 width=1014)
+ predicate:ca_address_sk is not null
+ TableScan [TS_3] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state","ca_zip"]
+ <-Reducer 8 [SIMPLE_EDGE]
+ SHUFFLE [RS_20]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_76] (rows=316788826 width=135)
+ Conds:RS_94._col0=RS_86._col0(Inner),Output:["_col1","_col2"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_86]
+ PartitionCols:_col0
+ Select Operator [SEL_85] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_84] (rows=18262 width=1119)
+ predicate:((d_qoy = 2) and (d_year = 2000) and d_date_sk is not null)
+ TableScan [TS_9] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
+ <-Map 7 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_94]
+ PartitionCols:_col0
+ Select Operator [SEL_93] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_92] (rows=287989836 width=135)
+ predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_13_date_dim_d_date_sk_min) AND DynamicValue(RS_13_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_13_date_dim_d_date_sk_bloom_filter))) and cs_bill_customer_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_6] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_sales_price"]
+ <-Reducer 10 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_91]
+ Group By Operator [GBY_90] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_89]
+ Group By Operator [GBY_88] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_87] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_85]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query17.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query17.q.out b/ql/src/test/results/clientpositive/perf/tez/query17.q.out
index 620d88a..e185775 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query17.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query17.q.out
@@ -112,199 +112,201 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_269]
- Limit [LIM_268] (rows=100 width=88)
+ File Output Operator [FS_270]
+ Limit [LIM_269] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_267] (rows=421657640 width=88)
+ Select Operator [SEL_268] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_266]
- Select Operator [SEL_265] (rows=421657640 width=88)
+ SHUFFLE [RS_267]
+ Select Operator [SEL_266] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- Group By Operator [GBY_264] (rows=421657640 width=88)
+ Group By Operator [GBY_265] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"],aggregations:["count(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","count(VALUE._col4)","sum(VALUE._col5)","sum(VALUE._col6)","sum(VALUE._col7)","count(VALUE._col8)","sum(VALUE._col9)","sum(VALUE._col10)","sum(VALUE._col11)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_50]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_49] (rows=843315281 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"],aggregations:["count(_col3)","sum(_col3)","sum(_col7)","sum(_col6)","count(_col4)","sum(_col4)","sum(_col9)","sum(_col8)","count(_col5)","sum(_col5)","sum(_col11)","sum(_col10)"],keys:_col0, _col1, _col2
- Select Operator [SEL_47] (rows=843315281 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
- Merge Join Operator [MERGEJOIN_212] (rows=843315281 width=88)
- Conds:RS_44._col3=RS_250._col0(Inner),Output:["_col5","_col9","_col10","_col14","_col21","_col25"]
- <-Map 20 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_250]
- PartitionCols:_col0
- Select Operator [SEL_249] (rows=1704 width=1910)
- Output:["_col0","_col1"]
- Filter Operator [FIL_248] (rows=1704 width=1910)
- predicate:s_store_sk is not null
- TableScan [TS_32] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_state"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_44]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_211] (rows=766650239 width=88)
- Conds:RS_41._col1, _col2, _col4=RS_42._col7, _col8, _col9(Inner),Output:["_col3","_col5","_col9","_col10","_col14","_col21"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_42]
- PartitionCols:_col7, _col8, _col9
- Merge Join Operator [MERGEJOIN_210] (rows=348467716 width=135)
- Conds:RS_28._col2, _col1=RS_29._col1, _col2(Inner),Output:["_col3","_col7","_col8","_col9","_col10"]
- <-Reducer 13 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_29]
- PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_209] (rows=63350266 width=77)
- Conds:RS_241._col0=RS_223._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_223]
- PartitionCols:_col0
- Select Operator [SEL_218] (rows=36525 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_215] (rows=36525 width=1119)
- predicate:((d_quarter_name) IN ('2000Q1', '2000Q2', '2000Q3') and d_date_sk is not null)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_quarter_name"]
- <-Map 19 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_241]
- PartitionCols:_col0
- Select Operator [SEL_240] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_239] (rows=57591150 width=77)
- predicate:(sr_customer_sk is not null and sr_item_sk is not null and sr_returned_date_sk is not null and sr_ticket_number is not null)
- TableScan [TS_15] (rows=57591150 width=77)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_return_quantity"]
- <-Reducer 10 [SIMPLE_EDGE]
- SHUFFLE [RS_28]
- PartitionCols:_col2, _col1
- Merge Join Operator [MERGEJOIN_208] (rows=316788826 width=135)
- Conds:RS_263._col0=RS_221._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_221]
- PartitionCols:_col0
- Select Operator [SEL_217] (rows=36525 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_214] (rows=36525 width=1119)
- predicate:((d_quarter_name) IN ('2000Q1', '2000Q2', '2000Q3') and d_date_sk is not null)
- Please refer to the previous TableScan [TS_3]
- <-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_263]
- PartitionCols:_col0
- Select Operator [SEL_262] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_261] (rows=287989836 width=135)
- predicate:((cs_bill_customer_sk BETWEEN DynamicValue(RS_29_store_returns_sr_customer_sk_min) AND DynamicValue(RS_29_store_returns_sr_customer_sk_max) and in_bloom_filter(cs_bill_customer_sk, DynamicValue(RS_29_store_returns_sr_customer_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_29_store_returns_sr_item_sk_min) AND DynamicValue(RS_29_store_returns_sr_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_29_store_returns_sr_item_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_39_item_i_item_sk_min) AND DynamicValue(RS_39_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_39_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_26_d3_d_date_sk_min) AND DynamicValue(RS_26_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_26_d3_d_date_sk_bloom_filter))) and cs_bill_customer_sk is not null and cs_item_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_9] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_quantity"]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_244]
- Group By Operator [GBY_242] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_110]
- Group By Operator [GBY_109] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_108] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_209]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_247]
- Group By Operator [GBY_245] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_125]
- Group By Operator [GBY_124] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_123] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_209]
- <-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_238]
- Group By Operator [GBY_236] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 16 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_235]
- Group By Operator [GBY_234] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_233] (rows=462000 width=1436)
- Output:["_col0"]
- Select Operator [SEL_231] (rows=462000 width=1436)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_230] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- TableScan [TS_6] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc"]
- <-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_260]
- Group By Operator [GBY_259] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_227]
- Group By Operator [GBY_225] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_222] (rows=36525 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_217]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_41]
- PartitionCols:_col1, _col2, _col4
- Merge Join Operator [MERGEJOIN_207] (rows=696954748 width=88)
- Conds:RS_38._col1=RS_232._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col9","_col10"]
- <-Map 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_232]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_231]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_38]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_206] (rows=633595212 width=88)
- Conds:RS_258._col0=RS_219._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_219]
- PartitionCols:_col0
- Select Operator [SEL_216] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_213] (rows=36524 width=1119)
- predicate:((d_quarter_name = '2000Q1') and d_date_sk is not null)
- Please refer to the previous TableScan [TS_3]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_258]
- PartitionCols:_col0
- Select Operator [SEL_257] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_256] (rows=575995635 width=88)
- predicate:((ss_customer_sk BETWEEN DynamicValue(RS_29_store_returns_sr_customer_sk_min) AND DynamicValue(RS_29_store_returns_sr_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_29_store_returns_sr_customer_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_29_store_returns_sr_item_sk_min) AND DynamicValue(RS_29_store_returns_sr_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_29_store_returns_sr_item_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_39_item_i_item_sk_min) AND DynamicValue(RS_39_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_39_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_36_d1_d_date_sk_min) AND DynamicValue(RS_36_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_36_d1_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_45_store_s_store_sk_min) AND DynamicValue(RS_45_store_s_store_sk_max)
and in_bloom_filter(ss_store_sk, DynamicValue(RS_45_store_s_store_sk_bloom_filter))) and ss_customer_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null and ss_ticket_number is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_quantity"]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_243]
- Please refer to the previous Group By Operator [GBY_242]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_246]
- Please refer to the previous Group By Operator [GBY_245]
- <-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_237]
- Please refer to the previous Group By Operator [GBY_236]
- <-Reducer 21 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_255]
- Group By Operator [GBY_254] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_253]
- Group By Operator [GBY_252] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_251] (rows=1704 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_249]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_229]
- Group By Operator [GBY_228] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_226]
- Group By Operator [GBY_224] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_220] (rows=36524 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_216]
+ Top N Key Operator [TNK_93] (rows=843315281 width=88)
+ keys:_col0, _col1, _col2,sort order:+++,top n:100
+ Select Operator [SEL_47] (rows=843315281 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Merge Join Operator [MERGEJOIN_213] (rows=843315281 width=88)
+ Conds:RS_44._col3=RS_251._col0(Inner),Output:["_col5","_col9","_col10","_col14","_col21","_col25"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_251]
+ PartitionCols:_col0
+ Select Operator [SEL_250] (rows=1704 width=1910)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_249] (rows=1704 width=1910)
+ predicate:s_store_sk is not null
+ TableScan [TS_32] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_state"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_44]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_212] (rows=766650239 width=88)
+ Conds:RS_41._col1, _col2, _col4=RS_42._col7, _col8, _col9(Inner),Output:["_col3","_col5","_col9","_col10","_col14","_col21"]
+ <-Reducer 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_42]
+ PartitionCols:_col7, _col8, _col9
+ Merge Join Operator [MERGEJOIN_211] (rows=348467716 width=135)
+ Conds:RS_28._col2, _col1=RS_29._col1, _col2(Inner),Output:["_col3","_col7","_col8","_col9","_col10"]
+ <-Reducer 13 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_29]
+ PartitionCols:_col1, _col2
+ Merge Join Operator [MERGEJOIN_210] (rows=63350266 width=77)
+ Conds:RS_242._col0=RS_224._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_224]
+ PartitionCols:_col0
+ Select Operator [SEL_219] (rows=36525 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_216] (rows=36525 width=1119)
+ predicate:((d_quarter_name) IN ('2000Q1', '2000Q2', '2000Q3') and d_date_sk is not null)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_quarter_name"]
+ <-Map 19 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_242]
+ PartitionCols:_col0
+ Select Operator [SEL_241] (rows=57591150 width=77)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_240] (rows=57591150 width=77)
+ predicate:(sr_customer_sk is not null and sr_item_sk is not null and sr_returned_date_sk is not null and sr_ticket_number is not null)
+ TableScan [TS_15] (rows=57591150 width=77)
+ default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_return_quantity"]
+ <-Reducer 10 [SIMPLE_EDGE]
+ SHUFFLE [RS_28]
+ PartitionCols:_col2, _col1
+ Merge Join Operator [MERGEJOIN_209] (rows=316788826 width=135)
+ Conds:RS_264._col0=RS_222._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_222]
+ PartitionCols:_col0
+ Select Operator [SEL_218] (rows=36525 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_215] (rows=36525 width=1119)
+ predicate:((d_quarter_name) IN ('2000Q1', '2000Q2', '2000Q3') and d_date_sk is not null)
+ Please refer to the previous TableScan [TS_3]
+ <-Map 18 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_264]
+ PartitionCols:_col0
+ Select Operator [SEL_263] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_262] (rows=287989836 width=135)
+ predicate:((cs_bill_customer_sk BETWEEN DynamicValue(RS_29_store_returns_sr_customer_sk_min) AND DynamicValue(RS_29_store_returns_sr_customer_sk_max) and in_bloom_filter(cs_bill_customer_sk, DynamicValue(RS_29_store_returns_sr_customer_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_29_store_returns_sr_item_sk_min) AND DynamicValue(RS_29_store_returns_sr_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_29_store_returns_sr_item_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_39_item_i_item_sk_min) AND DynamicValue(RS_39_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_39_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_26_d3_d_date_sk_min) AND DynamicValue(RS_26_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_26_d3_d_date_sk_bloom_filter))) and cs_bill_customer_sk is not null and cs_item_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_9] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_quantity"]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_245]
+ Group By Operator [GBY_243] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_111]
+ Group By Operator [GBY_110] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_109] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_210]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_248]
+ Group By Operator [GBY_246] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_126]
+ Group By Operator [GBY_125] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_124] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_210]
+ <-Reducer 17 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_239]
+ Group By Operator [GBY_237] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 16 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_236]
+ Group By Operator [GBY_235] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_234] (rows=462000 width=1436)
+ Output:["_col0"]
+ Select Operator [SEL_232] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_231] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_6] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc"]
+ <-Reducer 12 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_261]
+ Group By Operator [GBY_260] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_228]
+ Group By Operator [GBY_226] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_223] (rows=36525 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_218]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_41]
+ PartitionCols:_col1, _col2, _col4
+ Merge Join Operator [MERGEJOIN_208] (rows=696954748 width=88)
+ Conds:RS_38._col1=RS_233._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5","_col9","_col10"]
+ <-Map 16 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_233]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_232]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_38]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_207] (rows=633595212 width=88)
+ Conds:RS_259._col0=RS_220._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_220]
+ PartitionCols:_col0
+ Select Operator [SEL_217] (rows=36524 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_214] (rows=36524 width=1119)
+ predicate:((d_quarter_name = '2000Q1') and d_date_sk is not null)
+ Please refer to the previous TableScan [TS_3]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_259]
+ PartitionCols:_col0
+ Select Operator [SEL_258] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_257] (rows=575995635 width=88)
+ predicate:((ss_customer_sk BETWEEN DynamicValue(RS_29_store_returns_sr_customer_sk_min) AND DynamicValue(RS_29_store_returns_sr_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_29_store_returns_sr_customer_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_29_store_returns_sr_item_sk_min) AND DynamicValue(RS_29_store_returns_sr_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_29_store_returns_sr_item_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_39_item_i_item_sk_min) AND DynamicValue(RS_39_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_39_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_36_d1_d_date_sk_min) AND DynamicValue(RS_36_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_36_d1_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_45_store_s_store_sk_min) AND DynamicValue(RS_45_store_s_store_sk_ma
x) and in_bloom_filter(ss_store_sk, DynamicValue(RS_45_store_s_store_sk_bloom_filter))) and ss_customer_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null and ss_ticket_number is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_quantity"]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_244]
+ Please refer to the previous Group By Operator [GBY_243]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_247]
+ Please refer to the previous Group By Operator [GBY_246]
+ <-Reducer 17 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_238]
+ Please refer to the previous Group By Operator [GBY_237]
+ <-Reducer 21 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_256]
+ Group By Operator [GBY_255] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_254]
+ Group By Operator [GBY_253] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_252] (rows=1704 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_250]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_230]
+ Group By Operator [GBY_229] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_227]
+ Group By Operator [GBY_225] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_221] (rows=36524 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_217]
[15/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vector_topnkey.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_topnkey.q.out b/ql/src/test/results/clientpositive/llap/vector_topnkey.q.out
new file mode 100644
index 0000000..16803c9
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/vector_topnkey.q.out
@@ -0,0 +1,592 @@
+PREHOOK: query: explain vectorization detail
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+PLAN VECTORIZATION:
+ enabled: true
+ enabledConditionsMet: [hive.vectorized.execution.enabled IS true]
+
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:key:string, 1:value:string, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
+ Select Operator
+ expressions: key (type: string), UDFToInteger(substr(value, 5)) (type: int)
+ outputColumnNames: _col0, _col1
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0, 4]
+ selectExpressions: CastStringToLong(col 3:string)(children: StringSubstrColStart(col 1:string, start 4) -> 3:string) -> 4:int
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Top N Key Operator
+ sort order: +
+ keys: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 5
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:string
+ native: true
+ Group By Operator
+ aggregations: sum(_col1)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 4:int) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: [1]
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
+ Execution mode: vectorized, llap
+ LLAP IO: no inputs
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: [DECIMAL_64]
+ featureSupportInUse: [DECIMAL_64]
+ inputFileFormats: org.apache.hadoop.mapred.TextInputFormat
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ includeColumns: [0, 1]
+ dataColumns: key:string, value:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: [string, bigint]
+ Reducer 2
+ Execution mode: vectorized, llap
+ Reduce Vectorization:
+ enabled: true
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true
+ reduceColumnNullOrder: a
+ reduceColumnSortOrder: +
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ dataColumns: KEY._col0:string, VALUE._col0:bigint
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: sum(VALUE._col0)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 1:bigint) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: MERGEPARTIAL
+ keyExpressions: col 0:string
+ native: false
+ vectorProcessingMode: MERGE_PARTIAL
+ projectedOutputColumnNums: [0]
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkObjectHashOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: [1]
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Vectorization:
+ enabled: true
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true
+ reduceColumnNullOrder: a
+ reduceColumnSortOrder: +
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ dataColumns: KEY.reducesinkkey0:string, VALUE._col0:bigint
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint)
+ outputColumnNames: _col0, _col1
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0, 1]
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Limit Vectorization:
+ className: VectorLimitOperator
+ native: true
+ Statistics: Num rows: 5 Data size: 475 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ File Sink Vectorization:
+ className: VectorFileSinkOperator
+ native: false
+ Statistics: Num rows: 5 Data size: 475 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0 0
+10 10
+100 200
+103 206
+104 208
+PREHOOK: query: explain vectorization detail
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+PLAN VECTORIZATION:
+ enabled: true
+ enabledConditionsMet: [hive.vectorized.execution.enabled IS true]
+
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:key:string, 1:value:string, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: key
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0]
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Top N Key Operator
+ sort order: +
+ keys: key (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 5
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:string
+ native: true
+ Group By Operator
+ Group By Vectorization:
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: []
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: []
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ Execution mode: vectorized, llap
+ LLAP IO: no inputs
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: [DECIMAL_64]
+ featureSupportInUse: [DECIMAL_64]
+ inputFileFormats: org.apache.hadoop.mapred.TextInputFormat
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ includeColumns: [0]
+ dataColumns: key:string, value:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reducer 2
+ Execution mode: vectorized, llap
+ Reduce Vectorization:
+ enabled: true
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true
+ reduceColumnNullOrder: a
+ reduceColumnSortOrder: +
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 1
+ dataColumns: KEY._col0:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Operator Tree:
+ Group By Operator
+ Group By Vectorization:
+ className: VectorGroupByOperator
+ groupByMode: MERGEPARTIAL
+ keyExpressions: col 0:string
+ native: false
+ vectorProcessingMode: MERGE_PARTIAL
+ projectedOutputColumnNums: []
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkObjectHashOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: []
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Vectorization:
+ enabled: true
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true
+ reduceColumnNullOrder: a
+ reduceColumnSortOrder: +
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 1
+ dataColumns: KEY.reducesinkkey0:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string)
+ outputColumnNames: _col0
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0]
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Limit Vectorization:
+ className: VectorLimitOperator
+ native: true
+ Statistics: Num rows: 5 Data size: 435 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ File Sink Vectorization:
+ className: VectorFileSinkOperator
+ native: false
+ Statistics: Num rows: 5 Data size: 435 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0
+10
+100
+103
+104
+PREHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+PLAN VECTORIZATION:
+ enabled: true
+ enabledConditionsMet: [hive.vectorized.execution.enabled IS true]
+
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src1
+ filterExpr: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:key:string, 1:value:string, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
+ Filter Operator
+ Filter Vectorization:
+ className: VectorFilterOperator
+ native: true
+ predicateExpression: SelectColumnIsNotNull(col 0:string)
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0]
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: []
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: vectorized, llap
+ LLAP IO: no inputs
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: [DECIMAL_64]
+ featureSupportInUse: [DECIMAL_64]
+ inputFileFormats: org.apache.hadoop.mapred.TextInputFormat
+ allNative: true
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ includeColumns: [0]
+ dataColumns: key:string, value:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ filterExpr: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TableScan Vectorization:
+ native: true
+ vectorizationSchemaColumns: [0:key:string, 1:value:string, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
+ Filter Operator
+ Filter Vectorization:
+ className: VectorFilterOperator
+ native: true
+ predicateExpression: SelectColumnIsNotNull(col 0:string)
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0, 1]
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: [1]
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: string)
+ Execution mode: vectorized, llap
+ LLAP IO: no inputs
+ Map Vectorization:
+ enabled: true
+ enabledConditionsMet: hive.vectorized.use.vector.serde.deserialize IS true
+ inputFormatFeatureSupport: [DECIMAL_64]
+ featureSupportInUse: [DECIMAL_64]
+ inputFileFormats: org.apache.hadoop.mapred.TextInputFormat
+ allNative: true
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ includeColumns: [0, 1]
+ dataColumns: key:string, value:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col2
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: string), _col2 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: string)
+ Reducer 3
+ Execution mode: vectorized, llap
+ Reduce Vectorization:
+ enabled: true
+ enableConditionsMet: hive.vectorized.execution.reduce.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true
+ reduceColumnNullOrder: a
+ reduceColumnSortOrder: +
+ allNative: false
+ usesVectorUDFAdaptor: false
+ vectorized: true
+ rowBatchContext:
+ dataColumnCount: 2
+ dataColumns: KEY.reducesinkkey0:string, VALUE._col0:string
+ partitionColumnCount: 0
+ scratchColumnTypeNames: []
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
+ outputColumnNames: _col0, _col1
+ Select Vectorization:
+ className: VectorSelectOperator
+ native: true
+ projectedOutputColumnNums: [0, 1]
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Limit Vectorization:
+ className: VectorLimitOperator
+ native: true
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ File Sink Vectorization:
+ className: VectorFileSinkOperator
+ native: false
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out b/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out
index e79cdf7..f7c00f8 100644
--- a/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out
+++ b/ql/src/test/results/clientpositive/llap/vectorization_limit.q.out
@@ -491,31 +491,40 @@ STAGE PLANS:
native: true
projectedOutputColumnNums: [0]
Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- Group By Vectorization:
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 0:tinyint
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: []
+ Top N Key Operator
+ sort order: +
keys: ctinyint (type: tinyint)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 131 Data size: 396 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: tinyint)
- sort order: +
- Map-reduce partition columns: _col0 (type: tinyint)
- Reduce Sink Vectorization:
- className: VectorReduceSinkObjectHashOperator
- keyColumnNums: [0]
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- partitionColumnNums: [0]
- valueColumnNums: []
- Statistics: Num rows: 131 Data size: 396 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:tinyint
+ native: true
+ Group By Operator
+ Group By Vectorization:
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:tinyint
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: []
+ keys: ctinyint (type: tinyint)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 131 Data size: 264 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: tinyint)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: tinyint)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkObjectHashOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ partitionColumnNums: [0]
+ valueColumnNums: []
+ Statistics: Num rows: 131 Data size: 264 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
@@ -560,19 +569,19 @@ STAGE PLANS:
keys: KEY._col0 (type: tinyint)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 131 Data size: 396 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 131 Data size: 264 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
Limit Vectorization:
className: VectorLimitOperator
native: true
- Statistics: Num rows: 20 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 20 Data size: 44 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
File Sink Vectorization:
className: VectorFileSinkOperator
native: false
- Statistics: Num rows: 20 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 20 Data size: 44 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query10.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query10.q.out b/ql/src/test/results/clientpositive/perf/tez/query10.q.out
index 1b6adee..a8f097f 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query10.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query10.q.out
@@ -140,188 +140,190 @@ Stage-0
limit:100
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_224]
- Limit [LIM_223] (rows=100 width=88)
+ File Output Operator [FS_225]
+ Limit [LIM_224] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_222] (rows=1045432122 width=88)
+ Select Operator [SEL_223] (rows=1045432122 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_221]
- Select Operator [SEL_220] (rows=1045432122 width=88)
+ SHUFFLE [RS_222]
+ Select Operator [SEL_221] (rows=1045432122 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col6","_col8","_col10","_col12"]
- Group By Operator [GBY_219] (rows=1045432122 width=88)
+ Group By Operator [GBY_220] (rows=1045432122 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5, KEY._col6, KEY._col7
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_63]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
Group By Operator [GBY_62] (rows=2090864244 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["count()"],keys:_col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13
- Select Operator [SEL_61] (rows=2090864244 width=88)
- Output:["_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- Filter Operator [FIL_60] (rows=2090864244 width=88)
- predicate:(_col15 is not null or _col17 is not null)
- Merge Join Operator [MERGEJOIN_172] (rows=2090864244 width=88)
- Conds:RS_55._col0=RS_56._col0(Left Semi),RS_55._col0=RS_210._col0(Left Outer),RS_55._col0=RS_218._col0(Left Outer),Output:["_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col15","_col17"]
- <-Reducer 3 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_55]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_168] (rows=96800003 width=860)
- Conds:RS_50._col1=RS_181._col0(Inner),Output:["_col0","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_181]
- PartitionCols:_col0
- Select Operator [SEL_180] (rows=1861800 width=385)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- Filter Operator [FIL_179] (rows=1861800 width=385)
- predicate:cd_demo_sk is not null
- TableScan [TS_6] (rows=1861800 width=385)
- default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status","cd_purchase_estimate","cd_credit_rating","cd_dep_count","cd_dep_employed_count","cd_dep_college_count"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_50]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_167] (rows=88000001 width=860)
- Conds:RS_175._col2=RS_178._col0(Inner),Output:["_col0","_col1"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_175]
- PartitionCols:_col2
- Select Operator [SEL_174] (rows=80000000 width=860)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_173] (rows=80000000 width=860)
- predicate:(c_current_addr_sk is not null and c_current_cdemo_sk is not null and c_customer_sk is not null)
- TableScan [TS_0] (rows=80000000 width=860)
- default@customer,c,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_cdemo_sk","c_current_addr_sk"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_178]
- PartitionCols:_col0
- Select Operator [SEL_177] (rows=20000000 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_176] (rows=20000000 width=1014)
- predicate:((ca_county) IN ('Walker County', 'Richland County', 'Gaines County', 'Douglas County', 'Dona Ana County') and ca_address_sk is not null)
- TableScan [TS_3] (rows=40000000 width=1014)
- default@customer_address,ca,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_county"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_56]
- PartitionCols:_col0
- Group By Operator [GBY_54] (rows=633595212 width=88)
- Output:["_col0"],keys:_col0
- Select Operator [SEL_18] (rows=633595212 width=88)
- Output:["_col0"]
- Merge Join Operator [MERGEJOIN_169] (rows=633595212 width=88)
- Conds:RS_202._col0=RS_184._col0(Inner),Output:["_col1"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_184]
+ Top N Key Operator [TNK_103] (rows=2090864244 width=88)
+ keys:_col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13,sort order:++++++++,top n:100
+ Select Operator [SEL_61] (rows=2090864244 width=88)
+ Output:["_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
+ Filter Operator [FIL_60] (rows=2090864244 width=88)
+ predicate:(_col15 is not null or _col17 is not null)
+ Merge Join Operator [MERGEJOIN_173] (rows=2090864244 width=88)
+ Conds:RS_55._col0=RS_56._col0(Left Semi),RS_55._col0=RS_211._col0(Left Outer),RS_55._col0=RS_219._col0(Left Outer),Output:["_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col15","_col17"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_55]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_169] (rows=96800003 width=860)
+ Conds:RS_50._col1=RS_182._col0(Inner),Output:["_col0","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_182]
+ PartitionCols:_col0
+ Select Operator [SEL_181] (rows=1861800 width=385)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
+ Filter Operator [FIL_180] (rows=1861800 width=385)
+ predicate:cd_demo_sk is not null
+ TableScan [TS_6] (rows=1861800 width=385)
+ default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status","cd_purchase_estimate","cd_credit_rating","cd_dep_count","cd_dep_employed_count","cd_dep_college_count"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_50]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_168] (rows=88000001 width=860)
+ Conds:RS_176._col2=RS_179._col0(Inner),Output:["_col0","_col1"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_176]
+ PartitionCols:_col2
+ Select Operator [SEL_175] (rows=80000000 width=860)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_174] (rows=80000000 width=860)
+ predicate:(c_current_addr_sk is not null and c_current_cdemo_sk is not null and c_customer_sk is not null)
+ TableScan [TS_0] (rows=80000000 width=860)
+ default@customer,c,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_cdemo_sk","c_current_addr_sk"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_179]
+ PartitionCols:_col0
+ Select Operator [SEL_178] (rows=20000000 width=1014)
+ Output:["_col0"]
+ Filter Operator [FIL_177] (rows=20000000 width=1014)
+ predicate:((ca_county) IN ('Walker County', 'Richland County', 'Gaines County', 'Douglas County', 'Dona Ana County') and ca_address_sk is not null)
+ TableScan [TS_3] (rows=40000000 width=1014)
+ default@customer_address,ca,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_county"]
+ <-Reducer 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_56]
+ PartitionCols:_col0
+ Group By Operator [GBY_54] (rows=633595212 width=88)
+ Output:["_col0"],keys:_col0
+ Select Operator [SEL_18] (rows=633595212 width=88)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_170] (rows=633595212 width=88)
+ Conds:RS_203._col0=RS_185._col0(Inner),Output:["_col1"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_185]
+ PartitionCols:_col0
+ Select Operator [SEL_184] (rows=4058 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_183] (rows=4058 width=1119)
+ predicate:((d_year = 2002) and d_date_sk is not null and d_moy BETWEEN 4 AND 7)
+ TableScan [TS_12] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_203]
+ PartitionCols:_col0
+ Select Operator [SEL_202] (rows=575995635 width=88)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_201] (rows=575995635 width=88)
+ predicate:((ss_customer_sk BETWEEN DynamicValue(RS_55_c_c_customer_sk_min) AND DynamicValue(RS_55_c_c_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_55_c_c_customer_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_16_date_dim_d_date_sk_min) AND DynamicValue(RS_16_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_16_date_dim_d_date_sk_bloom_filter))) and ss_customer_sk is not null and ss_sold_date_sk is not null)
+ TableScan [TS_9] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_customer_sk"]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_198]
+ Group By Operator [GBY_197] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_194]
+ Group By Operator [GBY_191] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_186] (rows=4058 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_184]
+ <-Reducer 7 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_200]
+ Group By Operator [GBY_199] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=96800000)"]
+ <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_136]
+ Group By Operator [GBY_135] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=96800000)"]
+ Select Operator [SEL_134] (rows=96800003 width=860)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_169]
+ <-Reducer 15 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_211]
+ PartitionCols:_col0
+ Select Operator [SEL_210] (rows=79201469 width=135)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_209] (rows=79201469 width=135)
+ Output:["_col0"],keys:KEY._col0
+ <-Reducer 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_30]
PartitionCols:_col0
- Select Operator [SEL_183] (rows=4058 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_182] (rows=4058 width=1119)
- predicate:((d_year = 2002) and d_date_sk is not null and d_moy BETWEEN 4 AND 7)
- TableScan [TS_12] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_202]
+ Group By Operator [GBY_29] (rows=158402938 width=135)
+ Output:["_col0"],keys:_col1
+ Merge Join Operator [MERGEJOIN_171] (rows=158402938 width=135)
+ Conds:RS_208._col0=RS_187._col0(Inner),Output:["_col1"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_187]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_184]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_208]
+ PartitionCols:_col0
+ Select Operator [SEL_207] (rows=144002668 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_206] (rows=144002668 width=135)
+ predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_26_date_dim_d_date_sk_min) AND DynamicValue(RS_26_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_26_date_dim_d_date_sk_bloom_filter))) and ws_bill_customer_sk is not null and ws_sold_date_sk is not null)
+ TableScan [TS_19] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_bill_customer_sk"]
+ <-Reducer 16 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_205]
+ Group By Operator [GBY_204] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_195]
+ Group By Operator [GBY_192] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_188] (rows=4058 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_184]
+ <-Reducer 18 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_219]
+ PartitionCols:_col0
+ Select Operator [SEL_218] (rows=158394413 width=135)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_217] (rows=158394413 width=135)
+ Output:["_col0"],keys:KEY._col0
+ <-Reducer 17 [SIMPLE_EDGE]
+ SHUFFLE [RS_44]
PartitionCols:_col0
- Select Operator [SEL_201] (rows=575995635 width=88)
- Output:["_col0","_col1"]
- Filter Operator [FIL_200] (rows=575995635 width=88)
- predicate:((ss_customer_sk BETWEEN DynamicValue(RS_55_c_c_customer_sk_min) AND DynamicValue(RS_55_c_c_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_55_c_c_customer_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_16_date_dim_d_date_sk_min) AND DynamicValue(RS_16_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_16_date_dim_d_date_sk_bloom_filter))) and ss_customer_sk is not null and ss_sold_date_sk is not null)
- TableScan [TS_9] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_customer_sk"]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_197]
- Group By Operator [GBY_196] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_193]
- Group By Operator [GBY_190] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_185] (rows=4058 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_183]
- <-Reducer 7 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_199]
- Group By Operator [GBY_198] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=96800000)"]
- <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_135]
- Group By Operator [GBY_134] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=96800000)"]
- Select Operator [SEL_133] (rows=96800003 width=860)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_168]
- <-Reducer 15 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_210]
- PartitionCols:_col0
- Select Operator [SEL_209] (rows=79201469 width=135)
- Output:["_col0","_col1"]
- Group By Operator [GBY_208] (rows=79201469 width=135)
- Output:["_col0"],keys:KEY._col0
- <-Reducer 14 [SIMPLE_EDGE]
- SHUFFLE [RS_30]
- PartitionCols:_col0
- Group By Operator [GBY_29] (rows=158402938 width=135)
- Output:["_col0"],keys:_col1
- Merge Join Operator [MERGEJOIN_170] (rows=158402938 width=135)
- Conds:RS_207._col0=RS_186._col0(Inner),Output:["_col1"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_186]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_183]
- <-Map 20 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_207]
- PartitionCols:_col0
- Select Operator [SEL_206] (rows=144002668 width=135)
- Output:["_col0","_col1"]
- Filter Operator [FIL_205] (rows=144002668 width=135)
- predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_26_date_dim_d_date_sk_min) AND DynamicValue(RS_26_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_26_date_dim_d_date_sk_bloom_filter))) and ws_bill_customer_sk is not null and ws_sold_date_sk is not null)
- TableScan [TS_19] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_bill_customer_sk"]
- <-Reducer 16 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_204]
- Group By Operator [GBY_203] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_194]
- Group By Operator [GBY_191] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_187] (rows=4058 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_183]
- <-Reducer 18 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_218]
- PartitionCols:_col0
- Select Operator [SEL_217] (rows=158394413 width=135)
- Output:["_col0","_col1"]
- Group By Operator [GBY_216] (rows=158394413 width=135)
- Output:["_col0"],keys:KEY._col0
- <-Reducer 17 [SIMPLE_EDGE]
- SHUFFLE [RS_44]
- PartitionCols:_col0
- Group By Operator [GBY_43] (rows=316788826 width=135)
- Output:["_col0"],keys:_col1
- Merge Join Operator [MERGEJOIN_171] (rows=316788826 width=135)
- Conds:RS_215._col0=RS_188._col0(Inner),Output:["_col1"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_188]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_183]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_215]
- PartitionCols:_col0
- Select Operator [SEL_214] (rows=287989836 width=135)
- Output:["_col0","_col1"]
- Filter Operator [FIL_213] (rows=287989836 width=135)
- predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_40_date_dim_d_date_sk_min) AND DynamicValue(RS_40_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_40_date_dim_d_date_sk_bloom_filter))) and cs_ship_customer_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_33] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_ship_customer_sk"]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_212]
- Group By Operator [GBY_211] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_195]
- Group By Operator [GBY_192] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_189] (rows=4058 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_183]
+ Group By Operator [GBY_43] (rows=316788826 width=135)
+ Output:["_col0"],keys:_col1
+ Merge Join Operator [MERGEJOIN_172] (rows=316788826 width=135)
+ Conds:RS_216._col0=RS_189._col0(Inner),Output:["_col1"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_189]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_184]
+ <-Map 21 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_216]
+ PartitionCols:_col0
+ Select Operator [SEL_215] (rows=287989836 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_214] (rows=287989836 width=135)
+ predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_40_date_dim_d_date_sk_min) AND DynamicValue(RS_40_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_40_date_dim_d_date_sk_bloom_filter))) and cs_ship_customer_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_33] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_ship_customer_sk"]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_213]
+ Group By Operator [GBY_212] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_196]
+ Group By Operator [GBY_193] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_190] (rows=4058 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_184]
[18/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
HIVE-17896: TopNKey: Create a standalone vectorizable TopNKey operator (Teddy Choi, reviewed by Jesus Camacho Rodriguez)
Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/851c8aba
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/851c8aba
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/851c8aba
Branch: refs/heads/master
Commit: 851c8aba86aa027cc5aa21e8b71e04a1243c35b9
Parents: e867d1c
Author: Teddy Choi <pu...@gmail.com>
Authored: Thu Jul 19 13:55:57 2018 -0700
Committer: Jesus Camacho Rodriguez <jc...@apache.org>
Committed: Thu Jul 19 13:55:57 2018 -0700
----------------------------------------------------------------------
.../org/apache/hadoop/hive/conf/HiveConf.java | 2 +
.../test/resources/testconfiguration.properties | 6 +-
.../hadoop/hive/ql/plan/api/OperatorType.java | 5 +-
.../hadoop/hive/ql/exec/KeyWrapperFactory.java | 2 +-
.../hadoop/hive/ql/exec/OperatorFactory.java | 4 +
.../hadoop/hive/ql/exec/TopNKeyOperator.java | 214 ++
.../ql/exec/vector/VectorTopNKeyOperator.java | 304 +++
.../hive/ql/optimizer/TopNKeyProcessor.java | 109 +
.../hive/ql/optimizer/physical/Vectorizer.java | 37 +
.../hadoop/hive/ql/parse/TezCompiler.java | 27 +
.../apache/hadoop/hive/ql/plan/TopNKeyDesc.java | 139 ++
.../hadoop/hive/ql/plan/VectorTopNKeyDesc.java | 39 +
ql/src/test/queries/clientpositive/topnkey.q | 31 +
.../queries/clientpositive/vector_topnkey.q | 30 +
.../clientpositive/llap/bucket_groupby.q.out | 274 ++-
.../clientpositive/llap/check_constraint.q.out | 27 +-
.../clientpositive/llap/explainuser_1.q.out | 28 +-
.../clientpositive/llap/explainuser_2.q.out | 406 ++--
.../clientpositive/llap/limit_pushdown.q.out | 135 +-
.../clientpositive/llap/limit_pushdown3.q.out | 89 +-
.../llap/llap_decimal64_reader.q.out | 46 +-
.../clientpositive/llap/offset_limit.q.out | 27 +-
.../llap/offset_limit_ppd_optimizer.q.out | 85 +-
.../llap/orc_struct_type_vectorization.q.out | 53 +-
.../parquet_complex_types_vectorization.q.out | 159 +-
.../llap/parquet_map_type_vectorization.q.out | 53 +-
.../parquet_struct_type_vectorization.q.out | 53 +-
.../results/clientpositive/llap/topnkey.q.out | 318 +++
.../llap/vector_cast_constant.q.out | 55 +-
.../clientpositive/llap/vector_char_2.q.out | 110 +-
.../vector_groupby_grouping_sets_limit.q.out | 346 +--
.../llap/vector_groupby_reduce.q.out | 49 +-
.../llap/vector_mr_diff_schema_alias.q.out | 25 +-
.../llap/vector_reduce_groupby_decimal.q.out | 53 +-
.../llap/vector_string_concat.q.out | 47 +-
.../clientpositive/llap/vector_topnkey.q.out | 592 +++++
.../llap/vectorization_limit.q.out | 63 +-
.../clientpositive/perf/tez/query10.q.out | 346 +--
.../clientpositive/perf/tez/query14.q.out | 2198 +++++++++---------
.../clientpositive/perf/tez/query15.q.out | 138 +-
.../clientpositive/perf/tez/query17.q.out | 372 +--
.../clientpositive/perf/tez/query25.q.out | 366 +--
.../clientpositive/perf/tez/query26.q.out | 226 +-
.../clientpositive/perf/tez/query27.q.out | 230 +-
.../clientpositive/perf/tez/query29.q.out | 374 +--
.../clientpositive/perf/tez/query35.q.out | 346 +--
.../clientpositive/perf/tez/query37.q.out | 142 +-
.../clientpositive/perf/tez/query40.q.out | 206 +-
.../clientpositive/perf/tez/query43.q.out | 128 +-
.../clientpositive/perf/tez/query45.q.out | 272 +--
.../clientpositive/perf/tez/query49.q.out | 478 ++--
.../clientpositive/perf/tez/query5.q.out | 542 ++---
.../clientpositive/perf/tez/query50.q.out | 250 +-
.../clientpositive/perf/tez/query60.q.out | 546 ++---
.../clientpositive/perf/tez/query66.q.out | 452 ++--
.../clientpositive/perf/tez/query69.q.out | 364 +--
.../clientpositive/perf/tez/query7.q.out | 226 +-
.../clientpositive/perf/tez/query76.q.out | 356 +--
.../clientpositive/perf/tez/query77.q.out | 562 ++---
.../clientpositive/perf/tez/query8.q.out | 276 +--
.../clientpositive/perf/tez/query80.q.out | 756 +++---
.../clientpositive/perf/tez/query82.q.out | 142 +-
.../clientpositive/perf/tez/query99.q.out | 230 +-
.../results/clientpositive/tez/topnkey.q.out | 162 ++
.../clientpositive/tez/vector_topnkey.q.out | 162 ++
.../test/results/clientpositive/topnkey.q.out | 301 +++
.../results/clientpositive/vector_topnkey.q.out | 480 ++++
.../objectinspector/ObjectInspectorUtils.java | 19 +
68 files changed, 9530 insertions(+), 6160 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
----------------------------------------------------------------------
diff --git a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
index 4ed1636..e630e88 100644
--- a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+++ b/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
@@ -2233,6 +2233,8 @@ public class HiveConf extends Configuration {
"If the skew information is correctly stored in the metadata, hive.optimize.skewjoin.compiletime\n" +
"would change the query plan to take care of it, and hive.optimize.skewjoin will be a no-op."),
+ HIVE_OPTIMIZE_TOPNKEY("hive.optimize.topnkey", true, "Whether to enable top n key optimizer."),
+
HIVE_SHARED_WORK_OPTIMIZATION("hive.optimize.shared.work", true,
"Whether to enable shared work optimizer. The optimizer finds scan operator over the same table\n" +
"and follow-up operators in the query plan and merges them if they meet some preconditions. Tez only."),
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/itests/src/test/resources/testconfiguration.properties
----------------------------------------------------------------------
diff --git a/itests/src/test/resources/testconfiguration.properties b/itests/src/test/resources/testconfiguration.properties
index 870a9b6..d5a33bd 100644
--- a/itests/src/test/resources/testconfiguration.properties
+++ b/itests/src/test/resources/testconfiguration.properties
@@ -26,9 +26,11 @@ disabled.query.files=ql_rewrite_gbtoidx.q,\
minitez.query.files.shared=delete_orig_table.q,\
orc_merge12.q,\
orc_vectorization_ppd.q,\
+ topnkey.q,\
update_orig_table.q,\
vector_join_part_col_char.q,\
- vector_non_string_partition.q
+ vector_non_string_partition.q,\
+ vector_topnkey.q
# NOTE: Add tests to minitez only if it is very
# specific to tez and cannot be added to minillap.
@@ -209,6 +211,7 @@ minillaplocal.shared.query.files=alter_merge_2_orc.q,\
subquery_exists.q,\
subquery_in.q,\
temp_table.q,\
+ topnkey.q,\
union2.q,\
union3.q,\
union4.q,\
@@ -315,6 +318,7 @@ minillaplocal.shared.query.files=alter_merge_2_orc.q,\
vector_reduce_groupby_duplicate_cols.q,\
vector_string_concat.q,\
vector_struct_in.q,\
+ vector_topnkey.q,\
vector_udf_character_length.q,\
vector_udf_octet_length.q,\
vector_varchar_4.q,\
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java
----------------------------------------------------------------------
diff --git a/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java b/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java
index a002348..f8328be 100644
--- a/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java
+++ b/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java
@@ -37,7 +37,8 @@ public enum OperatorType implements org.apache.thrift.TEnum {
ORCFILEMERGE(22),
RCFILEMERGE(23),
MERGEJOIN(24),
- SPARKPRUNINGSINK(25);
+ SPARKPRUNINGSINK(25),
+ TOPNKEY(26);
private final int value;
@@ -110,6 +111,8 @@ public enum OperatorType implements org.apache.thrift.TEnum {
return MERGEJOIN;
case 25:
return SPARKPRUNINGSINK;
+ case 26:
+ return TOPNKEY;
default:
return null;
}
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java
index 71ee25d..f1bf902 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapperFactory.java
@@ -168,7 +168,6 @@ public class KeyWrapperFactory {
}
}
- transient Object[] singleEleArray = new Object[1];
transient StringObjectInspector soi_new, soi_copy;
class TextKeyWrapper extends KeyWrapper {
@@ -180,6 +179,7 @@ public class KeyWrapperFactory {
int hashcode;
Object key;
boolean isCopy;
+ transient Object[] singleEleArray = new Object[1];
public TextKeyWrapper(boolean isCopy) {
this(-1, null, isCopy);
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
index 7bb6590..b61d37e 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
@@ -38,6 +38,7 @@ import org.apache.hadoop.hive.ql.exec.vector.VectorSMBMapJoinOperator;
import org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator;
import org.apache.hadoop.hive.ql.exec.vector.VectorSparkHashTableSinkOperator;
import org.apache.hadoop.hive.ql.exec.vector.VectorSparkPartitionPruningSinkOperator;
+import org.apache.hadoop.hive.ql.exec.vector.VectorTopNKeyOperator;
import org.apache.hadoop.hive.ql.exec.vector.VectorizationContext;
import org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkCommonOperator;
import org.apache.hadoop.hive.ql.exec.vector.ptf.VectorPTFOperator;
@@ -76,6 +77,7 @@ import org.apache.hadoop.hive.ql.plan.ScriptDesc;
import org.apache.hadoop.hive.ql.plan.SelectDesc;
import org.apache.hadoop.hive.ql.plan.SparkHashTableSinkDesc;
import org.apache.hadoop.hive.ql.plan.TableScanDesc;
+import org.apache.hadoop.hive.ql.plan.TopNKeyDesc;
import org.apache.hadoop.hive.ql.plan.UDTFDesc;
import org.apache.hadoop.hive.ql.plan.UnionDesc;
import org.apache.hadoop.hive.ql.plan.VectorDesc;
@@ -126,6 +128,7 @@ public final class OperatorFactory {
opvec.put(OrcFileMergeDesc.class, OrcFileMergeOperator.class);
opvec.put(CommonMergeJoinDesc.class, CommonMergeJoinOperator.class);
opvec.put(ListSinkDesc.class, ListSinkOperator.class);
+ opvec.put(TopNKeyDesc.class, TopNKeyOperator.class);
}
static {
@@ -143,6 +146,7 @@ public final class OperatorFactory {
vectorOpvec.put(LimitDesc.class, VectorLimitOperator.class);
vectorOpvec.put(PTFDesc.class, VectorPTFOperator.class);
vectorOpvec.put(SparkHashTableSinkDesc.class, VectorSparkHashTableSinkOperator.class);
+ vectorOpvec.put(TopNKeyDesc.class, VectorTopNKeyOperator.class);
}
public static <T extends OperatorDesc> Operator<T> getVectorOperator(
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java
new file mode 100644
index 0000000..3dfeeaf
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/TopNKeyOperator.java
@@ -0,0 +1,214 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hive.ql.CompilationOpContext;
+import org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDesc;
+import org.apache.hadoop.hive.ql.plan.TopNKeyDesc;
+import org.apache.hadoop.hive.ql.plan.api.OperatorType;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils;
+
+import java.io.Serializable;
+import java.util.Comparator;
+import java.util.PriorityQueue;
+
+import static org.apache.hadoop.hive.ql.plan.api.OperatorType.TOPNKEY;
+
+/**
+ * TopNKeyOperator passes rows that contains top N keys only.
+ */
+public class TopNKeyOperator extends Operator<TopNKeyDesc> implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+
+ // Maximum number of keys to hold
+ private transient int topN;
+
+ // Priority queue that holds occurred keys
+ private transient PriorityQueue<KeyWrapper> priorityQueue;
+
+ // Fast key wrapper in input format for fast comparison
+ private transient KeyWrapper keyWrapper;
+
+ // Standard key wrapper in standard format for output
+ private transient KeyWrapper standardKeyWrapper;
+
+ // Maximum number of rows
+ private transient int rowLimit;
+
+ // Current number of rows
+ private transient int rowSize;
+
+ // Rows
+ private transient Object[] rows;
+
+ /** Kryo ctor. */
+ public TopNKeyOperator() {
+ super();
+ }
+
+ public TopNKeyOperator(CompilationOpContext ctx) {
+ super(ctx);
+ }
+
+ public static class KeyWrapperComparator implements Comparator<KeyWrapper> {
+ private ObjectInspector[] objectInspectors1;
+ private ObjectInspector[] objectInspectors2;
+ private boolean[] columnSortOrderIsDesc;
+
+ public KeyWrapperComparator(ObjectInspector[] objectInspectors1, ObjectInspector[]
+ objectInspectors2, boolean[] columnSortOrderIsDesc) {
+ this.objectInspectors1 = objectInspectors1;
+ this.objectInspectors2 = objectInspectors2;
+ this.columnSortOrderIsDesc = columnSortOrderIsDesc;
+ }
+
+ @Override
+ public int compare(KeyWrapper key1, KeyWrapper key2) {
+ return ObjectInspectorUtils.compare(key1.getKeyArray(), objectInspectors1,
+ key2.getKeyArray(), objectInspectors2, columnSortOrderIsDesc);
+ }
+ }
+
+ @Override
+ protected void initializeOp(Configuration hconf) throws HiveException {
+ super.initializeOp(hconf);
+
+ this.topN = conf.getTopN();
+
+ String columnSortOrder = conf.getColumnSortOrder();
+ boolean[] columnSortOrderIsDesc = new boolean[columnSortOrder.length()];
+ for (int i = 0; i < columnSortOrderIsDesc.length; i++) {
+ columnSortOrderIsDesc[i] = (columnSortOrder.charAt(i) == '-');
+ }
+
+ ObjectInspector rowInspector = inputObjInspectors[0];
+ outputObjInspector = ObjectInspectorUtils.getStandardObjectInspector(rowInspector);
+
+ // init keyFields
+ int numKeys = conf.getKeyColumns().size();
+ ExprNodeEvaluator[] keyFields = new ExprNodeEvaluator[numKeys];
+ ObjectInspector[] keyObjectInspectors = new ObjectInspector[numKeys];
+ ExprNodeEvaluator[] standardKeyFields = new ExprNodeEvaluator[numKeys];
+ ObjectInspector[] standardKeyObjectInspectors = new ObjectInspector[numKeys];
+
+ for (int i = 0; i < numKeys; i++) {
+ ExprNodeDesc key = conf.getKeyColumns().get(i);
+ keyFields[i] = ExprNodeEvaluatorFactory.get(key, hconf);
+ keyObjectInspectors[i] = keyFields[i].initialize(rowInspector);
+ standardKeyFields[i] = ExprNodeEvaluatorFactory.get(key, hconf);
+ standardKeyObjectInspectors[i] = standardKeyFields[i].initialize(outputObjInspector);
+ }
+
+ priorityQueue = new PriorityQueue<>(topN + 1, new TopNKeyOperator.KeyWrapperComparator(
+ standardKeyObjectInspectors, standardKeyObjectInspectors, columnSortOrderIsDesc));
+
+ keyWrapper = new KeyWrapperFactory(keyFields, keyObjectInspectors,
+ standardKeyObjectInspectors).getKeyWrapper();
+ standardKeyWrapper = new KeyWrapperFactory(standardKeyFields, standardKeyObjectInspectors,
+ standardKeyObjectInspectors).getKeyWrapper();
+
+ rowLimit = VectorizedRowBatch.DEFAULT_SIZE;
+ rows = new Object[rowLimit];
+ rowSize = 0;
+ }
+
+ @Override
+ public void process(Object row, int tag) throws HiveException {
+ keyWrapper.getNewKey(row, inputObjInspectors[0]);
+ keyWrapper.setHashKey();
+
+ if (!priorityQueue.contains(keyWrapper)) {
+ priorityQueue.offer(keyWrapper.copyKey());
+ }
+ if (priorityQueue.size() > topN) {
+ priorityQueue.poll();
+ }
+
+ rows[rowSize] = ObjectInspectorUtils.copyToStandardObject(row, inputObjInspectors[0]);
+ rowSize++;
+
+ if (rowSize % rowLimit == 0) {
+ processRows();
+ }
+ }
+
+ private void processRows() throws HiveException {
+ for (int i = 0; i < rowSize; i++) {
+ Object row = rows[i];
+
+ standardKeyWrapper.getNewKey(row, outputObjInspector);
+ standardKeyWrapper.setHashKey();
+
+ if (priorityQueue.contains(standardKeyWrapper)) {
+ forward(row, outputObjInspector);
+ }
+ }
+ priorityQueue.clear();
+ rowSize = 0;
+ }
+
+ @Override
+ protected final void closeOp(boolean abort) throws HiveException {
+ processRows();
+ super.closeOp(abort);
+ }
+
+ @Override
+ public String getName() {
+ return getOperatorName();
+ }
+
+ static public String getOperatorName() {
+ return "TNK";
+ }
+
+ @Override
+ public OperatorType getType() {
+ return TOPNKEY;
+ }
+
+ // Because a TopNKeyOperator works like a FilterOperator with top n key condition, its properties
+ // for optimizers has same values. Following methods are same with FilterOperator;
+ // supportSkewJoinOptimization, columnNamesRowResolvedCanBeObtained,
+ // supportAutomaticSortMergeJoin, and supportUnionRemoveOptimization.
+ @Override
+ public boolean supportSkewJoinOptimization() {
+ return true;
+ }
+
+ @Override
+ public boolean columnNamesRowResolvedCanBeObtained() {
+ return true;
+ }
+
+ @Override
+ public boolean supportAutomaticSortMergeJoin() {
+ return true;
+ }
+
+ @Override
+ public boolean supportUnionRemoveOptimization() {
+ return true;
+ }
+}
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java
new file mode 100644
index 0000000..6f29f88
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorTopNKeyOperator.java
@@ -0,0 +1,304 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.exec.vector;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Joiner;
+import com.google.common.primitives.Ints;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hive.ql.CompilationOpContext;
+import org.apache.hadoop.hive.ql.exec.Operator;
+import org.apache.hadoop.hive.ql.exec.TopNKeyOperator;
+import org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.ql.plan.OperatorDesc;
+import org.apache.hadoop.hive.ql.plan.TopNKeyDesc;
+import org.apache.hadoop.hive.ql.plan.VectorDesc;
+import org.apache.hadoop.hive.ql.plan.VectorTopNKeyDesc;
+import org.apache.hadoop.hive.ql.plan.api.OperatorType;
+import org.apache.hadoop.hive.serde.serdeConstants;
+import org.apache.hadoop.hive.serde2.SerDeException;
+import org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory;
+import org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfo;
+import org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils;
+import org.apache.hadoop.io.Writable;
+import org.apache.hadoop.io.WritableUtils;
+
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.PriorityQueue;
+import java.util.Properties;
+
+import static org.apache.hadoop.hive.ql.plan.api.OperatorType.TOPNKEY;
+
+/**
+ * VectorTopNKeyOperator passes rows that contains top N keys only.
+ */
+public class VectorTopNKeyOperator extends Operator<TopNKeyDesc> implements VectorizationOperator {
+
+ private static final long serialVersionUID = 1L;
+
+ private VectorTopNKeyDesc vectorDesc;
+ private VectorizationContext vContext;
+
+ // Key column info
+ private int[] keyColumnNums;
+ private TypeInfo[] keyTypeInfos;
+
+ // Extract row
+ private transient Object[] singleRow;
+ private transient VectorExtractRow vectorExtractRow;
+
+ // Serialization
+ private transient BinarySortableSerDe binarySortableSerDe;
+ private transient StructObjectInspector keyObjectInspector;
+
+ // Batch processing
+ private transient boolean firstBatch;
+ private transient PriorityQueue<Writable> priorityQueue;
+ private transient int[] temporarySelected;
+
+ public VectorTopNKeyOperator(CompilationOpContext ctx, OperatorDesc conf,
+ VectorizationContext vContext, VectorDesc vectorDesc) {
+
+ this(ctx);
+ this.conf = (TopNKeyDesc) conf;
+ this.vContext = vContext;
+ this.vectorDesc = (VectorTopNKeyDesc) vectorDesc;
+
+ VectorExpression[] keyExpressions = this.vectorDesc.getKeyExpressions();
+ final int numKeys = keyExpressions.length;
+ keyColumnNums = new int[numKeys];
+ keyTypeInfos = new TypeInfo[numKeys];
+
+ for (int i = 0; i < numKeys; i++) {
+ keyColumnNums[i] = keyExpressions[i].getOutputColumnNum();
+ keyTypeInfos[i] = keyExpressions[i].getOutputTypeInfo();
+ }
+ }
+
+ /** Kryo ctor. */
+ @VisibleForTesting
+ public VectorTopNKeyOperator() {
+ super();
+ }
+
+ public VectorTopNKeyOperator(CompilationOpContext ctx) {
+ super(ctx);
+ }
+
+ @Override
+ protected void initializeOp(Configuration hconf) throws HiveException {
+ super.initializeOp(hconf);
+
+ VectorExpression.doTransientInit(vectorDesc.getKeyExpressions());
+ for (VectorExpression keyExpression : vectorDesc.getKeyExpressions()) {
+ keyExpression.init(hconf);
+ }
+
+ this.firstBatch = true;
+
+ VectorExpression[] keyExpressions = vectorDesc.getKeyExpressions();
+ final int size = keyExpressions.length;
+ ObjectInspector[] fieldObjectInspectors = new ObjectInspector[size];
+
+ for (int i = 0; i < size; i++) {
+ VectorExpression keyExpression = keyExpressions[i];
+ fieldObjectInspectors[i] = TypeInfoUtils.getStandardWritableObjectInspectorFromTypeInfo(
+ keyExpression.getOutputTypeInfo());
+ }
+
+ keyObjectInspector = ObjectInspectorFactory.getStandardStructObjectInspector(
+ this.conf.getKeyColumnNames(), Arrays.asList(fieldObjectInspectors));
+
+ temporarySelected = new int [VectorizedRowBatch.DEFAULT_SIZE];
+ }
+
+ @Override
+ public void process(Object data, int tag) throws HiveException {
+ VectorizedRowBatch batch = (VectorizedRowBatch) data;
+
+ // The selected vector represents selected rows.
+ // Clone the selected vector
+ System.arraycopy(batch.selected, 0, temporarySelected, 0, batch.size);
+ int [] selectedBackup = batch.selected;
+ batch.selected = temporarySelected;
+ int sizeBackup = batch.size;
+ boolean selectedInUseBackup = batch.selectedInUse;
+
+ for (VectorExpression keyExpression : vectorDesc.getKeyExpressions()) {
+ keyExpression.evaluate(batch);
+ }
+
+ if (firstBatch) {
+ vectorExtractRow = new VectorExtractRow();
+ vectorExtractRow.init(keyObjectInspector, Ints.asList(keyColumnNums));
+
+ singleRow = new Object[vectorExtractRow.getCount()];
+ Comparator comparator = Comparator.reverseOrder();
+ priorityQueue = new PriorityQueue<Writable>(comparator);
+
+ try {
+ binarySortableSerDe = new BinarySortableSerDe();
+ Properties properties = new Properties();
+ Joiner joiner = Joiner.on(',');
+ properties.setProperty(serdeConstants.LIST_COLUMNS, joiner.join(conf.getKeyColumnNames()));
+ properties.setProperty(serdeConstants.LIST_COLUMN_TYPES, joiner.join(keyTypeInfos));
+ properties.setProperty(serdeConstants.SERIALIZATION_SORT_ORDER,
+ conf.getColumnSortOrder());
+ binarySortableSerDe.initialize(getConfiguration(), properties);
+ } catch (SerDeException e) {
+ throw new HiveException(e);
+ }
+
+ firstBatch = false;
+ }
+
+ // Clear the priority queue
+ priorityQueue.clear();
+
+ // Get top n keys
+ for (int i = 0; i < batch.size; i++) {
+
+ // Get keys
+ int j;
+ if (batch.selectedInUse) {
+ j = batch.selected[i];
+ } else {
+ j = i;
+ }
+ vectorExtractRow.extractRow(batch, j, singleRow);
+
+ Writable keysWritable;
+ try {
+ keysWritable = binarySortableSerDe.serialize(singleRow, keyObjectInspector);
+ } catch (SerDeException e) {
+ throw new HiveException(e);
+ }
+
+ // Put the copied keys into the priority queue
+ if (!priorityQueue.contains(keysWritable)) {
+ priorityQueue.offer(WritableUtils.clone(keysWritable, getConfiguration()));
+ }
+
+ // Limit the queue size
+ if (priorityQueue.size() > conf.getTopN()) {
+ priorityQueue.poll();
+ }
+ }
+
+ // Filter rows with top n keys
+ int size = 0;
+ int[] selected = new int[batch.selected.length];
+ for (int i = 0; i < batch.size; i++) {
+ int j;
+ if (batch.selectedInUse) {
+ j = batch.selected[i];
+ } else {
+ j = i;
+ }
+
+ // Get keys
+ vectorExtractRow.extractRow(batch, j, singleRow);
+
+ Writable keysWritable;
+ try {
+ keysWritable = binarySortableSerDe.serialize(singleRow, keyObjectInspector);
+ } catch (SerDeException e) {
+ throw new HiveException(e);
+ }
+
+ // Select a row in the priority queue
+ if (priorityQueue.contains(keysWritable)) {
+ selected[size++] = j;
+ }
+ }
+
+ // Apply selection to batch
+ if (batch.size != size) {
+ batch.selectedInUse = true;
+ batch.selected = selected;
+ batch.size = size;
+ }
+
+ // Forward the result
+ if (size > 0) {
+ forward(batch, null, true);
+ }
+
+ // Restore the original selected vector
+ batch.selected = selectedBackup;
+ batch.size = sizeBackup;
+ batch.selectedInUse = selectedInUseBackup;
+ }
+
+ @Override
+ public String getName() {
+ return TopNKeyOperator.getOperatorName();
+ }
+
+ @Override
+ public OperatorType getType() {
+ return TOPNKEY;
+ }
+
+ @Override
+ public VectorizationContext getInputVectorizationContext() {
+ return vContext;
+ }
+
+ @Override
+ public VectorDesc getVectorDesc() {
+ return vectorDesc;
+ }
+
+ // Because a TopNKeyOperator works like a FilterOperator with top n key condition, its properties
+ // for optimizers has same values. Following methods are same with FilterOperator;
+ // supportSkewJoinOptimization, columnNamesRowResolvedCanBeObtained,
+ // supportAutomaticSortMergeJoin, and supportUnionRemoveOptimization.
+ @Override
+ public boolean supportSkewJoinOptimization() {
+ return true;
+ }
+
+ @Override
+ public boolean columnNamesRowResolvedCanBeObtained() {
+ return true;
+ }
+
+ @Override
+ public boolean supportAutomaticSortMergeJoin() {
+ return true;
+ }
+
+ @Override
+ public boolean supportUnionRemoveOptimization() {
+ return true;
+ }
+
+ // Must send on to VectorPTFOperator...
+ @Override
+ public void setNextVectorBatchGroupStatus(boolean isLastGroupBatch) throws HiveException {
+ for (Operator<? extends OperatorDesc> op : childOperators) {
+ op.setNextVectorBatchGroupStatus(isLastGroupBatch);
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java
new file mode 100644
index 0000000..721a9b9
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer;
+
+import org.apache.hadoop.hive.ql.exec.GroupByOperator;
+import org.apache.hadoop.hive.ql.exec.Operator;
+import org.apache.hadoop.hive.ql.exec.OperatorFactory;
+import org.apache.hadoop.hive.ql.exec.ReduceSinkOperator;
+import org.apache.hadoop.hive.ql.exec.RowSchema;
+import org.apache.hadoop.hive.ql.exec.TopNKeyOperator;
+import org.apache.hadoop.hive.ql.lib.Node;
+import org.apache.hadoop.hive.ql.lib.NodeProcessor;
+import org.apache.hadoop.hive.ql.lib.NodeProcessorCtx;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDesc;
+import org.apache.hadoop.hive.ql.plan.ExprNodeDescUtils;
+import org.apache.hadoop.hive.ql.plan.GroupByDesc;
+import org.apache.hadoop.hive.ql.plan.OperatorDesc;
+import org.apache.hadoop.hive.ql.plan.ReduceSinkDesc;
+import org.apache.hadoop.hive.ql.plan.TopNKeyDesc;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Stack;
+
+/**
+ * TopNKeyProcessor is a processor for TopNKeyOperator. A TopNKeyOperator will be placed between
+ * a GroupByOperator and its following ReduceSinkOperator. If there already is a TopNKeyOperator,
+ * then it will be skipped.
+ */
+public class TopNKeyProcessor implements NodeProcessor {
+ private static final Logger LOG = LoggerFactory.getLogger(TopNKeyProcessor.class);
+
+ public TopNKeyProcessor() {
+ }
+
+ @Override
+ public Object process(Node nd, Stack<Node> stack, NodeProcessorCtx procCtx,
+ Object... nodeOutputs) throws SemanticException {
+
+ // Get ReduceSinkOperator
+ ReduceSinkOperator reduceSinkOperator = (ReduceSinkOperator) nd;
+ ReduceSinkDesc reduceSinkDesc = reduceSinkOperator.getConf();
+
+ // Get GroupByOperator
+ GroupByOperator groupByOperator = (GroupByOperator) reduceSinkOperator.getParentOperators().get(0);
+ GroupByDesc groupByDesc = groupByOperator.getConf();
+
+ // Check whether the reduce sink operator contains top n
+ if (!reduceSinkDesc.isOrdering() || reduceSinkDesc.getTopN() < 0) {
+ return null;
+ }
+
+ // Check whether the group by operator is in hash mode
+ if (groupByDesc.getMode() != GroupByDesc.Mode.HASH) {
+ return null;
+ }
+
+ // Check whether the group by operator has distinct aggregations
+ if (groupByDesc.isDistinct()) {
+ return null;
+ }
+
+ // Check whether RS keys are same as GBY keys
+ List<ExprNodeDesc> groupByKeyColumns = groupByDesc.getKeys();
+ List<ExprNodeDesc> mappedColumns = new ArrayList<>();
+ for (ExprNodeDesc columns : reduceSinkDesc.getKeyCols()) {
+ mappedColumns.add(groupByDesc.getColumnExprMap().get(columns.getExprString()));
+ }
+ if (!ExprNodeDescUtils.isSame(mappedColumns, groupByKeyColumns)) {
+ return null;
+ }
+
+ // Check whether there already is a top n key operator
+ Operator<? extends OperatorDesc> parentOperator = groupByOperator.getParentOperators().get(0);
+ if (parentOperator instanceof TopNKeyOperator) {
+ return null;
+ }
+
+ // Insert a new top n key operator between the group by operator and its parent
+ TopNKeyDesc topNKeyDesc = new TopNKeyDesc(reduceSinkDesc.getTopN(), reduceSinkDesc.getOrder(),
+ groupByKeyColumns);
+ Operator<? extends OperatorDesc> newOperator = OperatorFactory.getAndMakeChild(
+ groupByOperator.getCompilationOpContext(), (OperatorDesc) topNKeyDesc,
+ new RowSchema(groupByOperator.getSchema()), groupByOperator.getParentOperators());
+ newOperator.getChildOperators().add(groupByOperator);
+ groupByOperator.getParentOperators().add(newOperator);
+ parentOperator.removeChild(groupByOperator);
+
+ return null;
+ }
+}
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
index 7ec80e6..40bd075 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
@@ -124,6 +124,7 @@ import org.apache.hadoop.hive.ql.plan.MapredWork;
import org.apache.hadoop.hive.ql.plan.OperatorDesc;
import org.apache.hadoop.hive.ql.plan.PTFDesc;
import org.apache.hadoop.hive.ql.plan.SelectDesc;
+import org.apache.hadoop.hive.ql.plan.TopNKeyDesc;
import org.apache.hadoop.hive.ql.plan.VectorAppMasterEventDesc;
import org.apache.hadoop.hive.ql.plan.VectorDesc;
import org.apache.hadoop.hive.ql.plan.VectorFileSinkDesc;
@@ -135,6 +136,7 @@ import org.apache.hadoop.hive.ql.plan.VectorTableScanDesc;
import org.apache.hadoop.hive.ql.plan.VectorGroupByDesc.ProcessingMode;
import org.apache.hadoop.hive.ql.plan.VectorSparkHashTableSinkDesc;
import org.apache.hadoop.hive.ql.plan.VectorSparkPartitionPruningSinkDesc;
+import org.apache.hadoop.hive.ql.plan.VectorTopNKeyDesc;
import org.apache.hadoop.hive.ql.plan.VectorLimitDesc;
import org.apache.hadoop.hive.ql.plan.VectorMapJoinInfo;
import org.apache.hadoop.hive.ql.plan.VectorSMBJoinDesc;
@@ -2555,6 +2557,10 @@ public class Vectorizer implements PhysicalPlanResolver {
desc, "Predicate", VectorExpressionDescriptor.Mode.FILTER, /* allowComplex */ true);
}
+ private boolean validateTopNKeyOperator(TopNKeyOperator op) {
+ List<ExprNodeDesc> keyColumns = op.getConf().getKeyColumns();
+ return validateExprNodeDesc(keyColumns, "Key columns");
+ }
private boolean validateGroupByOperator(GroupByOperator op, boolean isReduce,
boolean isTezOrSpark, VectorGroupByDesc vectorGroupByDesc) {
@@ -4155,6 +4161,20 @@ public class Vectorizer implements PhysicalPlanResolver {
vContext, vectorFilterDesc);
}
+ private static Operator<? extends OperatorDesc> vectorizeTopNKeyOperator(
+ Operator<? extends OperatorDesc> topNKeyOperator, VectorizationContext vContext,
+ VectorTopNKeyDesc vectorTopNKeyDesc) throws HiveException {
+
+ TopNKeyDesc topNKeyDesc = (TopNKeyDesc) topNKeyOperator.getConf();
+
+ List<ExprNodeDesc> keyColumns = topNKeyDesc.getKeyColumns();
+ VectorExpression[] keyExpressions = vContext.getVectorExpressions(keyColumns);
+ vectorTopNKeyDesc.setKeyExpressions(keyExpressions);
+ return OperatorFactory.getVectorOperator(
+ topNKeyOperator.getCompilationOpContext(), topNKeyDesc,
+ vContext, vectorTopNKeyDesc);
+ }
+
private static Class<? extends VectorAggregateExpression> findVecAggrClass(
Class<? extends VectorAggregateExpression>[] vecAggrClasses,
String aggregateName, ColumnVector.Type inputColVectorType,
@@ -5051,6 +5071,23 @@ public class Vectorizer implements PhysicalPlanResolver {
}
}
break;
+ case TOPNKEY:
+ {
+ if (!validateTopNKeyOperator((TopNKeyOperator) op)) {
+ throw new VectorizerCannotVectorizeException();
+ }
+
+ VectorTopNKeyDesc vectorTopNKeyDesc = new VectorTopNKeyDesc();
+ vectorOp = vectorizeTopNKeyOperator(op, vContext, vectorTopNKeyDesc);
+ isNative = true;
+ if (vectorTaskColumnInfo != null) {
+ VectorExpression[] keyExpressions = vectorTopNKeyDesc.getKeyExpressions();
+ if (usesVectorUDFAdaptor(keyExpressions)) {
+ vectorTaskColumnInfo.setUsesVectorUDFAdaptor(true);
+ }
+ }
+ }
+ break;
case SELECT:
{
if (!validateSelectOperator((SelectOperator) op)) {
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java b/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
index 1b433c7..1661aec 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
@@ -56,6 +56,7 @@ import org.apache.hadoop.hive.ql.exec.tez.TezTask;
import org.apache.hadoop.hive.ql.hooks.ReadEntity;
import org.apache.hadoop.hive.ql.hooks.WriteEntity;
import org.apache.hadoop.hive.ql.lib.CompositeProcessor;
+import org.apache.hadoop.hive.ql.lib.DefaultGraphWalker;
import org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher;
import org.apache.hadoop.hive.ql.lib.Dispatcher;
import org.apache.hadoop.hive.ql.lib.ForwardWalker;
@@ -77,6 +78,7 @@ import org.apache.hadoop.hive.ql.optimizer.ReduceSinkMapJoinProc;
import org.apache.hadoop.hive.ql.optimizer.RemoveDynamicPruningBySize;
import org.apache.hadoop.hive.ql.optimizer.SetReducerParallelism;
import org.apache.hadoop.hive.ql.optimizer.SharedWorkOptimizer;
+import org.apache.hadoop.hive.ql.optimizer.TopNKeyProcessor;
import org.apache.hadoop.hive.ql.optimizer.correlation.ReduceSinkJoinDeDuplication;
import org.apache.hadoop.hive.ql.optimizer.metainfo.annotation.AnnotateWithOpTraits;
import org.apache.hadoop.hive.ql.optimizer.physical.AnnotateRunTimeStatsOptimizer;
@@ -144,6 +146,10 @@ public class TezCompiler extends TaskCompiler {
OptimizeTezProcContext procCtx = new OptimizeTezProcContext(conf, pCtx, inputs, outputs);
perfLogger.PerfLogBegin(this.getClass().getName(), PerfLogger.TEZ_COMPILER);
+ runTopNKeyOptimization(procCtx);
+ perfLogger.PerfLogEnd(this.getClass().getName(), PerfLogger.TEZ_COMPILER, "Run top n key optimization");
+
+ perfLogger.PerfLogBegin(this.getClass().getName(), PerfLogger.TEZ_COMPILER);
// setup dynamic partition pruning where possible
runDynamicPartitionPruning(procCtx, inputs, outputs);
perfLogger.PerfLogEnd(this.getClass().getName(), PerfLogger.TEZ_COMPILER, "Setup dynamic partition pruning");
@@ -1231,6 +1237,27 @@ public class TezCompiler extends TaskCompiler {
}
}
+ private static void runTopNKeyOptimization(OptimizeTezProcContext procCtx)
+ throws SemanticException {
+ if (!procCtx.conf.getBoolVar(ConfVars.HIVE_OPTIMIZE_TOPNKEY)) {
+ return;
+ }
+
+ Map<Rule, NodeProcessor> opRules = new LinkedHashMap<Rule, NodeProcessor>();
+ opRules.put(
+ new RuleRegExp("Top n key optimization", GroupByOperator.getOperatorName() + "%" +
+ ReduceSinkOperator.getOperatorName() + "%"),
+ new TopNKeyProcessor());
+
+ // The dispatcher fires the processor corresponding to the closest matching
+ // rule and passes the context along
+ Dispatcher disp = new DefaultRuleDispatcher(null, opRules, procCtx);
+ List<Node> topNodes = new ArrayList<Node>();
+ topNodes.addAll(procCtx.parseContext.getTopOps().values());
+ GraphWalker ogw = new DefaultGraphWalker(disp);
+ ogw.startWalking(topNodes, null);
+ }
+
private boolean findParallelSemiJoinBranch(Operator<?> mapjoin, TableScanOperator bigTableTS,
ParseContext parseContext,
Map<ReduceSinkOperator, TableScanOperator> semijoins) {
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/plan/TopNKeyDesc.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/plan/TopNKeyDesc.java b/ql/src/java/org/apache/hadoop/hive/ql/plan/TopNKeyDesc.java
new file mode 100644
index 0000000..c62c4a9
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/plan/TopNKeyDesc.java
@@ -0,0 +1,139 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.plan;
+
+import org.apache.hadoop.hive.ql.plan.Explain.Level;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+/**
+ * TopNKeyDesc.
+ *
+ */
+@Explain(displayName = "Top N Key Operator", explainLevels = { Level.USER, Level.DEFAULT, Level.EXTENDED })
+public class TopNKeyDesc extends AbstractOperatorDesc {
+ private static final long serialVersionUID = 1L;
+
+ private int topN;
+ private String columnSortOrder;
+ private List<ExprNodeDesc> keyColumns;
+
+ public TopNKeyDesc() {
+ }
+
+ public TopNKeyDesc(
+ final int topN,
+ final String columnSortOrder,
+ final List<ExprNodeDesc> keyColumns) {
+
+ this.topN = topN;
+ this.columnSortOrder = columnSortOrder;
+ this.keyColumns = keyColumns;
+ }
+
+ @Explain(displayName = "top n", explainLevels = { Level.DEFAULT, Level.EXTENDED, Level.USER })
+ public int getTopN() {
+ return topN;
+ }
+
+ public void setTopN(int topN) {
+ this.topN = topN;
+ }
+
+ @Explain(displayName = "sort order", explainLevels = { Level.DEFAULT, Level.EXTENDED, Level.USER })
+ public String getColumnSortOrder() {
+ return columnSortOrder;
+ }
+
+ public void setColumnSortOrder(String columnSortOrder) {
+ this.columnSortOrder = columnSortOrder;
+ }
+
+ @Explain(displayName = "keys")
+ public String getKeyString() {
+ return PlanUtils.getExprListString(keyColumns);
+ }
+
+ @Explain(displayName = "keys", explainLevels = { Level.USER })
+ public String getUserLevelExplainKeyString() {
+ return PlanUtils.getExprListString(keyColumns, true);
+ }
+
+ public List<ExprNodeDesc> getKeyColumns() {
+ return keyColumns;
+ }
+
+ public void setKeyColumns(List<ExprNodeDesc> keyColumns) {
+ this.keyColumns = keyColumns;
+ }
+
+ public List<String> getKeyColumnNames() {
+ List<String> ret = new ArrayList<>();
+ for (ExprNodeDesc keyColumn : keyColumns) {
+ ret.add(keyColumn.getExprString());
+ }
+ return ret;
+ }
+
+ @Override
+ public boolean isSame(OperatorDesc other) {
+ if (getClass().getName().equals(other.getClass().getName())) {
+ TopNKeyDesc otherDesc = (TopNKeyDesc) other;
+ return getTopN() == otherDesc.getTopN() &&
+ Objects.equals(columnSortOrder, otherDesc.columnSortOrder) &&
+ ExprNodeDescUtils.isSame(keyColumns, otherDesc.keyColumns);
+ }
+ return false;
+ }
+
+ @Override
+ public Object clone() {
+ TopNKeyDesc ret = new TopNKeyDesc();
+ ret.setTopN(topN);
+ ret.setColumnSortOrder(columnSortOrder);
+ ret.setKeyColumns(getKeyColumns() == null ? null : new ArrayList<>(getKeyColumns()));
+ return ret;
+ }
+
+ public class TopNKeyDescExplainVectorization extends OperatorExplainVectorization {
+ private final TopNKeyDesc topNKeyDesc;
+ private final VectorTopNKeyDesc vectorTopNKeyDesc;
+
+ public TopNKeyDescExplainVectorization(TopNKeyDesc topNKeyDesc, VectorTopNKeyDesc vectorTopNKeyDesc) {
+ super(vectorTopNKeyDesc, true);
+ this.topNKeyDesc = topNKeyDesc;
+ this.vectorTopNKeyDesc = vectorTopNKeyDesc;
+ }
+
+ @Explain(vectorization = Explain.Vectorization.OPERATOR, displayName = "keyExpressions", explainLevels = { Level.DEFAULT, Level.EXTENDED })
+ public List<String> getKeyExpressions() {
+ return vectorExpressionsToStringList(vectorTopNKeyDesc.getKeyExpressions());
+ }
+ }
+
+ @Explain(vectorization = Explain.Vectorization.OPERATOR, displayName = "Top N Key Vectorization", explainLevels = { Level.DEFAULT, Level.EXTENDED })
+ public TopNKeyDescExplainVectorization getTopNKeyVectorization() {
+ VectorTopNKeyDesc vectorTopNKeyDesc = (VectorTopNKeyDesc) getVectorDesc();
+ if (vectorTopNKeyDesc == null) {
+ return null;
+ }
+ return new TopNKeyDescExplainVectorization(this, vectorTopNKeyDesc);
+ }
+}
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/java/org/apache/hadoop/hive/ql/plan/VectorTopNKeyDesc.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/plan/VectorTopNKeyDesc.java b/ql/src/java/org/apache/hadoop/hive/ql/plan/VectorTopNKeyDesc.java
new file mode 100644
index 0000000..9a266a0
--- /dev/null
+++ b/ql/src/java/org/apache/hadoop/hive/ql/plan/VectorTopNKeyDesc.java
@@ -0,0 +1,39 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.plan;
+
+import org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression;
+
+public class VectorTopNKeyDesc extends AbstractVectorDesc {
+
+ private static final long serialVersionUID = 1L;
+
+ private VectorExpression[] keyExpressions;
+
+ public VectorTopNKeyDesc() {
+ }
+
+ public VectorExpression[] getKeyExpressions() {
+ return keyExpressions;
+ }
+
+ public void setKeyExpressions(VectorExpression[] keyExpressions) {
+ this.keyExpressions = keyExpressions;
+ }
+}
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/queries/clientpositive/topnkey.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/topnkey.q b/ql/src/test/queries/clientpositive/topnkey.q
new file mode 100644
index 0000000..e02a41d
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/topnkey.q
@@ -0,0 +1,31 @@
+--! qt:dataset:src
+set hive.mapred.mode=nonstrict;
+set hive.vectorized.execution.enabled=false;
+set hive.optimize.topnkey=true;
+
+set hive.optimize.ppd=true;
+set hive.ppd.remove.duplicatefilters=true;
+set hive.tez.dynamic.partition.pruning=true;
+set hive.optimize.metadataonly=false;
+set hive.optimize.index.filter=true;
+set hive.tez.min.bloom.filter.entries=1;
+
+set hive.tez.dynamic.partition.pruning=true;
+set hive.stats.fetch.column.stats=true;
+set hive.cbo.enable=true;
+
+EXPLAIN
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5;
+
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5;
+
+EXPLAIN
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5;
+
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5;
+
+explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5;
+
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5;
+
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/queries/clientpositive/vector_topnkey.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vector_topnkey.q b/ql/src/test/queries/clientpositive/vector_topnkey.q
new file mode 100644
index 0000000..e1b7d26
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/vector_topnkey.q
@@ -0,0 +1,30 @@
+--! qt:dataset:src
+set hive.mapred.mode=nonstrict;
+set hive.vectorized.execution.enabled=true;
+set hive.optimize.topnkey=true;
+
+set hive.optimize.ppd=true;
+set hive.ppd.remove.duplicatefilters=true;
+set hive.tez.dynamic.partition.pruning=true;
+set hive.optimize.metadataonly=false;
+set hive.optimize.index.filter=true;
+set hive.tez.min.bloom.filter.entries=1;
+
+set hive.tez.dynamic.partition.pruning=true;
+set hive.stats.fetch.column.stats=true;
+set hive.cbo.enable=true;
+
+explain vectorization detail
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5;
+
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5;
+
+explain vectorization detail
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5;
+
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5;
+
+explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5;
+
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5;
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out b/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out
index bee7889..726d46b 100644
--- a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out
+++ b/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out
@@ -68,19 +68,24 @@ STAGE PLANS:
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: key (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -196,19 +201,24 @@ STAGE PLANS:
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: key (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -298,19 +308,24 @@ STAGE PLANS:
expressions: length(key) (type: int)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 3000 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 3000 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 250 Data size: 3000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -380,19 +395,24 @@ STAGE PLANS:
expressions: abs(length(key)) (type: int)
outputColumnNames: _col0
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 3000 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 3000 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 250 Data size: 3000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -463,19 +483,24 @@ STAGE PLANS:
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: key (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -566,19 +591,24 @@ STAGE PLANS:
expressions: value (type: string)
outputColumnNames: value
Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: value (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ keys: value (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -1167,20 +1197,25 @@ STAGE PLANS:
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- bucketGroup: true
+ Top N Key Operator
+ sort order: +
keys: key (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ bucketGroup: true
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -1271,19 +1306,24 @@ STAGE PLANS:
expressions: value (type: string)
outputColumnNames: value
Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: value (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ keys: value (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -1475,20 +1515,25 @@ STAGE PLANS:
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
- bucketGroup: true
+ Top N Key Operator
+ sort order: +
keys: key (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ bucketGroup: true
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -1579,19 +1624,24 @@ STAGE PLANS:
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: ++
keys: key (type: string), value (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string)
- sort order: ++
- Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: count()
+ keys: key (type: string), value (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col2 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string)
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
+ Statistics: Num rows: 250 Data size: 46500 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col2 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/check_constraint.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/check_constraint.q.out b/ql/src/test/results/clientpositive/llap/check_constraint.q.out
index e4cd97e..123a3e4 100644
--- a/ql/src/test/results/clientpositive/llap/check_constraint.q.out
+++ b/ql/src/test/results/clientpositive/llap/check_constraint.q.out
@@ -1675,19 +1675,24 @@ STAGE PLANS:
expressions: key (type: string), value (type: string), UDFToInteger(key) (type: int), CAST( key AS decimal(5,2)) (type: decimal(5,2))
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: min(_col2), max(_col3)
+ Top N Key Operator
+ sort order: ++
keys: _col0 (type: string), _col1 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 250 Data size: 73500 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string)
- sort order: ++
- Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Group By Operator
+ aggregations: min(_col2), max(_col3)
+ keys: _col0 (type: string), _col1 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 250 Data size: 73500 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col2 (type: int), _col3 (type: decimal(5,2))
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string)
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
+ Statistics: Num rows: 250 Data size: 73500 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col2 (type: int), _col3 (type: decimal(5,2))
Execution mode: vectorized, llap
LLAP IO: no inputs
Reducer 2
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
index 6a2ae62..f9018b4 100644
--- a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
@@ -1264,19 +1264,21 @@ Stage-0
PartitionCols:_col0, _col1
Group By Operator [GBY_7] (rows=5 width=20)
Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col1, _col0
- Select Operator [SEL_5] (rows=10 width=101)
- Output:["_col0","_col1"]
- Group By Operator [GBY_4] (rows=10 width=101)
- Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
- <-Map 1 [SIMPLE_EDGE] llap
- SHUFFLE [RS_3]
- PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_2] (rows=10 width=101)
- Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(c_int)"],keys:key, c_int, c_float
- Select Operator [SEL_1] (rows=20 width=88)
- Output:["key","c_int","c_float"]
- TableScan [TS_0] (rows=20 width=88)
- default@cbo_t1,cbo_t1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int","c_float"]
+ Top N Key Operator [TNK_15] (rows=10 width=101)
+ keys:_col1, _col0,sort order:++,top n:1
+ Select Operator [SEL_5] (rows=10 width=101)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_4] (rows=10 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_3]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_2] (rows=10 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(c_int)"],keys:key, c_int, c_float
+ Select Operator [SEL_1] (rows=20 width=88)
+ Output:["key","c_int","c_float"]
+ TableScan [TS_0] (rows=20 width=88)
+ default@cbo_t1,cbo_t1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int","c_float"]
PREHOOK: query: explain select key from(select key from (select key from cbo_t1 limit 5)cbo_t2 limit 5)cbo_t3 limit 5
PREHOOK: type: QUERY
[04/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query76.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query76.q.out b/ql/src/test/results/clientpositive/perf/tez/query76.q.out
index b32d7e4..c1d7b97 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query76.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query76.q.out
@@ -70,193 +70,199 @@ Stage-0
limit:100
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_195]
- Limit [LIM_194] (rows=100 width=108)
+ File Output Operator [FS_199]
+ Limit [LIM_198] (rows=100 width=108)
Number of rows:100
- Select Operator [SEL_193] (rows=304916424 width=108)
+ Select Operator [SEL_197] (rows=304916424 width=108)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_192]
- Group By Operator [GBY_191] (rows=304916424 width=108)
+ SHUFFLE [RS_196]
+ Group By Operator [GBY_195] (rows=304916424 width=108)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["count(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
<-Union 4 [SIMPLE_EDGE]
<-Reducer 12 [CONTAINS]
- Reduce Output Operator [RS_163]
+ Reduce Output Operator [RS_167]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_162] (rows=609832848 width=108)
+ Group By Operator [GBY_166] (rows=609832848 width=108)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["count()","sum(_col5)"],keys:_col0, _col1, _col2, _col3, _col4
- Select Operator [SEL_160] (rows=174233858 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_159] (rows=174233858 width=135)
- Conds:RS_45._col0=RS_213._col0(Inner),Output:["_col3","_col5","_col7","_col8"]
- <-Map 21 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_213]
- PartitionCols:_col0
- Select Operator [SEL_212] (rows=73049 width=1119)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_211] (rows=73049 width=1119)
- predicate:d_date_sk is not null
- TableScan [TS_39] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_45]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_147] (rows=158394413 width=135)
- Conds:RS_221._col2=RS_170._col0(Inner),Output:["_col0","_col3","_col5"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_170]
- PartitionCols:_col0
- Select Operator [SEL_165] (rows=462000 width=1436)
- Output:["_col0","_col1"]
- Filter Operator [FIL_164] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- TableScan [TS_0] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_category"]
- <-Map 20 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_221]
- PartitionCols:_col2
- Select Operator [SEL_220] (rows=143994918 width=135)
- Output:["_col0","_col2","_col3"]
- Filter Operator [FIL_219] (rows=143994918 width=135)
- predicate:((cs_item_sk BETWEEN DynamicValue(RS_43_item_i_item_sk_min) AND DynamicValue(RS_43_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_43_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_46_date_dim_d_date_sk_min) AND DynamicValue(RS_46_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_46_date_dim_d_date_sk_bloom_filter))) and cs_item_sk is not null and cs_sold_date_sk is not null and cs_warehouse_sk is null)
- TableScan [TS_33] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_warehouse_sk","cs_item_sk","cs_ext_sales_price"]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_210]
- Group By Operator [GBY_209] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_177]
- Group By Operator [GBY_174] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_171] (rows=462000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_165]
- <-Reducer 22 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_218]
- Group By Operator [GBY_217] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_216]
- Group By Operator [GBY_215] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_214] (rows=73049 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_212]
+ Top N Key Operator [TNK_165] (rows=609832848 width=108)
+ keys:_col0, _col1, _col2, _col3, _col4,sort order:+++++,top n:100
+ Select Operator [SEL_163] (rows=174233858 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_162] (rows=174233858 width=135)
+ Conds:RS_45._col0=RS_217._col0(Inner),Output:["_col3","_col5","_col7","_col8"]
+ <-Map 21 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_217]
+ PartitionCols:_col0
+ Select Operator [SEL_216] (rows=73049 width=1119)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_215] (rows=73049 width=1119)
+ predicate:d_date_sk is not null
+ TableScan [TS_39] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
+ <-Reducer 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_45]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_148] (rows=158394413 width=135)
+ Conds:RS_225._col2=RS_174._col0(Inner),Output:["_col0","_col3","_col5"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_174]
+ PartitionCols:_col0
+ Select Operator [SEL_169] (rows=462000 width=1436)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_168] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_0] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_category"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_225]
+ PartitionCols:_col2
+ Select Operator [SEL_224] (rows=143994918 width=135)
+ Output:["_col0","_col2","_col3"]
+ Filter Operator [FIL_223] (rows=143994918 width=135)
+ predicate:((cs_item_sk BETWEEN DynamicValue(RS_43_item_i_item_sk_min) AND DynamicValue(RS_43_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_43_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_46_date_dim_d_date_sk_min) AND DynamicValue(RS_46_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_46_date_dim_d_date_sk_bloom_filter))) and cs_item_sk is not null and cs_sold_date_sk is not null and cs_warehouse_sk is null)
+ TableScan [TS_33] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_warehouse_sk","cs_item_sk","cs_ext_sales_price"]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_214]
+ Group By Operator [GBY_213] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_181]
+ Group By Operator [GBY_178] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_175] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_169]
+ <-Reducer 22 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_222]
+ Group By Operator [GBY_221] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 21 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_220]
+ Group By Operator [GBY_219] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_218] (rows=73049 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_216]
<-Reducer 3 [CONTAINS]
- Reduce Output Operator [RS_153]
+ Reduce Output Operator [RS_155]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_152] (rows=609832848 width=108)
+ Group By Operator [GBY_154] (rows=609832848 width=108)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["count()","sum(_col5)"],keys:_col0, _col1, _col2, _col3, _col4
- Select Operator [SEL_150] (rows=348477373 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_149] (rows=348477373 width=88)
- Conds:RS_12._col2=RS_182._col0(Inner),Output:["_col1","_col5","_col7","_col8"]
- <-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_182]
- PartitionCols:_col0
- Select Operator [SEL_181] (rows=73049 width=1119)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_180] (rows=73049 width=1119)
- predicate:d_date_sk is not null
- TableScan [TS_6] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_12]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_143] (rows=316797605 width=88)
- Conds:RS_166._col0=RS_190._col1(Inner),Output:["_col1","_col2","_col5"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_166]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_165]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_190]
- PartitionCols:_col1
- Select Operator [SEL_189] (rows=287997817 width=88)
- Output:["_col0","_col1","_col3"]
- Filter Operator [FIL_188] (rows=287997817 width=88)
- predicate:((ss_item_sk BETWEEN DynamicValue(RS_9_item_i_item_sk_min) AND DynamicValue(RS_9_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_9_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_13_date_dim_d_date_sk_min) AND DynamicValue(RS_13_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_13_date_dim_d_date_sk_bloom_filter))) and ss_addr_sk is null and ss_item_sk is not null and ss_sold_date_sk is not null)
- TableScan [TS_3] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_addr_sk","ss_ext_sales_price"]
- <-Reducer 16 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_187]
- Group By Operator [GBY_186] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 15 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_185]
- Group By Operator [GBY_184] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_183] (rows=73049 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_181]
- <-Reducer 7 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_179]
- Group By Operator [GBY_178] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_175]
- Group By Operator [GBY_172] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_167] (rows=462000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_165]
+ Top N Key Operator [TNK_153] (rows=609832848 width=108)
+ keys:_col0, _col1, _col2, _col3, _col4,sort order:+++++,top n:100
+ Select Operator [SEL_151] (rows=348477373 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_150] (rows=348477373 width=88)
+ Conds:RS_12._col2=RS_186._col0(Inner),Output:["_col1","_col5","_col7","_col8"]
+ <-Map 15 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_186]
+ PartitionCols:_col0
+ Select Operator [SEL_185] (rows=73049 width=1119)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_184] (rows=73049 width=1119)
+ predicate:d_date_sk is not null
+ TableScan [TS_6] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_12]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_144] (rows=316797605 width=88)
+ Conds:RS_170._col0=RS_194._col1(Inner),Output:["_col1","_col2","_col5"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_170]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_169]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_194]
+ PartitionCols:_col1
+ Select Operator [SEL_193] (rows=287997817 width=88)
+ Output:["_col0","_col1","_col3"]
+ Filter Operator [FIL_192] (rows=287997817 width=88)
+ predicate:((ss_item_sk BETWEEN DynamicValue(RS_9_item_i_item_sk_min) AND DynamicValue(RS_9_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_9_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_13_date_dim_d_date_sk_min) AND DynamicValue(RS_13_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_13_date_dim_d_date_sk_bloom_filter))) and ss_addr_sk is null and ss_item_sk is not null and ss_sold_date_sk is not null)
+ TableScan [TS_3] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_addr_sk","ss_ext_sales_price"]
+ <-Reducer 16 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_191]
+ Group By Operator [GBY_190] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 15 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_189]
+ Group By Operator [GBY_188] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_187] (rows=73049 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_185]
+ <-Reducer 7 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_183]
+ Group By Operator [GBY_182] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_179]
+ Group By Operator [GBY_176] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_171] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_169]
<-Reducer 9 [CONTAINS]
- Reduce Output Operator [RS_158]
+ Reduce Output Operator [RS_161]
PartitionCols:_col0, _col1, _col2, _col3, _col4
- Group By Operator [GBY_157] (rows=609832848 width=108)
+ Group By Operator [GBY_160] (rows=609832848 width=108)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["count()","sum(_col5)"],keys:_col0, _col1, _col2, _col3, _col4
- Select Operator [SEL_155] (rows=87121617 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_154] (rows=87121617 width=135)
- Conds:RS_28._col0=RS_200._col0(Inner),Output:["_col3","_col5","_col7","_col8"]
- <-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_200]
- PartitionCols:_col0
- Select Operator [SEL_199] (rows=73049 width=1119)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_198] (rows=73049 width=1119)
- predicate:d_date_sk is not null
- TableScan [TS_22] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
- <-Reducer 8 [SIMPLE_EDGE]
- SHUFFLE [RS_28]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_145] (rows=79201469 width=135)
- Conds:RS_208._col1=RS_168._col0(Inner),Output:["_col0","_col3","_col5"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_168]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_165]
- <-Map 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_208]
- PartitionCols:_col1
- Select Operator [SEL_207] (rows=72001334 width=135)
- Output:["_col0","_col1","_col3"]
- Filter Operator [FIL_206] (rows=72001334 width=135)
- predicate:((ws_item_sk BETWEEN DynamicValue(RS_26_item_i_item_sk_min) AND DynamicValue(RS_26_item_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_26_item_i_item_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_29_date_dim_d_date_sk_min) AND DynamicValue(RS_29_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_29_date_dim_d_date_sk_bloom_filter))) and ws_item_sk is not null and ws_sold_date_sk is not null and ws_web_page_sk is null)
- TableScan [TS_16] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_web_page_sk","ws_ext_sales_price"]
- <-Reducer 10 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_197]
- Group By Operator [GBY_196] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_176]
- Group By Operator [GBY_173] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_169] (rows=462000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_165]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_205]
- Group By Operator [GBY_204] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 18 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_203]
- Group By Operator [GBY_202] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_201] (rows=73049 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_199]
+ Top N Key Operator [TNK_159] (rows=609832848 width=108)
+ keys:_col0, _col1, _col2, _col3, _col4,sort order:+++++,top n:100
+ Select Operator [SEL_157] (rows=87121617 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_156] (rows=87121617 width=135)
+ Conds:RS_28._col0=RS_204._col0(Inner),Output:["_col3","_col5","_col7","_col8"]
+ <-Map 18 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_204]
+ PartitionCols:_col0
+ Select Operator [SEL_203] (rows=73049 width=1119)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_202] (rows=73049 width=1119)
+ predicate:d_date_sk is not null
+ TableScan [TS_22] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_qoy"]
+ <-Reducer 8 [SIMPLE_EDGE]
+ SHUFFLE [RS_28]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_146] (rows=79201469 width=135)
+ Conds:RS_212._col1=RS_172._col0(Inner),Output:["_col0","_col3","_col5"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_172]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_169]
+ <-Map 17 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_212]
+ PartitionCols:_col1
+ Select Operator [SEL_211] (rows=72001334 width=135)
+ Output:["_col0","_col1","_col3"]
+ Filter Operator [FIL_210] (rows=72001334 width=135)
+ predicate:((ws_item_sk BETWEEN DynamicValue(RS_26_item_i_item_sk_min) AND DynamicValue(RS_26_item_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_26_item_i_item_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_29_date_dim_d_date_sk_min) AND DynamicValue(RS_29_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_29_date_dim_d_date_sk_bloom_filter))) and ws_item_sk is not null and ws_sold_date_sk is not null and ws_web_page_sk is null)
+ TableScan [TS_16] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_web_page_sk","ws_ext_sales_price"]
+ <-Reducer 10 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_201]
+ Group By Operator [GBY_200] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_180]
+ Group By Operator [GBY_177] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_173] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_169]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_209]
+ Group By Operator [GBY_208] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 18 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_207]
+ Group By Operator [GBY_206] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_205] (rows=73049 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_203]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query77.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query77.q.out b/ql/src/test/results/clientpositive/perf/tez/query77.q.out
index 915d4fd..617a473 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query77.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query77.q.out
@@ -1,4 +1,4 @@
-Warning: Shuffle Join MERGEJOIN[315][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 16' is a cross product
+Warning: Shuffle Join MERGEJOIN[317][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 16' is a cross product
PREHOOK: query: explain
with ss as
(select s_store_sk,
@@ -249,296 +249,302 @@ Stage-0
limit:100
Stage-1
Reducer 8 vectorized
- File Output Operator [FS_368]
- Limit [LIM_367] (rows=100 width=163)
+ File Output Operator [FS_372]
+ Limit [LIM_371] (rows=100 width=163)
Number of rows:100
- Select Operator [SEL_366] (rows=956329968 width=163)
+ Select Operator [SEL_370] (rows=956329968 width=163)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_365]
- Select Operator [SEL_364] (rows=956329968 width=163)
+ SHUFFLE [RS_369]
+ Select Operator [SEL_368] (rows=956329968 width=163)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_363] (rows=956329968 width=163)
+ Group By Operator [GBY_367] (rows=956329968 width=163)
Output:["_col0","_col1","_col3","_col4","_col5"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 6 [SIMPLE_EDGE]
<-Reducer 16 [CONTAINS]
- Reduce Output Operator [RS_319]
+ Reduce Output Operator [RS_322]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_318] (rows=1912659936 width=163)
+ Group By Operator [GBY_321] (rows=1912659936 width=163)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_316] (rows=158394413 width=360)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_315] (rows=158394413 width=360)
- Conds:(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
- <-Reducer 15 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_375]
- Group By Operator [GBY_374] (rows=158394413 width=135)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
- <-Reducer 14 [SIMPLE_EDGE]
- SHUFFLE [RS_55]
- PartitionCols:_col0
- Group By Operator [GBY_54] (rows=316788826 width=135)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col1
- Merge Join Operator [MERGEJOIN_301] (rows=316788826 width=135)
- Conds:RS_373._col0=RS_330._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_330]
- PartitionCols:_col0
- Select Operator [SEL_326] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_325] (rows=8116 width=1119)
- predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-08-04 00:00:00' AND TIMESTAMP'1998-09-03 00:00:00' and d_date_sk is not null)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
- <-Map 31 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_373]
- PartitionCols:_col0
- Select Operator [SEL_372] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_371] (rows=287989836 width=135)
- predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_51_date_dim_d_date_sk_min) AND DynamicValue(RS_51_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_51_date_dim_d_date_sk_bloom_filter))) and cs_sold_date_sk is not null)
- TableScan [TS_44] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_call_center_sk","cs_ext_sales_price","cs_net_profit"]
- <-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_370]
- Group By Operator [GBY_369] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_340]
- Group By Operator [GBY_337] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_331] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_326]
- <-Reducer 19 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_380]
- Group By Operator [GBY_379] (rows=1 width=224)
- Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
- <-Reducer 18 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_69]
- Group By Operator [GBY_68] (rows=1 width=224)
- Output:["_col0","_col1"],aggregations:["sum(_col1)","sum(_col2)"]
- Merge Join Operator [MERGEJOIN_302] (rows=31678769 width=106)
- Conds:RS_378._col0=RS_332._col0(Inner),Output:["_col1","_col2"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_332]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_326]
- <-Map 32 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_378]
- PartitionCols:_col0
- Select Operator [SEL_377] (rows=28798881 width=106)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_376] (rows=28798881 width=106)
- predicate:cr_returned_date_sk is not null
- TableScan [TS_58] (rows=28798881 width=106)
- default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_returned_date_sk","cr_return_amount","cr_net_loss"]
+ Top N Key Operator [TNK_320] (rows=637553312 width=163)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_318] (rows=158394413 width=360)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_317] (rows=158394413 width=360)
+ Conds:(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
+ <-Reducer 15 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_379]
+ Group By Operator [GBY_378] (rows=158394413 width=135)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
+ <-Reducer 14 [SIMPLE_EDGE]
+ SHUFFLE [RS_55]
+ PartitionCols:_col0
+ Group By Operator [GBY_54] (rows=316788826 width=135)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col1
+ Merge Join Operator [MERGEJOIN_302] (rows=316788826 width=135)
+ Conds:RS_377._col0=RS_334._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_334]
+ PartitionCols:_col0
+ Select Operator [SEL_330] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_329] (rows=8116 width=1119)
+ predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-08-04 00:00:00' AND TIMESTAMP'1998-09-03 00:00:00' and d_date_sk is not null)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
+ <-Map 31 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_377]
+ PartitionCols:_col0
+ Select Operator [SEL_376] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_375] (rows=287989836 width=135)
+ predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_51_date_dim_d_date_sk_min) AND DynamicValue(RS_51_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_51_date_dim_d_date_sk_bloom_filter))) and cs_sold_date_sk is not null)
+ TableScan [TS_44] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_call_center_sk","cs_ext_sales_price","cs_net_profit"]
+ <-Reducer 17 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_374]
+ Group By Operator [GBY_373] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_344]
+ Group By Operator [GBY_341] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_335] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_330]
+ <-Reducer 19 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_384]
+ Group By Operator [GBY_383] (rows=1 width=224)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
+ <-Reducer 18 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_69]
+ Group By Operator [GBY_68] (rows=1 width=224)
+ Output:["_col0","_col1"],aggregations:["sum(_col1)","sum(_col2)"]
+ Merge Join Operator [MERGEJOIN_303] (rows=31678769 width=106)
+ Conds:RS_382._col0=RS_336._col0(Inner),Output:["_col1","_col2"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_336]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_330]
+ <-Map 32 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_382]
+ PartitionCols:_col0
+ Select Operator [SEL_381] (rows=28798881 width=106)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_380] (rows=28798881 width=106)
+ predicate:cr_returned_date_sk is not null
+ TableScan [TS_58] (rows=28798881 width=106)
+ default@catalog_returns,catalog_returns,Tbl:COMPLETE,Col:NONE,Output:["cr_returned_date_sk","cr_return_amount","cr_net_loss"]
<-Reducer 23 [CONTAINS]
- Reduce Output Operator [RS_324]
+ Reduce Output Operator [RS_328]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_323] (rows=1912659936 width=163)
+ Group By Operator [GBY_327] (rows=1912659936 width=163)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_321] (rows=95833780 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_320] (rows=95833780 width=135)
- Conds:RS_396._col0=RS_401._col0(Left Outer),Output:["_col0","_col1","_col2","_col4","_col5"]
- <-Reducer 22 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_396]
- PartitionCols:_col0
- Group By Operator [GBY_395] (rows=87121617 width=135)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
- <-Reducer 21 [SIMPLE_EDGE]
- SHUFFLE [RS_94]
- PartitionCols:_col0
- Group By Operator [GBY_93] (rows=174243235 width=135)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col6
- Merge Join Operator [MERGEJOIN_304] (rows=174243235 width=135)
- Conds:RS_89._col1=RS_385._col0(Inner),Output:["_col2","_col3","_col6"]
- <-Map 34 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_385]
- PartitionCols:_col0
- Select Operator [SEL_384] (rows=4602 width=585)
- Output:["_col0"]
- Filter Operator [FIL_383] (rows=4602 width=585)
- predicate:wp_web_page_sk is not null
- TableScan [TS_83] (rows=4602 width=585)
- default@web_page,web_page,Tbl:COMPLETE,Col:NONE,Output:["wp_web_page_sk"]
- <-Reducer 20 [SIMPLE_EDGE]
- SHUFFLE [RS_89]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_303] (rows=158402938 width=135)
- Conds:RS_394._col0=RS_333._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_333]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_326]
- <-Map 33 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_394]
- PartitionCols:_col0
- Select Operator [SEL_393] (rows=144002668 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_392] (rows=144002668 width=135)
- predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_87_date_dim_d_date_sk_min) AND DynamicValue(RS_87_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_87_date_dim_d_date_sk_bloom_filter))) and (ws_web_page_sk BETWEEN DynamicValue(RS_90_web_page_wp_web_page_sk_min) AND DynamicValue(RS_90_web_page_wp_web_page_sk_max) and in_bloom_filter(ws_web_page_sk, DynamicValue(RS_90_web_page_wp_web_page_sk_bloom_filter))) and ws_sold_date_sk is not null and ws_web_page_sk is not null)
- TableScan [TS_77] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_web_page_sk","ws_ext_sales_price","ws_net_profit"]
- <-Reducer 24 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_382]
- Group By Operator [GBY_381] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_341]
- Group By Operator [GBY_338] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_334] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_326]
- <-Reducer 35 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_391]
- Group By Operator [GBY_390] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 34 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_389]
- Group By Operator [GBY_388] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_386] (rows=4602 width=585)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_384]
- <-Reducer 27 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_401]
- PartitionCols:_col0
- Group By Operator [GBY_400] (rows=8711072 width=92)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
- <-Reducer 26 [SIMPLE_EDGE]
- SHUFFLE [RS_114]
- PartitionCols:_col0
- Group By Operator [GBY_113] (rows=17422145 width=92)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col6
- Merge Join Operator [MERGEJOIN_306] (rows=17422145 width=92)
- Conds:RS_109._col1=RS_387._col0(Inner),Output:["_col2","_col3","_col6"]
- <-Map 34 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_387]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_384]
- <-Reducer 25 [SIMPLE_EDGE]
- SHUFFLE [RS_109]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_305] (rows=15838314 width=92)
- Conds:RS_399._col0=RS_335._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_335]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_326]
- <-Map 36 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_399]
- PartitionCols:_col0
- Select Operator [SEL_398] (rows=14398467 width=92)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_397] (rows=14398467 width=92)
- predicate:(wr_returned_date_sk is not null and wr_web_page_sk is not null)
- TableScan [TS_97] (rows=14398467 width=92)
- default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_web_page_sk","wr_return_amt","wr_net_loss"]
+ Top N Key Operator [TNK_326] (rows=637553312 width=163)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_324] (rows=95833780 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_323] (rows=95833780 width=135)
+ Conds:RS_400._col0=RS_405._col0(Left Outer),Output:["_col0","_col1","_col2","_col4","_col5"]
+ <-Reducer 22 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_400]
+ PartitionCols:_col0
+ Group By Operator [GBY_399] (rows=87121617 width=135)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
+ <-Reducer 21 [SIMPLE_EDGE]
+ SHUFFLE [RS_94]
+ PartitionCols:_col0
+ Group By Operator [GBY_93] (rows=174243235 width=135)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col6
+ Merge Join Operator [MERGEJOIN_305] (rows=174243235 width=135)
+ Conds:RS_89._col1=RS_389._col0(Inner),Output:["_col2","_col3","_col6"]
+ <-Map 34 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_389]
+ PartitionCols:_col0
+ Select Operator [SEL_388] (rows=4602 width=585)
+ Output:["_col0"]
+ Filter Operator [FIL_387] (rows=4602 width=585)
+ predicate:wp_web_page_sk is not null
+ TableScan [TS_83] (rows=4602 width=585)
+ default@web_page,web_page,Tbl:COMPLETE,Col:NONE,Output:["wp_web_page_sk"]
+ <-Reducer 20 [SIMPLE_EDGE]
+ SHUFFLE [RS_89]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_304] (rows=158402938 width=135)
+ Conds:RS_398._col0=RS_337._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_337]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_330]
+ <-Map 33 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_398]
+ PartitionCols:_col0
+ Select Operator [SEL_397] (rows=144002668 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_396] (rows=144002668 width=135)
+ predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_87_date_dim_d_date_sk_min) AND DynamicValue(RS_87_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_87_date_dim_d_date_sk_bloom_filter))) and (ws_web_page_sk BETWEEN DynamicValue(RS_90_web_page_wp_web_page_sk_min) AND DynamicValue(RS_90_web_page_wp_web_page_sk_max) and in_bloom_filter(ws_web_page_sk, DynamicValue(RS_90_web_page_wp_web_page_sk_bloom_filter))) and ws_sold_date_sk is not null and ws_web_page_sk is not null)
+ TableScan [TS_77] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_web_page_sk","ws_ext_sales_price","ws_net_profit"]
+ <-Reducer 24 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_386]
+ Group By Operator [GBY_385] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_345]
+ Group By Operator [GBY_342] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_338] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_330]
+ <-Reducer 35 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_395]
+ Group By Operator [GBY_394] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 34 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_393]
+ Group By Operator [GBY_392] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_390] (rows=4602 width=585)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_388]
+ <-Reducer 27 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_405]
+ PartitionCols:_col0
+ Group By Operator [GBY_404] (rows=8711072 width=92)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
+ <-Reducer 26 [SIMPLE_EDGE]
+ SHUFFLE [RS_114]
+ PartitionCols:_col0
+ Group By Operator [GBY_113] (rows=17422145 width=92)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col6
+ Merge Join Operator [MERGEJOIN_307] (rows=17422145 width=92)
+ Conds:RS_109._col1=RS_391._col0(Inner),Output:["_col2","_col3","_col6"]
+ <-Map 34 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_391]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_388]
+ <-Reducer 25 [SIMPLE_EDGE]
+ SHUFFLE [RS_109]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_306] (rows=15838314 width=92)
+ Conds:RS_403._col0=RS_339._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_339]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_330]
+ <-Map 36 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_403]
+ PartitionCols:_col0
+ Select Operator [SEL_402] (rows=14398467 width=92)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_401] (rows=14398467 width=92)
+ predicate:(wr_returned_date_sk is not null and wr_web_page_sk is not null)
+ TableScan [TS_97] (rows=14398467 width=92)
+ default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_web_page_sk","wr_return_amt","wr_net_loss"]
<-Reducer 5 [CONTAINS]
- Reduce Output Operator [RS_314]
+ Reduce Output Operator [RS_316]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_313] (rows=1912659936 width=163)
+ Group By Operator [GBY_315] (rows=1912659936 width=163)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_311] (rows=383325119 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_310] (rows=383325119 width=88)
- Conds:RS_357._col0=RS_362._col0(Left Outer),Output:["_col0","_col1","_col2","_col4","_col5"]
- <-Reducer 13 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_362]
- PartitionCols:_col0
- Group By Operator [GBY_361] (rows=34842647 width=77)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
- <-Reducer 12 [SIMPLE_EDGE]
- SHUFFLE [RS_37]
- PartitionCols:_col0
- Group By Operator [GBY_36] (rows=69685294 width=77)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col6
- Merge Join Operator [MERGEJOIN_300] (rows=69685294 width=77)
- Conds:RS_32._col1=RS_348._col0(Inner),Output:["_col2","_col3","_col6"]
- <-Map 28 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_348]
- PartitionCols:_col0
- Select Operator [SEL_345] (rows=1704 width=1910)
- Output:["_col0"]
- Filter Operator [FIL_344] (rows=1704 width=1910)
- predicate:s_store_sk is not null
- TableScan [TS_6] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_32]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_299] (rows=63350266 width=77)
- Conds:RS_360._col0=RS_329._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_329]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_326]
- <-Map 30 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_360]
- PartitionCols:_col0
- Select Operator [SEL_359] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_358] (rows=57591150 width=77)
- predicate:(sr_returned_date_sk is not null and sr_store_sk is not null)
- TableScan [TS_20] (rows=57591150 width=77)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_store_sk","sr_return_amt","sr_net_loss"]
- <-Reducer 4 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_357]
- PartitionCols:_col0
- Group By Operator [GBY_356] (rows=348477374 width=88)
- Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_17]
- PartitionCols:_col0
- Group By Operator [GBY_16] (rows=696954748 width=88)
- Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col6
- Merge Join Operator [MERGEJOIN_298] (rows=696954748 width=88)
- Conds:RS_12._col1=RS_346._col0(Inner),Output:["_col2","_col3","_col6"]
- <-Map 28 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_346]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_345]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_12]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_297] (rows=633595212 width=88)
- Conds:RS_355._col0=RS_327._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_327]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_326]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_355]
- PartitionCols:_col0
- Select Operator [SEL_354] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_353] (rows=575995635 width=88)
- predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_10_date_dim_d_date_sk_min) AND DynamicValue(RS_10_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_10_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_13_store_s_store_sk_min) AND DynamicValue(RS_13_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_13_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_store_sk","ss_ext_sales_price","ss_net_profit"]
- <-Reducer 10 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_343]
- Group By Operator [GBY_342] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_339]
- Group By Operator [GBY_336] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_328] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_326]
- <-Reducer 29 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_352]
- Group By Operator [GBY_351] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 28 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_350]
- Group By Operator [GBY_349] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_347] (rows=1704 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_345]
+ Top N Key Operator [TNK_314] (rows=637553312 width=163)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_312] (rows=383325119 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Merge Join Operator [MERGEJOIN_311] (rows=383325119 width=88)
+ Conds:RS_361._col0=RS_366._col0(Left Outer),Output:["_col0","_col1","_col2","_col4","_col5"]
+ <-Reducer 13 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_366]
+ PartitionCols:_col0
+ Group By Operator [GBY_365] (rows=34842647 width=77)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_37]
+ PartitionCols:_col0
+ Group By Operator [GBY_36] (rows=69685294 width=77)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col6
+ Merge Join Operator [MERGEJOIN_301] (rows=69685294 width=77)
+ Conds:RS_32._col1=RS_352._col0(Inner),Output:["_col2","_col3","_col6"]
+ <-Map 28 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_352]
+ PartitionCols:_col0
+ Select Operator [SEL_349] (rows=1704 width=1910)
+ Output:["_col0"]
+ Filter Operator [FIL_348] (rows=1704 width=1910)
+ predicate:s_store_sk is not null
+ TableScan [TS_6] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk"]
+ <-Reducer 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_32]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_300] (rows=63350266 width=77)
+ Conds:RS_364._col0=RS_333._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_333]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_330]
+ <-Map 30 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_364]
+ PartitionCols:_col0
+ Select Operator [SEL_363] (rows=57591150 width=77)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_362] (rows=57591150 width=77)
+ predicate:(sr_returned_date_sk is not null and sr_store_sk is not null)
+ TableScan [TS_20] (rows=57591150 width=77)
+ default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_store_sk","sr_return_amt","sr_net_loss"]
+ <-Reducer 4 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_361]
+ PartitionCols:_col0
+ Group By Operator [GBY_360] (rows=348477374 width=88)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_17]
+ PartitionCols:_col0
+ Group By Operator [GBY_16] (rows=696954748 width=88)
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","sum(_col3)"],keys:_col6
+ Merge Join Operator [MERGEJOIN_299] (rows=696954748 width=88)
+ Conds:RS_12._col1=RS_350._col0(Inner),Output:["_col2","_col3","_col6"]
+ <-Map 28 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_350]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_349]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_12]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_298] (rows=633595212 width=88)
+ Conds:RS_359._col0=RS_331._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_331]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_330]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_359]
+ PartitionCols:_col0
+ Select Operator [SEL_358] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_357] (rows=575995635 width=88)
+ predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_10_date_dim_d_date_sk_min) AND DynamicValue(RS_10_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_10_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_13_store_s_store_sk_min) AND DynamicValue(RS_13_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_13_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_store_sk","ss_ext_sales_price","ss_net_profit"]
+ <-Reducer 10 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_347]
+ Group By Operator [GBY_346] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 9 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_343]
+ Group By Operator [GBY_340] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_332] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_330]
+ <-Reducer 29 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_356]
+ Group By Operator [GBY_355] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 28 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_354]
+ Group By Operator [GBY_353] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_351] (rows=1704 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_349]
[12/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query25.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query25.q.out b/ql/src/test/results/clientpositive/perf/tez/query25.q.out
index 77a9a07..b68c54a 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query25.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query25.q.out
@@ -118,195 +118,197 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_269]
- Limit [LIM_268] (rows=100 width=88)
+ File Output Operator [FS_270]
+ Limit [LIM_269] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_267] (rows=421657640 width=88)
+ Select Operator [SEL_268] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_266]
- Group By Operator [GBY_265] (rows=421657640 width=88)
+ SHUFFLE [RS_267]
+ Group By Operator [GBY_266] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_49]
PartitionCols:_col0, _col1, _col2, _col3
Group By Operator [GBY_48] (rows=843315281 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"],aggregations:["sum(_col5)","sum(_col20)","sum(_col12)"],keys:_col25, _col26, _col28, _col29
- Merge Join Operator [MERGEJOIN_213] (rows=843315281 width=88)
- Conds:RS_44._col3=RS_251._col0(Inner),Output:["_col5","_col12","_col20","_col25","_col26","_col28","_col29"]
- <-Map 20 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_251]
- PartitionCols:_col0
- Select Operator [SEL_250] (rows=1704 width=1910)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_249] (rows=1704 width=1910)
- predicate:s_store_sk is not null
- TableScan [TS_32] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id","s_store_name"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_44]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_212] (rows=766650239 width=88)
- Conds:RS_41._col1=RS_242._col0(Inner),Output:["_col3","_col5","_col12","_col20","_col25","_col26"]
- <-Map 18 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_242]
- PartitionCols:_col0
- Select Operator [SEL_241] (rows=462000 width=1436)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_240] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- TableScan [TS_29] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_41]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_211] (rows=696954748 width=88)
- Conds:RS_38._col1, _col2, _col4=RS_39._col8, _col9, _col10(Inner),Output:["_col1","_col3","_col5","_col12","_col20"]
- <-Reducer 11 [SIMPLE_EDGE]
- SHUFFLE [RS_39]
- PartitionCols:_col8, _col9, _col10
- Merge Join Operator [MERGEJOIN_210] (rows=348467716 width=135)
- Conds:RS_25._col2, _col1=RS_26._col1, _col2(Inner),Output:["_col3","_col8","_col9","_col10","_col11"]
- <-Reducer 13 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_26]
- PartitionCols:_col1, _col2
- Merge Join Operator [MERGEJOIN_209] (rows=63350266 width=77)
- Conds:RS_233._col0=RS_224._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_224]
- PartitionCols:_col0
- Select Operator [SEL_219] (rows=4058 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_216] (rows=4058 width=1119)
- predicate:((d_year = 2000) and d_date_sk is not null and d_moy BETWEEN 4 AND 10)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
- <-Map 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_233]
- PartitionCols:_col0
- Select Operator [SEL_232] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_231] (rows=57591150 width=77)
- predicate:(sr_customer_sk is not null and sr_item_sk is not null and sr_returned_date_sk is not null and sr_ticket_number is not null)
- TableScan [TS_12] (rows=57591150 width=77)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_net_loss"]
- <-Reducer 10 [SIMPLE_EDGE]
- SHUFFLE [RS_25]
- PartitionCols:_col2, _col1
- Merge Join Operator [MERGEJOIN_208] (rows=316788826 width=135)
- Conds:RS_264._col0=RS_222._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_222]
- PartitionCols:_col0
- Select Operator [SEL_218] (rows=4058 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_215] (rows=4058 width=1119)
- predicate:((d_year = 2000) and d_date_sk is not null and d_moy BETWEEN 4 AND 10)
- Please refer to the previous TableScan [TS_3]
- <-Map 16 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_264]
- PartitionCols:_col0
- Select Operator [SEL_263] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_262] (rows=287989836 width=135)
- predicate:((cs_bill_customer_sk BETWEEN DynamicValue(RS_26_store_returns_sr_customer_sk_min) AND DynamicValue(RS_26_store_returns_sr_customer_sk_max) and in_bloom_filter(cs_bill_customer_sk, DynamicValue(RS_26_store_returns_sr_customer_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_26_store_returns_sr_item_sk_min) AND DynamicValue(RS_26_store_returns_sr_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_26_store_returns_sr_item_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_42_item_i_item_sk_min) AND DynamicValue(RS_42_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_42_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_23_d3_d_date_sk_min) AND DynamicValue(RS_23_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_23_d3_d_date_sk_bloom_filter))) and cs_bill_customer_sk is not null and cs_item_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_6] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_net_profit"]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_236]
- Group By Operator [GBY_234] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_107]
- Group By Operator [GBY_106] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_105] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_209]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_239]
- Group By Operator [GBY_237] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_122]
- Group By Operator [GBY_121] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_120] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_209]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_248]
- Group By Operator [GBY_246] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 18 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_245]
- Group By Operator [GBY_244] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_243] (rows=462000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_241]
- <-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_261]
- Group By Operator [GBY_260] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_228]
- Group By Operator [GBY_226] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_223] (rows=4058 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_218]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_38]
- PartitionCols:_col1, _col2, _col4
- Merge Join Operator [MERGEJOIN_207] (rows=633595212 width=88)
- Conds:RS_259._col0=RS_220._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_220]
- PartitionCols:_col0
- Select Operator [SEL_217] (rows=18262 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_214] (rows=18262 width=1119)
- predicate:((d_moy = 4) and (d_year = 2000) and d_date_sk is not null)
- Please refer to the previous TableScan [TS_3]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_259]
- PartitionCols:_col0
- Select Operator [SEL_258] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_257] (rows=575995635 width=88)
- predicate:((ss_customer_sk BETWEEN DynamicValue(RS_26_store_returns_sr_customer_sk_min) AND DynamicValue(RS_26_store_returns_sr_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_26_store_returns_sr_customer_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_26_store_returns_sr_item_sk_min) AND DynamicValue(RS_26_store_returns_sr_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_26_store_returns_sr_item_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_42_item_i_item_sk_min) AND DynamicValue(RS_42_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_42_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_36_d1_d_date_sk_min) AND DynamicValue(RS_36_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_36_d1_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_45_store_s_store_sk_min) AND DynamicValue(RS_45_store_s_store_sk_max) and
in_bloom_filter(ss_store_sk, DynamicValue(RS_45_store_s_store_sk_bloom_filter))) and ss_customer_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null and ss_ticket_number is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_net_profit"]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_235]
- Please refer to the previous Group By Operator [GBY_234]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_238]
- Please refer to the previous Group By Operator [GBY_237]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_247]
- Please refer to the previous Group By Operator [GBY_246]
- <-Reducer 21 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_256]
- Group By Operator [GBY_255] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_254]
- Group By Operator [GBY_253] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_252] (rows=1704 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_250]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_230]
- Group By Operator [GBY_229] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_227]
- Group By Operator [GBY_225] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_221] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_217]
+ Top N Key Operator [TNK_95] (rows=843315281 width=88)
+ keys:_col25, _col26, _col28, _col29,sort order:++++,top n:100
+ Merge Join Operator [MERGEJOIN_214] (rows=843315281 width=88)
+ Conds:RS_44._col3=RS_252._col0(Inner),Output:["_col5","_col12","_col20","_col25","_col26","_col28","_col29"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_252]
+ PartitionCols:_col0
+ Select Operator [SEL_251] (rows=1704 width=1910)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_250] (rows=1704 width=1910)
+ predicate:s_store_sk is not null
+ TableScan [TS_32] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id","s_store_name"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_44]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_213] (rows=766650239 width=88)
+ Conds:RS_41._col1=RS_243._col0(Inner),Output:["_col3","_col5","_col12","_col20","_col25","_col26"]
+ <-Map 18 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_243]
+ PartitionCols:_col0
+ Select Operator [SEL_242] (rows=462000 width=1436)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_241] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_29] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id","i_item_desc"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_41]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_212] (rows=696954748 width=88)
+ Conds:RS_38._col1, _col2, _col4=RS_39._col8, _col9, _col10(Inner),Output:["_col1","_col3","_col5","_col12","_col20"]
+ <-Reducer 11 [SIMPLE_EDGE]
+ SHUFFLE [RS_39]
+ PartitionCols:_col8, _col9, _col10
+ Merge Join Operator [MERGEJOIN_211] (rows=348467716 width=135)
+ Conds:RS_25._col2, _col1=RS_26._col1, _col2(Inner),Output:["_col3","_col8","_col9","_col10","_col11"]
+ <-Reducer 13 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_26]
+ PartitionCols:_col1, _col2
+ Merge Join Operator [MERGEJOIN_210] (rows=63350266 width=77)
+ Conds:RS_234._col0=RS_225._col0(Inner),Output:["_col1","_col2","_col3","_col4"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_225]
+ PartitionCols:_col0
+ Select Operator [SEL_220] (rows=4058 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_217] (rows=4058 width=1119)
+ predicate:((d_year = 2000) and d_date_sk is not null and d_moy BETWEEN 4 AND 10)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Map 17 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_234]
+ PartitionCols:_col0
+ Select Operator [SEL_233] (rows=57591150 width=77)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_232] (rows=57591150 width=77)
+ predicate:(sr_customer_sk is not null and sr_item_sk is not null and sr_returned_date_sk is not null and sr_ticket_number is not null)
+ TableScan [TS_12] (rows=57591150 width=77)
+ default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number","sr_net_loss"]
+ <-Reducer 10 [SIMPLE_EDGE]
+ SHUFFLE [RS_25]
+ PartitionCols:_col2, _col1
+ Merge Join Operator [MERGEJOIN_209] (rows=316788826 width=135)
+ Conds:RS_265._col0=RS_223._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_223]
+ PartitionCols:_col0
+ Select Operator [SEL_219] (rows=4058 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_216] (rows=4058 width=1119)
+ predicate:((d_year = 2000) and d_date_sk is not null and d_moy BETWEEN 4 AND 10)
+ Please refer to the previous TableScan [TS_3]
+ <-Map 16 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_265]
+ PartitionCols:_col0
+ Select Operator [SEL_264] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_263] (rows=287989836 width=135)
+ predicate:((cs_bill_customer_sk BETWEEN DynamicValue(RS_26_store_returns_sr_customer_sk_min) AND DynamicValue(RS_26_store_returns_sr_customer_sk_max) and in_bloom_filter(cs_bill_customer_sk, DynamicValue(RS_26_store_returns_sr_customer_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_26_store_returns_sr_item_sk_min) AND DynamicValue(RS_26_store_returns_sr_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_26_store_returns_sr_item_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_42_item_i_item_sk_min) AND DynamicValue(RS_42_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_42_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_23_d3_d_date_sk_min) AND DynamicValue(RS_23_d3_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_23_d3_d_date_sk_bloom_filter))) and cs_bill_customer_sk is not null and cs_item_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_6] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_customer_sk","cs_item_sk","cs_net_profit"]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_237]
+ Group By Operator [GBY_235] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_108]
+ Group By Operator [GBY_107] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_106] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_210]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_240]
+ Group By Operator [GBY_238] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 13 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_123]
+ Group By Operator [GBY_122] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_121] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_210]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_249]
+ Group By Operator [GBY_247] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 18 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_246]
+ Group By Operator [GBY_245] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_244] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_242]
+ <-Reducer 12 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_262]
+ Group By Operator [GBY_261] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_229]
+ Group By Operator [GBY_227] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_224] (rows=4058 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_219]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_38]
+ PartitionCols:_col1, _col2, _col4
+ Merge Join Operator [MERGEJOIN_208] (rows=633595212 width=88)
+ Conds:RS_260._col0=RS_221._col0(Inner),Output:["_col1","_col2","_col3","_col4","_col5"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_221]
+ PartitionCols:_col0
+ Select Operator [SEL_218] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_215] (rows=18262 width=1119)
+ predicate:((d_moy = 4) and (d_year = 2000) and d_date_sk is not null)
+ Please refer to the previous TableScan [TS_3]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_260]
+ PartitionCols:_col0
+ Select Operator [SEL_259] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_258] (rows=575995635 width=88)
+ predicate:((ss_customer_sk BETWEEN DynamicValue(RS_26_store_returns_sr_customer_sk_min) AND DynamicValue(RS_26_store_returns_sr_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_26_store_returns_sr_customer_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_26_store_returns_sr_item_sk_min) AND DynamicValue(RS_26_store_returns_sr_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_26_store_returns_sr_item_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_42_item_i_item_sk_min) AND DynamicValue(RS_42_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_42_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_36_d1_d_date_sk_min) AND DynamicValue(RS_36_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_36_d1_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_45_store_s_store_sk_min) AND DynamicValue(RS_45_store_s_store_sk_max) a
nd in_bloom_filter(ss_store_sk, DynamicValue(RS_45_store_s_store_sk_bloom_filter))) and ss_customer_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null and ss_ticket_number is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number","ss_net_profit"]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_236]
+ Please refer to the previous Group By Operator [GBY_235]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_239]
+ Please refer to the previous Group By Operator [GBY_238]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_248]
+ Please refer to the previous Group By Operator [GBY_247]
+ <-Reducer 21 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_257]
+ Group By Operator [GBY_256] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_255]
+ Group By Operator [GBY_254] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_253] (rows=1704 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_251]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_231]
+ Group By Operator [GBY_230] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_228]
+ Group By Operator [GBY_226] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_222] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_218]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query26.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query26.q.out b/ql/src/test/results/clientpositive/perf/tez/query26.q.out
index 0c6c0f8..a90e09f 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query26.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query26.q.out
@@ -58,126 +58,128 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_139]
- Limit [LIM_138] (rows=100 width=135)
+ File Output Operator [FS_140]
+ Limit [LIM_139] (rows=100 width=135)
Number of rows:100
- Select Operator [SEL_137] (rows=210822976 width=135)
+ Select Operator [SEL_138] (rows=210822976 width=135)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_136]
- Select Operator [SEL_135] (rows=210822976 width=135)
+ SHUFFLE [RS_137]
+ Select Operator [SEL_136] (rows=210822976 width=135)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_134] (rows=210822976 width=135)
+ Group By Operator [GBY_135] (rows=210822976 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)","sum(VALUE._col2)","count(VALUE._col3)","sum(VALUE._col4)","count(VALUE._col5)","sum(VALUE._col6)","count(VALUE._col7)"],keys:KEY._col0
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_29]
PartitionCols:_col0
Group By Operator [GBY_28] (rows=421645953 width=135)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["sum(_col4)","count(_col4)","sum(_col5)","count(_col5)","sum(_col7)","count(_col7)","sum(_col6)","count(_col6)"],keys:_col18
- Merge Join Operator [MERGEJOIN_98] (rows=421645953 width=135)
- Conds:RS_24._col2=RS_125._col0(Inner),Output:["_col4","_col5","_col6","_col7","_col18"]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_125]
- PartitionCols:_col0
- Select Operator [SEL_124] (rows=462000 width=1436)
- Output:["_col0","_col1"]
- Filter Operator [FIL_123] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- TableScan [TS_12] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_24]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_97] (rows=383314495 width=135)
- Conds:RS_21._col3=RS_117._col0(Inner),Output:["_col2","_col4","_col5","_col6","_col7"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_117]
- PartitionCols:_col0
- Select Operator [SEL_116] (rows=2300 width=1179)
- Output:["_col0"]
- Filter Operator [FIL_115] (rows=2300 width=1179)
- predicate:(((p_channel_email = 'N') or (p_channel_event = 'N')) and p_promo_sk is not null)
- TableScan [TS_9] (rows=2300 width=1179)
- default@promotion,promotion,Tbl:COMPLETE,Col:NONE,Output:["p_promo_sk","p_channel_email","p_channel_event"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_96] (rows=348467716 width=135)
- Conds:RS_18._col0=RS_109._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6","_col7"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_109]
- PartitionCols:_col0
- Select Operator [SEL_108] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_107] (rows=36524 width=1119)
- predicate:((d_year = 1998) and d_date_sk is not null)
- TableScan [TS_6] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_95] (rows=316788826 width=135)
- Conds:RS_133._col1=RS_101._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6","_col7"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_101]
- PartitionCols:_col0
- Select Operator [SEL_100] (rows=232725 width=385)
- Output:["_col0"]
- Filter Operator [FIL_99] (rows=232725 width=385)
- predicate:((cd_education_status = 'Primary') and (cd_gender = 'F') and (cd_marital_status = 'W') and cd_demo_sk is not null)
- TableScan [TS_3] (rows=1861800 width=385)
- default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_133]
- PartitionCols:_col1
- Select Operator [SEL_132] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- Filter Operator [FIL_131] (rows=287989836 width=135)
- predicate:((cs_bill_cdemo_sk BETWEEN DynamicValue(RS_16_customer_demographics_cd_demo_sk_min) AND DynamicValue(RS_16_customer_demographics_cd_demo_sk_max) and in_bloom_filter(cs_bill_cdemo_sk, DynamicValue(RS_16_customer_demographics_cd_demo_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_25_item_i_item_sk_min) AND DynamicValue(RS_25_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_25_item_i_item_sk_bloom_filter))) and (cs_promo_sk BETWEEN DynamicValue(RS_22_promotion_p_promo_sk_min) AND DynamicValue(RS_22_promotion_p_promo_sk_max) and in_bloom_filter(cs_promo_sk, DynamicValue(RS_22_promotion_p_promo_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and cs_bill_cdemo_sk is not null and cs_item_sk is not null and cs_promo_sk is not null
and cs_sold_date_sk is not null)
- TableScan [TS_0] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_cdemo_sk","cs_item_sk","cs_promo_sk","cs_quantity","cs_list_price","cs_sales_price","cs_coupon_amt"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_114]
- Group By Operator [GBY_113] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_112]
- Group By Operator [GBY_111] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_110] (rows=36524 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_108]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_122]
- Group By Operator [GBY_121] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_120]
- Group By Operator [GBY_119] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_118] (rows=2300 width=1179)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_116]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_130]
- Group By Operator [GBY_129] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 14 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_128]
- Group By Operator [GBY_127] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_126] (rows=462000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_124]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_106]
- Group By Operator [GBY_105] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_104]
- Group By Operator [GBY_103] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_102] (rows=232725 width=385)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_100]
+ Top N Key Operator [TNK_55] (rows=421645953 width=135)
+ keys:_col18,sort order:+,top n:100
+ Merge Join Operator [MERGEJOIN_99] (rows=421645953 width=135)
+ Conds:RS_24._col2=RS_126._col0(Inner),Output:["_col4","_col5","_col6","_col7","_col18"]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_126]
+ PartitionCols:_col0
+ Select Operator [SEL_125] (rows=462000 width=1436)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_124] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_12] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_24]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_98] (rows=383314495 width=135)
+ Conds:RS_21._col3=RS_118._col0(Inner),Output:["_col2","_col4","_col5","_col6","_col7"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_118]
+ PartitionCols:_col0
+ Select Operator [SEL_117] (rows=2300 width=1179)
+ Output:["_col0"]
+ Filter Operator [FIL_116] (rows=2300 width=1179)
+ predicate:(((p_channel_email = 'N') or (p_channel_event = 'N')) and p_promo_sk is not null)
+ TableScan [TS_9] (rows=2300 width=1179)
+ default@promotion,promotion,Tbl:COMPLETE,Col:NONE,Output:["p_promo_sk","p_channel_email","p_channel_event"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_97] (rows=348467716 width=135)
+ Conds:RS_18._col0=RS_110._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col6","_col7"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_110]
+ PartitionCols:_col0
+ Select Operator [SEL_109] (rows=36524 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_108] (rows=36524 width=1119)
+ predicate:((d_year = 1998) and d_date_sk is not null)
+ TableScan [TS_6] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_96] (rows=316788826 width=135)
+ Conds:RS_134._col1=RS_102._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5","_col6","_col7"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_102]
+ PartitionCols:_col0
+ Select Operator [SEL_101] (rows=232725 width=385)
+ Output:["_col0"]
+ Filter Operator [FIL_100] (rows=232725 width=385)
+ predicate:((cd_education_status = 'Primary') and (cd_gender = 'F') and (cd_marital_status = 'W') and cd_demo_sk is not null)
+ TableScan [TS_3] (rows=1861800 width=385)
+ default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_134]
+ PartitionCols:_col1
+ Select Operator [SEL_133] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
+ Filter Operator [FIL_132] (rows=287989836 width=135)
+ predicate:((cs_bill_cdemo_sk BETWEEN DynamicValue(RS_16_customer_demographics_cd_demo_sk_min) AND DynamicValue(RS_16_customer_demographics_cd_demo_sk_max) and in_bloom_filter(cs_bill_cdemo_sk, DynamicValue(RS_16_customer_demographics_cd_demo_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_25_item_i_item_sk_min) AND DynamicValue(RS_25_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_25_item_i_item_sk_bloom_filter))) and (cs_promo_sk BETWEEN DynamicValue(RS_22_promotion_p_promo_sk_min) AND DynamicValue(RS_22_promotion_p_promo_sk_max) and in_bloom_filter(cs_promo_sk, DynamicValue(RS_22_promotion_p_promo_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and cs_bill_cdemo_sk is not null and cs_item_sk is not null and cs_promo_sk is not nul
l and cs_sold_date_sk is not null)
+ TableScan [TS_0] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_cdemo_sk","cs_item_sk","cs_promo_sk","cs_quantity","cs_list_price","cs_sales_price","cs_coupon_amt"]
+ <-Reducer 11 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_115]
+ Group By Operator [GBY_114] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_113]
+ Group By Operator [GBY_112] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_111] (rows=36524 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_109]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_123]
+ Group By Operator [GBY_122] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_121]
+ Group By Operator [GBY_120] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_119] (rows=2300 width=1179)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_117]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_131]
+ Group By Operator [GBY_130] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 14 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_129]
+ Group By Operator [GBY_128] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_127] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_125]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_107]
+ Group By Operator [GBY_106] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_105]
+ Group By Operator [GBY_104] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_103] (rows=232725 width=385)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_101]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query27.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query27.q.out b/ql/src/test/results/clientpositive/perf/tez/query27.q.out
index df1e15f..7ea13c8 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query27.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query27.q.out
@@ -62,128 +62,130 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_140]
- Limit [LIM_139] (rows=100 width=88)
+ File Output Operator [FS_141]
+ Limit [LIM_140] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_138] (rows=1264972921 width=88)
+ Select Operator [SEL_139] (rows=1264972921 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_137]
- Select Operator [SEL_136] (rows=1264972921 width=88)
+ SHUFFLE [RS_138]
+ Select Operator [SEL_137] (rows=1264972921 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- Group By Operator [GBY_135] (rows=1264972921 width=88)
+ Group By Operator [GBY_136] (rows=1264972921 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)","sum(VALUE._col2)","count(VALUE._col3)","sum(VALUE._col4)","count(VALUE._col5)","sum(VALUE._col6)","count(VALUE._col7)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_30]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_29] (rows=2529945843 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"],aggregations:["sum(_col2)","count(_col2)","sum(_col3)","count(_col3)","sum(_col4)","count(_col4)","sum(_col5)","count(_col5)"],keys:_col0, _col1, 0L
- Select Operator [SEL_27] (rows=843315281 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_99] (rows=843315281 width=88)
- Conds:RS_24._col1=RS_126._col0(Inner),Output:["_col4","_col5","_col6","_col7","_col15","_col17"]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_126]
- PartitionCols:_col0
- Select Operator [SEL_125] (rows=462000 width=1436)
- Output:["_col0","_col1"]
- Filter Operator [FIL_124] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- TableScan [TS_12] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_24]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_98] (rows=766650239 width=88)
- Conds:RS_21._col3=RS_118._col0(Inner),Output:["_col1","_col4","_col5","_col6","_col7","_col15"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_118]
- PartitionCols:_col0
- Select Operator [SEL_117] (rows=852 width=1910)
- Output:["_col0","_col1"]
- Filter Operator [FIL_116] (rows=852 width=1910)
- predicate:((s_state) IN ('SD', 'FL', 'MI', 'LA', 'MO', 'SC') and s_store_sk is not null)
- TableScan [TS_9] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_state"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_97] (rows=696954748 width=88)
- Conds:RS_18._col0=RS_110._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_110]
- PartitionCols:_col0
- Select Operator [SEL_109] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_108] (rows=36524 width=1119)
- predicate:((d_year = 2001) and d_date_sk is not null)
- TableScan [TS_6] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_96] (rows=633595212 width=88)
- Conds:RS_134._col2=RS_102._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_102]
- PartitionCols:_col0
- Select Operator [SEL_101] (rows=232725 width=385)
- Output:["_col0"]
- Filter Operator [FIL_100] (rows=232725 width=385)
- predicate:((cd_education_status = '2 yr Degree') and (cd_gender = 'M') and (cd_marital_status = 'U') and cd_demo_sk is not null)
- TableScan [TS_3] (rows=1861800 width=385)
- default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_134]
- PartitionCols:_col2
- Select Operator [SEL_133] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- Filter Operator [FIL_132] (rows=575995635 width=88)
- predicate:((ss_cdemo_sk BETWEEN DynamicValue(RS_16_customer_demographics_cd_demo_sk_min) AND DynamicValue(RS_16_customer_demographics_cd_demo_sk_max) and in_bloom_filter(ss_cdemo_sk, DynamicValue(RS_16_customer_demographics_cd_demo_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_25_item_i_item_sk_min) AND DynamicValue(RS_25_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_25_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_22_store_s_store_sk_min) AND DynamicValue(RS_22_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_22_store_s_store_sk_bloom_filter))) and ss_cdemo_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is no
t null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_cdemo_sk","ss_store_sk","ss_quantity","ss_list_price","ss_sales_price","ss_coupon_amt"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_115]
- Group By Operator [GBY_114] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_113]
- Group By Operator [GBY_112] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_111] (rows=36524 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_109]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_123]
- Group By Operator [GBY_122] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_121]
- Group By Operator [GBY_120] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_119] (rows=852 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_117]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_131]
- Group By Operator [GBY_130] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 14 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_129]
- Group By Operator [GBY_128] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_127] (rows=462000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_125]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_107]
- Group By Operator [GBY_106] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_105]
- Group By Operator [GBY_104] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_103] (rows=232725 width=385)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_101]
+ Top N Key Operator [TNK_56] (rows=843315281 width=88)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_27] (rows=843315281 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_100] (rows=843315281 width=88)
+ Conds:RS_24._col1=RS_127._col0(Inner),Output:["_col4","_col5","_col6","_col7","_col15","_col17"]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_127]
+ PartitionCols:_col0
+ Select Operator [SEL_126] (rows=462000 width=1436)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_125] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_12] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_24]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_99] (rows=766650239 width=88)
+ Conds:RS_21._col3=RS_119._col0(Inner),Output:["_col1","_col4","_col5","_col6","_col7","_col15"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_119]
+ PartitionCols:_col0
+ Select Operator [SEL_118] (rows=852 width=1910)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_117] (rows=852 width=1910)
+ predicate:((s_state) IN ('SD', 'FL', 'MI', 'LA', 'MO', 'SC') and s_store_sk is not null)
+ TableScan [TS_9] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_state"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_98] (rows=696954748 width=88)
+ Conds:RS_18._col0=RS_111._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_111]
+ PartitionCols:_col0
+ Select Operator [SEL_110] (rows=36524 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_109] (rows=36524 width=1119)
+ predicate:((d_year = 2001) and d_date_sk is not null)
+ TableScan [TS_6] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_97] (rows=633595212 width=88)
+ Conds:RS_135._col2=RS_103._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_103]
+ PartitionCols:_col0
+ Select Operator [SEL_102] (rows=232725 width=385)
+ Output:["_col0"]
+ Filter Operator [FIL_101] (rows=232725 width=385)
+ predicate:((cd_education_status = '2 yr Degree') and (cd_gender = 'M') and (cd_marital_status = 'U') and cd_demo_sk is not null)
+ TableScan [TS_3] (rows=1861800 width=385)
+ default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_135]
+ PartitionCols:_col2
+ Select Operator [SEL_134] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
+ Filter Operator [FIL_133] (rows=575995635 width=88)
+ predicate:((ss_cdemo_sk BETWEEN DynamicValue(RS_16_customer_demographics_cd_demo_sk_min) AND DynamicValue(RS_16_customer_demographics_cd_demo_sk_max) and in_bloom_filter(ss_cdemo_sk, DynamicValue(RS_16_customer_demographics_cd_demo_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_25_item_i_item_sk_min) AND DynamicValue(RS_25_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_25_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_22_store_s_store_sk_min) AND DynamicValue(RS_22_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_22_store_s_store_sk_bloom_filter))) and ss_cdemo_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is
not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_cdemo_sk","ss_store_sk","ss_quantity","ss_list_price","ss_sales_price","ss_coupon_amt"]
+ <-Reducer 11 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_116]
+ Group By Operator [GBY_115] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_114]
+ Group By Operator [GBY_113] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_112] (rows=36524 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_110]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_124]
+ Group By Operator [GBY_123] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_122]
+ Group By Operator [GBY_121] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_120] (rows=852 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_118]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_132]
+ Group By Operator [GBY_131] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 14 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_130]
+ Group By Operator [GBY_129] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_128] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_126]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_108]
+ Group By Operator [GBY_107] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_106]
+ Group By Operator [GBY_105] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_104] (rows=232725 width=385)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_102]
[17/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out b/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
index 3675d8d..5f5f5f6 100644
--- a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
@@ -303,124 +303,126 @@ Stage-0
limit:100
Stage-1
Reducer 5 vectorized, llap
- File Output Operator [FS_216]
- Limit [LIM_215] (rows=12 width=285)
+ File Output Operator [FS_217]
+ Limit [LIM_216] (rows=12 width=285)
Number of rows:100
- Select Operator [SEL_214] (rows=12 width=285)
+ Select Operator [SEL_215] (rows=12 width=285)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 4 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_213]
- Group By Operator [GBY_212] (rows=12 width=285)
+ SHUFFLE [RS_214]
+ Group By Operator [GBY_213] (rows=12 width=285)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(VALUE._col0)","count(VALUE._col1)","count(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 3 [SIMPLE_EDGE] llap
SHUFFLE [RS_49]
PartitionCols:_col0, _col1, _col2
Group By Operator [GBY_48] (rows=12 width=285)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(_col11)","count(_col21)","count(_col3)"],keys:_col10, _col20, _col2
- Merge Join Operator [MERGEJOIN_187] (rows=9275 width=534)
- Conds:RS_44._col1, _col3=RS_45._col15, _col17(Inner),Output:["_col2","_col3","_col10","_col11","_col20","_col21"]
- <-Reducer 10 [SIMPLE_EDGE] llap
- SHUFFLE [RS_45]
- PartitionCols:_col15, _col17
- Select Operator [SEL_40] (rows=420 width=447)
- Output:["_col4","_col5","_col14","_col15","_col17"]
- Merge Join Operator [MERGEJOIN_186] (rows=420 width=447)
- Conds:RS_37._col4, _col2=RS_38._col4, _col2(Inner),Output:["_col0","_col1","_col14","_col15","_col17"]
- <-Reducer 11 [SIMPLE_EDGE] llap
- SHUFFLE [RS_38]
- PartitionCols:_col4, _col2
- Merge Join Operator [MERGEJOIN_185] (rows=10 width=356)
- Conds:RS_211._col0=RS_199._col0(Inner),Output:["_col2","_col3","_col4","_col5"]
- <-Map 6 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_199]
- PartitionCols:_col0
- Select Operator [SEL_196] (rows=25 width=178)
- Output:["_col0"]
- Filter Operator [FIL_193] (rows=25 width=178)
- predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
- TableScan [TS_3] (rows=500 width=178)
- default@src,d3,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
- <-Map 15 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_211]
- PartitionCols:_col0
- Select Operator [SEL_210] (rows=7 width=531)
- Output:["_col0","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_209] (rows=7 width=534)
- predicate:((v1 = 'srv1') and k1 is not null and k2 is not null and k3 is not null and v2 is not null and v3 is not null)
- TableScan [TS_18] (rows=85 width=534)
- default@sr,sr,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v1","k2","v2","k3","v3"]
- <-Reducer 9 [SIMPLE_EDGE] llap
- SHUFFLE [RS_37]
- PartitionCols:_col4, _col2
- Merge Join Operator [MERGEJOIN_184] (rows=42 width=352)
- Conds:RS_34._col1=RS_208._col1(Inner),Output:["_col0","_col1","_col2","_col4"]
- <-Map 14 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_208]
- PartitionCols:_col1
- Select Operator [SEL_207] (rows=2 width=180)
- Output:["_col1"]
- Filter Operator [FIL_206] (rows=2 width=175)
- predicate:((key = 'src1key') and value is not null)
- TableScan [TS_15] (rows=25 width=175)
- default@src1,src1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
- <-Reducer 8 [SIMPLE_EDGE] llap
- SHUFFLE [RS_34]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_183] (rows=42 width=352)
- Conds:RS_31._col3=RS_205._col1(Inner),Output:["_col0","_col1","_col2","_col4"]
- <-Map 13 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_205]
- PartitionCols:_col1
- Select Operator [SEL_204] (rows=6 width=185)
- Output:["_col1"]
- Filter Operator [FIL_203] (rows=6 width=178)
- predicate:((key = 'srcpartkey') and value is not null)
- TableScan [TS_12] (rows=2000 width=178)
- default@srcpart,srcpart,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
- <-Reducer 7 [SIMPLE_EDGE] llap
- SHUFFLE [RS_31]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_182] (rows=7 width=443)
- Conds:RS_202._col0=RS_198._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
- <-Map 6 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_198]
- PartitionCols:_col0
- Select Operator [SEL_195] (rows=2 width=178)
- Output:["_col0"]
- Filter Operator [FIL_192] (rows=2 width=178)
- predicate:((value = 'd1value') and key is not null)
- Please refer to the previous TableScan [TS_3]
- <-Map 12 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_202]
- PartitionCols:_col0
- Select Operator [SEL_201] (rows=7 width=531)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_200] (rows=7 width=534)
- predicate:((v3 = 'ssv3') and k1 is not null and k2 is not null and k3 is not null and v1 is not null and v2 is not null)
- TableScan [TS_6] (rows=85 width=534)
- default@ss_n1,ss_n1,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v1","k2","v2","k3","v3"]
- <-Reducer 2 [SIMPLE_EDGE] llap
- SHUFFLE [RS_44]
- PartitionCols:_col1, _col3
- Merge Join Operator [MERGEJOIN_181] (rows=265 width=269)
- Conds:RS_190._col0=RS_197._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 6 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_197]
- PartitionCols:_col0
- Select Operator [SEL_194] (rows=25 width=178)
- Output:["_col0"]
- Filter Operator [FIL_191] (rows=25 width=178)
- predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
- Please refer to the previous TableScan [TS_3]
- <-Map 1 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_190]
- PartitionCols:_col0
- Select Operator [SEL_189] (rows=170 width=356)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_188] (rows=170 width=356)
- predicate:(k1 is not null and v2 is not null and v3 is not null)
- TableScan [TS_0] (rows=170 width=356)
- default@cs,cs,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v2","k3","v3"]
+ Top N Key Operator [TNK_91] (rows=9275 width=534)
+ keys:_col10, _col20, _col2,sort order:+++,top n:100
+ Merge Join Operator [MERGEJOIN_188] (rows=9275 width=534)
+ Conds:RS_44._col1, _col3=RS_45._col15, _col17(Inner),Output:["_col2","_col3","_col10","_col11","_col20","_col21"]
+ <-Reducer 10 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_45]
+ PartitionCols:_col15, _col17
+ Select Operator [SEL_40] (rows=420 width=447)
+ Output:["_col4","_col5","_col14","_col15","_col17"]
+ Merge Join Operator [MERGEJOIN_187] (rows=420 width=447)
+ Conds:RS_37._col4, _col2=RS_38._col4, _col2(Inner),Output:["_col0","_col1","_col14","_col15","_col17"]
+ <-Reducer 11 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_38]
+ PartitionCols:_col4, _col2
+ Merge Join Operator [MERGEJOIN_186] (rows=10 width=356)
+ Conds:RS_212._col0=RS_200._col0(Inner),Output:["_col2","_col3","_col4","_col5"]
+ <-Map 6 [SIMPLE_EDGE] vectorized, llap
+ SHUFFLE [RS_200]
+ PartitionCols:_col0
+ Select Operator [SEL_197] (rows=25 width=178)
+ Output:["_col0"]
+ Filter Operator [FIL_194] (rows=25 width=178)
+ predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
+ TableScan [TS_3] (rows=500 width=178)
+ default@src,d3,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Map 15 [SIMPLE_EDGE] vectorized, llap
+ SHUFFLE [RS_212]
+ PartitionCols:_col0
+ Select Operator [SEL_211] (rows=7 width=531)
+ Output:["_col0","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_210] (rows=7 width=534)
+ predicate:((v1 = 'srv1') and k1 is not null and k2 is not null and k3 is not null and v2 is not null and v3 is not null)
+ TableScan [TS_18] (rows=85 width=534)
+ default@sr,sr,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v1","k2","v2","k3","v3"]
+ <-Reducer 9 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_37]
+ PartitionCols:_col4, _col2
+ Merge Join Operator [MERGEJOIN_185] (rows=42 width=352)
+ Conds:RS_34._col1=RS_209._col1(Inner),Output:["_col0","_col1","_col2","_col4"]
+ <-Map 14 [SIMPLE_EDGE] vectorized, llap
+ SHUFFLE [RS_209]
+ PartitionCols:_col1
+ Select Operator [SEL_208] (rows=2 width=180)
+ Output:["_col1"]
+ Filter Operator [FIL_207] (rows=2 width=175)
+ predicate:((key = 'src1key') and value is not null)
+ TableScan [TS_15] (rows=25 width=175)
+ default@src1,src1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 8 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_34]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_184] (rows=42 width=352)
+ Conds:RS_31._col3=RS_206._col1(Inner),Output:["_col0","_col1","_col2","_col4"]
+ <-Map 13 [SIMPLE_EDGE] vectorized, llap
+ SHUFFLE [RS_206]
+ PartitionCols:_col1
+ Select Operator [SEL_205] (rows=6 width=185)
+ Output:["_col1"]
+ Filter Operator [FIL_204] (rows=6 width=178)
+ predicate:((key = 'srcpartkey') and value is not null)
+ TableScan [TS_12] (rows=2000 width=178)
+ default@srcpart,srcpart,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 7 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_31]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_183] (rows=7 width=443)
+ Conds:RS_203._col0=RS_199._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
+ <-Map 6 [SIMPLE_EDGE] vectorized, llap
+ SHUFFLE [RS_199]
+ PartitionCols:_col0
+ Select Operator [SEL_196] (rows=2 width=178)
+ Output:["_col0"]
+ Filter Operator [FIL_193] (rows=2 width=178)
+ predicate:((value = 'd1value') and key is not null)
+ Please refer to the previous TableScan [TS_3]
+ <-Map 12 [SIMPLE_EDGE] vectorized, llap
+ SHUFFLE [RS_203]
+ PartitionCols:_col0
+ Select Operator [SEL_202] (rows=7 width=531)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_201] (rows=7 width=534)
+ predicate:((v3 = 'ssv3') and k1 is not null and k2 is not null and k3 is not null and v1 is not null and v2 is not null)
+ TableScan [TS_6] (rows=85 width=534)
+ default@ss_n1,ss_n1,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v1","k2","v2","k3","v3"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_44]
+ PartitionCols:_col1, _col3
+ Merge Join Operator [MERGEJOIN_182] (rows=265 width=269)
+ Conds:RS_191._col0=RS_198._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 6 [SIMPLE_EDGE] vectorized, llap
+ SHUFFLE [RS_198]
+ PartitionCols:_col0
+ Select Operator [SEL_195] (rows=25 width=178)
+ Output:["_col0"]
+ Filter Operator [FIL_192] (rows=25 width=178)
+ predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
+ Please refer to the previous TableScan [TS_3]
+ <-Map 1 [SIMPLE_EDGE] vectorized, llap
+ SHUFFLE [RS_191]
+ PartitionCols:_col0
+ Select Operator [SEL_190] (rows=170 width=356)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_189] (rows=170 width=356)
+ predicate:(k1 is not null and v2 is not null and v3 is not null)
+ TableScan [TS_0] (rows=170 width=356)
+ default@cs,cs,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v2","k3","v3"]
PREHOOK: query: explain
SELECT x.key, z.value, y.value
@@ -992,105 +994,107 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized, llap
- File Output Operator [FS_232]
- Limit [LIM_231] (rows=100 width=10)
+ File Output Operator [FS_234]
+ Limit [LIM_233] (rows=100 width=10)
Number of rows:100
- Select Operator [SEL_230] (rows=732 width=10)
+ Select Operator [SEL_232] (rows=732 width=10)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
<-Reducer 6 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_229]
- Group By Operator [GBY_228] (rows=732 width=10)
+ SHUFFLE [RS_231]
+ Group By Operator [GBY_230] (rows=732 width=10)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(VALUE._col0)","count(VALUE._col1)","count(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Map 5 [SIMPLE_EDGE] vectorized, llap
- SHUFFLE [RS_227]
+ SHUFFLE [RS_229]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_226] (rows=1464 width=10)
+ Group By Operator [GBY_228] (rows=1464 width=10)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(_col11)","count(_col21)","count(_col3)"],keys:_col10, _col20, _col2
- Map Join Operator [MAPJOIN_225] (rows=1464 width=10)
- Conds:RS_208._col1, _col3=SEL_224._col15, _col17(Inner),Output:["_col2","_col3","_col10","_col11","_col20","_col21"]
- <-Map 2 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_208]
- PartitionCols:_col1, _col3
- Map Join Operator [MAPJOIN_207] (rows=275 width=10)
- Conds:RS_204._col0=SEL_206._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 1 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_204]
- PartitionCols:_col0
- Select Operator [SEL_203] (rows=170 width=34)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_202] (rows=170 width=34)
- predicate:(k1 is not null and v2 is not null and v3 is not null)
- TableScan [TS_0] (rows=170 width=34)
- default@cs,cs,Tbl:COMPLETE,Col:NONE,Output:["k1","v2","k3","v3"]
- <-Select Operator [SEL_206] (rows=250 width=10)
- Output:["_col0"]
- Filter Operator [FIL_205] (rows=250 width=10)
- predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
- TableScan [TS_3] (rows=500 width=10)
- default@src,d3,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
- <-Select Operator [SEL_224] (rows=1331 width=10)
- Output:["_col4","_col5","_col14","_col15","_col17"]
- Map Join Operator [MAPJOIN_223] (rows=1331 width=10)
- Conds:MAPJOIN_222._col4, _col2=RS_218._col4, _col2(Inner),Output:["_col0","_col1","_col14","_col15","_col17"]
- <-Map 10 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_218]
- PartitionCols:_col4, _col2
- Map Join Operator [MAPJOIN_217] (rows=275 width=10)
- Conds:RS_214._col0=SEL_216._col0(Inner),Output:["_col2","_col3","_col4","_col5"]
- <-Map 9 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_214]
- PartitionCols:_col0
- Select Operator [SEL_213] (rows=42 width=34)
- Output:["_col0","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_212] (rows=42 width=34)
- predicate:((v1 = 'srv1') and k1 is not null and k2 is not null and k3 is not null and v2 is not null and v3 is not null)
- TableScan [TS_18] (rows=85 width=34)
- default@sr,sr,Tbl:COMPLETE,Col:NONE,Output:["k1","v1","k2","v2","k3","v3"]
- <-Select Operator [SEL_216] (rows=250 width=10)
- Output:["_col0"]
- Filter Operator [FIL_215] (rows=250 width=10)
- predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
- TableScan [TS_21] (rows=500 width=10)
- default@src,d2,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
- <-Map Join Operator [MAPJOIN_222] (rows=1210 width=10)
- Conds:MAPJOIN_221._col1=RS_211._col1(Inner),Output:["_col0","_col1","_col2","_col4"]
- <-Map 8 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_211]
- PartitionCols:_col1
- Select Operator [SEL_210] (rows=12 width=7)
- Output:["_col1"]
- Filter Operator [FIL_209] (rows=12 width=7)
- predicate:((key = 'src1key') and value is not null)
- TableScan [TS_15] (rows=25 width=7)
- default@src1,src1,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
- <-Map Join Operator [MAPJOIN_221] (rows=1100 width=10)
- Conds:RS_201._col3=SEL_220._col1(Inner),Output:["_col0","_col1","_col2","_col4"]
- <-Map 4 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_201]
- PartitionCols:_col3
- Map Join Operator [MAPJOIN_200] (rows=275 width=10)
- Conds:RS_197._col0=SEL_199._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
- <-Map 3 [BROADCAST_EDGE] vectorized, llap
- BROADCAST [RS_197]
- PartitionCols:_col0
- Select Operator [SEL_196] (rows=42 width=34)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_195] (rows=42 width=34)
- predicate:((v3 = 'ssv3') and k1 is not null and k2 is not null and k3 is not null and v1 is not null and v2 is not null)
- TableScan [TS_6] (rows=85 width=34)
- default@ss_n1,ss_n1,Tbl:COMPLETE,Col:NONE,Output:["k1","v1","k2","v2","k3","v3"]
- <-Select Operator [SEL_199] (rows=250 width=10)
- Output:["_col0"]
- Filter Operator [FIL_198] (rows=250 width=10)
- predicate:((value = 'd1value') and key is not null)
- TableScan [TS_9] (rows=500 width=10)
- default@src,d1,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
- <-Select Operator [SEL_220] (rows=1000 width=10)
- Output:["_col1"]
- Filter Operator [FIL_219] (rows=1000 width=10)
- predicate:((key = 'srcpartkey') and value is not null)
- TableScan [TS_12] (rows=2000 width=10)
- default@srcpart,srcpart,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
+ Top N Key Operator [TNK_227] (rows=1464 width=10)
+ keys:_col10, _col20, _col2,sort order:+++,top n:100
+ Map Join Operator [MAPJOIN_226] (rows=1464 width=10)
+ Conds:RS_209._col1, _col3=SEL_225._col15, _col17(Inner),Output:["_col2","_col3","_col10","_col11","_col20","_col21"]
+ <-Map 2 [BROADCAST_EDGE] vectorized, llap
+ BROADCAST [RS_209]
+ PartitionCols:_col1, _col3
+ Map Join Operator [MAPJOIN_208] (rows=275 width=10)
+ Conds:RS_205._col0=SEL_207._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 1 [BROADCAST_EDGE] vectorized, llap
+ BROADCAST [RS_205]
+ PartitionCols:_col0
+ Select Operator [SEL_204] (rows=170 width=34)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_203] (rows=170 width=34)
+ predicate:(k1 is not null and v2 is not null and v3 is not null)
+ TableScan [TS_0] (rows=170 width=34)
+ default@cs,cs,Tbl:COMPLETE,Col:NONE,Output:["k1","v2","k3","v3"]
+ <-Select Operator [SEL_207] (rows=250 width=10)
+ Output:["_col0"]
+ Filter Operator [FIL_206] (rows=250 width=10)
+ predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
+ TableScan [TS_3] (rows=500 width=10)
+ default@src,d3,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
+ <-Select Operator [SEL_225] (rows=1331 width=10)
+ Output:["_col4","_col5","_col14","_col15","_col17"]
+ Map Join Operator [MAPJOIN_224] (rows=1331 width=10)
+ Conds:MAPJOIN_223._col4, _col2=RS_219._col4, _col2(Inner),Output:["_col0","_col1","_col14","_col15","_col17"]
+ <-Map 10 [BROADCAST_EDGE] vectorized, llap
+ BROADCAST [RS_219]
+ PartitionCols:_col4, _col2
+ Map Join Operator [MAPJOIN_218] (rows=275 width=10)
+ Conds:RS_215._col0=SEL_217._col0(Inner),Output:["_col2","_col3","_col4","_col5"]
+ <-Map 9 [BROADCAST_EDGE] vectorized, llap
+ BROADCAST [RS_215]
+ PartitionCols:_col0
+ Select Operator [SEL_214] (rows=42 width=34)
+ Output:["_col0","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_213] (rows=42 width=34)
+ predicate:((v1 = 'srv1') and k1 is not null and k2 is not null and k3 is not null and v2 is not null and v3 is not null)
+ TableScan [TS_18] (rows=85 width=34)
+ default@sr,sr,Tbl:COMPLETE,Col:NONE,Output:["k1","v1","k2","v2","k3","v3"]
+ <-Select Operator [SEL_217] (rows=250 width=10)
+ Output:["_col0"]
+ Filter Operator [FIL_216] (rows=250 width=10)
+ predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
+ TableScan [TS_21] (rows=500 width=10)
+ default@src,d2,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
+ <-Map Join Operator [MAPJOIN_223] (rows=1210 width=10)
+ Conds:MAPJOIN_222._col1=RS_212._col1(Inner),Output:["_col0","_col1","_col2","_col4"]
+ <-Map 8 [BROADCAST_EDGE] vectorized, llap
+ BROADCAST [RS_212]
+ PartitionCols:_col1
+ Select Operator [SEL_211] (rows=12 width=7)
+ Output:["_col1"]
+ Filter Operator [FIL_210] (rows=12 width=7)
+ predicate:((key = 'src1key') and value is not null)
+ TableScan [TS_15] (rows=25 width=7)
+ default@src1,src1,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
+ <-Map Join Operator [MAPJOIN_222] (rows=1100 width=10)
+ Conds:RS_202._col3=SEL_221._col1(Inner),Output:["_col0","_col1","_col2","_col4"]
+ <-Map 4 [BROADCAST_EDGE] vectorized, llap
+ BROADCAST [RS_202]
+ PartitionCols:_col3
+ Map Join Operator [MAPJOIN_201] (rows=275 width=10)
+ Conds:RS_198._col0=SEL_200._col0(Inner),Output:["_col0","_col1","_col2","_col3","_col4"]
+ <-Map 3 [BROADCAST_EDGE] vectorized, llap
+ BROADCAST [RS_198]
+ PartitionCols:_col0
+ Select Operator [SEL_197] (rows=42 width=34)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_196] (rows=42 width=34)
+ predicate:((v3 = 'ssv3') and k1 is not null and k2 is not null and k3 is not null and v1 is not null and v2 is not null)
+ TableScan [TS_6] (rows=85 width=34)
+ default@ss_n1,ss_n1,Tbl:COMPLETE,Col:NONE,Output:["k1","v1","k2","v2","k3","v3"]
+ <-Select Operator [SEL_200] (rows=250 width=10)
+ Output:["_col0"]
+ Filter Operator [FIL_199] (rows=250 width=10)
+ predicate:((value = 'd1value') and key is not null)
+ TableScan [TS_9] (rows=500 width=10)
+ default@src,d1,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
+ <-Select Operator [SEL_221] (rows=1000 width=10)
+ Output:["_col1"]
+ Filter Operator [FIL_220] (rows=1000 width=10)
+ predicate:((key = 'srcpartkey') and value is not null)
+ TableScan [TS_12] (rows=2000 width=10)
+ default@srcpart,srcpart,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
PREHOOK: query: explain
SELECT x.key, z.value, y.value
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out b/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out
index 1463b32..252831e 100644
--- a/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out
+++ b/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out
@@ -199,19 +199,24 @@ STAGE PLANS:
expressions: value (type: string), (UDFToDouble(key) + 1.0D) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1)
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Group By Operator
+ aggregations: sum(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: double)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
+ value expressions: _col1 (type: double)
Execution mode: vectorized, llap
LLAP IO: no inputs
Reducer 2
@@ -295,19 +300,24 @@ STAGE PLANS:
expressions: value (type: string), (UDFToDouble(key) + 1.0D) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1), count(_col1)
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Group By Operator
+ aggregations: sum(_col1), count(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: double), _col2 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
+ value expressions: _col1 (type: double), _col2 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: no inputs
Reducer 2
@@ -395,17 +405,22 @@ STAGE PLANS:
expressions: cdouble (type: double)
outputColumnNames: cdouble
Statistics: Num rows: 12288 Data size: 73400 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
+ Top N Key Operator
+ sort order: +
keys: cdouble (type: double)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: double)
- sort order: +
- Map-reduce partition columns: _col0 (type: double)
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
+ Statistics: Num rows: 12288 Data size: 73400 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Group By Operator
+ keys: cdouble (type: double)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: double)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: double)
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
@@ -415,13 +430,13 @@ STAGE PLANS:
keys: KEY._col0 (type: double)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 128 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 20 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 128 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 20 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -921,26 +936,42 @@ STAGE PLANS:
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: key (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 2
+ Group By Operator
+ aggregations: count()
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: bigint)
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
+ value expressions: _col1 (type: bigint)
+ Top N Key Operator
+ sort order: +
+ keys: key (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 3
+ Group By Operator
+ aggregations: count()
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
+ value expressions: _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: no inputs
Reducer 2
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out b/ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out
index 260ce3d..1027bfe 100644
--- a/ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out
+++ b/ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out
@@ -200,19 +200,24 @@ STAGE PLANS:
expressions: value (type: string), (UDFToDouble(key) + 1.0D) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1)
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Group By Operator
+ aggregations: sum(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: double)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
+ value expressions: _col1 (type: double)
Execution mode: vectorized, llap
LLAP IO: no inputs
Reducer 2
@@ -310,19 +315,24 @@ STAGE PLANS:
expressions: value (type: string), (UDFToDouble(key) + 1.0D) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1), count(_col1)
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Group By Operator
+ aggregations: sum(_col1), count(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: double), _col2 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
+ value expressions: _col1 (type: double), _col2 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: no inputs
Reducer 2
@@ -424,17 +434,22 @@ STAGE PLANS:
expressions: cdouble (type: double)
outputColumnNames: cdouble
Statistics: Num rows: 12288 Data size: 73400 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
+ Top N Key Operator
+ sort order: +
keys: cdouble (type: double)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: double)
- sort order: +
- Map-reduce partition columns: _col0 (type: double)
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
+ Statistics: Num rows: 12288 Data size: 73400 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Group By Operator
+ keys: cdouble (type: double)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: double)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: double)
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
Execution mode: vectorized, llap
LLAP IO: all inputs
Reducer 2
@@ -444,11 +459,11 @@ STAGE PLANS:
keys: KEY._col0 (type: double)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double)
sort order: +
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
Reducer 3
Execution mode: vectorized, llap
@@ -456,13 +471,13 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: double)
outputColumnNames: _col0
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 128 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 20 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 128 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 20 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out b/ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out
index 0041206..88ddd9c 100644
--- a/ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out
+++ b/ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out
@@ -131,17 +131,22 @@ STAGE PLANS:
Filter Operator
predicate: ((cdecimal1 = 3.35) or (cdecimal1 = 4.46)) (type: boolean)
Statistics: Num rows: 24576 Data size: 5505024 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
+ Top N Key Operator
+ sort order: ++
keys: cdecimal1 (type: decimal(10,2)), cdecimal2 (type: decimal(38,5))
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: decimal(10,2)), _col1 (type: decimal(38,5))
- sort order: ++
- Map-reduce partition columns: _col0 (type: decimal(10,2)), _col1 (type: decimal(38,5))
+ Statistics: Num rows: 24576 Data size: 5505024 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 2
+ Group By Operator
+ keys: cdecimal1 (type: decimal(10,2)), cdecimal2 (type: decimal(38,5))
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
+ Reduce Output Operator
+ key expressions: _col0 (type: decimal(10,2)), _col1 (type: decimal(38,5))
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: decimal(10,2)), _col1 (type: decimal(38,5))
+ Statistics: Num rows: 2 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
@@ -227,17 +232,22 @@ STAGE PLANS:
Filter Operator
predicate: ((cdecimal1 = 3.35) or (cdecimal1 = 4.46)) (type: boolean)
Statistics: Num rows: 24576 Data size: 5505024 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
+ Top N Key Operator
+ sort order: ++
keys: cdecimal1 (type: decimal(10,2)), cdecimal2 (type: decimal(38,5))
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: decimal(10,2)), _col1 (type: decimal(38,5))
- sort order: ++
- Map-reduce partition columns: _col0 (type: decimal(10,2)), _col1 (type: decimal(38,5))
+ Statistics: Num rows: 24576 Data size: 5505024 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 2
+ Group By Operator
+ keys: cdecimal1 (type: decimal(10,2)), cdecimal2 (type: decimal(38,5))
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 2 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
+ Reduce Output Operator
+ key expressions: _col0 (type: decimal(10,2)), _col1 (type: decimal(38,5))
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: decimal(10,2)), _col1 (type: decimal(38,5))
+ Statistics: Num rows: 2 Data size: 448 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/offset_limit.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/offset_limit.q.out b/ql/src/test/results/clientpositive/llap/offset_limit.q.out
index 0e718fd..97d2ac2 100644
--- a/ql/src/test/results/clientpositive/llap/offset_limit.q.out
+++ b/ql/src/test/results/clientpositive/llap/offset_limit.q.out
@@ -26,19 +26,24 @@ STAGE PLANS:
expressions: key (type: string), substr(value, 5) (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1)
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Group By Operator
+ aggregations: sum(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: double)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: double)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out b/ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out
index 4c62d7f..2e8d5f3 100644
--- a/ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out
+++ b/ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out
@@ -201,19 +201,24 @@ STAGE PLANS:
expressions: value (type: string), (UDFToDouble(key) + 1.0D) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1)
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 30
+ Group By Operator
+ aggregations: sum(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: double)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 24750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
+ value expressions: _col1 (type: double)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -298,19 +303,24 @@ STAGE PLANS:
expressions: value (type: string), (UDFToDouble(key) + 1.0D) (type: double)
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1), count(_col1)
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 30
+ Group By Operator
+ aggregations: sum(_col1), count(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: double), _col2 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
+ value expressions: _col1 (type: double), _col2 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -399,17 +409,22 @@ STAGE PLANS:
expressions: cdouble (type: double)
outputColumnNames: cdouble
Statistics: Num rows: 12288 Data size: 73400 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
+ Top N Key Operator
+ sort order: +
keys: cdouble (type: double)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: double)
- sort order: +
- Map-reduce partition columns: _col0 (type: double)
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.3
+ Statistics: Num rows: 12288 Data size: 73400 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 30
+ Group By Operator
+ keys: cdouble (type: double)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: double)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: double)
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.3
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -419,14 +434,14 @@ STAGE PLANS:
keys: KEY._col0 (type: double)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 5528 Data size: 33024 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5528 Data size: 21816 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
Offset of rows: 10
- Statistics: Num rows: 20 Data size: 128 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 20 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 128 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 20 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out b/ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out
index 5c6b481..a804e3c 100644
--- a/ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out
@@ -238,31 +238,40 @@ STAGE PLANS:
projectedOutputColumnNums: [4]
selectExpressions: VectorUDFStructField(col 1:struct<f1:int,f2:string>, col 0:int) -> 4:int
Statistics: Num rows: 341 Data size: 76542 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col0)
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(col 4:int) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 4:int
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 76542 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkLongOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 4:int
+ native: true
+ Group By Operator
+ aggregations: sum(_col0)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 4:int) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 4:int
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 76542 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkLongOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 341 Data size: 76542 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out b/ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out
index da9dd4e..baeb60e 100644
--- a/ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out
+++ b/ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out
@@ -214,31 +214,40 @@ STAGE PLANS:
projectedOutputColumnNums: [6, 7]
selectExpressions: ListIndexColScalar(col 2:array<int>, col 1:int) -> 6:int, ListIndexColScalar(col 2:array<int>, col 0:int) -> 7:int
Statistics: Num rows: 341 Data size: 38920 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col1)
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(col 7:int) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 6:int
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 38920 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkLongOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 6:int
+ native: true
+ Group By Operator
+ aggregations: sum(_col1)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 7:int) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 6:int
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 38920 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkLongOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 341 Data size: 38920 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs (cache only)
Map Vectorization:
@@ -503,31 +512,40 @@ STAGE PLANS:
projectedOutputColumnNums: [6, 7]
selectExpressions: ListIndexColScalar(col 2:array<int>, col 1:int) -> 6:int, ListIndexColScalar(col 2:array<int>, col 0:int) -> 7:int
Statistics: Num rows: 341 Data size: 38921 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col1)
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(col 7:int) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 6:int
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 38921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkLongOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 6:int
+ native: true
+ Group By Operator
+ aggregations: sum(_col1)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 7:int) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 6:int
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 38921 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkLongOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 341 Data size: 38921 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs (cache only)
Map Vectorization:
@@ -792,31 +810,40 @@ STAGE PLANS:
projectedOutputColumnNums: [6, 7]
selectExpressions: ListIndexColScalar(col 2:array<int>, col 1:int) -> 6:int, ListIndexColScalar(col 2:array<int>, col 0:int) -> 7:int
Statistics: Num rows: 341 Data size: 38923 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col1)
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(col 7:int) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 6:int
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 38923 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkLongOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 6:int
+ native: true
+ Group By Operator
+ aggregations: sum(_col1)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 7:int) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 6:int
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 38923 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkLongOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 341 Data size: 38923 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs (cache only)
Map Vectorization:
[05/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query69.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query69.q.out b/ql/src/test/results/clientpositive/perf/tez/query69.q.out
index aad5b81..738508a 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query69.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query69.q.out
@@ -117,197 +117,199 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_232]
- Limit [LIM_231] (rows=100 width=88)
+ File Output Operator [FS_233]
+ Limit [LIM_232] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_230] (rows=191662559 width=88)
+ Select Operator [SEL_231] (rows=191662559 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_229]
- Select Operator [SEL_228] (rows=191662559 width=88)
+ SHUFFLE [RS_230]
+ Select Operator [SEL_229] (rows=191662559 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col6"]
- Group By Operator [GBY_227] (rows=191662559 width=88)
+ Group By Operator [GBY_228] (rows=191662559 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_67]
PartitionCols:_col0, _col1, _col2, _col3, _col4
Group By Operator [GBY_66] (rows=383325119 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count()"],keys:_col6, _col7, _col8, _col9, _col10
- Select Operator [SEL_65] (rows=383325119 width=88)
- Output:["_col6","_col7","_col8","_col9","_col10"]
- Filter Operator [FIL_64] (rows=383325119 width=88)
- predicate:_col14 is null
- Merge Join Operator [MERGEJOIN_180] (rows=766650239 width=88)
- Conds:RS_61._col0=RS_226._col0(Left Outer),Output:["_col6","_col7","_col8","_col9","_col10","_col14"]
- <-Reducer 19 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_226]
- PartitionCols:_col0
- Select Operator [SEL_225] (rows=158394413 width=135)
- Output:["_col0","_col1"]
- Group By Operator [GBY_224] (rows=158394413 width=135)
- Output:["_col0"],keys:KEY._col0
- <-Reducer 18 [SIMPLE_EDGE]
- SHUFFLE [RS_58]
- PartitionCols:_col0
- Group By Operator [GBY_57] (rows=316788826 width=135)
- Output:["_col0"],keys:_col1
- Merge Join Operator [MERGEJOIN_178] (rows=316788826 width=135)
- Conds:RS_223._col0=RS_196._col0(Inner),Output:["_col1"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_196]
- PartitionCols:_col0
- Select Operator [SEL_191] (rows=4058 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_190] (rows=4058 width=1119)
- predicate:((d_year = 1999) and d_date_sk is not null and d_moy BETWEEN 1 AND 3)
- TableScan [TS_12] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
- <-Map 22 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_223]
- PartitionCols:_col0
- Select Operator [SEL_222] (rows=287989836 width=135)
- Output:["_col0","_col1"]
- Filter Operator [FIL_221] (rows=287989836 width=135)
- predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_54_date_dim_d_date_sk_min) AND DynamicValue(RS_54_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_54_date_dim_d_date_sk_bloom_filter))) and cs_ship_customer_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_47] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_ship_customer_sk"]
- <-Reducer 20 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_220]
- Group By Operator [GBY_219] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_203]
- Group By Operator [GBY_200] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_197] (rows=4058 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_191]
- <-Reducer 4 [ONE_TO_ONE_EDGE]
- FORWARD [RS_61]
- PartitionCols:_col0
- Select Operator [SEL_46] (rows=696954748 width=88)
- Output:["_col0","_col6","_col7","_col8","_col9","_col10"]
- Filter Operator [FIL_45] (rows=696954748 width=88)
- predicate:_col12 is null
- Merge Join Operator [MERGEJOIN_179] (rows=1393909496 width=88)
- Conds:RS_41._col0=RS_42._col0(Left Semi),RS_41._col0=RS_218._col0(Left Outer),Output:["_col0","_col6","_col7","_col8","_col9","_col10","_col12"]
- <-Reducer 3 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_41]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_175] (rows=96800003 width=860)
- Conds:RS_36._col1=RS_189._col0(Inner),Output:["_col0","_col6","_col7","_col8","_col9","_col10"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_189]
- PartitionCols:_col0
- Select Operator [SEL_188] (rows=1861800 width=385)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_187] (rows=1861800 width=385)
- predicate:cd_demo_sk is not null
- TableScan [TS_6] (rows=1861800 width=385)
- default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status","cd_purchase_estimate","cd_credit_rating"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_36]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_174] (rows=88000001 width=860)
- Conds:RS_183._col2=RS_186._col0(Inner),Output:["_col0","_col1"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_183]
- PartitionCols:_col2
- Select Operator [SEL_182] (rows=80000000 width=860)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_181] (rows=80000000 width=860)
- predicate:(c_current_addr_sk is not null and c_current_cdemo_sk is not null and c_customer_sk is not null)
- TableScan [TS_0] (rows=80000000 width=860)
- default@customer,c,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_cdemo_sk","c_current_addr_sk"]
- <-Map 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_186]
- PartitionCols:_col0
- Select Operator [SEL_185] (rows=20000000 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_184] (rows=20000000 width=1014)
- predicate:((ca_state) IN ('CO', 'IL', 'MN') and ca_address_sk is not null)
- TableScan [TS_3] (rows=40000000 width=1014)
- default@customer_address,ca,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
- <-Reducer 12 [SIMPLE_EDGE]
- SHUFFLE [RS_42]
- PartitionCols:_col0
- Group By Operator [GBY_40] (rows=633595212 width=88)
- Output:["_col0"],keys:_col0
- Select Operator [SEL_18] (rows=633595212 width=88)
- Output:["_col0"]
- Merge Join Operator [MERGEJOIN_176] (rows=633595212 width=88)
- Conds:RS_210._col0=RS_192._col0(Inner),Output:["_col1"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_192]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_191]
- <-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_210]
- PartitionCols:_col0
- Select Operator [SEL_209] (rows=575995635 width=88)
- Output:["_col0","_col1"]
- Filter Operator [FIL_208] (rows=575995635 width=88)
- predicate:((ss_customer_sk BETWEEN DynamicValue(RS_41_c_c_customer_sk_min) AND DynamicValue(RS_41_c_c_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_41_c_c_customer_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_16_date_dim_d_date_sk_min) AND DynamicValue(RS_16_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_16_date_dim_d_date_sk_bloom_filter))) and ss_customer_sk is not null and ss_sold_date_sk is not null)
- TableScan [TS_9] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_customer_sk"]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_205]
- Group By Operator [GBY_204] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_201]
- Group By Operator [GBY_198] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_193] (rows=4058 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_191]
- <-Reducer 8 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_207]
- Group By Operator [GBY_206] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=96800000)"]
- <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_137]
- Group By Operator [GBY_136] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=96800000)"]
- Select Operator [SEL_135] (rows=96800003 width=860)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_175]
- <-Reducer 16 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_218]
+ Top N Key Operator [TNK_105] (rows=383325119 width=88)
+ keys:_col6, _col7, _col8, _col9, _col10,sort order:+++++,top n:100
+ Select Operator [SEL_65] (rows=383325119 width=88)
+ Output:["_col6","_col7","_col8","_col9","_col10"]
+ Filter Operator [FIL_64] (rows=383325119 width=88)
+ predicate:_col14 is null
+ Merge Join Operator [MERGEJOIN_181] (rows=766650239 width=88)
+ Conds:RS_61._col0=RS_227._col0(Left Outer),Output:["_col6","_col7","_col8","_col9","_col10","_col14"]
+ <-Reducer 19 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_227]
+ PartitionCols:_col0
+ Select Operator [SEL_226] (rows=158394413 width=135)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_225] (rows=158394413 width=135)
+ Output:["_col0"],keys:KEY._col0
+ <-Reducer 18 [SIMPLE_EDGE]
+ SHUFFLE [RS_58]
PartitionCols:_col0
- Select Operator [SEL_217] (rows=79201469 width=135)
- Output:["_col0","_col1"]
- Group By Operator [GBY_216] (rows=79201469 width=135)
- Output:["_col0"],keys:KEY._col0
- <-Reducer 15 [SIMPLE_EDGE]
- SHUFFLE [RS_30]
+ Group By Operator [GBY_57] (rows=316788826 width=135)
+ Output:["_col0"],keys:_col1
+ Merge Join Operator [MERGEJOIN_179] (rows=316788826 width=135)
+ Conds:RS_224._col0=RS_197._col0(Inner),Output:["_col1"]
+ <-Map 13 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_197]
+ PartitionCols:_col0
+ Select Operator [SEL_192] (rows=4058 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_191] (rows=4058 width=1119)
+ predicate:((d_year = 1999) and d_date_sk is not null and d_moy BETWEEN 1 AND 3)
+ TableScan [TS_12] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Map 22 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_224]
+ PartitionCols:_col0
+ Select Operator [SEL_223] (rows=287989836 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_222] (rows=287989836 width=135)
+ predicate:((cs_sold_date_sk BETWEEN DynamicValue(RS_54_date_dim_d_date_sk_min) AND DynamicValue(RS_54_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_54_date_dim_d_date_sk_bloom_filter))) and cs_ship_customer_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_47] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_ship_customer_sk"]
+ <-Reducer 20 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_221]
+ Group By Operator [GBY_220] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_204]
+ Group By Operator [GBY_201] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_198] (rows=4058 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_192]
+ <-Reducer 4 [ONE_TO_ONE_EDGE]
+ FORWARD [RS_61]
+ PartitionCols:_col0
+ Select Operator [SEL_46] (rows=696954748 width=88)
+ Output:["_col0","_col6","_col7","_col8","_col9","_col10"]
+ Filter Operator [FIL_45] (rows=696954748 width=88)
+ predicate:_col12 is null
+ Merge Join Operator [MERGEJOIN_180] (rows=1393909496 width=88)
+ Conds:RS_41._col0=RS_42._col0(Left Semi),RS_41._col0=RS_219._col0(Left Outer),Output:["_col0","_col6","_col7","_col8","_col9","_col10","_col12"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_41]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_176] (rows=96800003 width=860)
+ Conds:RS_36._col1=RS_190._col0(Inner),Output:["_col0","_col6","_col7","_col8","_col9","_col10"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_190]
PartitionCols:_col0
- Group By Operator [GBY_29] (rows=158402938 width=135)
- Output:["_col0"],keys:_col1
- Merge Join Operator [MERGEJOIN_177] (rows=158402938 width=135)
- Conds:RS_215._col0=RS_194._col0(Inner),Output:["_col1"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_194]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_191]
- <-Map 21 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_215]
- PartitionCols:_col0
- Select Operator [SEL_214] (rows=144002668 width=135)
- Output:["_col0","_col1"]
- Filter Operator [FIL_213] (rows=144002668 width=135)
- predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_26_date_dim_d_date_sk_min) AND DynamicValue(RS_26_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_26_date_dim_d_date_sk_bloom_filter))) and ws_bill_customer_sk is not null and ws_sold_date_sk is not null)
- TableScan [TS_19] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_bill_customer_sk"]
- <-Reducer 17 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_212]
- Group By Operator [GBY_211] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_202]
- Group By Operator [GBY_199] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_195] (rows=4058 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_191]
+ Select Operator [SEL_189] (rows=1861800 width=385)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_188] (rows=1861800 width=385)
+ predicate:cd_demo_sk is not null
+ TableScan [TS_6] (rows=1861800 width=385)
+ default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status","cd_purchase_estimate","cd_credit_rating"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_36]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_175] (rows=88000001 width=860)
+ Conds:RS_184._col2=RS_187._col0(Inner),Output:["_col0","_col1"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_184]
+ PartitionCols:_col2
+ Select Operator [SEL_183] (rows=80000000 width=860)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_182] (rows=80000000 width=860)
+ predicate:(c_current_addr_sk is not null and c_current_cdemo_sk is not null and c_customer_sk is not null)
+ TableScan [TS_0] (rows=80000000 width=860)
+ default@customer,c,Tbl:COMPLETE,Col:NONE,Output:["c_customer_sk","c_current_cdemo_sk","c_current_addr_sk"]
+ <-Map 9 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_187]
+ PartitionCols:_col0
+ Select Operator [SEL_186] (rows=20000000 width=1014)
+ Output:["_col0"]
+ Filter Operator [FIL_185] (rows=20000000 width=1014)
+ predicate:((ca_state) IN ('CO', 'IL', 'MN') and ca_address_sk is not null)
+ TableScan [TS_3] (rows=40000000 width=1014)
+ default@customer_address,ca,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_state"]
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_42]
+ PartitionCols:_col0
+ Group By Operator [GBY_40] (rows=633595212 width=88)
+ Output:["_col0"],keys:_col0
+ Select Operator [SEL_18] (rows=633595212 width=88)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_177] (rows=633595212 width=88)
+ Conds:RS_211._col0=RS_193._col0(Inner),Output:["_col1"]
+ <-Map 13 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_193]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_192]
+ <-Map 11 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_211]
+ PartitionCols:_col0
+ Select Operator [SEL_210] (rows=575995635 width=88)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_209] (rows=575995635 width=88)
+ predicate:((ss_customer_sk BETWEEN DynamicValue(RS_41_c_c_customer_sk_min) AND DynamicValue(RS_41_c_c_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_41_c_c_customer_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_16_date_dim_d_date_sk_min) AND DynamicValue(RS_16_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_16_date_dim_d_date_sk_bloom_filter))) and ss_customer_sk is not null and ss_sold_date_sk is not null)
+ TableScan [TS_9] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_customer_sk"]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_206]
+ Group By Operator [GBY_205] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_202]
+ Group By Operator [GBY_199] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_194] (rows=4058 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_192]
+ <-Reducer 8 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_208]
+ Group By Operator [GBY_207] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=96800000)"]
+ <-Reducer 3 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_138]
+ Group By Operator [GBY_137] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=96800000)"]
+ Select Operator [SEL_136] (rows=96800003 width=860)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_176]
+ <-Reducer 16 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_219]
+ PartitionCols:_col0
+ Select Operator [SEL_218] (rows=79201469 width=135)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_217] (rows=79201469 width=135)
+ Output:["_col0"],keys:KEY._col0
+ <-Reducer 15 [SIMPLE_EDGE]
+ SHUFFLE [RS_30]
+ PartitionCols:_col0
+ Group By Operator [GBY_29] (rows=158402938 width=135)
+ Output:["_col0"],keys:_col1
+ Merge Join Operator [MERGEJOIN_178] (rows=158402938 width=135)
+ Conds:RS_216._col0=RS_195._col0(Inner),Output:["_col1"]
+ <-Map 13 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_195]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_192]
+ <-Map 21 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_216]
+ PartitionCols:_col0
+ Select Operator [SEL_215] (rows=144002668 width=135)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_214] (rows=144002668 width=135)
+ predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_26_date_dim_d_date_sk_min) AND DynamicValue(RS_26_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_26_date_dim_d_date_sk_bloom_filter))) and ws_bill_customer_sk is not null and ws_sold_date_sk is not null)
+ TableScan [TS_19] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_bill_customer_sk"]
+ <-Reducer 17 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_213]
+ Group By Operator [GBY_212] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_203]
+ Group By Operator [GBY_200] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_196] (rows=4058 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_192]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query7.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query7.q.out b/ql/src/test/results/clientpositive/perf/tez/query7.q.out
index 2bb39dd..c78e1e6 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query7.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query7.q.out
@@ -58,126 +58,128 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_139]
- Limit [LIM_138] (rows=100 width=88)
+ File Output Operator [FS_140]
+ Limit [LIM_139] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_137] (rows=421657640 width=88)
+ Select Operator [SEL_138] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_136]
- Select Operator [SEL_135] (rows=421657640 width=88)
+ SHUFFLE [RS_137]
+ Select Operator [SEL_136] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_134] (rows=421657640 width=88)
+ Group By Operator [GBY_135] (rows=421657640 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)","sum(VALUE._col2)","count(VALUE._col3)","sum(VALUE._col4)","count(VALUE._col5)","sum(VALUE._col6)","count(VALUE._col7)"],keys:KEY._col0
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_29]
PartitionCols:_col0
Group By Operator [GBY_28] (rows=843315281 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"],aggregations:["sum(_col4)","count(_col4)","sum(_col5)","count(_col5)","sum(_col7)","count(_col7)","sum(_col6)","count(_col6)"],keys:_col18
- Merge Join Operator [MERGEJOIN_98] (rows=843315281 width=88)
- Conds:RS_24._col1=RS_125._col0(Inner),Output:["_col4","_col5","_col6","_col7","_col18"]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_125]
- PartitionCols:_col0
- Select Operator [SEL_124] (rows=462000 width=1436)
- Output:["_col0","_col1"]
- Filter Operator [FIL_123] (rows=462000 width=1436)
- predicate:i_item_sk is not null
- TableScan [TS_12] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_24]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_97] (rows=766650239 width=88)
- Conds:RS_21._col3=RS_117._col0(Inner),Output:["_col1","_col4","_col5","_col6","_col7"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_117]
- PartitionCols:_col0
- Select Operator [SEL_116] (rows=2300 width=1179)
- Output:["_col0"]
- Filter Operator [FIL_115] (rows=2300 width=1179)
- predicate:(((p_channel_email = 'N') or (p_channel_event = 'N')) and p_promo_sk is not null)
- TableScan [TS_9] (rows=2300 width=1179)
- default@promotion,promotion,Tbl:COMPLETE,Col:NONE,Output:["p_promo_sk","p_channel_email","p_channel_event"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col3
- Merge Join Operator [MERGEJOIN_96] (rows=696954748 width=88)
- Conds:RS_18._col0=RS_109._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_109]
- PartitionCols:_col0
- Select Operator [SEL_108] (rows=36524 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_107] (rows=36524 width=1119)
- predicate:((d_year = 1998) and d_date_sk is not null)
- TableScan [TS_6] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_18]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_95] (rows=633595212 width=88)
- Conds:RS_133._col2=RS_101._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7"]
- <-Map 8 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_101]
- PartitionCols:_col0
- Select Operator [SEL_100] (rows=232725 width=385)
- Output:["_col0"]
- Filter Operator [FIL_99] (rows=232725 width=385)
- predicate:((cd_education_status = 'Primary') and (cd_gender = 'F') and (cd_marital_status = 'W') and cd_demo_sk is not null)
- TableScan [TS_3] (rows=1861800 width=385)
- default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_133]
- PartitionCols:_col2
- Select Operator [SEL_132] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- Filter Operator [FIL_131] (rows=575995635 width=88)
- predicate:((ss_cdemo_sk BETWEEN DynamicValue(RS_16_customer_demographics_cd_demo_sk_min) AND DynamicValue(RS_16_customer_demographics_cd_demo_sk_max) and in_bloom_filter(ss_cdemo_sk, DynamicValue(RS_16_customer_demographics_cd_demo_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_25_item_i_item_sk_min) AND DynamicValue(RS_25_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_25_item_i_item_sk_bloom_filter))) and (ss_promo_sk BETWEEN DynamicValue(RS_22_promotion_p_promo_sk_min) AND DynamicValue(RS_22_promotion_p_promo_sk_max) and in_bloom_filter(ss_promo_sk, DynamicValue(RS_22_promotion_p_promo_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and ss_cdemo_sk is not null and ss_item_sk is not null and ss_promo_sk is not null and ss_sold_dat
e_sk is not null)
- TableScan [TS_0] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_cdemo_sk","ss_promo_sk","ss_quantity","ss_list_price","ss_sales_price","ss_coupon_amt"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_114]
- Group By Operator [GBY_113] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_112]
- Group By Operator [GBY_111] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_110] (rows=36524 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_108]
- <-Reducer 13 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_122]
- Group By Operator [GBY_121] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_120]
- Group By Operator [GBY_119] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_118] (rows=2300 width=1179)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_116]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_130]
- Group By Operator [GBY_129] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 14 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_128]
- Group By Operator [GBY_127] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_126] (rows=462000 width=1436)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_124]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_106]
- Group By Operator [GBY_105] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_104]
- Group By Operator [GBY_103] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_102] (rows=232725 width=385)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_100]
+ Top N Key Operator [TNK_55] (rows=843315281 width=88)
+ keys:_col18,sort order:+,top n:100
+ Merge Join Operator [MERGEJOIN_99] (rows=843315281 width=88)
+ Conds:RS_24._col1=RS_126._col0(Inner),Output:["_col4","_col5","_col6","_col7","_col18"]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_126]
+ PartitionCols:_col0
+ Select Operator [SEL_125] (rows=462000 width=1436)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_124] (rows=462000 width=1436)
+ predicate:i_item_sk is not null
+ TableScan [TS_12] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_24]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_98] (rows=766650239 width=88)
+ Conds:RS_21._col3=RS_118._col0(Inner),Output:["_col1","_col4","_col5","_col6","_col7"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_118]
+ PartitionCols:_col0
+ Select Operator [SEL_117] (rows=2300 width=1179)
+ Output:["_col0"]
+ Filter Operator [FIL_116] (rows=2300 width=1179)
+ predicate:(((p_channel_email = 'N') or (p_channel_event = 'N')) and p_promo_sk is not null)
+ TableScan [TS_9] (rows=2300 width=1179)
+ default@promotion,promotion,Tbl:COMPLETE,Col:NONE,Output:["p_promo_sk","p_channel_email","p_channel_event"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_97] (rows=696954748 width=88)
+ Conds:RS_18._col0=RS_110._col0(Inner),Output:["_col1","_col3","_col4","_col5","_col6","_col7"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_110]
+ PartitionCols:_col0
+ Select Operator [SEL_109] (rows=36524 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_108] (rows=36524 width=1119)
+ predicate:((d_year = 1998) and d_date_sk is not null)
+ TableScan [TS_6] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_18]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_96] (rows=633595212 width=88)
+ Conds:RS_134._col2=RS_102._col0(Inner),Output:["_col0","_col1","_col3","_col4","_col5","_col6","_col7"]
+ <-Map 8 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_102]
+ PartitionCols:_col0
+ Select Operator [SEL_101] (rows=232725 width=385)
+ Output:["_col0"]
+ Filter Operator [FIL_100] (rows=232725 width=385)
+ predicate:((cd_education_status = 'Primary') and (cd_gender = 'F') and (cd_marital_status = 'W') and cd_demo_sk is not null)
+ TableScan [TS_3] (rows=1861800 width=385)
+ default@customer_demographics,customer_demographics,Tbl:COMPLETE,Col:NONE,Output:["cd_demo_sk","cd_gender","cd_marital_status","cd_education_status"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_134]
+ PartitionCols:_col2
+ Select Operator [SEL_133] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
+ Filter Operator [FIL_132] (rows=575995635 width=88)
+ predicate:((ss_cdemo_sk BETWEEN DynamicValue(RS_16_customer_demographics_cd_demo_sk_min) AND DynamicValue(RS_16_customer_demographics_cd_demo_sk_max) and in_bloom_filter(ss_cdemo_sk, DynamicValue(RS_16_customer_demographics_cd_demo_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_25_item_i_item_sk_min) AND DynamicValue(RS_25_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_25_item_i_item_sk_bloom_filter))) and (ss_promo_sk BETWEEN DynamicValue(RS_22_promotion_p_promo_sk_min) AND DynamicValue(RS_22_promotion_p_promo_sk_max) and in_bloom_filter(ss_promo_sk, DynamicValue(RS_22_promotion_p_promo_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_19_date_dim_d_date_sk_min) AND DynamicValue(RS_19_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_19_date_dim_d_date_sk_bloom_filter))) and ss_cdemo_sk is not null and ss_item_sk is not null and ss_promo_sk is not null and ss_sold_d
ate_sk is not null)
+ TableScan [TS_0] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_cdemo_sk","ss_promo_sk","ss_quantity","ss_list_price","ss_sales_price","ss_coupon_amt"]
+ <-Reducer 11 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_115]
+ Group By Operator [GBY_114] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_113]
+ Group By Operator [GBY_112] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_111] (rows=36524 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_109]
+ <-Reducer 13 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_123]
+ Group By Operator [GBY_122] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 12 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_121]
+ Group By Operator [GBY_120] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_119] (rows=2300 width=1179)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_117]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_131]
+ Group By Operator [GBY_130] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 14 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_129]
+ Group By Operator [GBY_128] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_127] (rows=462000 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_125]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_107]
+ Group By Operator [GBY_106] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 8 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_105]
+ Group By Operator [GBY_104] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_103] (rows=232725 width=385)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_101]
[07/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query60.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query60.q.out b/ql/src/test/results/clientpositive/perf/tez/query60.q.out
index f10a728..277dd5d 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query60.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query60.q.out
@@ -189,290 +189,296 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_370]
- Limit [LIM_369] (rows=100 width=108)
+ File Output Operator [FS_375]
+ Limit [LIM_374] (rows=100 width=108)
Number of rows:100
- Select Operator [SEL_368] (rows=335408073 width=108)
+ Select Operator [SEL_373] (rows=335408073 width=108)
Output:["_col0","_col1"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_367]
- Group By Operator [GBY_366] (rows=335408073 width=108)
+ SHUFFLE [RS_372]
+ Group By Operator [GBY_371] (rows=335408073 width=108)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
<-Union 5 [SIMPLE_EDGE]
<-Reducer 10 [CONTAINS] vectorized
- Reduce Output Operator [RS_382]
+ Reduce Output Operator [RS_388]
PartitionCols:_col0
- Group By Operator [GBY_381] (rows=670816147 width=108)
+ Group By Operator [GBY_387] (rows=670816147 width=108)
Output:["_col0","_col1"],aggregations:["sum(_col1)"],keys:_col0
- Group By Operator [GBY_380] (rows=191657247 width=135)
- Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
- <-Reducer 9 [SIMPLE_EDGE]
- SHUFFLE [RS_71]
- PartitionCols:_col0
- Group By Operator [GBY_70] (rows=383314495 width=135)
- Output:["_col0","_col1"],aggregations:["sum(_col8)"],keys:_col1
- Merge Join Operator [MERGEJOIN_303] (rows=383314495 width=135)
- Conds:RS_66._col0=RS_67._col4(Inner),Output:["_col1","_col8"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_66]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_293] (rows=508200 width=1436)
- Conds:RS_319._col1=RS_325._col0(Inner),Output:["_col0","_col1"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_319]
- PartitionCols:_col1
- Select Operator [SEL_318] (rows=462000 width=1436)
- Output:["_col0","_col1"]
- Filter Operator [FIL_317] (rows=462000 width=1436)
- predicate:(i_item_id is not null and i_item_sk is not null)
- TableScan [TS_0] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
- <-Reducer 16 [ONE_TO_ONE_EDGE] vectorized
- FORWARD [RS_325]
- PartitionCols:_col0
- Group By Operator [GBY_324] (rows=115500 width=1436)
- Output:["_col0"],keys:KEY._col0
- <-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_323]
- PartitionCols:_col0
- Group By Operator [GBY_322] (rows=231000 width=1436)
- Output:["_col0"],keys:i_item_id
- Select Operator [SEL_321] (rows=231000 width=1436)
- Output:["i_item_id"]
- Filter Operator [FIL_320] (rows=231000 width=1436)
- predicate:((i_category = 'Children') and i_item_id is not null)
- TableScan [TS_3] (rows=462000 width=1436)
- default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_id","i_category"]
- <-Reducer 23 [SIMPLE_EDGE]
- SHUFFLE [RS_67]
- PartitionCols:_col4
- Select Operator [SEL_62] (rows=348467716 width=135)
- Output:["_col4","_col5"]
- Merge Join Operator [MERGEJOIN_298] (rows=348467716 width=135)
- Conds:RS_59._col1=RS_346._col0(Inner),Output:["_col2","_col3"]
- <-Map 28 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_346]
- PartitionCols:_col0
- Select Operator [SEL_343] (rows=20000000 width=1014)
- Output:["_col0"]
- Filter Operator [FIL_342] (rows=20000000 width=1014)
- predicate:((ca_gmt_offset = -6) and ca_address_sk is not null)
- TableScan [TS_16] (rows=40000000 width=1014)
- default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_gmt_offset"]
- <-Reducer 22 [SIMPLE_EDGE]
- SHUFFLE [RS_59]
+ Top N Key Operator [TNK_386] (rows=670816147 width=108)
+ keys:_col0,sort order:+,top n:100
+ Group By Operator [GBY_385] (rows=191657247 width=135)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 9 [SIMPLE_EDGE]
+ SHUFFLE [RS_71]
+ PartitionCols:_col0
+ Group By Operator [GBY_70] (rows=383314495 width=135)
+ Output:["_col0","_col1"],aggregations:["sum(_col8)"],keys:_col1
+ Merge Join Operator [MERGEJOIN_304] (rows=383314495 width=135)
+ Conds:RS_66._col0=RS_67._col4(Inner),Output:["_col1","_col8"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_66]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_294] (rows=508200 width=1436)
+ Conds:RS_323._col1=RS_329._col0(Inner),Output:["_col0","_col1"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_323]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_297] (rows=316788826 width=135)
- Conds:RS_379._col0=RS_330._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 20 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_330]
- PartitionCols:_col0
- Select Operator [SEL_327] (rows=18262 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_326] (rows=18262 width=1119)
- predicate:((d_moy = 9) and (d_year = 1999) and d_date_sk is not null)
- TableScan [TS_13] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
- <-Map 32 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_379]
+ Select Operator [SEL_322] (rows=462000 width=1436)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_321] (rows=462000 width=1436)
+ predicate:(i_item_id is not null and i_item_sk is not null)
+ TableScan [TS_0] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_sk","i_item_id"]
+ <-Reducer 16 [ONE_TO_ONE_EDGE] vectorized
+ FORWARD [RS_329]
+ PartitionCols:_col0
+ Group By Operator [GBY_328] (rows=115500 width=1436)
+ Output:["_col0"],keys:KEY._col0
+ <-Map 15 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_327]
PartitionCols:_col0
- Select Operator [SEL_378] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_377] (rows=287989836 width=135)
- predicate:((cs_bill_addr_sk BETWEEN DynamicValue(RS_60_customer_address_ca_address_sk_min) AND DynamicValue(RS_60_customer_address_ca_address_sk_max) and in_bloom_filter(cs_bill_addr_sk, DynamicValue(RS_60_customer_address_ca_address_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_66_item_i_item_sk_min) AND DynamicValue(RS_66_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_66_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_57_date_dim_d_date_sk_min) AND DynamicValue(RS_57_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_57_date_dim_d_date_sk_bloom_filter))) and cs_bill_addr_sk is not null and cs_item_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_47] (rows=287989836 width=135)
- default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_addr_sk","cs_item_sk","cs_ext_sales_price"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_376]
- Group By Operator [GBY_375] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- SHUFFLE [RS_240]
- Group By Operator [GBY_239] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_238] (rows=508200 width=1436)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_293]
- <-Reducer 24 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_372]
- Group By Operator [GBY_371] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_338]
- Group By Operator [GBY_335] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_331] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_327]
- <-Reducer 30 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_374]
- Group By Operator [GBY_373] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=20000000)"]
- <-Map 28 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_354]
- Group By Operator [GBY_351] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=20000000)"]
- Select Operator [SEL_347] (rows=20000000 width=1014)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_343]
+ Group By Operator [GBY_326] (rows=231000 width=1436)
+ Output:["_col0"],keys:i_item_id
+ Select Operator [SEL_325] (rows=231000 width=1436)
+ Output:["i_item_id"]
+ Filter Operator [FIL_324] (rows=231000 width=1436)
+ predicate:((i_category = 'Children') and i_item_id is not null)
+ TableScan [TS_3] (rows=462000 width=1436)
+ default@item,item,Tbl:COMPLETE,Col:NONE,Output:["i_item_id","i_category"]
+ <-Reducer 23 [SIMPLE_EDGE]
+ SHUFFLE [RS_67]
+ PartitionCols:_col4
+ Select Operator [SEL_62] (rows=348467716 width=135)
+ Output:["_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_299] (rows=348467716 width=135)
+ Conds:RS_59._col1=RS_350._col0(Inner),Output:["_col2","_col3"]
+ <-Map 28 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_350]
+ PartitionCols:_col0
+ Select Operator [SEL_347] (rows=20000000 width=1014)
+ Output:["_col0"]
+ Filter Operator [FIL_346] (rows=20000000 width=1014)
+ predicate:((ca_gmt_offset = -6) and ca_address_sk is not null)
+ TableScan [TS_16] (rows=40000000 width=1014)
+ default@customer_address,customer_address,Tbl:COMPLETE,Col:NONE,Output:["ca_address_sk","ca_gmt_offset"]
+ <-Reducer 22 [SIMPLE_EDGE]
+ SHUFFLE [RS_59]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_298] (rows=316788826 width=135)
+ Conds:RS_384._col0=RS_334._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_334]
+ PartitionCols:_col0
+ Select Operator [SEL_331] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_330] (rows=18262 width=1119)
+ predicate:((d_moy = 9) and (d_year = 1999) and d_date_sk is not null)
+ TableScan [TS_13] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Map 32 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_384]
+ PartitionCols:_col0
+ Select Operator [SEL_383] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_382] (rows=287989836 width=135)
+ predicate:((cs_bill_addr_sk BETWEEN DynamicValue(RS_60_customer_address_ca_address_sk_min) AND DynamicValue(RS_60_customer_address_ca_address_sk_max) and in_bloom_filter(cs_bill_addr_sk, DynamicValue(RS_60_customer_address_ca_address_sk_bloom_filter))) and (cs_item_sk BETWEEN DynamicValue(RS_66_item_i_item_sk_min) AND DynamicValue(RS_66_item_i_item_sk_max) and in_bloom_filter(cs_item_sk, DynamicValue(RS_66_item_i_item_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_57_date_dim_d_date_sk_min) AND DynamicValue(RS_57_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_57_date_dim_d_date_sk_bloom_filter))) and cs_bill_addr_sk is not null and cs_item_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_47] (rows=287989836 width=135)
+ default@catalog_sales,catalog_sales,Tbl:COMPLETE,Col:NONE,Output:["cs_sold_date_sk","cs_bill_addr_sk","cs_item_sk","cs_ext_sales_price"]
+ <-Reducer 11 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_381]
+ Group By Operator [GBY_380] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
+ SHUFFLE [RS_241]
+ Group By Operator [GBY_240] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_239] (rows=508200 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_294]
+ <-Reducer 24 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_377]
+ Group By Operator [GBY_376] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_342]
+ Group By Operator [GBY_339] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_335] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_331]
+ <-Reducer 30 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_379]
+ Group By Operator [GBY_378] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=20000000)"]
+ <-Map 28 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_358]
+ Group By Operator [GBY_355] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=20000000)"]
+ Select Operator [SEL_351] (rows=20000000 width=1014)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_347]
<-Reducer 13 [CONTAINS] vectorized
- Reduce Output Operator [RS_394]
+ Reduce Output Operator [RS_401]
PartitionCols:_col0
- Group By Operator [GBY_393] (rows=670816147 width=108)
+ Group By Operator [GBY_400] (rows=670816147 width=108)
Output:["_col0","_col1"],aggregations:["sum(_col1)"],keys:_col0
- Group By Operator [GBY_392] (rows=95833781 width=135)
- Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
- <-Reducer 12 [SIMPLE_EDGE]
- SHUFFLE [RS_109]
- PartitionCols:_col0
- Group By Operator [GBY_108] (rows=191667562 width=135)
- Output:["_col0","_col1"],aggregations:["sum(_col8)"],keys:_col1
- Merge Join Operator [MERGEJOIN_304] (rows=191667562 width=135)
- Conds:RS_104._col0=RS_105._col3(Inner),Output:["_col1","_col8"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_104]
- PartitionCols:_col0
- Please refer to the previous Merge Join Operator [MERGEJOIN_293]
- <-Reducer 26 [SIMPLE_EDGE]
- SHUFFLE [RS_105]
- PartitionCols:_col3
- Select Operator [SEL_100] (rows=174243235 width=135)
- Output:["_col3","_col5"]
- Merge Join Operator [MERGEJOIN_301] (rows=174243235 width=135)
- Conds:RS_97._col2=RS_348._col0(Inner),Output:["_col1","_col3"]
- <-Map 28 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_348]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_343]
- <-Reducer 25 [SIMPLE_EDGE]
- SHUFFLE [RS_97]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_300] (rows=158402938 width=135)
- Conds:RS_391._col0=RS_332._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 20 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_332]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_327]
- <-Map 33 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_391]
- PartitionCols:_col0
- Select Operator [SEL_390] (rows=144002668 width=135)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_389] (rows=144002668 width=135)
- predicate:((ws_bill_addr_sk BETWEEN DynamicValue(RS_98_customer_address_ca_address_sk_min) AND DynamicValue(RS_98_customer_address_ca_address_sk_max) and in_bloom_filter(ws_bill_addr_sk, DynamicValue(RS_98_customer_address_ca_address_sk_bloom_filter))) and (ws_item_sk BETWEEN DynamicValue(RS_104_item_i_item_sk_min) AND DynamicValue(RS_104_item_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_104_item_i_item_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_95_date_dim_d_date_sk_min) AND DynamicValue(RS_95_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_95_date_dim_d_date_sk_bloom_filter))) and ws_bill_addr_sk is not null and ws_item_sk is not null and ws_sold_date_sk is not null)
- TableScan [TS_85] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_bill_addr_sk","ws_ext_sales_price"]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_388]
- Group By Operator [GBY_387] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- SHUFFLE [RS_280]
- Group By Operator [GBY_279] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_278] (rows=508200 width=1436)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_293]
- <-Reducer 27 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_384]
- Group By Operator [GBY_383] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_339]
- Group By Operator [GBY_336] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_333] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_327]
- <-Reducer 31 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_386]
- Group By Operator [GBY_385] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=20000000)"]
- <-Map 28 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_355]
- Group By Operator [GBY_352] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=20000000)"]
- Select Operator [SEL_349] (rows=20000000 width=1014)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_343]
+ Top N Key Operator [TNK_399] (rows=670816147 width=108)
+ keys:_col0,sort order:+,top n:100
+ Group By Operator [GBY_398] (rows=95833781 width=135)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_109]
+ PartitionCols:_col0
+ Group By Operator [GBY_108] (rows=191667562 width=135)
+ Output:["_col0","_col1"],aggregations:["sum(_col8)"],keys:_col1
+ Merge Join Operator [MERGEJOIN_305] (rows=191667562 width=135)
+ Conds:RS_104._col0=RS_105._col3(Inner),Output:["_col1","_col8"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_104]
+ PartitionCols:_col0
+ Please refer to the previous Merge Join Operator [MERGEJOIN_294]
+ <-Reducer 26 [SIMPLE_EDGE]
+ SHUFFLE [RS_105]
+ PartitionCols:_col3
+ Select Operator [SEL_100] (rows=174243235 width=135)
+ Output:["_col3","_col5"]
+ Merge Join Operator [MERGEJOIN_302] (rows=174243235 width=135)
+ Conds:RS_97._col2=RS_352._col0(Inner),Output:["_col1","_col3"]
+ <-Map 28 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_352]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_347]
+ <-Reducer 25 [SIMPLE_EDGE]
+ SHUFFLE [RS_97]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_301] (rows=158402938 width=135)
+ Conds:RS_397._col0=RS_336._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_336]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_331]
+ <-Map 33 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_397]
+ PartitionCols:_col0
+ Select Operator [SEL_396] (rows=144002668 width=135)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_395] (rows=144002668 width=135)
+ predicate:((ws_bill_addr_sk BETWEEN DynamicValue(RS_98_customer_address_ca_address_sk_min) AND DynamicValue(RS_98_customer_address_ca_address_sk_max) and in_bloom_filter(ws_bill_addr_sk, DynamicValue(RS_98_customer_address_ca_address_sk_bloom_filter))) and (ws_item_sk BETWEEN DynamicValue(RS_104_item_i_item_sk_min) AND DynamicValue(RS_104_item_i_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_104_item_i_item_sk_bloom_filter))) and (ws_sold_date_sk BETWEEN DynamicValue(RS_95_date_dim_d_date_sk_min) AND DynamicValue(RS_95_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_95_date_dim_d_date_sk_bloom_filter))) and ws_bill_addr_sk is not null and ws_item_sk is not null and ws_sold_date_sk is not null)
+ TableScan [TS_85] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_sold_date_sk","ws_item_sk","ws_bill_addr_sk","ws_ext_sales_price"]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_394]
+ Group By Operator [GBY_393] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
+ SHUFFLE [RS_281]
+ Group By Operator [GBY_280] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_279] (rows=508200 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_294]
+ <-Reducer 27 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_390]
+ Group By Operator [GBY_389] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_343]
+ Group By Operator [GBY_340] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_337] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_331]
+ <-Reducer 31 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_392]
+ Group By Operator [GBY_391] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=20000000)"]
+ <-Map 28 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_359]
+ Group By Operator [GBY_356] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=20000000)"]
+ Select Operator [SEL_353] (rows=20000000 width=1014)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_347]
<-Reducer 4 [CONTAINS] vectorized
- Reduce Output Operator [RS_365]
+ Reduce Output Operator [RS_370]
PartitionCols:_col0
- Group By Operator [GBY_364] (rows=670816147 width=108)
+ Group By Operator [GBY_369] (rows=670816147 width=108)
Output:["_col0","_col1"],aggregations:["sum(_col1)"],keys:_col0
- Group By Operator [GBY_363] (rows=383325119 width=88)
- Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_34]
- PartitionCols:_col0
- Group By Operator [GBY_33] (rows=766650239 width=88)
- Output:["_col0","_col1"],aggregations:["sum(_col8)"],keys:_col1
- Merge Join Operator [MERGEJOIN_302] (rows=766650239 width=88)
- Conds:RS_29._col0=RS_30._col3(Inner),Output:["_col1","_col8"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_29]
- PartitionCols:_col0
- Please refer to the previous Merge Join Operator [MERGEJOIN_293]
- <-Reducer 19 [SIMPLE_EDGE]
- SHUFFLE [RS_30]
- PartitionCols:_col3
- Select Operator [SEL_25] (rows=696954748 width=88)
- Output:["_col3","_col5"]
- Merge Join Operator [MERGEJOIN_295] (rows=696954748 width=88)
- Conds:RS_22._col2=RS_344._col0(Inner),Output:["_col1","_col3"]
- <-Map 28 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_344]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_343]
- <-Reducer 18 [SIMPLE_EDGE]
- SHUFFLE [RS_22]
- PartitionCols:_col2
- Merge Join Operator [MERGEJOIN_294] (rows=633595212 width=88)
- Conds:RS_362._col0=RS_328._col0(Inner),Output:["_col1","_col2","_col3"]
- <-Map 20 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_328]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_327]
- <-Map 17 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_362]
- PartitionCols:_col0
- Select Operator [SEL_361] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_360] (rows=575995635 width=88)
- predicate:((ss_addr_sk BETWEEN DynamicValue(RS_23_customer_address_ca_address_sk_min) AND DynamicValue(RS_23_customer_address_ca_address_sk_max) and in_bloom_filter(ss_addr_sk, DynamicValue(RS_23_customer_address_ca_address_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_29_item_i_item_sk_min) AND DynamicValue(RS_29_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_29_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_20_date_dim_d_date_sk_min) AND DynamicValue(RS_20_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_20_date_dim_d_date_sk_bloom_filter))) and ss_addr_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null)
- TableScan [TS_10] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_addr_sk","ss_ext_sales_price"]
- <-Reducer 21 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_341]
- Group By Operator [GBY_340] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_337]
- Group By Operator [GBY_334] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_329] (rows=18262 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_327]
- <-Reducer 29 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_357]
- Group By Operator [GBY_356] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=20000000)"]
- <-Map 28 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_353]
- Group By Operator [GBY_350] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=20000000)"]
- Select Operator [SEL_345] (rows=20000000 width=1014)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_343]
- <-Reducer 8 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_359]
- Group By Operator [GBY_358] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- SHUFFLE [RS_200]
- Group By Operator [GBY_199] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_198] (rows=508200 width=1436)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_293]
+ Top N Key Operator [TNK_368] (rows=670816147 width=108)
+ keys:_col0,sort order:+,top n:100
+ Group By Operator [GBY_367] (rows=383325119 width=88)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_34]
+ PartitionCols:_col0
+ Group By Operator [GBY_33] (rows=766650239 width=88)
+ Output:["_col0","_col1"],aggregations:["sum(_col8)"],keys:_col1
+ Merge Join Operator [MERGEJOIN_303] (rows=766650239 width=88)
+ Conds:RS_29._col0=RS_30._col3(Inner),Output:["_col1","_col8"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_29]
+ PartitionCols:_col0
+ Please refer to the previous Merge Join Operator [MERGEJOIN_294]
+ <-Reducer 19 [SIMPLE_EDGE]
+ SHUFFLE [RS_30]
+ PartitionCols:_col3
+ Select Operator [SEL_25] (rows=696954748 width=88)
+ Output:["_col3","_col5"]
+ Merge Join Operator [MERGEJOIN_296] (rows=696954748 width=88)
+ Conds:RS_22._col2=RS_348._col0(Inner),Output:["_col1","_col3"]
+ <-Map 28 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_348]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_347]
+ <-Reducer 18 [SIMPLE_EDGE]
+ SHUFFLE [RS_22]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_295] (rows=633595212 width=88)
+ Conds:RS_366._col0=RS_332._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_332]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_331]
+ <-Map 17 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_366]
+ PartitionCols:_col0
+ Select Operator [SEL_365] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_364] (rows=575995635 width=88)
+ predicate:((ss_addr_sk BETWEEN DynamicValue(RS_23_customer_address_ca_address_sk_min) AND DynamicValue(RS_23_customer_address_ca_address_sk_max) and in_bloom_filter(ss_addr_sk, DynamicValue(RS_23_customer_address_ca_address_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_29_item_i_item_sk_min) AND DynamicValue(RS_29_item_i_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_29_item_i_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_20_date_dim_d_date_sk_min) AND DynamicValue(RS_20_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_20_date_dim_d_date_sk_bloom_filter))) and ss_addr_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null)
+ TableScan [TS_10] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_addr_sk","ss_ext_sales_price"]
+ <-Reducer 21 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_345]
+ Group By Operator [GBY_344] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_341]
+ Group By Operator [GBY_338] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_333] (rows=18262 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_331]
+ <-Reducer 29 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_361]
+ Group By Operator [GBY_360] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=20000000)"]
+ <-Map 28 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_357]
+ Group By Operator [GBY_354] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=20000000)"]
+ Select Operator [SEL_349] (rows=20000000 width=1014)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_347]
+ <-Reducer 8 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_363]
+ Group By Operator [GBY_362] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
+ SHUFFLE [RS_201]
+ Group By Operator [GBY_200] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_199] (rows=508200 width=1436)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_294]
[08/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query5.q.out b/ql/src/test/results/clientpositive/perf/tez/query5.q.out
index 6b054f0..e554dd4 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query5.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query5.q.out
@@ -287,287 +287,293 @@ Stage-0
limit:100
Stage-1
Reducer 8 vectorized
- File Output Operator [FS_304]
- Limit [LIM_303] (rows=100 width=110)
+ File Output Operator [FS_309]
+ Limit [LIM_308] (rows=100 width=110)
Number of rows:100
- Select Operator [SEL_302] (rows=1136898901 width=110)
+ Select Operator [SEL_307] (rows=1136898901 width=110)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 7 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_301]
- Select Operator [SEL_300] (rows=1136898901 width=110)
+ SHUFFLE [RS_306]
+ Select Operator [SEL_305] (rows=1136898901 width=110)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_299] (rows=1136898901 width=110)
+ Group By Operator [GBY_304] (rows=1136898901 width=110)
Output:["_col0","_col1","_col3","_col4","_col5"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Union 6 [SIMPLE_EDGE]
<-Reducer 14 [CONTAINS] vectorized
- Reduce Output Operator [RS_317]
+ Reduce Output Operator [RS_323]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_316] (rows=2273797803 width=110)
+ Group By Operator [GBY_322] (rows=2273797803 width=110)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_315] (rows=191657181 width=132)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_314] (rows=191657181 width=132)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
- <-Reducer 13 [SIMPLE_EDGE]
- SHUFFLE [RS_47]
- PartitionCols:_col0
- Group By Operator [GBY_46] (rows=383314363 width=132)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col9
- Merge Join Operator [MERGEJOIN_221] (rows=383314363 width=132)
- Conds:RS_42._col0=RS_310._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col9"]
- <-Map 25 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_310]
- PartitionCols:_col0
- Select Operator [SEL_309] (rows=46000 width=460)
- Output:["_col0","_col1"]
- Filter Operator [FIL_308] (rows=46000 width=460)
- predicate:cp_catalog_page_sk is not null
- TableScan [TS_36] (rows=46000 width=460)
- default@catalog_page,catalog_page,Tbl:COMPLETE,Col:NONE,Output:["cp_catalog_page_sk","cp_catalog_page_id"]
- <-Reducer 12 [SIMPLE_EDGE]
- SHUFFLE [RS_42]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_220] (rows=348467596 width=132)
- Conds:Union 23._col1=RS_272._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_272]
- PartitionCols:_col0
- Select Operator [SEL_269] (rows=8116 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_268] (rows=8116 width=1119)
- predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-08-04 00:00:00' AND TIMESTAMP'1998-08-18 00:00:00' and d_date_sk is not null)
- TableScan [TS_8] (rows=73049 width=1119)
- default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
- <-Union 23 [SIMPLE_EDGE]
- <-Map 22 [CONTAINS] vectorized
- Reduce Output Operator [RS_334]
- PartitionCols:_col1
- Select Operator [SEL_333] (rows=287989836 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_332] (rows=287989836 width=135)
- predicate:((cs_catalog_page_sk BETWEEN DynamicValue(RS_43_catalog_page_cp_catalog_page_sk_min) AND DynamicValue(RS_43_catalog_page_cp_catalog_page_sk_max) and in_bloom_filter(cs_catalog_page_sk, DynamicValue(RS_43_catalog_page_cp_catalog_page_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_40_date_dim_d_date_sk_min) AND DynamicValue(RS_40_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_40_date_dim_d_date_sk_bloom_filter))) and cs_catalog_page_sk is not null and cs_sold_date_sk is not null)
- TableScan [TS_249] (rows=287989836 width=135)
- Output:["cs_sold_date_sk","cs_catalog_page_sk","cs_ext_sales_price","cs_net_profit"]
- <-Reducer 15 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_329]
- Group By Operator [GBY_328] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_280]
- Group By Operator [GBY_277] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_273] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_269]
- <-Reducer 26 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_331]
- Group By Operator [GBY_330] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 25 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_313]
- Group By Operator [GBY_312] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_311] (rows=46000 width=460)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_309]
- <-Map 24 [CONTAINS] vectorized
- Reduce Output Operator [RS_337]
- PartitionCols:_col1
- Select Operator [SEL_336] (rows=28798881 width=106)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_335] (rows=28798881 width=106)
- predicate:(cr_catalog_page_sk is not null and cr_returned_date_sk is not null)
- TableScan [TS_254] (rows=28798881 width=106)
- Output:["cr_returned_date_sk","cr_catalog_page_sk","cr_return_amount","cr_net_loss"]
+ Top N Key Operator [TNK_321] (rows=757932601 width=110)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_320] (rows=191657181 width=132)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Group By Operator [GBY_319] (rows=191657181 width=132)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
+ <-Reducer 13 [SIMPLE_EDGE]
+ SHUFFLE [RS_47]
+ PartitionCols:_col0
+ Group By Operator [GBY_46] (rows=383314363 width=132)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col9
+ Merge Join Operator [MERGEJOIN_222] (rows=383314363 width=132)
+ Conds:RS_42._col0=RS_315._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col9"]
+ <-Map 25 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_315]
+ PartitionCols:_col0
+ Select Operator [SEL_314] (rows=46000 width=460)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_313] (rows=46000 width=460)
+ predicate:cp_catalog_page_sk is not null
+ TableScan [TS_36] (rows=46000 width=460)
+ default@catalog_page,catalog_page,Tbl:COMPLETE,Col:NONE,Output:["cp_catalog_page_sk","cp_catalog_page_id"]
+ <-Reducer 12 [SIMPLE_EDGE]
+ SHUFFLE [RS_42]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_221] (rows=348467596 width=132)
+ Conds:Union 23._col1=RS_276._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_276]
+ PartitionCols:_col0
+ Select Operator [SEL_273] (rows=8116 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_272] (rows=8116 width=1119)
+ predicate:(CAST( d_date AS TIMESTAMP) BETWEEN TIMESTAMP'1998-08-04 00:00:00' AND TIMESTAMP'1998-08-18 00:00:00' and d_date_sk is not null)
+ TableScan [TS_8] (rows=73049 width=1119)
+ default@date_dim,date_dim,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_date"]
+ <-Union 23 [SIMPLE_EDGE]
+ <-Map 22 [CONTAINS] vectorized
+ Reduce Output Operator [RS_341]
+ PartitionCols:_col1
+ Select Operator [SEL_340] (rows=287989836 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_339] (rows=287989836 width=135)
+ predicate:((cs_catalog_page_sk BETWEEN DynamicValue(RS_43_catalog_page_cp_catalog_page_sk_min) AND DynamicValue(RS_43_catalog_page_cp_catalog_page_sk_max) and in_bloom_filter(cs_catalog_page_sk, DynamicValue(RS_43_catalog_page_cp_catalog_page_sk_bloom_filter))) and (cs_sold_date_sk BETWEEN DynamicValue(RS_40_date_dim_d_date_sk_min) AND DynamicValue(RS_40_date_dim_d_date_sk_max) and in_bloom_filter(cs_sold_date_sk, DynamicValue(RS_40_date_dim_d_date_sk_bloom_filter))) and cs_catalog_page_sk is not null and cs_sold_date_sk is not null)
+ TableScan [TS_253] (rows=287989836 width=135)
+ Output:["cs_sold_date_sk","cs_catalog_page_sk","cs_ext_sales_price","cs_net_profit"]
+ <-Reducer 15 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_336]
+ Group By Operator [GBY_335] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_284]
+ Group By Operator [GBY_281] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_277] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_273]
+ <-Reducer 26 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_338]
+ Group By Operator [GBY_337] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 25 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_318]
+ Group By Operator [GBY_317] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_316] (rows=46000 width=460)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_314]
+ <-Map 24 [CONTAINS] vectorized
+ Reduce Output Operator [RS_344]
+ PartitionCols:_col1
+ Select Operator [SEL_343] (rows=28798881 width=106)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_342] (rows=28798881 width=106)
+ predicate:(cr_catalog_page_sk is not null and cr_returned_date_sk is not null)
+ TableScan [TS_258] (rows=28798881 width=106)
+ Output:["cr_returned_date_sk","cr_catalog_page_sk","cr_return_amount","cr_net_loss"]
<-Reducer 18 [CONTAINS] vectorized
- Reduce Output Operator [RS_327]
+ Reduce Output Operator [RS_334]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_326] (rows=2273797803 width=110)
+ Group By Operator [GBY_333] (rows=2273797803 width=110)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_325] (rows=182955399 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_324] (rows=182955399 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
- <-Reducer 17 [SIMPLE_EDGE]
- SHUFFLE [RS_80]
- PartitionCols:_col0
- Group By Operator [GBY_79] (rows=365910798 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col9
- Merge Join Operator [MERGEJOIN_223] (rows=365910798 width=135)
- Conds:RS_75._col0=RS_320._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col9"]
- <-Map 34 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_320]
- PartitionCols:_col0
- Select Operator [SEL_319] (rows=84 width=1850)
- Output:["_col0","_col1"]
- Filter Operator [FIL_318] (rows=84 width=1850)
- predicate:web_site_sk is not null
- TableScan [TS_69] (rows=84 width=1850)
- default@web_site,web_site,Tbl:COMPLETE,Col:NONE,Output:["web_site_sk","web_site_id"]
- <-Reducer 16 [SIMPLE_EDGE]
- SHUFFLE [RS_75]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_222] (rows=332646173 width=135)
- Conds:Union 28._col1=RS_274._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_274]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_269]
- <-Union 28 [SIMPLE_EDGE]
- <-Map 27 [CONTAINS] vectorized
- Reduce Output Operator [RS_345]
- PartitionCols:_col1
- Select Operator [SEL_344] (rows=144002668 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_343] (rows=144002668 width=135)
- predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_73_date_dim_d_date_sk_min) AND DynamicValue(RS_73_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_73_date_dim_d_date_sk_bloom_filter))) and (ws_web_site_sk BETWEEN DynamicValue(RS_76_web_site_web_site_sk_min) AND DynamicValue(RS_76_web_site_web_site_sk_max) and in_bloom_filter(ws_web_site_sk, DynamicValue(RS_76_web_site_web_site_sk_bloom_filter))) and ws_sold_date_sk is not null and ws_web_site_sk is not null)
- TableScan [TS_259] (rows=144002668 width=135)
- Output:["ws_sold_date_sk","ws_web_site_sk","ws_ext_sales_price","ws_net_profit"]
- <-Reducer 35 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_341]
- Group By Operator [GBY_340] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 34 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_323]
- Group By Operator [GBY_322] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_321] (rows=84 width=1850)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_319]
- <-Reducer 19 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_339]
- Group By Operator [GBY_338] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_281]
- Group By Operator [GBY_278] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_275] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_269]
- <-Reducer 30 [CONTAINS]
- Reduce Output Operator [RS_267]
- PartitionCols:_col1
- Select Operator [SEL_265] (rows=158402938 width=135)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Merge Join Operator [MERGEJOIN_264] (rows=158402938 width=135)
- Conds:RS_361._col0, _col2=RS_348._col1, _col2(Inner),Output:["_col1","_col3","_col6","_col7"]
- <-Map 31 [SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_348]
- PartitionCols:_col1, _col2
- Select Operator [SEL_347] (rows=14398467 width=92)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_346] (rows=14398467 width=92)
- predicate:(wr_item_sk is not null and wr_order_number is not null and wr_returned_date_sk is not null)
- TableScan [TS_57] (rows=14398467 width=92)
- default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_item_sk","wr_order_number","wr_return_amt","wr_net_loss"]
- <-Map 29 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_361]
- PartitionCols:_col0, _col2
- Select Operator [SEL_360] (rows=144002668 width=135)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_359] (rows=144002668 width=135)
- predicate:((ws_item_sk BETWEEN DynamicValue(RS_61_web_returns_wr_item_sk_min) AND DynamicValue(RS_61_web_returns_wr_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_61_web_returns_wr_item_sk_bloom_filter))) and (ws_order_number BETWEEN DynamicValue(RS_61_web_returns_wr_order_number_min) AND DynamicValue(RS_61_web_returns_wr_order_number_max) and in_bloom_filter(ws_order_number, DynamicValue(RS_61_web_returns_wr_order_number_bloom_filter))) and (ws_web_site_sk BETWEEN DynamicValue(RS_76_web_site_web_site_sk_min) AND DynamicValue(RS_76_web_site_web_site_sk_max) and in_bloom_filter(ws_web_site_sk, DynamicValue(RS_76_web_site_web_site_sk_bloom_filter))) and ws_item_sk is not null and ws_order_number is not null and ws_web_site_sk is not null)
- TableScan [TS_54] (rows=144002668 width=135)
- default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_item_sk","ws_web_site_sk","ws_order_number"]
- <-Reducer 35 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_342]
- Please refer to the previous Group By Operator [GBY_340]
- <-Reducer 32 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_356]
- Group By Operator [GBY_355] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=14398467)"]
- <-Map 31 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_353]
- Group By Operator [GBY_351] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=14398467)"]
- Select Operator [SEL_349] (rows=14398467 width=92)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_347]
- <-Reducer 33 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_358]
- Group By Operator [GBY_357] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=14398467)"]
- <-Map 31 [CUSTOM_SIMPLE_EDGE] vectorized
- PARTITION_ONLY_SHUFFLE [RS_354]
- Group By Operator [GBY_352] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=14398467)"]
- Select Operator [SEL_350] (rows=14398467 width=92)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_347]
+ Top N Key Operator [TNK_332] (rows=757932601 width=110)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_331] (rows=182955399 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Group By Operator [GBY_330] (rows=182955399 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
+ <-Reducer 17 [SIMPLE_EDGE]
+ SHUFFLE [RS_80]
+ PartitionCols:_col0
+ Group By Operator [GBY_79] (rows=365910798 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col9
+ Merge Join Operator [MERGEJOIN_224] (rows=365910798 width=135)
+ Conds:RS_75._col0=RS_326._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col9"]
+ <-Map 34 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_326]
+ PartitionCols:_col0
+ Select Operator [SEL_325] (rows=84 width=1850)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_324] (rows=84 width=1850)
+ predicate:web_site_sk is not null
+ TableScan [TS_69] (rows=84 width=1850)
+ default@web_site,web_site,Tbl:COMPLETE,Col:NONE,Output:["web_site_sk","web_site_id"]
+ <-Reducer 16 [SIMPLE_EDGE]
+ SHUFFLE [RS_75]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_223] (rows=332646173 width=135)
+ Conds:Union 28._col1=RS_278._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_278]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_273]
+ <-Union 28 [SIMPLE_EDGE]
+ <-Map 27 [CONTAINS] vectorized
+ Reduce Output Operator [RS_352]
+ PartitionCols:_col1
+ Select Operator [SEL_351] (rows=144002668 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_350] (rows=144002668 width=135)
+ predicate:((ws_sold_date_sk BETWEEN DynamicValue(RS_73_date_dim_d_date_sk_min) AND DynamicValue(RS_73_date_dim_d_date_sk_max) and in_bloom_filter(ws_sold_date_sk, DynamicValue(RS_73_date_dim_d_date_sk_bloom_filter))) and (ws_web_site_sk BETWEEN DynamicValue(RS_76_web_site_web_site_sk_min) AND DynamicValue(RS_76_web_site_web_site_sk_max) and in_bloom_filter(ws_web_site_sk, DynamicValue(RS_76_web_site_web_site_sk_bloom_filter))) and ws_sold_date_sk is not null and ws_web_site_sk is not null)
+ TableScan [TS_263] (rows=144002668 width=135)
+ Output:["ws_sold_date_sk","ws_web_site_sk","ws_ext_sales_price","ws_net_profit"]
+ <-Reducer 35 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_348]
+ Group By Operator [GBY_347] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 34 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_329]
+ Group By Operator [GBY_328] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_327] (rows=84 width=1850)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_325]
+ <-Reducer 19 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_346]
+ Group By Operator [GBY_345] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_285]
+ Group By Operator [GBY_282] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_279] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_273]
+ <-Reducer 30 [CONTAINS]
+ Reduce Output Operator [RS_271]
+ PartitionCols:_col1
+ Select Operator [SEL_269] (rows=158402938 width=135)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Merge Join Operator [MERGEJOIN_268] (rows=158402938 width=135)
+ Conds:RS_368._col0, _col2=RS_355._col1, _col2(Inner),Output:["_col1","_col3","_col6","_col7"]
+ <-Map 31 [SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_355]
+ PartitionCols:_col1, _col2
+ Select Operator [SEL_354] (rows=14398467 width=92)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_353] (rows=14398467 width=92)
+ predicate:(wr_item_sk is not null and wr_order_number is not null and wr_returned_date_sk is not null)
+ TableScan [TS_57] (rows=14398467 width=92)
+ default@web_returns,web_returns,Tbl:COMPLETE,Col:NONE,Output:["wr_returned_date_sk","wr_item_sk","wr_order_number","wr_return_amt","wr_net_loss"]
+ <-Map 29 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_368]
+ PartitionCols:_col0, _col2
+ Select Operator [SEL_367] (rows=144002668 width=135)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_366] (rows=144002668 width=135)
+ predicate:((ws_item_sk BETWEEN DynamicValue(RS_61_web_returns_wr_item_sk_min) AND DynamicValue(RS_61_web_returns_wr_item_sk_max) and in_bloom_filter(ws_item_sk, DynamicValue(RS_61_web_returns_wr_item_sk_bloom_filter))) and (ws_order_number BETWEEN DynamicValue(RS_61_web_returns_wr_order_number_min) AND DynamicValue(RS_61_web_returns_wr_order_number_max) and in_bloom_filter(ws_order_number, DynamicValue(RS_61_web_returns_wr_order_number_bloom_filter))) and (ws_web_site_sk BETWEEN DynamicValue(RS_76_web_site_web_site_sk_min) AND DynamicValue(RS_76_web_site_web_site_sk_max) and in_bloom_filter(ws_web_site_sk, DynamicValue(RS_76_web_site_web_site_sk_bloom_filter))) and ws_item_sk is not null and ws_order_number is not null and ws_web_site_sk is not null)
+ TableScan [TS_54] (rows=144002668 width=135)
+ default@web_sales,web_sales,Tbl:COMPLETE,Col:NONE,Output:["ws_item_sk","ws_web_site_sk","ws_order_number"]
+ <-Reducer 35 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_349]
+ Please refer to the previous Group By Operator [GBY_347]
+ <-Reducer 32 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_363]
+ Group By Operator [GBY_362] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=14398467)"]
+ <-Map 31 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_360]
+ Group By Operator [GBY_358] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=14398467)"]
+ Select Operator [SEL_356] (rows=14398467 width=92)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_354]
+ <-Reducer 33 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_365]
+ Group By Operator [GBY_364] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=14398467)"]
+ <-Map 31 [CUSTOM_SIMPLE_EDGE] vectorized
+ PARTITION_ONLY_SHUFFLE [RS_361]
+ Group By Operator [GBY_359] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=14398467)"]
+ Select Operator [SEL_357] (rows=14398467 width=92)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_354]
<-Reducer 5 [CONTAINS] vectorized
- Reduce Output Operator [RS_298]
+ Reduce Output Operator [RS_303]
PartitionCols:_col0, _col1, _col2
- Group By Operator [GBY_297] (rows=2273797803 width=110)
+ Group By Operator [GBY_302] (rows=2273797803 width=110)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["sum(_col2)","sum(_col3)","sum(_col4)"],keys:_col0, _col1, 0L
- Select Operator [SEL_296] (rows=383320021 width=87)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Group By Operator [GBY_295] (rows=383320021 width=87)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_22]
- PartitionCols:_col0
- Group By Operator [GBY_21] (rows=766640042 width=87)
- Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col9
- Merge Join Operator [MERGEJOIN_219] (rows=766640042 width=87)
- Conds:RS_17._col0=RS_286._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col9"]
- <-Map 20 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_286]
- PartitionCols:_col0
- Select Operator [SEL_285] (rows=1704 width=1910)
- Output:["_col0","_col1"]
- Filter Operator [FIL_284] (rows=1704 width=1910)
- predicate:s_store_sk is not null
- TableScan [TS_11] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_17]
- PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_218] (rows=696945478 width=87)
- Conds:Union 2._col1=RS_270._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
- <-Map 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_270]
- PartitionCols:_col0
- Please refer to the previous Select Operator [SEL_269]
- <-Union 2 [SIMPLE_EDGE]
- <-Map 1 [CONTAINS] vectorized
- Reduce Output Operator [RS_294]
- PartitionCols:_col1
- Select Operator [SEL_293] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_292] (rows=575995635 width=88)
- predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_15_date_dim_d_date_sk_min) AND DynamicValue(RS_15_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_15_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_18_store_s_store_sk_min) AND DynamicValue(RS_18_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_18_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
- TableScan [TS_224] (rows=575995635 width=88)
- Output:["ss_sold_date_sk","ss_store_sk","ss_ext_sales_price","ss_net_profit"]
- <-Reducer 11 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_283]
- Group By Operator [GBY_282] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_279]
- Group By Operator [GBY_276] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_271] (rows=8116 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_269]
- <-Reducer 21 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_291]
- Group By Operator [GBY_290] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_289]
- Group By Operator [GBY_288] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_287] (rows=1704 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_285]
- <-Map 9 [CONTAINS] vectorized
- Reduce Output Operator [RS_307]
- PartitionCols:_col1
- Select Operator [SEL_306] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Filter Operator [FIL_305] (rows=57591150 width=77)
- predicate:(sr_returned_date_sk is not null and sr_store_sk is not null)
- TableScan [TS_234] (rows=57591150 width=77)
- Output:["sr_returned_date_sk","sr_store_sk","sr_return_amt","sr_net_loss"]
+ Top N Key Operator [TNK_301] (rows=757932601 width=110)
+ keys:_col0, _col1, 0L,sort order:+++,top n:100
+ Select Operator [SEL_300] (rows=383320021 width=87)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Group By Operator [GBY_299] (rows=383320021 width=87)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)"],keys:KEY._col0
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_22]
+ PartitionCols:_col0
+ Group By Operator [GBY_21] (rows=766640042 width=87)
+ Output:["_col0","_col1","_col2","_col3","_col4"],aggregations:["sum(_col2)","sum(_col4)","sum(_col3)","sum(_col5)"],keys:_col9
+ Merge Join Operator [MERGEJOIN_220] (rows=766640042 width=87)
+ Conds:RS_17._col0=RS_290._col0(Inner),Output:["_col2","_col3","_col4","_col5","_col9"]
+ <-Map 20 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_290]
+ PartitionCols:_col0
+ Select Operator [SEL_289] (rows=1704 width=1910)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_288] (rows=1704 width=1910)
+ predicate:s_store_sk is not null
+ TableScan [TS_11] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_id"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_17]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_219] (rows=696945478 width=87)
+ Conds:Union 2._col1=RS_274._col0(Inner),Output:["_col0","_col2","_col3","_col4","_col5"]
+ <-Map 10 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_274]
+ PartitionCols:_col0
+ Please refer to the previous Select Operator [SEL_273]
+ <-Union 2 [SIMPLE_EDGE]
+ <-Map 1 [CONTAINS] vectorized
+ Reduce Output Operator [RS_298]
+ PartitionCols:_col1
+ Select Operator [SEL_297] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_296] (rows=575995635 width=88)
+ predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_15_date_dim_d_date_sk_min) AND DynamicValue(RS_15_date_dim_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_15_date_dim_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_18_store_s_store_sk_min) AND DynamicValue(RS_18_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_18_store_s_store_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
+ TableScan [TS_225] (rows=575995635 width=88)
+ Output:["ss_sold_date_sk","ss_store_sk","ss_ext_sales_price","ss_net_profit"]
+ <-Reducer 11 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_287]
+ Group By Operator [GBY_286] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 10 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_283]
+ Group By Operator [GBY_280] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_275] (rows=8116 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_273]
+ <-Reducer 21 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_295]
+ Group By Operator [GBY_294] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 20 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_293]
+ Group By Operator [GBY_292] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_291] (rows=1704 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_289]
+ <-Map 9 [CONTAINS] vectorized
+ Reduce Output Operator [RS_312]
+ PartitionCols:_col1
+ Select Operator [SEL_311] (rows=57591150 width=77)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_310] (rows=57591150 width=77)
+ predicate:(sr_returned_date_sk is not null and sr_store_sk is not null)
+ TableScan [TS_236] (rows=57591150 width=77)
+ Output:["sr_returned_date_sk","sr_store_sk","sr_return_amt","sr_net_loss"]
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/perf/tez/query50.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query50.q.out b/ql/src/test/results/clientpositive/perf/tez/query50.q.out
index e723140..efbae5c 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query50.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query50.q.out
@@ -135,137 +135,139 @@ Stage-0
limit:100
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_155]
- Limit [LIM_154] (rows=100 width=88)
+ File Output Operator [FS_156]
+ Limit [LIM_155] (rows=100 width=88)
Number of rows:100
- Select Operator [SEL_153] (rows=383325119 width=88)
+ Select Operator [SEL_154] (rows=383325119 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_152]
- Group By Operator [GBY_151] (rows=383325119 width=88)
+ SHUFFLE [RS_153]
+ Group By Operator [GBY_152] (rows=383325119 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)","sum(VALUE._col2)","sum(VALUE._col3)","sum(VALUE._col4)"],keys:KEY._col0, KEY._col1, KEY._col2, KEY._col3, KEY._col4, KEY._col5, KEY._col6, KEY._col7, KEY._col8, KEY._col9
<-Reducer 5 [SIMPLE_EDGE]
SHUFFLE [RS_30]
PartitionCols:_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
Group By Operator [GBY_29] (rows=766650239 width=88)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"],aggregations:["sum(_col10)","sum(_col11)","sum(_col12)","sum(_col13)","sum(_col14)"],keys:_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
- Select Operator [SEL_27] (rows=766650239 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
- Merge Join Operator [MERGEJOIN_119] (rows=766650239 width=88)
- Conds:RS_24._col10=RS_142._col0(Inner),Output:["_col0","_col7","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
- <-Map 15 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_142]
- PartitionCols:_col0
- Select Operator [SEL_141] (rows=1704 width=1910)
- Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"]
- Filter Operator [FIL_140] (rows=1704 width=1910)
- predicate:s_store_sk is not null
- TableScan [TS_12] (rows=1704 width=1910)
- default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_name","s_company_id","s_street_number","s_street_name","s_street_type","s_suite_number","s_city","s_county","s_state","s_zip"]
- <-Reducer 4 [SIMPLE_EDGE]
- SHUFFLE [RS_24]
- PartitionCols:_col10
- Merge Join Operator [MERGEJOIN_118] (rows=696954748 width=88)
- Conds:RS_21._col7=RS_134._col0(Inner),Output:["_col0","_col7","_col10"]
- <-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_134]
- PartitionCols:_col0
- Select Operator [SEL_133] (rows=73049 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_132] (rows=73049 width=1119)
- predicate:d_date_sk is not null
- TableScan [TS_9] (rows=73049 width=1119)
- default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk"]
- <-Reducer 3 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col7
- Merge Join Operator [MERGEJOIN_117] (rows=633595212 width=88)
- Conds:RS_18._col1, _col2, _col3=RS_150._col1, _col2, _col4(Inner),Output:["_col0","_col7","_col10"]
- <-Reducer 2 [SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_18]
- PartitionCols:_col1, _col2, _col3
- Merge Join Operator [MERGEJOIN_116] (rows=63350266 width=77)
- Conds:RS_122._col0=RS_125._col0(Inner),Output:["_col0","_col1","_col2","_col3"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_122]
- PartitionCols:_col0
- Select Operator [SEL_121] (rows=57591150 width=77)
- Output:["_col0","_col1","_col2","_col3"]
- Filter Operator [FIL_120] (rows=57591150 width=77)
- predicate:(sr_customer_sk is not null and sr_item_sk is not null and sr_returned_date_sk is not null and sr_ticket_number is not null)
- TableScan [TS_0] (rows=57591150 width=77)
- default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number"]
- <-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_125]
- PartitionCols:_col0
- Select Operator [SEL_124] (rows=18262 width=1119)
- Output:["_col0"]
- Filter Operator [FIL_123] (rows=18262 width=1119)
- predicate:((d_moy = 9) and (d_year = 2000) and d_date_sk is not null)
- TableScan [TS_3] (rows=73049 width=1119)
- default@date_dim,d2,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
- <-Map 12 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_150]
- PartitionCols:_col1, _col2, _col4
- Select Operator [SEL_149] (rows=575995635 width=88)
- Output:["_col0","_col1","_col2","_col3","_col4"]
- Filter Operator [FIL_148] (rows=575995635 width=88)
- predicate:((ss_customer_sk BETWEEN DynamicValue(RS_18_store_returns_sr_customer_sk_min) AND DynamicValue(RS_18_store_returns_sr_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_18_store_returns_sr_customer_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_18_store_returns_sr_item_sk_min) AND DynamicValue(RS_18_store_returns_sr_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_18_store_returns_sr_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_22_d1_d_date_sk_min) AND DynamicValue(RS_22_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_22_d1_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_25_store_s_store_sk_min) AND DynamicValue(RS_25_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_25_store_s_store_sk_bloom_filter))) and (ss_ticket_number BETWEEN DynamicValue(RS_18_store_returns_sr_ticket_number_min) AND DynamicValue(RS_18_
store_returns_sr_ticket_number_max) and in_bloom_filter(ss_ticket_number, DynamicValue(RS_18_store_returns_sr_ticket_number_bloom_filter))) and ss_customer_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null and ss_ticket_number is not null)
- TableScan [TS_6] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number"]
- <-Reducer 10 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_131]
- Group By Operator [GBY_130] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_93]
- Group By Operator [GBY_92] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_91] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_116]
- <-Reducer 14 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_139]
- Group By Operator [GBY_138] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_137]
- Group By Operator [GBY_136] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_135] (rows=73049 width=1119)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_133]
- <-Reducer 16 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_147]
- Group By Operator [GBY_146] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 15 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_145]
- Group By Operator [GBY_144] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_143] (rows=1704 width=1910)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_141]
- <-Reducer 8 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_127]
- Group By Operator [GBY_126] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_83]
- Group By Operator [GBY_82] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_81] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_116]
- <-Reducer 9 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_129]
- Group By Operator [GBY_128] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
- <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_88]
- Group By Operator [GBY_87] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
- Select Operator [SEL_86] (rows=63350266 width=77)
- Output:["_col0"]
- Please refer to the previous Merge Join Operator [MERGEJOIN_116]
+ Top N Key Operator [TNK_56] (rows=766650239 width=88)
+ keys:_col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9,sort order:++++++++++,top n:100
+ Select Operator [SEL_27] (rows=766650239 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
+ Merge Join Operator [MERGEJOIN_120] (rows=766650239 width=88)
+ Conds:RS_24._col10=RS_143._col0(Inner),Output:["_col0","_col7","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Map 15 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_143]
+ PartitionCols:_col0
+ Select Operator [SEL_142] (rows=1704 width=1910)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"]
+ Filter Operator [FIL_141] (rows=1704 width=1910)
+ predicate:s_store_sk is not null
+ TableScan [TS_12] (rows=1704 width=1910)
+ default@store,store,Tbl:COMPLETE,Col:NONE,Output:["s_store_sk","s_store_name","s_company_id","s_street_number","s_street_name","s_street_type","s_suite_number","s_city","s_county","s_state","s_zip"]
+ <-Reducer 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_24]
+ PartitionCols:_col10
+ Merge Join Operator [MERGEJOIN_119] (rows=696954748 width=88)
+ Conds:RS_21._col7=RS_135._col0(Inner),Output:["_col0","_col7","_col10"]
+ <-Map 13 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_135]
+ PartitionCols:_col0
+ Select Operator [SEL_134] (rows=73049 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_133] (rows=73049 width=1119)
+ predicate:d_date_sk is not null
+ TableScan [TS_9] (rows=73049 width=1119)
+ default@date_dim,d1,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col7
+ Merge Join Operator [MERGEJOIN_118] (rows=633595212 width=88)
+ Conds:RS_18._col1, _col2, _col3=RS_151._col1, _col2, _col4(Inner),Output:["_col0","_col7","_col10"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_18]
+ PartitionCols:_col1, _col2, _col3
+ Merge Join Operator [MERGEJOIN_117] (rows=63350266 width=77)
+ Conds:RS_123._col0=RS_126._col0(Inner),Output:["_col0","_col1","_col2","_col3"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_123]
+ PartitionCols:_col0
+ Select Operator [SEL_122] (rows=57591150 width=77)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_121] (rows=57591150 width=77)
+ predicate:(sr_customer_sk is not null and sr_item_sk is not null and sr_returned_date_sk is not null and sr_ticket_number is not null)
+ TableScan [TS_0] (rows=57591150 width=77)
+ default@store_returns,store_returns,Tbl:COMPLETE,Col:NONE,Output:["sr_returned_date_sk","sr_item_sk","sr_customer_sk","sr_ticket_number"]
+ <-Map 11 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_126]
+ PartitionCols:_col0
+ Select Operator [SEL_125] (rows=18262 width=1119)
+ Output:["_col0"]
+ Filter Operator [FIL_124] (rows=18262 width=1119)
+ predicate:((d_moy = 9) and (d_year = 2000) and d_date_sk is not null)
+ TableScan [TS_3] (rows=73049 width=1119)
+ default@date_dim,d2,Tbl:COMPLETE,Col:NONE,Output:["d_date_sk","d_year","d_moy"]
+ <-Map 12 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_151]
+ PartitionCols:_col1, _col2, _col4
+ Select Operator [SEL_150] (rows=575995635 width=88)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_149] (rows=575995635 width=88)
+ predicate:((ss_customer_sk BETWEEN DynamicValue(RS_18_store_returns_sr_customer_sk_min) AND DynamicValue(RS_18_store_returns_sr_customer_sk_max) and in_bloom_filter(ss_customer_sk, DynamicValue(RS_18_store_returns_sr_customer_sk_bloom_filter))) and (ss_item_sk BETWEEN DynamicValue(RS_18_store_returns_sr_item_sk_min) AND DynamicValue(RS_18_store_returns_sr_item_sk_max) and in_bloom_filter(ss_item_sk, DynamicValue(RS_18_store_returns_sr_item_sk_bloom_filter))) and (ss_sold_date_sk BETWEEN DynamicValue(RS_22_d1_d_date_sk_min) AND DynamicValue(RS_22_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_22_d1_d_date_sk_bloom_filter))) and (ss_store_sk BETWEEN DynamicValue(RS_25_store_s_store_sk_min) AND DynamicValue(RS_25_store_s_store_sk_max) and in_bloom_filter(ss_store_sk, DynamicValue(RS_25_store_s_store_sk_bloom_filter))) and (ss_ticket_number BETWEEN DynamicValue(RS_18_store_returns_sr_ticket_number_min) AND DynamicValue(RS_1
8_store_returns_sr_ticket_number_max) and in_bloom_filter(ss_ticket_number, DynamicValue(RS_18_store_returns_sr_ticket_number_bloom_filter))) and ss_customer_sk is not null and ss_item_sk is not null and ss_sold_date_sk is not null and ss_store_sk is not null and ss_ticket_number is not null)
+ TableScan [TS_6] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_sold_date_sk","ss_item_sk","ss_customer_sk","ss_store_sk","ss_ticket_number"]
+ <-Reducer 10 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_132]
+ Group By Operator [GBY_131] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_94]
+ Group By Operator [GBY_93] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_92] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_117]
+ <-Reducer 14 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_140]
+ Group By Operator [GBY_139] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 13 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_138]
+ Group By Operator [GBY_137] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_136] (rows=73049 width=1119)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_134]
+ <-Reducer 16 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_148]
+ Group By Operator [GBY_147] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 15 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_146]
+ Group By Operator [GBY_145] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_144] (rows=1704 width=1910)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_142]
+ <-Reducer 8 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_128]
+ Group By Operator [GBY_127] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_84]
+ Group By Operator [GBY_83] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_82] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_117]
+ <-Reducer 9 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_130]
+ Group By Operator [GBY_129] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=63350264)"]
+ <-Reducer 2 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_89]
+ Group By Operator [GBY_88] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=63350264)"]
+ Select Operator [SEL_87] (rows=63350266 width=77)
+ Output:["_col0"]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_117]
[16/18] hive git commit: HIVE-17896: TopNKey: Create a standalone
vectorizable TopNKey operator (Teddy Choi,
reviewed by Jesus Camacho Rodriguez)
Posted by jc...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out b/ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out
index 6f65061..b278ecc 100644
--- a/ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out
+++ b/ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out
@@ -230,31 +230,40 @@ STAGE PLANS:
projectedOutputColumnNums: [8, 9, 10]
selectExpressions: VectorUDFMapIndexStringScalar(col 1:map<string,string>, key: k1) -> 8:string, VectorUDFMapIndexLongScalar(col 2:map<int,int>, key: 123) -> 9:int, VectorUDFMapIndexDoubleScalar(col 3:map<double,double>, key: 123.123) -> 10:double
Statistics: Num rows: 511 Data size: 995378 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col1), sum(_col2)
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(col 9:int) -> bigint, VectorUDAFSumDouble(col 10:double) -> double
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 8:string
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0, 1]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 511 Data size: 995378 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Reduce Sink Vectorization:
- className: VectorReduceSinkStringOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 8:string
+ native: true
+ Group By Operator
+ aggregations: sum(_col1), sum(_col2)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 9:int) -> bigint, VectorUDAFSumDouble(col 10:double) -> double
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 8:string
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0, 1]
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 511 Data size: 995378 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint), _col2 (type: double)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 511 Data size: 995378 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint), _col2 (type: double)
Execution mode: vectorized, llap
LLAP IO: all inputs (cache only)
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out b/ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out
index affb27e..fec8093 100644
--- a/ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out
+++ b/ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out
@@ -238,31 +238,40 @@ STAGE PLANS:
projectedOutputColumnNums: [4]
selectExpressions: VectorUDFStructField(col 1:struct<f1:int,f2:string>, col 0:int) -> 4:int
Statistics: Num rows: 341 Data size: 76542 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col0)
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(col 4:int) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 4:int
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 76542 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkLongOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 4:int
+ native: true
+ Group By Operator
+ aggregations: sum(_col0)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 4:int) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 4:int
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 341 Data size: 76542 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkLongOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 341 Data size: 76542 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs (cache only)
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/topnkey.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/topnkey.q.out b/ql/src/test/results/clientpositive/llap/topnkey.q.out
new file mode 100644
index 0000000..c1d8874
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/topnkey.q.out
@@ -0,0 +1,318 @@
+PREHOOK: query: EXPLAIN
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: EXPLAIN
+SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string), UDFToInteger(substr(value, 5)) (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Top N Key Operator
+ sort order: +
+ keys: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 5
+ Group By Operator
+ aggregations: sum(_col1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: sum(VALUE._col0)
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
+ Reducer 3
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 475 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 475 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT key, SUM(CAST(SUBSTR(value,5) AS INT)) FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0 0
+10 10
+100 200
+103 206
+104 208
+PREHOOK: query: EXPLAIN
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: EXPLAIN
+SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: key
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Top N Key Operator
+ sort order: +
+ keys: key (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 5
+ Group By Operator
+ keys: key (type: string)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ Reducer 3
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 250 Data size: 21750 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 435 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 435 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT key FROM src GROUP BY key ORDER BY key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0
+10
+100
+103
+104
+PREHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+POSTHOOK: query: explain vectorization detail
+SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+PLAN VECTORIZATION:
+ enabled: false
+ enabledConditionsNotMet: [hive.vectorized.execution.enabled IS false]
+
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src1
+ filterExpr: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ filterExpr: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: string)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col2
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col0 (type: string), _col2 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: string)
+ Reducer 3
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 791 Data size: 140798 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 5
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 5
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key = src2.key) ORDER BY src1.key LIMIT 5
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+0 val_0
+0 val_0
+0 val_0
+0 val_0
+0 val_0
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out b/ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out
index f801856..8c74a92 100644
--- a/ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out
+++ b/ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out
@@ -141,31 +141,40 @@ STAGE PLANS:
native: true
projectedOutputColumnNums: [2]
Statistics: Num rows: 1049 Data size: 4196 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(50), count(), sum(50.0D), count(50.0D), sum(50), count(50)
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(ConstantVectorExpression(val 50) -> 12:int) -> bigint, VectorUDAFCountStar(*) -> bigint, VectorUDAFSumDouble(ConstantVectorExpression(val 50.0) -> 13:double) -> double, VectorUDAFCount(ConstantVectorExpression(val 50.0) -> 14:double) -> bigint, VectorUDAFSumDecimal(ConstantVectorExpression(val 50) -> 15:decimal(10,0)) -> decimal(20,0), VectorUDAFCount(ConstantVectorExpression(val 50) -> 16:decimal(10,0)) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 2:int
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0, 1, 2, 3, 4, 5]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
- Statistics: Num rows: 257 Data size: 40092 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkLongOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 1049 Data size: 4196 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 2:int
+ native: true
+ Group By Operator
+ aggregations: sum(50), count(), sum(50.0D), count(50.0D), sum(50), count(50)
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(ConstantVectorExpression(val 50) -> 12:int) -> bigint, VectorUDAFCountStar(*) -> bigint, VectorUDAFSumDouble(ConstantVectorExpression(val 50.0) -> 13:double) -> double, VectorUDAFCount(ConstantVectorExpression(val 50.0) -> 14:double) -> bigint, VectorUDAFSumDecimal(ConstantVectorExpression(val 50) -> 15:decimal(10,0)) -> decimal(20,0), VectorUDAFCount(ConstantVectorExpression(val 50) -> 16:decimal(10,0)) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 2:int
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0, 1, 2, 3, 4, 5]
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 257 Data size: 40092 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint), _col2 (type: bigint), _col3 (type: double), _col4 (type: bigint), _col5 (type: decimal(12,0)), _col6 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkLongOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 257 Data size: 40092 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint), _col2 (type: bigint), _col3 (type: double), _col4 (type: bigint), _col5 (type: decimal(12,0)), _col6 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vector_char_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_char_2.q.out b/ql/src/test/results/clientpositive/llap/vector_char_2.q.out
index 73e8060..b58de03 100644
--- a/ql/src/test/results/clientpositive/llap/vector_char_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/vector_char_2.q.out
@@ -102,31 +102,40 @@ STAGE PLANS:
projectedOutputColumnNums: [1, 3]
selectExpressions: CastStringToLong(col 0:char(10)) -> 3:int
Statistics: Num rows: 501 Data size: 89178 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1), count()
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(col 3:int) -> bigint, VectorUDAFCountStar(*) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 1:char(20)
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0, 1]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: char(20))
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: char(20))
- sort order: +
- Map-reduce partition columns: _col0 (type: char(20))
- Reduce Sink Vectorization:
- className: VectorReduceSinkStringOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 501 Data size: 89178 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 5
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 1:char(20)
+ native: true
+ Group By Operator
+ aggregations: sum(_col1), count()
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 3:int) -> bigint, VectorUDAFCountStar(*) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 1:char(20)
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0, 1]
+ keys: _col0 (type: char(20))
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint), _col2 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: char(20))
+ sort order: +
+ Map-reduce partition columns: _col0 (type: char(20))
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint), _col2 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
@@ -298,31 +307,40 @@ STAGE PLANS:
projectedOutputColumnNums: [1, 3]
selectExpressions: CastStringToLong(col 0:char(10)) -> 3:int
Statistics: Num rows: 501 Data size: 89178 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- aggregations: sum(_col1), count()
- Group By Vectorization:
- aggregators: VectorUDAFSumLong(col 3:int) -> bigint, VectorUDAFCountStar(*) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 1:char(20)
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0, 1]
+ Top N Key Operator
+ sort order: -
keys: _col0 (type: char(20))
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: char(20))
- sort order: -
- Map-reduce partition columns: _col0 (type: char(20))
- Reduce Sink Vectorization:
- className: VectorReduceSinkStringOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 501 Data size: 89178 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 5
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 1:char(20)
+ native: true
+ Group By Operator
+ aggregations: sum(_col1), count()
+ Group By Vectorization:
+ aggregators: VectorUDAFSumLong(col 3:int) -> bigint, VectorUDAFCountStar(*) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 1:char(20)
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0, 1]
+ keys: _col0 (type: char(20))
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint), _col2 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: char(20))
+ sort order: -
+ Map-reduce partition columns: _col0 (type: char(20))
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 250 Data size: 26750 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint), _col2 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out b/ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out
index bddde5f..1f49804 100644
--- a/ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out
+++ b/ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out
@@ -68,33 +68,42 @@ STAGE PLANS:
native: true
projectedOutputColumnNums: [0, 1]
Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- Group By Vectorization:
- aggregators: VectorUDAFCountStar(*) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 4:bigint
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +++
keys: a (type: string), b (type: string), 0L (type: bigint)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 24 Data size: 8832 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
- sort order: +++
- Map-reduce partition columns: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
- Reduce Sink Vectorization:
- className: VectorReduceSinkMultiKeyOperator
- keyColumnNums: [0, 1, 2]
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- valueColumnNums: [3]
+ Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 4:bigint
+ native: true
+ Group By Operator
+ aggregations: count()
+ Group By Vectorization:
+ aggregators: VectorUDAFCountStar(*) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 5:bigint
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: a (type: string), b (type: string), 0L (type: bigint)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 24 Data size: 8832 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col3 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
+ sort order: +++
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkMultiKeyOperator
+ keyColumnNums: [0, 1, 2]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: [3]
+ Statistics: Num rows: 24 Data size: 8832 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col3 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
@@ -111,7 +120,7 @@ STAGE PLANS:
includeColumns: [0, 1]
dataColumns: a:string, b:string, c:string
partitionColumnCount: 0
- scratchColumnTypeNames: [bigint]
+ scratchColumnTypeNames: [bigint, bigint]
Reducer 2
Execution mode: vectorized, llap
Reduce Vectorization:
@@ -269,33 +278,42 @@ STAGE PLANS:
native: true
projectedOutputColumnNums: [0, 1]
Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- Group By Vectorization:
- aggregators: VectorUDAFCountStar(*) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 4:bigint
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +++
keys: a (type: string), b (type: string), 0L (type: bigint)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 24 Data size: 8832 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
- sort order: +++
- Map-reduce partition columns: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
- Reduce Sink Vectorization:
- className: VectorReduceSinkMultiKeyOperator
- keyColumnNums: [0, 1, 2]
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- valueColumnNums: [3]
+ Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 4:bigint
+ native: true
+ Group By Operator
+ aggregations: count()
+ Group By Vectorization:
+ aggregators: VectorUDAFCountStar(*) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 5:bigint
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: a (type: string), b (type: string), 0L (type: bigint)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 24 Data size: 8832 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col3 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
+ sort order: +++
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkMultiKeyOperator
+ keyColumnNums: [0, 1, 2]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: [3]
+ Statistics: Num rows: 24 Data size: 8832 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col3 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
@@ -312,7 +330,7 @@ STAGE PLANS:
includeColumns: [0, 1]
dataColumns: a:string, b:string, c:string
partitionColumnCount: 0
- scratchColumnTypeNames: [bigint]
+ scratchColumnTypeNames: [bigint, bigint]
Reducer 2
Execution mode: vectorized, llap
Reduce Vectorization:
@@ -470,33 +488,42 @@ STAGE PLANS:
native: true
projectedOutputColumnNums: [0, 1]
Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- Group By Vectorization:
- aggregators: VectorUDAFCountStar(*) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 4:bigint
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +++
keys: a (type: string), b (type: string), 0L (type: bigint)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 12 Data size: 4416 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
- sort order: +++
- Map-reduce partition columns: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
- Reduce Sink Vectorization:
- className: VectorReduceSinkMultiKeyOperator
- keyColumnNums: [0, 1, 2]
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- valueColumnNums: [3]
+ Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 4:bigint
+ native: true
+ Group By Operator
+ aggregations: count()
+ Group By Vectorization:
+ aggregators: VectorUDAFCountStar(*) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string, col 1:string, ConstantVectorExpression(val 0) -> 5:bigint
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: a (type: string), b (type: string), 0L (type: bigint)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 12 Data size: 4416 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col3 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
+ sort order: +++
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkMultiKeyOperator
+ keyColumnNums: [0, 1, 2]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: [3]
+ Statistics: Num rows: 12 Data size: 4416 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col3 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
@@ -513,7 +540,7 @@ STAGE PLANS:
includeColumns: [0, 1]
dataColumns: a:string, b:string, c:string
partitionColumnCount: 0
- scratchColumnTypeNames: [bigint]
+ scratchColumnTypeNames: [bigint, bigint]
Reducer 2
Execution mode: vectorized, llap
Reduce Vectorization:
@@ -671,30 +698,39 @@ STAGE PLANS:
native: true
projectedOutputColumnNums: [0, 1, 2]
Statistics: Num rows: 6 Data size: 3312 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- Group By Vectorization:
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 0:string, col 1:string, col 2:string, ConstantVectorExpression(val 0) -> 4:bigint
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: []
+ Top N Key Operator
+ sort order: ++++
keys: a (type: string), b (type: string), c (type: string), 0L (type: bigint)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 18 Data size: 9936 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: bigint)
- sort order: ++++
- Map-reduce partition columns: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: bigint)
- Reduce Sink Vectorization:
- className: VectorReduceSinkMultiKeyOperator
- keyColumnNums: [0, 1, 2, 3]
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- valueColumnNums: []
+ Statistics: Num rows: 6 Data size: 3312 Basic stats: COMPLETE Column stats: NONE
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:string, col 1:string, col 2:string, ConstantVectorExpression(val 0) -> 4:bigint
+ native: true
+ Group By Operator
+ Group By Vectorization:
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string, col 1:string, col 2:string, ConstantVectorExpression(val 0) -> 5:bigint
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: []
+ keys: a (type: string), b (type: string), c (type: string), 0L (type: bigint)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 18 Data size: 9936 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: bigint)
+ sort order: ++++
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: bigint)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkMultiKeyOperator
+ keyColumnNums: [0, 1, 2, 3]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: []
+ Statistics: Num rows: 18 Data size: 9936 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
@@ -711,7 +747,7 @@ STAGE PLANS:
includeColumns: [0, 1, 2]
dataColumns: a:string, b:string, c:string
partitionColumnCount: 0
- scratchColumnTypeNames: [bigint]
+ scratchColumnTypeNames: [bigint, bigint]
Reducer 2
Execution mode: vectorized, llap
Reduce Vectorization:
@@ -866,30 +902,39 @@ STAGE PLANS:
native: true
projectedOutputColumnNums: [0]
Statistics: Num rows: 6 Data size: 1104 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- Group By Vectorization:
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 0:string
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: []
+ Top N Key Operator
+ sort order: +
keys: a (type: string)
- mode: hash
- outputColumnNames: _col0
Statistics: Num rows: 6 Data size: 1104 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Reduce Sink Vectorization:
- className: VectorReduceSinkStringOperator
- keyColumnNums: [0]
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- valueColumnNums: []
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:string
+ native: true
+ Group By Operator
+ Group By Vectorization:
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:string
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: []
+ keys: a (type: string)
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 6 Data size: 1104 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: []
+ Statistics: Num rows: 6 Data size: 1104 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
@@ -1048,33 +1093,42 @@ STAGE PLANS:
projectedOutputColumnNums: [6]
selectExpressions: DoubleColAddDoubleColumn(col 4:double, col 5:double)(children: CastStringToDouble(col 0:string) -> 4:double, CastStringToDouble(col 1:string) -> 5:double) -> 6:double
Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- Group By Vectorization:
- aggregators: VectorUDAFCountStar(*) -> bigint
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 6:double
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: double)
- mode: hash
- outputColumnNames: _col0, _col1
Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: double)
- sort order: +
- Map-reduce partition columns: _col0 (type: double)
- Reduce Sink Vectorization:
- className: VectorReduceSinkMultiKeyOperator
- keyColumnNums: [0]
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- valueColumnNums: [1]
+ top n: 10
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 6:double
+ native: true
+ Group By Operator
+ aggregations: count()
+ Group By Vectorization:
+ aggregators: VectorUDAFCountStar(*) -> bigint
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 6:double
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: _col0 (type: double)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: double)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: double)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkMultiKeyOperator
+ keyColumnNums: [0]
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ valueColumnNums: [1]
+ Statistics: Num rows: 6 Data size: 2208 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out b/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
index 1235bda..bdcc286 100644
--- a/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
+++ b/ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
@@ -266,28 +266,37 @@ STAGE PLANS:
native: true
projectedOutputColumnNums: [9]
Statistics: Num rows: 1000 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- Group By Vectorization:
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 9:int
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: []
+ Top N Key Operator
+ sort order: +
keys: ss_ticket_number (type: int)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 85 Data size: 340 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Reduce Sink Vectorization:
- className: VectorReduceSinkLongOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 1000 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
+ top n: 20
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 9:int
+ native: true
+ Group By Operator
+ Group By Vectorization:
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 9:int
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: []
+ keys: ss_ticket_number (type: int)
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 85 Data size: 340 Basic stats: COMPLETE Column stats: COMPLETE
- TopN Hash Memory Usage: 0.1
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkLongOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 85 Data size: 340 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vector_mr_diff_schema_alias.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_mr_diff_schema_alias.q.out b/ql/src/test/results/clientpositive/llap/vector_mr_diff_schema_alias.q.out
index fee5a5f..e81d7df 100644
--- a/ql/src/test/results/clientpositive/llap/vector_mr_diff_schema_alias.q.out
+++ b/ql/src/test/results/clientpositive/llap/vector_mr_diff_schema_alias.q.out
@@ -365,19 +365,24 @@ STAGE PLANS:
1 _col0 (type: int)
outputColumnNames: _col4
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
+ Top N Key Operator
+ sort order: +
keys: _col4 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ top n: 100
+ Group By Operator
+ aggregations: count()
+ keys: _col4 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col1 (type: bigint)
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col1 (type: bigint)
Reducer 4
Execution mode: vectorized, llap
Reduce Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vector_reduce_groupby_decimal.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_reduce_groupby_decimal.q.out b/ql/src/test/results/clientpositive/llap/vector_reduce_groupby_decimal.q.out
index 983c71d..f65712a 100644
--- a/ql/src/test/results/clientpositive/llap/vector_reduce_groupby_decimal.q.out
+++ b/ql/src/test/results/clientpositive/llap/vector_reduce_groupby_decimal.q.out
@@ -60,31 +60,40 @@ STAGE PLANS:
predicateExpression: FilterExprAndExpr(children: SelectColumnIsNotNull(col 2:decimal(20,10)), SelectColumnIsNotNull(col 3:decimal(23,14)))
predicate: (cdecimal1 is not null and cdecimal2 is not null) (type: boolean)
Statistics: Num rows: 5492 Data size: 1231540 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: min(cdecimal1)
- Group By Vectorization:
- aggregators: VectorUDAFMinDecimal(col 2:decimal(20,10)) -> decimal(20,10)
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 0:int, col 1:double, col 2:decimal(20,10), col 3:decimal(23,14)
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: [0]
+ Top N Key Operator
+ sort order: ++++
keys: cint (type: int), cdouble (type: double), cdecimal1 (type: decimal(20,10)), cdecimal2 (type: decimal(23,14))
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 5492 Data size: 1231540 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: double), _col2 (type: decimal(20,10)), _col3 (type: decimal(23,14))
- sort order: ++++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: double), _col2 (type: decimal(20,10)), _col3 (type: decimal(23,14))
- Reduce Sink Vectorization:
- className: VectorReduceSinkMultiKeyOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ top n: 50
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 0:int, col 1:double, col 2:decimal(20,10), col 3:decimal(23,14)
+ native: true
+ Group By Operator
+ aggregations: min(cdecimal1)
+ Group By Vectorization:
+ aggregators: VectorUDAFMinDecimal(col 2:decimal(20,10)) -> decimal(20,10)
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 0:int, col 1:double, col 2:decimal(20,10), col 3:decimal(23,14)
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: [0]
+ keys: cint (type: int), cdouble (type: double), cdecimal1 (type: decimal(20,10)), cdecimal2 (type: decimal(23,14))
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 5492 Data size: 1231540 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
- value expressions: _col4 (type: decimal(20,10))
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: double), _col2 (type: decimal(20,10)), _col3 (type: decimal(23,14))
+ sort order: ++++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: double), _col2 (type: decimal(20,10)), _col3 (type: decimal(23,14))
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkMultiKeyOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 5492 Data size: 1231540 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col4 (type: decimal(20,10))
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization:
http://git-wip-us.apache.org/repos/asf/hive/blob/851c8aba/ql/src/test/results/clientpositive/llap/vector_string_concat.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vector_string_concat.q.out b/ql/src/test/results/clientpositive/llap/vector_string_concat.q.out
index 38d9172..c6b3dcc 100644
--- a/ql/src/test/results/clientpositive/llap/vector_string_concat.q.out
+++ b/ql/src/test/results/clientpositive/llap/vector_string_concat.q.out
@@ -352,28 +352,37 @@ STAGE PLANS:
projectedOutputColumnNums: [20]
selectExpressions: StringGroupConcatColCol(col 18:string, col 19:string)(children: StringGroupColConcatStringScalar(col 19:string, val -)(children: StringScalarConcatStringGroupCol(val Quarter , col 18:string)(children: CastLongToString(col 14:int)(children: CastDoubleToLong(col 16:double)(children: DoubleColAddDoubleScalar(col 17:double, val 1.0)(children: DoubleColDivideDoubleScalar(col 16:double, val 3.0)(children: CastLongToDouble(col 15:int)(children: LongColSubtractLongScalar(col 14:int, val 1)(children: VectorUDFMonthDate(col 12, field MONTH) -> 14:int) -> 15:int) -> 16:double) -> 17:double) -> 16:double) -> 14:int) -> 18:string) -> 19:string) -> 18:string, CastLongToString(col 14:int)(children: VectorUDFYearDate(col 12, field YEAR) -> 14:int) -> 19:string) -> 20:string
Statistics: Num rows: 2000 Data size: 106456 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- Group By Vectorization:
- className: VectorGroupByOperator
- groupByMode: HASH
- keyExpressions: col 20:string
- native: false
- vectorProcessingMode: HASH
- projectedOutputColumnNums: []
+ Top N Key Operator
+ sort order: +
keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0
Statistics: Num rows: 2000 Data size: 106456 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Reduce Sink Vectorization:
- className: VectorReduceSinkStringOperator
- native: true
- nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ top n: 50
+ Top N Key Vectorization:
+ className: VectorTopNKeyOperator
+ keyExpressions: col 20:string
+ native: true
+ Group By Operator
+ Group By Vectorization:
+ className: VectorGroupByOperator
+ groupByMode: HASH
+ keyExpressions: col 20:string
+ native: false
+ vectorProcessingMode: HASH
+ projectedOutputColumnNums: []
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 2000 Data size: 106456 Basic stats: COMPLETE Column stats: NONE
- TopN Hash Memory Usage: 0.1
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Reduce Sink Vectorization:
+ className: VectorReduceSinkStringOperator
+ native: true
+ nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
+ Statistics: Num rows: 2000 Data size: 106456 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized, llap
LLAP IO: all inputs
Map Vectorization: