You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by jc...@apache.org on 2018/03/22 16:57:37 UTC
[14/34] hive git commit: HIVE-18979: Enable
AggregateReduceFunctionsRule from Calcite (Jesus Camacho Rodriguez,
reviewed by Ashutosh Chauhan)
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/perf/tez/query9.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query9.q.out b/ql/src/test/results/clientpositive/perf/tez/query9.q.out
index 24de6f2..8df7ed9 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query9.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query9.q.out
@@ -151,69 +151,73 @@ Stage-0
Stage-1
Reducer 16
File Output Operator [FS_154]
- Select Operator [SEL_153] (rows=36 width=3135)
+ Select Operator [SEL_153] (rows=36 width=1455)
Output:["_col0","_col1","_col2","_col3","_col4"]
- Merge Join Operator [MERGEJOIN_185] (rows=36 width=3135)
+ Merge Join Operator [MERGEJOIN_185] (rows=36 width=1455)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
<-Reducer 15 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_150]
- Merge Join Operator [MERGEJOIN_184] (rows=36 width=2846)
+ Merge Join Operator [MERGEJOIN_184] (rows=36 width=1334)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
<-Reducer 14 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_147]
- Merge Join Operator [MERGEJOIN_183] (rows=36 width=2557)
+ Merge Join Operator [MERGEJOIN_183] (rows=36 width=1213)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
<-Reducer 13 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_144]
- Merge Join Operator [MERGEJOIN_182] (rows=36 width=2548)
+ Merge Join Operator [MERGEJOIN_182] (rows=36 width=1204)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12"]
<-Reducer 12 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_141]
- Merge Join Operator [MERGEJOIN_181] (rows=36 width=2259)
+ Merge Join Operator [MERGEJOIN_181] (rows=36 width=1083)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
<-Reducer 11 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_138]
- Merge Join Operator [MERGEJOIN_180] (rows=36 width=1970)
+ Merge Join Operator [MERGEJOIN_180] (rows=36 width=962)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10"]
<-Reducer 10 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_135]
- Merge Join Operator [MERGEJOIN_179] (rows=36 width=1961)
+ Merge Join Operator [MERGEJOIN_179] (rows=36 width=953)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9"]
<-Reducer 34 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_133]
- Group By Operator [GBY_64] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 29 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_63]
- Group By Operator [GBY_62] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_net_paid_inc_tax)"]
- Select Operator [SEL_61] (rows=63999515 width=88)
- Output:["ss_net_paid_inc_tax"]
- Filter Operator [FIL_164] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 41 AND 60
- TableScan [TS_80] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_net_paid_inc_tax"]
+ Select Operator [SEL_65] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_64] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 29 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_63]
+ Group By Operator [GBY_62] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_net_paid_inc_tax)","count(ss_net_paid_inc_tax)"]
+ Select Operator [SEL_61] (rows=63999515 width=88)
+ Output:["ss_net_paid_inc_tax"]
+ Filter Operator [FIL_164] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 41 AND 60
+ TableScan [TS_80] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_net_paid_inc_tax"]
<-Reducer 9 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_132]
- Merge Join Operator [MERGEJOIN_178] (rows=36 width=1672)
+ Merge Join Operator [MERGEJOIN_178] (rows=36 width=832)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
<-Reducer 28 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_130]
- Group By Operator [GBY_57] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 23 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_56]
- Group By Operator [GBY_55] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_54] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_163] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 41 AND 60
- TableScan [TS_73] (rows=575995635 width=88)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_ext_list_price"]
+ Select Operator [SEL_58] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_57] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 23 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_56]
+ Group By Operator [GBY_55] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_ext_list_price)","count(ss_ext_list_price)"]
+ Select Operator [SEL_54] (rows=63999515 width=88)
+ Output:["ss_ext_list_price"]
+ Filter Operator [FIL_163] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 41 AND 60
+ TableScan [TS_73] (rows=575995635 width=88)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity","ss_ext_list_price"]
<-Reducer 8 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_129]
- Merge Join Operator [MERGEJOIN_177] (rows=36 width=1383)
+ Merge Join Operator [MERGEJOIN_177] (rows=36 width=711)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
<-Reducer 22 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_127]
@@ -230,41 +234,45 @@ Stage-0
default@store_sales,store_sales,Tbl:COMPLETE,Col:NONE,Output:["ss_quantity"]
<-Reducer 7 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_126]
- Merge Join Operator [MERGEJOIN_176] (rows=36 width=1374)
+ Merge Join Operator [MERGEJOIN_176] (rows=36 width=702)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5","_col6"]
<-Reducer 33 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_124]
- Group By Operator [GBY_43] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 29 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_42]
- Group By Operator [GBY_41] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_net_paid_inc_tax)"]
- Select Operator [SEL_40] (rows=63999515 width=88)
- Output:["ss_net_paid_inc_tax"]
- Filter Operator [FIL_161] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 21 AND 40
- Please refer to the previous TableScan [TS_80]
+ Select Operator [SEL_44] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_43] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 29 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_42]
+ Group By Operator [GBY_41] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_net_paid_inc_tax)","count(ss_net_paid_inc_tax)"]
+ Select Operator [SEL_40] (rows=63999515 width=88)
+ Output:["ss_net_paid_inc_tax"]
+ Filter Operator [FIL_161] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 21 AND 40
+ Please refer to the previous TableScan [TS_80]
<-Reducer 6 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_123]
- Merge Join Operator [MERGEJOIN_175] (rows=36 width=1085)
+ Merge Join Operator [MERGEJOIN_175] (rows=36 width=581)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4","_col5"]
<-Reducer 27 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_121]
- Group By Operator [GBY_36] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 23 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_35]
- Group By Operator [GBY_34] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_33] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_160] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 21 AND 40
- Please refer to the previous TableScan [TS_73]
+ Select Operator [SEL_37] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_36] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 23 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_35]
+ Group By Operator [GBY_34] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_ext_list_price)","count(ss_ext_list_price)"]
+ Select Operator [SEL_33] (rows=63999515 width=88)
+ Output:["ss_ext_list_price"]
+ Filter Operator [FIL_160] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 21 AND 40
+ Please refer to the previous TableScan [TS_73]
<-Reducer 5 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_120]
- Merge Join Operator [MERGEJOIN_174] (rows=36 width=796)
+ Merge Join Operator [MERGEJOIN_174] (rows=36 width=460)
Conds:(Left Outer),Output:["_col1","_col2","_col3","_col4"]
<-Reducer 21 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_118]
@@ -280,11 +288,11 @@ Stage-0
Please refer to the previous TableScan [TS_66]
<-Reducer 4 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_117]
- Merge Join Operator [MERGEJOIN_173] (rows=36 width=787)
+ Merge Join Operator [MERGEJOIN_173] (rows=36 width=451)
Conds:(Left Outer),Output:["_col1","_col2","_col3"]
<-Reducer 3 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_114]
- Merge Join Operator [MERGEJOIN_172] (rows=36 width=498)
+ Merge Join Operator [MERGEJOIN_172] (rows=36 width=330)
Conds:(Left Outer),Output:["_col1","_col2"]
<-Reducer 2 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_111]
@@ -311,30 +319,34 @@ Stage-0
Please refer to the previous TableScan [TS_66]
<-Reducer 26 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_112]
- Group By Operator [GBY_15] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 23 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_14]
- Group By Operator [GBY_13] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_12] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_157] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 1 AND 20
- Please refer to the previous TableScan [TS_73]
+ Select Operator [SEL_16] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_15] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 23 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_14]
+ Group By Operator [GBY_13] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_ext_list_price)","count(ss_ext_list_price)"]
+ Select Operator [SEL_12] (rows=63999515 width=88)
+ Output:["ss_ext_list_price"]
+ Filter Operator [FIL_157] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 1 AND 20
+ Please refer to the previous TableScan [TS_73]
<-Reducer 32 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_115]
- Group By Operator [GBY_22] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 29 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_21]
- Group By Operator [GBY_20] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_net_paid_inc_tax)"]
- Select Operator [SEL_19] (rows=63999515 width=88)
- Output:["ss_net_paid_inc_tax"]
- Filter Operator [FIL_158] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 1 AND 20
- Please refer to the previous TableScan [TS_80]
+ Select Operator [SEL_23] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_22] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 29 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_21]
+ Group By Operator [GBY_20] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_net_paid_inc_tax)","count(ss_net_paid_inc_tax)"]
+ Select Operator [SEL_19] (rows=63999515 width=88)
+ Output:["ss_net_paid_inc_tax"]
+ Filter Operator [FIL_158] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 1 AND 20
+ Please refer to the previous TableScan [TS_80]
<-Reducer 18 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_136]
Group By Operator [GBY_71] (rows=1 width=8)
@@ -349,30 +361,34 @@ Stage-0
Please refer to the previous TableScan [TS_66]
<-Reducer 24 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_139]
- Group By Operator [GBY_78] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 23 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_77]
- Group By Operator [GBY_76] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_75] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_166] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 61 AND 80
- Please refer to the previous TableScan [TS_73]
+ Select Operator [SEL_79] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_78] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 23 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_77]
+ Group By Operator [GBY_76] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_ext_list_price)","count(ss_ext_list_price)"]
+ Select Operator [SEL_75] (rows=63999515 width=88)
+ Output:["ss_ext_list_price"]
+ Filter Operator [FIL_166] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 61 AND 80
+ Please refer to the previous TableScan [TS_73]
<-Reducer 30 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_142]
- Group By Operator [GBY_85] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 29 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_84]
- Group By Operator [GBY_83] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_net_paid_inc_tax)"]
- Select Operator [SEL_82] (rows=63999515 width=88)
- Output:["ss_net_paid_inc_tax"]
- Filter Operator [FIL_167] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 61 AND 80
- Please refer to the previous TableScan [TS_80]
+ Select Operator [SEL_86] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_85] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 29 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_84]
+ Group By Operator [GBY_83] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_net_paid_inc_tax)","count(ss_net_paid_inc_tax)"]
+ Select Operator [SEL_82] (rows=63999515 width=88)
+ Output:["ss_net_paid_inc_tax"]
+ Filter Operator [FIL_167] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 61 AND 80
+ Please refer to the previous TableScan [TS_80]
<-Reducer 19 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_145]
Group By Operator [GBY_92] (rows=1 width=8)
@@ -387,28 +403,32 @@ Stage-0
Please refer to the previous TableScan [TS_66]
<-Reducer 25 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_148]
- Group By Operator [GBY_99] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 23 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_98]
- Group By Operator [GBY_97] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_ext_list_price)"]
- Select Operator [SEL_96] (rows=63999515 width=88)
- Output:["ss_ext_list_price"]
- Filter Operator [FIL_169] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 81 AND 100
- Please refer to the previous TableScan [TS_73]
+ Select Operator [SEL_100] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_99] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 23 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_98]
+ Group By Operator [GBY_97] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_ext_list_price)","count(ss_ext_list_price)"]
+ Select Operator [SEL_96] (rows=63999515 width=88)
+ Output:["ss_ext_list_price"]
+ Filter Operator [FIL_169] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 81 AND 100
+ Please refer to the previous TableScan [TS_73]
<-Reducer 31 [CUSTOM_SIMPLE_EDGE]
PARTITION_ONLY_SHUFFLE [RS_151]
- Group By Operator [GBY_106] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(VALUE._col0)"]
- <-Map 29 [CUSTOM_SIMPLE_EDGE]
- PARTITION_ONLY_SHUFFLE [RS_105]
- Group By Operator [GBY_104] (rows=1 width=288)
- Output:["_col0"],aggregations:["avg(ss_net_paid_inc_tax)"]
- Select Operator [SEL_103] (rows=63999515 width=88)
- Output:["ss_net_paid_inc_tax"]
- Filter Operator [FIL_170] (rows=63999515 width=88)
- predicate:ss_quantity BETWEEN 81 AND 100
- Please refer to the previous TableScan [TS_80]
+ Select Operator [SEL_107] (rows=1 width=120)
+ Output:["_col0"]
+ Group By Operator [GBY_106] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"]
+ <-Map 29 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_105]
+ Group By Operator [GBY_104] (rows=1 width=120)
+ Output:["_col0","_col1"],aggregations:["sum(ss_net_paid_inc_tax)","count(ss_net_paid_inc_tax)"]
+ Select Operator [SEL_103] (rows=63999515 width=88)
+ Output:["ss_net_paid_inc_tax"]
+ Filter Operator [FIL_170] (rows=63999515 width=88)
+ predicate:ss_quantity BETWEEN 81 AND 100
+ Please refer to the previous TableScan [TS_80]
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/perf/tez/query92.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query92.q.out b/ql/src/test/results/clientpositive/perf/tez/query92.q.out
index 6009fdf..5118ebe 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query92.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query92.q.out
@@ -135,11 +135,11 @@ Stage-0
Select Operator [SEL_19] (rows=79201469 width=135)
Output:["_col0","_col1"]
Group By Operator [GBY_18] (rows=79201469 width=135)
- Output:["_col0","_col1"],aggregations:["avg(VALUE._col0)"],keys:KEY._col0
+ Output:["_col0","_col1","_col2"],aggregations:["sum(VALUE._col0)","count(VALUE._col1)"],keys:KEY._col0
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_17]
PartitionCols:_col0
Group By Operator [GBY_16] (rows=158402938 width=135)
- Output:["_col0","_col1"],aggregations:["avg(_col2)"],keys:_col1
+ Output:["_col0","_col1","_col2"],aggregations:["sum(_col2)","count(_col2)"],keys:_col1
Please refer to the previous Merge Join Operator [MERGEJOIN_61]
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out b/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out
index cf78358..3faa06a 100644
--- a/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out
+++ b/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out
@@ -925,35 +925,39 @@ STAGE PLANS:
1 _col0 (type: int)
outputColumnNames: _col2, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Group By Operator
- aggregations: stddev_samp(_col2), avg(_col2)
- keys: _col5 (type: int), _col4 (type: int), _col6 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Select Operator
+ expressions: _col6 (type: string), _col5 (type: int), _col4 (type: int), _col2 (type: int), UDFToDouble(_col2) (type: double), (UDFToDouble(_col2) * UDFToDouble(_col2)) (type: double)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: string)
- sort order: +++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: string)
+ Group By Operator
+ aggregations: sum(_col3), count(_col3), sum(_col5), sum(_col4)
+ keys: _col1 (type: int), _col2 (type: int), _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- value expressions: _col3 (type: struct<count:bigint,sum:double,variance:double>), _col4 (type: struct<count:bigint,sum:double,input:int>)
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: string)
+ sort order: +++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: string)
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ value expressions: _col3 (type: bigint), _col4 (type: bigint), _col5 (type: double), _col6 (type: double)
Reducer 15
Reduce Operator Tree:
Group By Operator
- aggregations: stddev_samp(VALUE._col0), avg(VALUE._col1)
+ aggregations: sum(VALUE._col0), count(VALUE._col1), sum(VALUE._col2), sum(VALUE._col3)
keys: KEY._col0 (type: int), KEY._col1 (type: int), KEY._col2 (type: string)
mode: mergepartial
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col0 (type: int), _col1 (type: int), _col3 (type: double), _col4 (type: double)
- outputColumnNames: _col1, _col2, _col3, _col4
+ expressions: _col0 (type: int), _col1 (type: int), _col3 (type: bigint), _col4 (type: bigint), _col5 (type: double), _col6 (type: double)
+ outputColumnNames: _col0, _col1, _col3, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
- predicate: CASE WHEN ((_col4 = 0.0D)) THEN (false) ELSE (((_col3 / _col4) > 1.0D)) END (type: boolean)
+ predicate: CASE WHEN (((_col3 / _col4) = 0)) THEN (false) ELSE (((power(((_col5 - ((_col6 * _col6) / _col4)) / CASE WHEN ((_col4 = 1L)) THEN (null) ELSE ((_col4 - 1)) END), 0.5) / (_col3 / _col4)) > 1.0D)) END (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col4 (type: double), CASE WHEN ((_col4 = 0.0D)) THEN (null) ELSE ((_col3 / _col4)) END (type: double)
+ expressions: _col0 (type: int), _col1 (type: int), (_col3 / _col4) (type: double), CASE WHEN (((_col3 / _col4) = 0)) THEN (null) ELSE ((power(((_col5 - ((_col6 * _col6) / _col4)) / CASE WHEN ((_col4 = 1L)) THEN (null) ELSE ((_col4 - 1)) END), 0.5) / (_col3 / _col4))) END (type: double)
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Reduce Output Operator
@@ -1004,35 +1008,39 @@ STAGE PLANS:
1 _col0 (type: int)
outputColumnNames: _col2, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Group By Operator
- aggregations: stddev_samp(_col2), avg(_col2)
- keys: _col5 (type: int), _col4 (type: int), _col6 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ Select Operator
+ expressions: _col6 (type: string), _col5 (type: int), _col4 (type: int), _col2 (type: int), UDFToDouble(_col2) (type: double), (UDFToDouble(_col2) * UDFToDouble(_col2)) (type: double)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: string)
- sort order: +++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: string)
+ Group By Operator
+ aggregations: sum(_col3), count(_col3), sum(_col5), sum(_col4)
+ keys: _col1 (type: int), _col2 (type: int), _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- value expressions: _col3 (type: struct<count:bigint,sum:double,variance:double>), _col4 (type: struct<count:bigint,sum:double,input:int>)
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: string)
+ sort order: +++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: int), _col2 (type: string)
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ value expressions: _col3 (type: bigint), _col4 (type: bigint), _col5 (type: double), _col6 (type: double)
Reducer 5
Reduce Operator Tree:
Group By Operator
- aggregations: stddev_samp(VALUE._col0), avg(VALUE._col1)
+ aggregations: sum(VALUE._col0), count(VALUE._col1), sum(VALUE._col2), sum(VALUE._col3)
keys: KEY._col0 (type: int), KEY._col1 (type: int), KEY._col2 (type: string)
mode: mergepartial
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col0 (type: int), _col1 (type: int), _col3 (type: double), _col4 (type: double)
- outputColumnNames: _col1, _col2, _col3, _col4
+ expressions: _col0 (type: int), _col1 (type: int), _col3 (type: bigint), _col4 (type: bigint), _col5 (type: double), _col6 (type: double)
+ outputColumnNames: _col0, _col1, _col3, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
- predicate: CASE WHEN ((_col4 = 0.0D)) THEN (false) ELSE (((_col3 / _col4) > 1.0D)) END (type: boolean)
+ predicate: CASE WHEN (((_col3 / _col4) = 0)) THEN (false) ELSE (((power(((_col5 - ((_col6 * _col6) / _col4)) / CASE WHEN ((_col4 = 1L)) THEN (null) ELSE ((_col4 - 1)) END), 0.5) / (_col3 / _col4)) > 1.0D)) END (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col4 (type: double), CASE WHEN ((_col4 = 0.0D)) THEN (null) ELSE ((_col3 / _col4)) END (type: double)
+ expressions: _col0 (type: int), _col1 (type: int), (_col3 / _col4) (type: double), CASE WHEN (((_col3 / _col4) = 0)) THEN (null) ELSE ((power(((_col5 - ((_col6 * _col6) / _col4)) / CASE WHEN ((_col4 = 1L)) THEN (null) ELSE ((_col4 - 1)) END), 0.5) / (_col3 / _col4))) END (type: double)
outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Reduce Output Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/spark/groupby3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/groupby3.q.out b/ql/src/test/results/clientpositive/spark/groupby3.q.out
index c314763..7bdc3a0 100644
--- a/ql/src/test/results/clientpositive/spark/groupby3.q.out
+++ b/ql/src/test/results/clientpositive/spark/groupby3.q.out
@@ -51,13 +51,13 @@ STAGE PLANS:
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: substr(value, 5) (type: string)
- outputColumnNames: _col0
+ expressions: value (type: string)
+ outputColumnNames: value
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: _col0 (type: string)
+ key expressions: substr(value, 5) (type: string)
sort order: +
- Map-reduce partition columns: _col0 (type: string)
+ Map-reduce partition columns: substr(value, 5) (type: string)
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Reducer 2
Reduce Operator Tree:
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/spark/groupby3_map.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/groupby3_map.q.out b/ql/src/test/results/clientpositive/spark/groupby3_map.q.out
index 9caf47b..0d5f352 100644
--- a/ql/src/test/results/clientpositive/spark/groupby3_map.q.out
+++ b/ql/src/test/results/clientpositive/spark/groupby3_map.q.out
@@ -50,34 +50,34 @@ STAGE PLANS:
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: substr(value, 5) (type: string)
- outputColumnNames: _col0
+ expressions: substr(value, 5) (type: string), UDFToDouble(substr(value, 5)) (type: double), (UDFToDouble(substr(value, 5)) * UDFToDouble(substr(value, 5))) (type: double)
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- aggregations: sum(_col0), avg(_col0), avg(DISTINCT _col0), max(_col0), min(_col0), std(_col0), stddev_samp(_col0), variance(_col0), var_samp(_col0)
+ aggregations: sum(_col0), count(_col0), sum(DISTINCT _col0), count(DISTINCT _col0), max(_col0), min(_col0), sum(_col2), sum(_col1)
keys: _col0 (type: string)
mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: double), _col2 (type: struct<count:bigint,sum:double,input:string>), _col4 (type: string), _col5 (type: string), _col6 (type: struct<count:bigint,sum:double,variance:double>), _col7 (type: struct<count:bigint,sum:double,variance:double>), _col8 (type: struct<count:bigint,sum:double,variance:double>), _col9 (type: struct<count:bigint,sum:double,variance:double>)
+ value expressions: _col1 (type: double), _col2 (type: bigint), _col5 (type: string), _col6 (type: string), _col7 (type: double), _col8 (type: double)
Reducer 2
Reduce Operator Tree:
Group By Operator
- aggregations: sum(VALUE._col0), avg(VALUE._col1), avg(DISTINCT KEY._col0:0._col0), max(VALUE._col3), min(VALUE._col4), std(VALUE._col5), stddev_samp(VALUE._col6), variance(VALUE._col7), var_samp(VALUE._col8)
+ aggregations: sum(VALUE._col0), count(VALUE._col1), sum(DISTINCT KEY._col0:0._col0), count(DISTINCT KEY._col0:1._col0), max(VALUE._col4), min(VALUE._col5), sum(VALUE._col6), sum(VALUE._col7)
mode: mergepartial
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 1 Data size: 1216 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: double), _col1 (type: double), _col2 (type: double), UDFToDouble(_col3) (type: double), UDFToDouble(_col4) (type: double), _col5 (type: double), _col6 (type: double), _col7 (type: double), _col8 (type: double)
+ expressions: _col0 (type: double), (_col0 / _col1) (type: double), (_col2 / _col3) (type: double), UDFToDouble(_col4) (type: double), UDFToDouble(_col5) (type: double), power(((_col6 - ((_col7 * _col7) / _col1)) / _col1), 0.5) (type: double), power(((_col6 - ((_col7 * _col7) / _col1)) / CASE WHEN ((_col1 = 1L)) THEN (null) ELSE ((_col1 - 1)) END), 0.5) (type: double), ((_col6 - ((_col7 * _col7) / _col1)) / _col1) (type: double), ((_col6 - ((_col7 * _col7) / _col1)) / CASE WHEN ((_col1 = 1L)) THEN (null) ELSE ((_col1 - 1)) END) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 1 Data size: 1216 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 1216 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -128,7 +128,7 @@ POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest1
POSTHOOK: Lineage: dest1.c1 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c2 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
-POSTHOOK: Lineage: dest1.c3 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: dest1.c3 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), (src)src.null, ]
POSTHOOK: Lineage: dest1.c4 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c5 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c6 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/spark/groupby3_map_multi_distinct.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/groupby3_map_multi_distinct.q.out b/ql/src/test/results/clientpositive/spark/groupby3_map_multi_distinct.q.out
index 6782761..d5974ce 100644
--- a/ql/src/test/results/clientpositive/spark/groupby3_map_multi_distinct.q.out
+++ b/ql/src/test/results/clientpositive/spark/groupby3_map_multi_distinct.q.out
@@ -54,34 +54,34 @@ STAGE PLANS:
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: substr(value, 5) (type: string)
- outputColumnNames: _col0
+ expressions: substr(value, 5) (type: string), UDFToDouble(substr(value, 5)) (type: double), (UDFToDouble(substr(value, 5)) * UDFToDouble(substr(value, 5))) (type: double)
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- aggregations: sum(_col0), avg(_col0), avg(DISTINCT _col0), max(_col0), min(_col0), std(_col0), stddev_samp(_col0), variance(_col0), var_samp(_col0), sum(DISTINCT _col0), count(DISTINCT _col0)
+ aggregations: sum(_col0), count(_col0), sum(DISTINCT _col0), count(DISTINCT _col0), max(_col0), min(_col0), sum(_col2), sum(_col1)
keys: _col0 (type: string)
mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: double), _col2 (type: struct<count:bigint,sum:double,input:string>), _col4 (type: string), _col5 (type: string), _col6 (type: struct<count:bigint,sum:double,variance:double>), _col7 (type: struct<count:bigint,sum:double,variance:double>), _col8 (type: struct<count:bigint,sum:double,variance:double>), _col9 (type: struct<count:bigint,sum:double,variance:double>)
+ value expressions: _col1 (type: double), _col2 (type: bigint), _col5 (type: string), _col6 (type: string), _col7 (type: double), _col8 (type: double)
Reducer 2
Reduce Operator Tree:
Group By Operator
- aggregations: sum(VALUE._col0), avg(VALUE._col1), avg(DISTINCT KEY._col0:0._col0), max(VALUE._col3), min(VALUE._col4), std(VALUE._col5), stddev_samp(VALUE._col6), variance(VALUE._col7), var_samp(VALUE._col8), sum(DISTINCT KEY._col0:1._col0), count(DISTINCT KEY._col0:2._col0)
+ aggregations: sum(VALUE._col0), count(VALUE._col1), sum(DISTINCT KEY._col0:0._col0), count(DISTINCT KEY._col0:1._col0), max(VALUE._col4), min(VALUE._col5), sum(VALUE._col6), sum(VALUE._col7)
mode: mergepartial
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
- Statistics: Num rows: 1 Data size: 1232 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: double), _col1 (type: double), _col2 (type: double), UDFToDouble(_col3) (type: double), UDFToDouble(_col4) (type: double), _col5 (type: double), _col6 (type: double), _col7 (type: double), _col8 (type: double), _col9 (type: double), UDFToDouble(_col10) (type: double)
+ expressions: _col0 (type: double), (_col0 / _col1) (type: double), (_col2 / _col3) (type: double), UDFToDouble(_col4) (type: double), UDFToDouble(_col5) (type: double), power(((_col6 - ((_col7 * _col7) / _col1)) / _col1), 0.5) (type: double), power(((_col6 - ((_col7 * _col7) / _col1)) / CASE WHEN ((_col1 = 1L)) THEN (null) ELSE ((_col1 - 1)) END), 0.5) (type: double), ((_col6 - ((_col7 * _col7) / _col1)) / _col1) (type: double), ((_col6 - ((_col7 * _col7) / _col1)) / CASE WHEN ((_col1 = 1L)) THEN (null) ELSE ((_col1 - 1)) END) (type: double), _col2 (type: double), UDFToDouble(_col3) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
- Statistics: Num rows: 1 Data size: 1232 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 1232 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 516 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -135,10 +135,10 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest1
POSTHOOK: Lineage: dest1.c1 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
-POSTHOOK: Lineage: dest1.c10 EXPRESSION [(src)src.null, ]
+POSTHOOK: Lineage: dest1.c10 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c11 EXPRESSION [(src)src.null, ]
POSTHOOK: Lineage: dest1.c2 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
-POSTHOOK: Lineage: dest1.c3 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: dest1.c3 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), (src)src.null, ]
POSTHOOK: Lineage: dest1.c4 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c5 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c6 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
@@ -153,4 +153,4 @@ POSTHOOK: query: SELECT dest1.* FROM dest1
POSTHOOK: type: QUERY
POSTHOOK: Input: default@dest1
#### A masked pattern was here ####
-130091.0 260.182 256.10355987055016 98.0 0.0 142.9268095075238 143.06995106518906 20428.072876 20469.01089779559 79136.0 309.0
+130091.0 260.182 256.10355987055016 98.0 0.0 142.9268095075238 143.06995106518906 20428.072876000002 20469.010897795593 79136.0 309.0
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/spark/groupby3_map_skew.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/groupby3_map_skew.q.out b/ql/src/test/results/clientpositive/spark/groupby3_map_skew.q.out
index 2f10160..c7c8d6c 100644
--- a/ql/src/test/results/clientpositive/spark/groupby3_map_skew.q.out
+++ b/ql/src/test/results/clientpositive/spark/groupby3_map_skew.q.out
@@ -51,12 +51,12 @@ STAGE PLANS:
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: substr(value, 5) (type: string)
- outputColumnNames: _col0
+ expressions: value (type: string)
+ outputColumnNames: value
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- aggregations: sum(_col0), avg(_col0), avg(DISTINCT _col0), max(_col0), min(_col0), std(_col0), stddev_samp(_col0), variance(_col0), var_samp(_col0)
- keys: _col0 (type: string)
+ aggregations: sum(substr(value, 5)), avg(substr(value, 5)), avg(DISTINCT substr(value, 5)), max(substr(value, 5)), min(substr(value, 5)), std(substr(value, 5)), stddev_samp(substr(value, 5)), variance(substr(value, 5)), var_samp(substr(value, 5))
+ keys: substr(value, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/spark/groupby3_noskew.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/groupby3_noskew.q.out b/ql/src/test/results/clientpositive/spark/groupby3_noskew.q.out
index d07eea5..904fcf3 100644
--- a/ql/src/test/results/clientpositive/spark/groupby3_noskew.q.out
+++ b/ql/src/test/results/clientpositive/spark/groupby3_noskew.q.out
@@ -50,27 +50,28 @@ STAGE PLANS:
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: substr(value, 5) (type: string)
- outputColumnNames: _col0
+ expressions: substr(value, 5) (type: string), UDFToDouble(substr(value, 5)) (type: double), (UDFToDouble(substr(value, 5)) * UDFToDouble(substr(value, 5))) (type: double)
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col2 (type: double), _col1 (type: double)
Reducer 2
Reduce Operator Tree:
Group By Operator
- aggregations: sum(KEY._col0:0._col0), avg(KEY._col0:0._col0), avg(DISTINCT KEY._col0:0._col0), max(KEY._col0:0._col0), min(KEY._col0:0._col0), std(KEY._col0:0._col0), stddev_samp(KEY._col0:0._col0), variance(KEY._col0:0._col0), var_samp(KEY._col0:0._col0)
+ aggregations: sum(KEY._col0:0._col0), count(KEY._col0:0._col0), sum(DISTINCT KEY._col0:0._col0), count(DISTINCT KEY._col0:1._col0), max(KEY._col0:0._col0), min(KEY._col0:0._col0), sum(VALUE._col0), sum(VALUE._col1)
mode: complete
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 1 Data size: 424 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
+ Statistics: Num rows: 1 Data size: 416 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: double), _col1 (type: double), _col2 (type: double), UDFToDouble(_col3) (type: double), UDFToDouble(_col4) (type: double), _col5 (type: double), _col6 (type: double), _col7 (type: double), _col8 (type: double)
+ expressions: _col0 (type: double), (_col0 / _col1) (type: double), (_col2 / _col3) (type: double), UDFToDouble(_col4) (type: double), UDFToDouble(_col5) (type: double), power(((_col6 - ((_col7 * _col7) / _col1)) / _col1), 0.5) (type: double), power(((_col6 - ((_col7 * _col7) / _col1)) / CASE WHEN ((_col1 = 1L)) THEN (null) ELSE ((_col1 - 1)) END), 0.5) (type: double), ((_col6 - ((_col7 * _col7) / _col1)) / _col1) (type: double), ((_col6 - ((_col7 * _col7) / _col1)) / CASE WHEN ((_col1 = 1L)) THEN (null) ELSE ((_col1 - 1)) END) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 1 Data size: 424 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 416 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 424 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 416 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -121,7 +122,7 @@ POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest1
POSTHOOK: Lineage: dest1.c1 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c2 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
-POSTHOOK: Lineage: dest1.c3 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: dest1.c3 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), (src)src.null, ]
POSTHOOK: Lineage: dest1.c4 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c5 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c6 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
@@ -136,4 +137,4 @@ POSTHOOK: query: SELECT dest1.* FROM dest1
POSTHOOK: type: QUERY
POSTHOOK: Input: default@dest1
#### A masked pattern was here ####
-130091.0 260.182 256.10355987055016 98.0 0.0 142.92680950752379 143.06995106518903 20428.07287599999 20469.010897795582
+130091.0 260.182 256.10355987055016 98.0 0.0 142.9268095075238 143.06995106518906 20428.072876000002 20469.010897795593
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/spark/groupby3_noskew_multi_distinct.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/groupby3_noskew_multi_distinct.q.out b/ql/src/test/results/clientpositive/spark/groupby3_noskew_multi_distinct.q.out
index 8bcb912..1dc870d 100644
--- a/ql/src/test/results/clientpositive/spark/groupby3_noskew_multi_distinct.q.out
+++ b/ql/src/test/results/clientpositive/spark/groupby3_noskew_multi_distinct.q.out
@@ -54,27 +54,28 @@ STAGE PLANS:
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: substr(value, 5) (type: string)
- outputColumnNames: _col0
+ expressions: substr(value, 5) (type: string), UDFToDouble(substr(value, 5)) (type: double), (UDFToDouble(substr(value, 5)) * UDFToDouble(substr(value, 5))) (type: double)
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col2 (type: double), _col1 (type: double)
Reducer 2
Reduce Operator Tree:
Group By Operator
- aggregations: sum(KEY._col0:0._col0), avg(KEY._col0:0._col0), avg(DISTINCT KEY._col0:0._col0), max(KEY._col0:0._col0), min(KEY._col0:0._col0), std(KEY._col0:0._col0), stddev_samp(KEY._col0:0._col0), variance(KEY._col0:0._col0), var_samp(KEY._col0:0._col0), sum(DISTINCT KEY._col0:1._col0), count(DISTINCT KEY._col0:2._col0)
+ aggregations: sum(KEY._col0:0._col0), count(KEY._col0:0._col0), sum(DISTINCT KEY._col0:0._col0), count(DISTINCT KEY._col0:1._col0), max(KEY._col0:0._col0), min(KEY._col0:0._col0), sum(VALUE._col0), sum(VALUE._col1)
mode: complete
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
- Statistics: Num rows: 1 Data size: 440 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
+ Statistics: Num rows: 1 Data size: 416 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: double), _col1 (type: double), _col2 (type: double), UDFToDouble(_col3) (type: double), UDFToDouble(_col4) (type: double), _col5 (type: double), _col6 (type: double), _col7 (type: double), _col8 (type: double), _col9 (type: double), UDFToDouble(_col10) (type: double)
+ expressions: _col0 (type: double), (_col0 / _col1) (type: double), (_col2 / _col3) (type: double), UDFToDouble(_col4) (type: double), UDFToDouble(_col5) (type: double), power(((_col6 - ((_col7 * _col7) / _col1)) / _col1), 0.5) (type: double), power(((_col6 - ((_col7 * _col7) / _col1)) / CASE WHEN ((_col1 = 1L)) THEN (null) ELSE ((_col1 - 1)) END), 0.5) (type: double), ((_col6 - ((_col7 * _col7) / _col1)) / _col1) (type: double), ((_col6 - ((_col7 * _col7) / _col1)) / CASE WHEN ((_col1 = 1L)) THEN (null) ELSE ((_col1 - 1)) END) (type: double), _col2 (type: double), UDFToDouble(_col3) (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10
- Statistics: Num rows: 1 Data size: 440 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 416 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 440 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 416 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -128,10 +129,10 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest1
POSTHOOK: Lineage: dest1.c1 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
-POSTHOOK: Lineage: dest1.c10 EXPRESSION [(src)src.null, ]
+POSTHOOK: Lineage: dest1.c10 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c11 EXPRESSION [(src)src.null, ]
POSTHOOK: Lineage: dest1.c2 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
-POSTHOOK: Lineage: dest1.c3 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: dest1.c3 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), (src)src.null, ]
POSTHOOK: Lineage: dest1.c4 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c5 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
POSTHOOK: Lineage: dest1.c6 EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
@@ -146,4 +147,4 @@ POSTHOOK: query: SELECT dest1.* FROM dest1
POSTHOOK: type: QUERY
POSTHOOK: Input: default@dest1
#### A masked pattern was here ####
-130091.0 260.182 256.10355987055016 98.0 0.0 142.92680950752379 143.06995106518903 20428.07287599999 20469.010897795582 79136.0 309.0
+130091.0 260.182 256.10355987055016 98.0 0.0 142.9268095075238 143.06995106518906 20428.072876000002 20469.010897795593 79136.0 309.0
http://git-wip-us.apache.org/repos/asf/hive/blob/5cb8867b/ql/src/test/results/clientpositive/spark/limit_pushdown.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/limit_pushdown.q.out b/ql/src/test/results/clientpositive/spark/limit_pushdown.q.out
index 82ba94a..19d28c5 100644
--- a/ql/src/test/results/clientpositive/spark/limit_pushdown.q.out
+++ b/ql/src/test/results/clientpositive/spark/limit_pushdown.q.out
@@ -283,10 +283,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- aggregations: avg(_col1)
+ aggregations: sum(_col1), count(_col1)
keys: _col0 (type: string)
mode: hash
- outputColumnNames: _col0, _col1
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
@@ -294,25 +294,29 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
TopN Hash Memory Usage: 0.3
- value expressions: _col1 (type: struct<count:bigint,sum:double,input:double>)
+ value expressions: _col1 (type: double), _col2 (type: bigint)
Reducer 2
Reduce Operator Tree:
Group By Operator
- aggregations: avg(VALUE._col0)
+ aggregations: sum(VALUE._col0), count(VALUE._col1)
keys: KEY._col0 (type: string)
mode: mergepartial
- outputColumnNames: _col0, _col1
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Limit
- Number of rows: 20
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
+ Select Operator
+ expressions: _col0 (type: string), (_col1 / _col2) (type: double)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Limit
+ Number of rows: 20
Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator