You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by br...@apache.org on 2014/11/29 04:44:28 UTC
svn commit: r1642395 [2/22] - in /hive/branches/spark/ql/src:
java/org/apache/hadoop/hive/ql/exec/spark/
java/org/apache/hadoop/hive/ql/exec/spark/session/
test/results/clientpositive/ test/results/clientpositive/spark/
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join18_multi_distinct.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join18_multi_distinct.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join18_multi_distinct.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join18_multi_distinct.q.out Sat Nov 29 03:44:22 2014
@@ -34,8 +34,8 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 2 <- Map 1 (GROUP PARTITION-LEVEL SORT, 1)
- Reducer 6 <- Map 5 (GROUP, 1)
+ Reducer 2 <- Map 1 (GROUP, 1)
+ Reducer 6 <- Map 5 (GROUP PARTITION-LEVEL SORT, 1)
Reducer 3 <- Reducer 2 (PARTITION-LEVEL SORT, 1), Reducer 6 (PARTITION-LEVEL SORT, 1)
Reducer 4 <- Reducer 3 (GROUP, 1)
#### A masked pattern was here ####
@@ -43,26 +43,6 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: src2
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: key, value
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(DISTINCT value), count(DISTINCT key)
- keys: key (type: string), value (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string)
- sort order: ++
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- Map 5
- Map Operator Tree:
- TableScan
alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
@@ -81,24 +61,40 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint)
+ Map 5
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: key, value
+ Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count(DISTINCT value), count(DISTINCT key)
+ keys: key (type: string), value (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string)
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
Reducer 2
Reduce Operator Tree:
Group By Operator
- aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0)
+ aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 12 Data size: 91 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: bigint), _col2 (type: bigint)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 12 Data size: 91 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 12 Data size: 91 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint), _col2 (type: bigint)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
Reducer 3
Reduce Operator Tree:
Join Operator
@@ -108,20 +104,16 @@ STAGE PLANS:
0 {KEY.reducesinkkey0} {VALUE._col0}
1 {KEY.reducesinkkey0} {VALUE._col0} {VALUE._col1}
outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: bigint), _col2 (type: string), _col3 (type: bigint), _col4 (type: bigint)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(hash(_col0,_col1,_col2,_col3,_col4))
- mode: hash
- outputColumnNames: _col0
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: sum(hash(_col0,_col1,_col2,_col3,_col4))
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ value expressions: _col0 (type: bigint)
Reducer 4
Reduce Operator Tree:
Group By Operator
@@ -143,21 +135,17 @@ STAGE PLANS:
Reducer 6
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
+ aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: bigint)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint), _col2 (type: bigint)
Stage: Stage-0
Fetch Operator
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join19.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join19.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join19.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join19.q.out Sat Nov 29 03:44:22 2014
@@ -27,7 +27,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
alias: src2
@@ -49,7 +49,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -68,7 +68,7 @@ STAGE PLANS:
1 key (type: string)
outputColumnNames: _col0, _col8
input vertices:
- 1 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col8 (type: string)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join2.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join2.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join2.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join2.q.out Sat Nov 29 03:44:22 2014
@@ -26,7 +26,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
alias: src2
@@ -48,7 +48,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -67,7 +67,7 @@ STAGE PLANS:
1 key (type: string)
outputColumnNames: _col0, _col5
input vertices:
- 1 Map 1
+ 1 Map 2
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (_col0 + _col5) is not null (type: boolean)
@@ -86,7 +86,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -105,7 +105,7 @@ STAGE PLANS:
1 UDFToDouble(key) (type: double)
outputColumnNames: _col0, _col11
input vertices:
- 0 Map 3
+ 0 Map 1
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col11 (type: string)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join20.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join20.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join20.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join20.q.out Sat Nov 29 03:44:22 2014
@@ -24,18 +24,18 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 4
+ Map 1
Map Operator Tree:
TableScan
- alias: src2
+ alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {key} {value}
- 1 {value}
+ 0 {value}
+ 1 {key} {value}
2 {key} {value}
filter predicates:
0
@@ -47,18 +47,18 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 5
+ Map 2
Map Operator Tree:
TableScan
- alias: src1
+ alias: src2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {value}
- 1 {key} {value}
+ 0 {key} {value}
+ 1 {value}
2 {key} {value}
filter predicates:
0
@@ -74,11 +74,11 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 2 <- Map 1 (SORT, 1)
- Reducer 3 <- Reducer 2 (GROUP, 1)
+ Reducer 4 <- Map 3 (SORT, 1)
+ Reducer 5 <- Reducer 4 (GROUP, 1)
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -101,8 +101,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 5
- 1 Map 4
+ 0 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -114,7 +114,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 2
+ Reducer 4
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -129,7 +129,7 @@ STAGE PLANS:
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
- Reducer 3
+ Reducer 5
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
@@ -199,18 +199,18 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 4
+ Map 1
Map Operator Tree:
TableScan
- alias: src2
+ alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((key < 10) and (key < 15)) (type: boolean)
+ predicate: ((key < 15) and (key < 10)) (type: boolean)
Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {key} {value}
- 1 {value}
+ 0 {value}
+ 1 {key} {value}
2 {key} {value}
filter predicates:
0
@@ -222,18 +222,18 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 5
+ Map 2
Map Operator Tree:
TableScan
- alias: src1
+ alias: src2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((key < 15) and (key < 10)) (type: boolean)
+ predicate: ((key < 10) and (key < 15)) (type: boolean)
Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {value}
- 1 {key} {value}
+ 0 {key} {value}
+ 1 {value}
2 {key} {value}
filter predicates:
0
@@ -249,11 +249,11 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 2 <- Map 1 (SORT, 3)
- Reducer 3 <- Reducer 2 (GROUP, 1)
+ Reducer 4 <- Map 3 (SORT, 3)
+ Reducer 5 <- Reducer 4 (GROUP, 1)
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -276,8 +276,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 5
- 1 Map 4
+ 0 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -289,7 +289,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 2
+ Reducer 4
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -304,7 +304,7 @@ STAGE PLANS:
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
- Reducer 3
+ Reducer 5
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join21.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join21.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join21.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join21.q.out Sat Nov 29 03:44:22 2014
@@ -17,6 +17,26 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
+ alias: src1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {value}
+ 1 {key} {value}
+ 2 {key} {value}
+ filter predicates:
+ 0 {(key < 10)}
+ 1
+ 2 {(key < 10)}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ 2 key (type: string)
+ Local Work:
+ Map Reduce Local Work
+ Map 2
+ Map Operator Tree:
+ TableScan
alias: src2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
@@ -37,34 +57,14 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 4
- Map Operator Tree:
- TableScan
- alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Spark HashTable Sink Operator
- condition expressions:
- 0 {value}
- 1 {key} {value}
- 2 {key} {value}
- filter predicates:
- 0 {(key < 10)}
- 1
- 2 {(key < 10)}
- keys:
- 0 key (type: string)
- 1 key (type: string)
- 2 key (type: string)
- Local Work:
- Map Reduce Local Work
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (SORT, 1)
+ Reducer 4 <- Map 3 (SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -87,8 +87,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 1 Map 1
+ 0 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -100,7 +100,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 4
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join22.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join22.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join22.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join22.q.out Sat Nov 29 03:44:22 2014
@@ -14,7 +14,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src2
@@ -52,10 +52,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (GROUP, 1)
+ Reducer 2 <- Map 1 (GROUP, 1)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -74,7 +74,7 @@ STAGE PLANS:
1 key (type: string)
outputColumnNames: _col0, _col1
input vertices:
- 1 Map 1
+ 1 Map 3
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
@@ -108,7 +108,7 @@ STAGE PLANS:
value expressions: _col0 (type: bigint)
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 2
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join23.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join23.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join23.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join23.q.out Sat Nov 29 03:44:22 2014
@@ -14,7 +14,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src2
@@ -35,10 +35,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (SORT, 1)
+ Reducer 2 <- Map 1 (SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -57,7 +57,7 @@ STAGE PLANS:
1
outputColumnNames: _col0, _col1, _col5, _col6
input vertices:
- 1 Map 1
+ 1 Map 3
Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string)
@@ -69,7 +69,7 @@ STAGE PLANS:
Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 2
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join24.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join24.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join24.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join24.q.out Sat Nov 29 03:44:22 2014
@@ -34,7 +34,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: b
@@ -55,10 +55,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (GROUP, 1)
+ Reducer 2 <- Map 1 (GROUP, 1)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 1
Map Operator Tree:
TableScan
alias: a
@@ -77,24 +77,20 @@ STAGE PLANS:
1 key (type: string)
outputColumnNames: _col1
input vertices:
- 1 Map 1
+ 1 Map 3
Statistics: Num rows: 170 Data size: 817 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: int)
- outputColumnNames: _col1
- Statistics: Num rows: 170 Data size: 817 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col1)
- mode: hash
- outputColumnNames: _col0
+ Group By Operator
+ aggregations: sum(_col1)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ value expressions: _col0 (type: bigint)
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 2
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join26.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join26.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join26.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join26.q.out Sat Nov 29 03:44:22 2014
@@ -29,7 +29,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: x
@@ -50,10 +50,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 2 <- Map 1 (GROUP, 1)
+ Reducer 3 <- Map 2 (GROUP, 1)
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
alias: y
@@ -72,27 +72,23 @@ STAGE PLANS:
1 key (type: string)
outputColumnNames: _col0
input vertices:
- 0 Map 3
+ 0 Map 1
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
+ Group By Operator
+ aggregations: count(1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ value expressions: _col1 (type: bigint)
Local Work:
Map Reduce Local Work
- Reducer 2
+ Reducer 3
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join27.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join27.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join27.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join27.q.out Sat Nov 29 03:44:22 2014
@@ -30,38 +30,16 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 2 <- Map 1 (GROUP, 1)
- Reducer 4 <- Map 7 (PARTITION-LEVEL SORT, 1), Union 3 (PARTITION-LEVEL SORT, 1)
- Union 3 <- Map 6 (NONE, 0), Reducer 2 (NONE, 0)
- Reducer 5 <- Reducer 4 (GROUP, 1)
+ Reducer 3 <- Map 7 (PARTITION-LEVEL SORT, 1), Union 2 (PARTITION-LEVEL SORT, 1)
+ Reducer 6 <- Map 5 (GROUP, 1)
+ Reducer 4 <- Reducer 3 (GROUP, 1)
+ Union 2 <- Map 1 (NONE, 0), Reducer 6 (NONE, 0)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key < 200) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: key, value
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: key (type: string), value (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string)
- sort order: ++
- Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- Map 6
- Map Operator Tree:
- TableScan
- alias: src
Filter Operator
predicate: (key < 200) (type: boolean)
Select Operator
@@ -71,6 +49,24 @@ STAGE PLANS:
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
+ Map 5
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (key < 200) (type: boolean)
+ Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: key (type: string), value (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string)
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
+ Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Map 7
Map Operator Tree:
TableScan
@@ -88,20 +84,7 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- Reducer 2
- Reduce Operator Tree:
- Group By Operator
- keys: KEY._col0 (type: string), KEY._col1 (type: string)
- mode: mergepartial
- outputColumnNames: _col0, _col1
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Reducer 4
+ Reducer 3
Reduce Operator Tree:
Join Operator
condition map:
@@ -110,18 +93,16 @@ STAGE PLANS:
0
1
Statistics: Num rows: 273 Data size: 2908 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 273 Data size: 2908 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- mode: hash
- outputColumnNames: _col0
+ Group By Operator
+ aggregations: count(1)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
- Reducer 5
+ value expressions: _col0 (type: bigint)
+ Reducer 4
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -139,8 +120,21 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Union 3
- Vertex: Union 3
+ Reducer 6
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: string), KEY._col1 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Select Operator
+ expressions: _col0 (type: string)
+ outputColumnNames: _col0
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Union 2
+ Vertex: Union 2
Stage: Stage-0
Fetch Operator
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join28.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join28.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join28.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join28.q.out Sat Nov 29 03:44:22 2014
@@ -17,6 +17,26 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
+ alias: src1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {value}
+ 1 {key} {value}
+ 2 {key} {value}
+ filter predicates:
+ 0 {(key < 10)}
+ 1
+ 2 {(key < 10)}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ 2 key (type: string)
+ Local Work:
+ Map Reduce Local Work
+ Map 2
+ Map Operator Tree:
+ TableScan
alias: src2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
@@ -37,34 +57,14 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 4
- Map Operator Tree:
- TableScan
- alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Spark HashTable Sink Operator
- condition expressions:
- 0 {value}
- 1 {key} {value}
- 2 {key} {value}
- filter predicates:
- 0 {(key < 10)}
- 1
- 2 {(key < 10)}
- keys:
- 0 key (type: string)
- 1 key (type: string)
- 2 key (type: string)
- Local Work:
- Map Reduce Local Work
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (SORT, 1)
+ Reducer 4 <- Map 3 (SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -87,8 +87,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 1 Map 1
+ 0 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -100,7 +100,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 4
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -136,7 +136,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src2
@@ -159,7 +159,7 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 2
+ Map 4
Map Operator Tree:
TableScan
alias: src3
@@ -186,10 +186,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 4 <- Map 3 (SORT, 1)
+ Reducer 2 <- Map 1 (SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -212,8 +212,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 1 Map 1
- 2 Map 2
+ 1 Map 3
+ 2 Map 4
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -225,7 +225,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 4
+ Reducer 2
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -261,19 +261,19 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
- alias: src3
+ alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {key} {value}
+ 0 {value}
1 {key} {value}
- 2 {value}
+ 2 {key} {value}
filter predicates:
0
1 {(key > 10)}
@@ -287,16 +287,16 @@ STAGE PLANS:
Map 4
Map Operator Tree:
TableScan
- alias: src1
+ alias: src3
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {value}
+ 0 {key} {value}
1 {key} {value}
- 2 {key} {value}
+ 2 {value}
filter predicates:
0
1 {(key > 10)}
@@ -311,10 +311,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 2 <- Map 1 (SORT, 1)
+ Reducer 3 <- Map 2 (SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
alias: src2
@@ -337,8 +337,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 2 Map 3
+ 0 Map 1
+ 2 Map 4
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -350,7 +350,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 2
+ Reducer 3
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -389,26 +389,6 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Spark HashTable Sink Operator
- condition expressions:
- 0 {key} {value}
- 1 {value}
- 2 {key} {value}
- filter predicates:
- 0
- 1 {(key > 10)}
- 2 {(key < 10)}
- keys:
- 0 key (type: string)
- 1 key (type: string)
- 2 key (type: string)
- Local Work:
- Map Reduce Local Work
- Map 4
- Map Operator Tree:
- TableScan
alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
@@ -429,14 +409,34 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {value}
+ 2 {key} {value}
+ filter predicates:
+ 0
+ 1 {(key > 10)}
+ 2 {(key < 10)}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ 2 key (type: string)
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (SORT, 1)
+ Reducer 4 <- Map 3 (SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -459,8 +459,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 1 Map 1
+ 0 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -472,7 +472,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 4
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join29.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join29.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join29.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join29.q.out Sat Nov 29 03:44:22 2014
@@ -17,6 +17,26 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
+ alias: src1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {value}
+ 1 {key} {value}
+ 2 {key} {value}
+ filter predicates:
+ 0 {(key < 10)}
+ 1
+ 2 {(key < 10)}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ 2 key (type: string)
+ Local Work:
+ Map Reduce Local Work
+ Map 2
+ Map Operator Tree:
+ TableScan
alias: src2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
@@ -37,34 +57,14 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 4
- Map Operator Tree:
- TableScan
- alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Spark HashTable Sink Operator
- condition expressions:
- 0 {value}
- 1 {key} {value}
- 2 {key} {value}
- filter predicates:
- 0 {(key < 10)}
- 1
- 2 {(key < 10)}
- keys:
- 0 key (type: string)
- 1 key (type: string)
- 2 key (type: string)
- Local Work:
- Map Reduce Local Work
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (SORT, 1)
+ Reducer 4 <- Map 3 (SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -87,8 +87,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 1 Map 1
+ 0 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -100,7 +100,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 4
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -644,7 +644,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src2
@@ -667,7 +667,7 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 2
+ Map 4
Map Operator Tree:
TableScan
alias: src3
@@ -694,10 +694,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 4 <- Map 3 (SORT, 3)
+ Reducer 2 <- Map 1 (SORT, 3)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -720,8 +720,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 1 Map 1
- 2 Map 2
+ 1 Map 3
+ 2 Map 4
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -733,7 +733,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 4
+ Reducer 2
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -1277,19 +1277,19 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
- alias: src3
+ alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {key} {value}
+ 0 {value}
1 {key} {value}
- 2 {value}
+ 2 {key} {value}
filter predicates:
0
1 {(key > 10)}
@@ -1303,16 +1303,16 @@ STAGE PLANS:
Map 4
Map Operator Tree:
TableScan
- alias: src1
+ alias: src3
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {value}
+ 0 {key} {value}
1 {key} {value}
- 2 {key} {value}
+ 2 {value}
filter predicates:
0
1 {(key > 10)}
@@ -1327,10 +1327,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 2 <- Map 1 (SORT, 3)
+ Reducer 3 <- Map 2 (SORT, 3)
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
alias: src2
@@ -1353,8 +1353,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 2 Map 3
+ 0 Map 1
+ 2 Map 4
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -1366,7 +1366,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 2
+ Reducer 3
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -1925,26 +1925,6 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Spark HashTable Sink Operator
- condition expressions:
- 0 {key} {value}
- 1 {value}
- 2 {key} {value}
- filter predicates:
- 0
- 1 {(key > 10)}
- 2 {(key < 10)}
- keys:
- 0 key (type: string)
- 1 key (type: string)
- 2 key (type: string)
- Local Work:
- Map Reduce Local Work
- Map 4
- Map Operator Tree:
- TableScan
alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
@@ -1965,14 +1945,34 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {value}
+ 2 {key} {value}
+ filter predicates:
+ 0
+ 1 {(key > 10)}
+ 2 {(key < 10)}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ 2 key (type: string)
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (SORT, 3)
+ Reducer 4 <- Map 3 (SORT, 3)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -1995,8 +1995,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 1 Map 1
+ 0 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -2008,7 +2008,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 4
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -2564,7 +2564,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src2
@@ -2583,7 +2583,7 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 2
+ Map 4
Map Operator Tree:
TableScan
alias: src3
@@ -2606,10 +2606,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 4 <- Map 3 (SORT, 3)
+ Reducer 2 <- Map 1 (SORT, 3)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -2631,8 +2631,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 1 Map 1
- 2 Map 2
+ 1 Map 3
+ 2 Map 4
Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -2644,7 +2644,7 @@ STAGE PLANS:
Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 4
+ Reducer 2
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -2691,15 +2691,15 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: src2
+ alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((key < 10) and (key > 10)) (type: boolean)
+ predicate: ((key > 10) and (key < 10)) (type: boolean)
Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {key} {value}
- 1 {value}
+ 0 {value}
+ 1 {key} {value}
2 {key} {value}
filter predicates:
0
@@ -2711,18 +2711,18 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 4
+ Map 2
Map Operator Tree:
TableScan
- alias: src1
+ alias: src2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((key > 10) and (key < 10)) (type: boolean)
+ predicate: ((key < 10) and (key > 10)) (type: boolean)
Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {value}
- 1 {key} {value}
+ 0 {key} {value}
+ 1 {value}
2 {key} {value}
filter predicates:
0
@@ -2738,10 +2738,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 3 <- Map 2 (SORT, 3)
+ Reducer 4 <- Map 3 (SORT, 3)
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -2764,8 +2764,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 1 Map 1
+ 0 Map 1
+ 1 Map 2
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -2777,7 +2777,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 3
+ Reducer 4
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -3321,7 +3321,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src2
@@ -3344,7 +3344,7 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 2
+ Map 4
Map Operator Tree:
TableScan
alias: src3
@@ -3371,10 +3371,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 4 <- Map 3 (SORT, 3)
+ Reducer 2 <- Map 1 (SORT, 3)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -3397,8 +3397,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 1 Map 1
- 2 Map 2
+ 1 Map 3
+ 2 Map 4
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -3410,7 +3410,7 @@ STAGE PLANS:
Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 4
+ Reducer 2
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -3454,19 +3454,19 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
- alias: src3
+ alias: src1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {key} {value}
+ 0 {value}
1 {key} {value}
- 2 {value}
+ 2 {key} {value}
filter predicates:
0
1 {(key > 10)}
@@ -3480,16 +3480,16 @@ STAGE PLANS:
Map 4
Map Operator Tree:
TableScan
- alias: src1
+ alias: src3
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
condition expressions:
- 0 {value}
+ 0 {key} {value}
1 {key} {value}
- 2 {key} {value}
+ 2 {value}
filter predicates:
0
1 {(key > 10)}
@@ -3504,10 +3504,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 2 <- Map 1 (SORT, 3)
+ Reducer 3 <- Map 2 (SORT, 3)
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
alias: src2
@@ -3533,8 +3533,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 0 Map 4
- 2 Map 3
+ 0 Map 1
+ 2 Map 4
Statistics: Num rows: 365 Data size: 3878 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -3546,7 +3546,7 @@ STAGE PLANS:
Statistics: Num rows: 365 Data size: 3878 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 2
+ Reducer 3
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
@@ -3612,7 +3612,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 3
Map Operator Tree:
TableScan
alias: src2
@@ -3631,7 +3631,7 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 2
+ Map 4
Map Operator Tree:
TableScan
alias: src3
@@ -3654,10 +3654,10 @@ STAGE PLANS:
Stage: Stage-1
Spark
Edges:
- Reducer 4 <- Map 3 (SORT, 3)
+ Reducer 2 <- Map 1 (SORT, 3)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -3679,8 +3679,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
input vertices:
- 1 Map 1
- 2 Map 2
+ 1 Map 3
+ 2 Map 4
Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
@@ -3692,7 +3692,7 @@ STAGE PLANS:
Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
- Reducer 4
+ Reducer 2
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join3.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join3.q.out?rev=1642395&r1=1642394&r2=1642395&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join3.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/auto_join3.q.out Sat Nov 29 03:44:22 2014
@@ -25,7 +25,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
alias: src2
@@ -44,7 +44,7 @@ STAGE PLANS:
2 key (type: string)
Local Work:
Map Reduce Local Work
- Map 2
+ Map 3
Map Operator Tree:
TableScan
alias: src3
@@ -68,7 +68,7 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src1
@@ -90,8 +90,8 @@ STAGE PLANS:
2 key (type: string)
outputColumnNames: _col0, _col11
input vertices:
- 1 Map 1
- 2 Map 2
+ 1 Map 2
+ 2 Map 3
Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col11 (type: string)