You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by xu...@apache.org on 2014/11/25 14:03:23 UTC
svn commit: r1641614 [19/43] - in /hive/branches/spark:
itests/src/test/resources/
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/
ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/
ql/src/java/org/apache/hadoop/hive/ql/parse/spark/ ql/sr...
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/join_merge_multi_expressions.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/join_merge_multi_expressions.q.out?rev=1641614&r1=1641613&r2=1641614&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/join_merge_multi_expressions.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/join_merge_multi_expressions.q.out Tue Nov 25 13:03:18 2014
@@ -5,15 +5,13 @@ POSTHOOK: query: explain
select count(*) from srcpart a join srcpart b on a.key = b.key and a.hr = b.hr join srcpart c on a.hr = c.hr and a.key = c.key
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 4 (PARTITION-LEVEL SORT, 1), Map 5 (PARTITION-LEVEL SORT, 1)
- Reducer 3 <- Reducer 2 (GROUP, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -24,12 +22,18 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string), hr (type: string)
- sort order: ++
- Map-reduce partition columns: key (type: string), hr (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
- Map 4
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0
+ 1
+ 2
+ keys:
+ 0 key (type: string), hr (type: string)
+ 1 key (type: string), hr (type: string)
+ 2 key (type: string), hr (type: string)
+ Local Work:
+ Map Reduce Local Work
+ Map 2
Map Operator Tree:
TableScan
alias: c
@@ -37,12 +41,25 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string), hr (type: string)
- sort order: ++
- Map-reduce partition columns: key (type: string), hr (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
- Map 5
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0
+ 1
+ 2
+ keys:
+ 0 key (type: string), hr (type: string)
+ 1 key (type: string), hr (type: string)
+ 2 key (type: string), hr (type: string)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+ Edges:
+ Reducer 4 <- Map 3 (GROUP, 1)
+#### A masked pattern was here ####
+ Vertices:
+ Map 3
Map Operator Tree:
TableScan
alias: a
@@ -50,34 +67,36 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string), hr (type: string)
- sort order: ++
- Map-reduce partition columns: key (type: string), hr (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- Inner Join 0 to 2
- condition expressions:
- 0
- 1
- 2
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
- Reducer 3
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Inner Join 0 to 2
+ condition expressions:
+ 0
+ 1
+ 2
+ keys:
+ 0 key (type: string), hr (type: string)
+ 1 key (type: string), hr (type: string)
+ 2 key (type: string), hr (type: string)
+ input vertices:
+ 1 Map 1
+ 2 Map 2
+ Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Local Work:
+ Map Reduce Local Work
+ Reducer 4
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/join_merging.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/join_merging.q.out?rev=1641614&r1=1641613&r2=1641614&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/join_merging.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/join_merging.q.out Tue Nov 25 13:03:18 2014
@@ -9,38 +9,32 @@ from part p1 left outer join part p2 on
p1.p_size > 10
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1), Map 4 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: p3
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: p_partkey (type: int)
- sort order: +
- Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Map 3
+ Map 2
Map Operator Tree:
TableScan
alias: p2
Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: p_partkey (type: int)
- sort order: +
- Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- value expressions: p_size (type: int)
- Map 4
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {p_size}
+ 1 {p_size}
+ 2
+ keys:
+ 0 p_partkey (type: int)
+ 1 p_partkey (type: int)
+ 2 p_partkey (type: int)
+ Local Work:
+ Map Reduce Local Work
+ Map 3
Map Operator Tree:
TableScan
alias: p1
@@ -48,35 +42,57 @@ STAGE PLANS:
Filter Operator
predicate: (p_size > 10) (type: boolean)
Statistics: Num rows: 8 Data size: 968 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: p_partkey (type: int)
- sort order: +
- Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 8 Data size: 968 Basic stats: COMPLETE Column stats: NONE
- value expressions: p_size (type: int)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Left Outer Join0 to 1
- Right Outer Join1 to 2
- condition expressions:
- 0 {VALUE._col4}
- 1 {VALUE._col4}
- 2
- outputColumnNames: _col5, _col17
- Statistics: Num rows: 57 Data size: 6923 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col5 (type: int), _col17 (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 57 Data size: 6923 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {p_size}
+ 1 {p_size}
+ 2
+ keys:
+ 0 p_partkey (type: int)
+ 1 p_partkey (type: int)
+ 2 p_partkey (type: int)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: p3
+ Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ Right Outer Join1 to 2
+ condition expressions:
+ 0 {p_size}
+ 1 {p_size}
+ 2
+ keys:
+ 0 p_partkey (type: int)
+ 1 p_partkey (type: int)
+ 2 p_partkey (type: int)
+ outputColumnNames: _col5, _col17
+ input vertices:
+ 0 Map 3
+ 1 Map 2
Statistics: Num rows: 57 Data size: 6923 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ expressions: _col5 (type: int), _col17 (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 57 Data size: 6923 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 57 Data size: 6923 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -95,38 +111,32 @@ from part p1 left outer join part p2 on
p1.p_size > 10 and p1.p_size > p2.p_size + 10
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1), Map 4 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: p3
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: p_partkey (type: int)
- sort order: +
- Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Map 3
+ Map 2
Map Operator Tree:
TableScan
alias: p2
Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: p_partkey (type: int)
- sort order: +
- Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- value expressions: p_size (type: int)
- Map 4
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {p_size}
+ 1 {p_size}
+ 2
+ keys:
+ 0 p_partkey (type: int)
+ 1 p_partkey (type: int)
+ 2 p_partkey (type: int)
+ Local Work:
+ Map Reduce Local Work
+ Map 3
Map Operator Tree:
TableScan
alias: p1
@@ -134,38 +144,60 @@ STAGE PLANS:
Filter Operator
predicate: (p_size > 10) (type: boolean)
Statistics: Num rows: 8 Data size: 968 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: p_partkey (type: int)
- sort order: +
- Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 8 Data size: 968 Basic stats: COMPLETE Column stats: NONE
- value expressions: p_size (type: int)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Left Outer Join0 to 1
- Right Outer Join1 to 2
- condition expressions:
- 0 {VALUE._col4}
- 1 {VALUE._col4}
- 2
- outputColumnNames: _col5, _col17
- Statistics: Num rows: 57 Data size: 6923 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (_col5 > (_col17 + 10)) (type: boolean)
- Statistics: Num rows: 19 Data size: 2307 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col5 (type: int), _col17 (type: int)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 19 Data size: 2307 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {p_size}
+ 1 {p_size}
+ 2
+ keys:
+ 0 p_partkey (type: int)
+ 1 p_partkey (type: int)
+ 2 p_partkey (type: int)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: p3
+ Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ Right Outer Join1 to 2
+ condition expressions:
+ 0 {p_size}
+ 1 {p_size}
+ 2
+ keys:
+ 0 p_partkey (type: int)
+ 1 p_partkey (type: int)
+ 2 p_partkey (type: int)
+ outputColumnNames: _col5, _col17
+ input vertices:
+ 0 Map 3
+ 1 Map 2
+ Statistics: Num rows: 57 Data size: 6923 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (_col5 > (_col17 + 10)) (type: boolean)
Statistics: Num rows: 19 Data size: 2307 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ expressions: _col5 (type: int), _col17 (type: int)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 19 Data size: 2307 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 19 Data size: 2307 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/join_nullsafe.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/join_nullsafe.q.out?rev=1641614&r1=1641613&r2=1641614&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/join_nullsafe.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/join_nullsafe.q.out Tue Nov 25 13:03:18 2014
@@ -25,14 +25,13 @@ POSTHOOK: query: -- merging
explain select * from myinput1 a join myinput1 b on a.key<=>b.value
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -40,45 +39,52 @@ STAGE PLANS:
TableScan
alias: b
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: int)
- sort order: +
- Map-reduce partition columns: value (type: int)
- Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: int)
- Map 3
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {key}
+ keys:
+ 0 key (type: int)
+ 1 value (type: int)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 2
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: int)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- condition expressions:
- 0 {KEY.reducesinkkey0} {VALUE._col0}
- 1 {VALUE._col0} {KEY.reducesinkkey0}
- nullSafes: [true]
- outputColumnNames: _col0, _col1, _col5, _col6
- Statistics: Num rows: 3 Data size: 28 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int)
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 3 Data size: 28 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ keys:
+ 0 key (type: int)
+ 1 value (type: int)
+ nullSafes: [true]
+ outputColumnNames: _col0, _col1, _col5, _col6
+ input vertices:
+ 1 Map 1
Statistics: Num rows: 3 Data size: 28 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 3 Data size: 28 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 3 Data size: 28 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -110,14 +116,13 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain select * from myinput1 a join myinput1 b on a.key<=>b.value join myinput1 c on a.key=c.key
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1), Map 4 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -128,13 +133,18 @@ STAGE PLANS:
Filter Operator
predicate: value is not null (type: boolean)
Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: int)
- sort order: +
- Map-reduce partition columns: value (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: int)
- Map 3
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {key}
+ 2 {key} {value}
+ keys:
+ 0 key (type: int)
+ 1 value (type: int)
+ 2 key (type: int)
+ Local Work:
+ Map Reduce Local Work
+ Map 2
Map Operator Tree:
TableScan
alias: c
@@ -142,13 +152,23 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: int)
- Map 4
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ 2 {value}
+ keys:
+ 0 key (type: int)
+ 1 value (type: int)
+ 2 key (type: int)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 3
Map Operator Tree:
TableScan
alias: a
@@ -156,35 +176,36 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: int)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- Inner Join 0 to 2
- condition expressions:
- 0 {KEY.reducesinkkey0} {VALUE._col0}
- 1 {VALUE._col0} {KEY.reducesinkkey0}
- 2 {KEY.reducesinkkey0} {VALUE._col0}
- outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Inner Join 0 to 2
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ 2 {key} {value}
+ keys:
+ 0 key (type: int)
+ 1 value (type: int)
+ 2 key (type: int)
+ outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
+ input vertices:
+ 1 Map 1
+ 2 Map 2
+ Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -207,14 +228,13 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain select * from myinput1 a join myinput1 b on a.key<=>b.value join myinput1 c on a.key<=>c.key
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1), Map 4 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -222,58 +242,74 @@ STAGE PLANS:
TableScan
alias: b
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: int)
- sort order: +
- Map-reduce partition columns: value (type: int)
- Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: int)
- Map 3
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {key}
+ 2 {key} {value}
+ keys:
+ 0 key (type: int)
+ 1 value (type: int)
+ 2 key (type: int)
+ Local Work:
+ Map Reduce Local Work
+ Map 2
Map Operator Tree:
TableScan
alias: c
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: int)
- Map 4
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ 2 {value}
+ keys:
+ 0 key (type: int)
+ 1 value (type: int)
+ 2 key (type: int)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: int)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- Inner Join 0 to 2
- condition expressions:
- 0 {KEY.reducesinkkey0} {VALUE._col0}
- 1 {VALUE._col0} {KEY.reducesinkkey0}
- 2 {KEY.reducesinkkey0} {VALUE._col0}
- nullSafes: [true]
- outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Inner Join 0 to 2
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ 2 {key} {value}
+ keys:
+ 0 key (type: int)
+ 1 value (type: int)
+ 2 key (type: int)
+ nullSafes: [true]
+ outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
+ input vertices:
+ 1 Map 1
+ 2 Map 2
Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -323,14 +359,13 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain select * from myinput1 a join myinput1 b on a.key<=>b.value AND a.value=b.key join myinput1 c on a.key<=>c.key AND a.value=c.value
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1), Map 4 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -341,12 +376,18 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: int), key (type: int)
- sort order: ++
- Map-reduce partition columns: value (type: int), key (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Map 3
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1
+ 2 {key} {value}
+ keys:
+ 0 key (type: int), value (type: int)
+ 1 value (type: int), key (type: int)
+ 2 key (type: int), value (type: int)
+ Local Work:
+ Map Reduce Local Work
+ Map 2
Map Operator Tree:
TableScan
alias: c
@@ -354,12 +395,23 @@ STAGE PLANS:
Filter Operator
predicate: value is not null (type: boolean)
Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int), value (type: int)
- sort order: ++
- Map-reduce partition columns: key (type: int), value (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Map 4
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ 2
+ keys:
+ 0 key (type: int), value (type: int)
+ 1 value (type: int), key (type: int)
+ 2 key (type: int), value (type: int)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 3
Map Operator Tree:
TableScan
alias: a
@@ -367,35 +419,37 @@ STAGE PLANS:
Filter Operator
predicate: value is not null (type: boolean)
Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int), value (type: int)
- sort order: ++
- Map-reduce partition columns: key (type: int), value (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- Inner Join 0 to 2
- condition expressions:
- 0 {KEY.reducesinkkey0} {KEY.reducesinkkey1}
- 1 {KEY.reducesinkkey1} {KEY.reducesinkkey0}
- 2 {KEY.reducesinkkey0} {KEY.reducesinkkey1}
- nullSafes: [true, false]
- outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Inner Join 0 to 2
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ 2 {key} {value}
+ keys:
+ 0 key (type: int), value (type: int)
+ 1 value (type: int), key (type: int)
+ 2 key (type: int), value (type: int)
+ nullSafes: [true, false]
+ outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
+ input vertices:
+ 1 Map 1
+ 2 Map 2
+ Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -418,14 +472,13 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain select * from myinput1 a join myinput1 b on a.key<=>b.value AND a.value<=>b.key join myinput1 c on a.key<=>c.key AND a.value<=>c.value
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1), Map 4 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -433,55 +486,74 @@ STAGE PLANS:
TableScan
alias: b
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: int), key (type: int)
- sort order: ++
- Map-reduce partition columns: value (type: int), key (type: int)
- Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Map 3
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1
+ 2 {key} {value}
+ keys:
+ 0 key (type: int), value (type: int)
+ 1 value (type: int), key (type: int)
+ 2 key (type: int), value (type: int)
+ Local Work:
+ Map Reduce Local Work
+ Map 2
Map Operator Tree:
TableScan
alias: c
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int), value (type: int)
- sort order: ++
- Map-reduce partition columns: key (type: int), value (type: int)
- Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Map 4
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ 2
+ keys:
+ 0 key (type: int), value (type: int)
+ 1 value (type: int), key (type: int)
+ 2 key (type: int), value (type: int)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int), value (type: int)
- sort order: ++
- Map-reduce partition columns: key (type: int), value (type: int)
- Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- Inner Join 0 to 2
- condition expressions:
- 0 {KEY.reducesinkkey0} {KEY.reducesinkkey1}
- 1 {KEY.reducesinkkey1} {KEY.reducesinkkey0}
- 2 {KEY.reducesinkkey0} {KEY.reducesinkkey1}
- nullSafes: [true, true]
- outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Inner Join 0 to 2
+ condition expressions:
+ 0 {key} {value}
+ 1 {key} {value}
+ 2 {key} {value}
+ keys:
+ 0 key (type: int), value (type: int)
+ 1 value (type: int), key (type: int)
+ 2 key (type: int), value (type: int)
+ nullSafes: [true, true]
+ outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
+ input vertices:
+ 1 Map 1
+ 2 Map 2
Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -1557,14 +1629,13 @@ POSTHOOK: query: --HIVE-3315 join predic
explain select * from myinput1 a join myinput1 b on a.key<=>b.value AND a.key is NULL
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1575,12 +1646,21 @@ STAGE PLANS:
Filter Operator
predicate: value is null (type: boolean)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: null (type: void)
- sort order: +
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: int)
- Map 3
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {value}
+ 1 {key}
+ keys:
+ 0 null (type: void)
+ 1 null (type: void)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 2
Map Operator Tree:
TableScan
alias: a
@@ -1588,33 +1668,33 @@ STAGE PLANS:
Filter Operator
predicate: key is null (type: boolean)
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: null (type: void)
- sort order: +
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ condition expressions:
+ 0 {value}
+ 1 {key}
+ keys:
+ 0 null (type: void)
+ 1 null (type: void)
+ nullSafes: [true]
+ outputColumnNames: _col1, _col5
+ input vertices:
+ 1 Map 1
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: int)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- condition expressions:
- 0 {VALUE._col0}
- 1 {VALUE._col0}
- nullSafes: [true]
- outputColumnNames: _col1, _col5
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: null (type: void), _col1 (type: int), _col5 (type: int), null (type: void)
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ expressions: null (type: void), _col1 (type: int), _col5 (type: int), null (type: void)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/join_rc.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/join_rc.q.out?rev=1641614&r1=1641613&r2=1641614&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/join_rc.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/join_rc.q.out Tue Nov 25 13:03:18 2014
@@ -47,14 +47,13 @@ select join_rc1.key, join_rc2.value
FROM join_rc1 JOIN join_rc2 ON join_rc1.key = join_rc2.key
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-3
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -65,13 +64,21 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2406 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 250 Data size: 2406 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: string)
- Map 3
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key}
+ 1 {value}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 2
Map Operator Tree:
TableScan
alias: join_rc1
@@ -79,32 +86,32 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2406 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 250 Data size: 2406 Basic stats: COMPLETE Column stats: NONE
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- condition expressions:
- 0 {KEY.reducesinkkey0}
- 1 {VALUE._col0}
- outputColumnNames: _col0, _col6
- Statistics: Num rows: 275 Data size: 2646 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col6 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2646 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 275 Data size: 2646 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ condition expressions:
+ 0 {key}
+ 1 {value}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ outputColumnNames: _col0, _col6
+ input vertices:
+ 1 Map 1
+ Statistics: Num rows: 275 Data size: 2646 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col6 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 275 Data size: 2646 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 275 Data size: 2646 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/join_reorder.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/join_reorder.q.out?rev=1641614&r1=1641613&r2=1641614&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/join_reorder.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/join_reorder.q.out Tue Nov 25 13:03:18 2014
@@ -53,14 +53,35 @@ POSTHOOK: query: EXPLAIN FROM T1 a JOIN
SELECT a.key, a.val, c.key
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
+ Stage: Stage-3
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
+ Filter Operator
+ predicate: UDFToDouble(key) is not null (type: boolean)
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {val}
+ 1 {key}
+ keys:
+ 0 UDFToDouble(key) (type: double)
+ 1 (key + 1) (type: double)
+ Local Work:
+ Map Reduce Local Work
+
Stage: Stage-1
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -71,47 +92,32 @@ STAGE PLANS:
Filter Operator
predicate: (key + 1) is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: (key + 1) (type: double)
- sort order: +
- Map-reduce partition columns: (key + 1) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: string)
- Map 3
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: UDFToDouble(key) is not null (type: boolean)
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(key) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(key) (type: double)
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
- value expressions: key (type: string), val (type: string)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- condition expressions:
- 0 {VALUE._col0} {VALUE._col1}
- 1 {VALUE._col0}
- outputColumnNames: _col0, _col1, _col5
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ condition expressions:
+ 0 {key} {val}
+ 1 {key}
+ keys:
+ 0 UDFToDouble(key) (type: double)
+ 1 (key + 1) (type: double)
+ outputColumnNames: _col0, _col1, _col5
+ input vertices:
+ 0 Map 2
+ Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -126,14 +132,35 @@ POSTHOOK: query: EXPLAIN FROM T1 a JOIN
SELECT /*+ STREAMTABLE(a) */ a.key, a.val, c.key
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-3 is a root stage
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
+ Stage: Stage-3
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
+ Filter Operator
+ predicate: UDFToDouble(key) is not null (type: boolean)
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {val}
+ 1 {key}
+ keys:
+ 0 UDFToDouble(key) (type: double)
+ 1 (key + 1) (type: double)
+ Local Work:
+ Map Reduce Local Work
+
Stage: Stage-1
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 3 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -144,47 +171,32 @@ STAGE PLANS:
Filter Operator
predicate: (key + 1) is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: (key + 1) (type: double)
- sort order: +
- Map-reduce partition columns: (key + 1) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: string)
- Map 3
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: UDFToDouble(key) is not null (type: boolean)
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(key) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(key) (type: double)
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
- value expressions: key (type: string), val (type: string)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Inner Join 0 to 1
- condition expressions:
- 0 {VALUE._col0} {VALUE._col1}
- 1 {VALUE._col0}
- outputColumnNames: _col0, _col1, _col5
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ condition expressions:
+ 0 {key} {val}
+ 1 {key}
+ keys:
+ 0 UDFToDouble(key) (type: double)
+ 1 (key + 1) (type: double)
+ outputColumnNames: _col0, _col1, _col5
+ input vertices:
+ 0 Map 2
+ Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -235,15 +247,14 @@ POSTHOOK: query: EXPLAIN FROM T1 a
SELECT a.key, b.key, a.val, c.val
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-4 is a root stage
+ Stage-3 depends on stages: Stage-4
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-4
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 5 (PARTITION-LEVEL SORT, 1)
- Reducer 3 <- Map 4 (PARTITION-LEVEL SORT, 1), Reducer 2 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -251,69 +262,83 @@ STAGE PLANS:
TableScan
alias: b
Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- Map 4
- Map Operator Tree:
- TableScan
- alias: c
- Statistics: Num rows: 0 Data size: 20 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: val (type: string)
- sort order: +
- Map-reduce partition columns: val (type: string)
- Statistics: Num rows: 0 Data size: 20 Basic stats: PARTIAL Column stats: NONE
- Map 5
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {val}
+ 1
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-3
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- value expressions: val (type: string)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Left Outer Join0 to 1
- condition expressions:
- 0 {KEY.reducesinkkey0} {VALUE._col0}
- 1 {KEY.reducesinkkey0}
- outputColumnNames: _col0, _col1, _col5
- Statistics: Num rows: 0 Data size: 33 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col1 (type: string)
- sort order: +
- Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 0 Data size: 33 Basic stats: PARTIAL Column stats: NONE
- value expressions: _col0 (type: string), _col5 (type: string)
- Reducer 3
- Reduce Operator Tree:
- Join Operator
- condition map:
- Right Outer Join0 to 1
- condition expressions:
- 0 {VALUE._col0} {KEY.reducesinkkey0} {VALUE._col4}
- 1 {KEY.reducesinkkey0}
- outputColumnNames: _col0, _col1, _col5, _col11
- Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col5 (type: string), _col1 (type: string), _col11 (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
+ Map Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ condition expressions:
+ 0 {key} {val}
+ 1 {key}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ outputColumnNames: _col0, _col1, _col5
+ input vertices:
+ 1 Map 1
+ Statistics: Num rows: 0 Data size: 33 Basic stats: PARTIAL Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {_col0} {_col5}
+ 1 {val}
+ keys:
+ 0 _col1 (type: string)
+ 1 val (type: string)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: c
+ Statistics: Num rows: 0 Data size: 20 Basic stats: PARTIAL Column stats: NONE
+ Map Join Operator
+ condition map:
+ Right Outer Join0 to 1
+ condition expressions:
+ 0 {_col0} {_col1} {_col5}
+ 1 {val}
+ keys:
+ 0 _col1 (type: string)
+ 1 val (type: string)
+ outputColumnNames: _col0, _col1, _col5, _col11
+ input vertices:
+ 0 Map 3
Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ expressions: _col0 (type: string), _col5 (type: string), _col1 (type: string), _col11 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator
@@ -332,15 +357,14 @@ POSTHOOK: query: EXPLAIN FROM T1 a
SELECT /*+ STREAMTABLE(a) */ a.key, b.key, a.val, c.val
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
+ Stage-4 is a root stage
+ Stage-3 depends on stages: Stage-4
+ Stage-1 depends on stages: Stage-3
Stage-0 depends on stages: Stage-1
STAGE PLANS:
- Stage: Stage-1
+ Stage: Stage-4
Spark
- Edges:
- Reducer 2 <- Map 1 (PARTITION-LEVEL SORT, 1), Map 5 (PARTITION-LEVEL SORT, 1)
- Reducer 3 <- Map 4 (PARTITION-LEVEL SORT, 1), Reducer 2 (PARTITION-LEVEL SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -348,69 +372,83 @@ STAGE PLANS:
TableScan
alias: b
Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- Map 4
- Map Operator Tree:
- TableScan
- alias: c
- Statistics: Num rows: 0 Data size: 20 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: val (type: string)
- sort order: +
- Map-reduce partition columns: val (type: string)
- Statistics: Num rows: 0 Data size: 20 Basic stats: PARTIAL Column stats: NONE
- Map 5
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {key} {val}
+ 1
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-3
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 0 Data size: 30 Basic stats: PARTIAL Column stats: NONE
- value expressions: val (type: string)
- Reducer 2
- Reduce Operator Tree:
- Join Operator
- condition map:
- Left Outer Join0 to 1
- condition expressions:
- 0 {KEY.reducesinkkey0} {VALUE._col0}
- 1 {KEY.reducesinkkey0}
- outputColumnNames: _col0, _col1, _col5
- Statistics: Num rows: 0 Data size: 33 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col1 (type: string)
- sort order: +
- Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 0 Data size: 33 Basic stats: PARTIAL Column stats: NONE
- value expressions: _col0 (type: string), _col5 (type: string)
- Reducer 3
- Reduce Operator Tree:
- Join Operator
- condition map:
- Right Outer Join0 to 1
- condition expressions:
- 0 {VALUE._col0} {KEY.reducesinkkey0} {VALUE._col4}
- 1 {KEY.reducesinkkey0}
- outputColumnNames: _col0, _col1, _col5, _col11
- Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col5 (type: string), _col1 (type: string), _col11 (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
+ Map Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ condition expressions:
+ 0 {key} {val}
+ 1 {key}
+ keys:
+ 0 key (type: string)
+ 1 key (type: string)
+ outputColumnNames: _col0, _col1, _col5
+ input vertices:
+ 1 Map 1
+ Statistics: Num rows: 0 Data size: 33 Basic stats: PARTIAL Column stats: NONE
+ Spark HashTable Sink Operator
+ condition expressions:
+ 0 {_col0} {_col5}
+ 1 {val}
+ keys:
+ 0 _col1 (type: string)
+ 1 val (type: string)
+ Local Work:
+ Map Reduce Local Work
+
+ Stage: Stage-1
+ Spark
+#### A masked pattern was here ####
+ Vertices:
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: c
+ Statistics: Num rows: 0 Data size: 20 Basic stats: PARTIAL Column stats: NONE
+ Map Join Operator
+ condition map:
+ Right Outer Join0 to 1
+ condition expressions:
+ 0 {_col0} {_col1} {_col5}
+ 1 {val}
+ keys:
+ 0 _col1 (type: string)
+ 1 val (type: string)
+ outputColumnNames: _col0, _col1, _col5, _col11
+ input vertices:
+ 0 Map 3
Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ expressions: _col0 (type: string), _col5 (type: string), _col1 (type: string), _col11 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 0 Data size: 36 Basic stats: PARTIAL Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Local Work:
+ Map Reduce Local Work
Stage: Stage-0
Fetch Operator