You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by xu...@apache.org on 2014/10/21 04:45:05 UTC
svn commit: r1633268 [3/7] - in /hive/branches/spark:
itests/src/test/resources/ ql/src/java/org/apache/hadoop/hive/ql/exec/
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/
ql/src/java/org/apache/hadoop/hive/ql/parse/spark/
ql/src/java/org/apache/had...
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/insert_into3.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/insert_into3.q.out?rev=1633268&r1=1633267&r2=1633268&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/insert_into3.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/insert_into3.q.out Tue Oct 21 02:45:04 2014
@@ -30,40 +30,33 @@ POSTHOOK: query: EXPLAIN FROM src INSERT
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
-#### A masked pattern was here ####
- Vertices:
- Map 3
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-4
- Spark
Edges:
- Reducer 4 <- Map 1 (GROUP SORT, 1)
+ Reducer 2 <- Map 1 (GROUP SORT, 1)
+ Reducer 3 <- Map 1 (GROUP SORT, 1)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: string)
+ sort order: ++
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
@@ -72,7 +65,7 @@ STAGE PLANS:
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Reducer 4
+ Reducer 2
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
@@ -93,6 +86,27 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert_into3a
+ Reducer 3
+ Reduce Operator Tree:
+ Select Operator
+ expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Limit
+ Number of rows: 100
+ Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.insert_into3b
Stage: Stage-3
Dependency Collection
@@ -107,7 +121,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert_into3a
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -120,47 +134,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert_into3b
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
- Edges:
- Reducer 5 <- Map 2 (GROUP SORT, 1)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string), _col1 (type: string)
- sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Reducer 5
- Reduce Operator Tree:
- Select Operator
- expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Limit
- Number of rows: 100
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.insert_into3b
+ Stats-Aggr Operator
PREHOOK: query: FROM src INSERT INTO TABLE insert_into3a SELECT * ORDER BY key, value LIMIT 50
INSERT INTO TABLE insert_into3b SELECT * ORDER BY key, value LIMIT 100
@@ -214,40 +189,36 @@ POSTHOOK: query: EXPLAIN FROM src INSERT
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
-#### A masked pattern was here ####
- Vertices:
- Map 3
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-4
- Spark
Edges:
- Reducer 4 <- Map 1 (GROUP, 1)
+ Reducer 2 <- Map 1 (GROUP, 1)
+ Reducer 3 <- Map 1 (GROUP, 1)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Limit
+ Number of rows: 10
+ Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string), _col1 (type: string)
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
@@ -259,7 +230,7 @@ STAGE PLANS:
sort order:
Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string), _col1 (type: string)
- Reducer 4
+ Reducer 2
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: string), VALUE._col1 (type: string)
@@ -280,6 +251,27 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert_into3a
+ Reducer 3
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: string), VALUE._col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Limit
+ Number of rows: 10
+ Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.insert_into3b
Stage: Stage-3
Dependency Collection
@@ -294,7 +286,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert_into3a
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -307,50 +299,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert_into3b
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
- Edges:
- Reducer 5 <- Map 2 (GROUP, 1)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Limit
- Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: string)
- Reducer 5
- Reduce Operator Tree:
- Select Operator
- expressions: VALUE._col0 (type: string), VALUE._col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
- Limit
- Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.insert_into3b
+ Stats-Aggr Operator
PREHOOK: query: FROM src INSERT OVERWRITE TABLE insert_into3a SELECT * LIMIT 10
INSERT INTO TABLE insert_into3b SELECT * LIMIT 10
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/load_dyn_part1.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/load_dyn_part1.q.out?rev=1633268&r1=1633267&r2=1633268&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/load_dyn_part1.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/load_dyn_part1.q.out Tue Oct 21 02:45:04 2014
@@ -58,38 +58,22 @@ insert overwrite table nzhang_part2 part
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: srcpart
Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (ds <= '2008-04-08') (type: boolean)
Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
@@ -105,6 +89,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.nzhang_part1
+ Filter Operator
+ predicate: (ds > '2008-04-08') (type: boolean)
+ Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string), hr (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.nzhang_part2
Stage: Stage-3
Dependency Collection
@@ -122,7 +121,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.nzhang_part1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -138,31 +137,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.nzhang_part2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: (ds > '2008-04-08') (type: boolean)
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string), hr (type: string)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.nzhang_part2
+ Stats-Aggr Operator
PREHOOK: query: from srcpart
insert overwrite table nzhang_part1 partition (ds, hr) select key, value, ds, hr where ds <= '2008-04-08'
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/load_dyn_part8.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/load_dyn_part8.q.out?rev=1633268&r1=1633267&r2=1633268&view=diff
==============================================================================
Files hive/branches/spark/ql/src/test/results/clientpositive/spark/load_dyn_part8.q.out (original) and hive/branches/spark/ql/src/test/results/clientpositive/spark/load_dyn_part8.q.out Tue Oct 21 02:45:04 2014 differ
Modified: hive/branches/spark/ql/src/test/results/clientpositive/spark/multi_insert.q.out
URL: http://svn.apache.org/viewvc/hive/branches/spark/ql/src/test/results/clientpositive/spark/multi_insert.q.out?rev=1633268&r1=1633267&r2=1633268&view=diff
==============================================================================
--- hive/branches/spark/ql/src/test/results/clientpositive/spark/multi_insert.q.out (original)
+++ hive/branches/spark/ql/src/test/results/clientpositive/spark/multi_insert.q.out Tue Oct 21 02:45:04 2014
@@ -30,38 +30,22 @@ insert overwrite table src_multi2 select
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
@@ -77,6 +61,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
+ Filter Operator
+ predicate: ((key > 10) and (key < 20)) (type: boolean)
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
Stage: Stage-3
Dependency Collection
@@ -91,7 +90,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -104,31 +103,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: ((key > 10) and (key < 20)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi2
+ Stats-Aggr Operator
PREHOOK: query: from src
insert overwrite table src_multi1 select * where key < 10
@@ -197,38 +173,22 @@ insert overwrite table src_multi2 select
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
@@ -244,6 +204,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
+ Filter Operator
+ predicate: ((key > 10) and (key < 20)) (type: boolean)
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
Stage: Stage-3
Dependency Collection
@@ -258,7 +233,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -271,31 +246,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: ((key > 10) and (key < 20)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi2
+ Stats-Aggr Operator
PREHOOK: query: from src
insert overwrite table src_multi1 select * where key < 10
@@ -364,38 +316,22 @@ insert overwrite table src_multi2 select
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
@@ -411,6 +347,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
+ Filter Operator
+ predicate: ((key > 10) and (key < 20)) (type: boolean)
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
Stage: Stage-3
Dependency Collection
@@ -425,7 +376,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -438,31 +389,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: ((key > 10) and (key < 20)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi2
+ Stats-Aggr Operator
PREHOOK: query: from src
insert overwrite table src_multi1 select * where key < 10
@@ -531,38 +459,22 @@ insert overwrite table src_multi2 select
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key < 10) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
@@ -578,6 +490,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
+ Filter Operator
+ predicate: ((key > 10) and (key < 20)) (type: boolean)
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
Stage: Stage-3
Dependency Collection
@@ -592,7 +519,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -605,31 +532,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: ((key > 10) and (key < 20)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi2
+ Stats-Aggr Operator
PREHOOK: query: from src
insert overwrite table src_multi1 select * where key < 10
@@ -1354,72 +1258,83 @@ insert overwrite table src_multi2 select
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
Edges:
- Union 4 <- Map 3 (NONE, 0), Map 5 (NONE, 0)
+ Union 2 <- Map 1 (NONE, 0), Map 3 (NONE, 0)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Map 5
+ Filter Operator
+ predicate: (_col0 < 10) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi1
+ Filter Operator
+ predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
+ Map 3
Map Operator Tree:
TableScan
alias: src
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Union 4
- Vertex: Union 4
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: (_col0 < 10) (type: boolean)
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi1
+ Filter Operator
+ predicate: (_col0 < 10) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi1
+ Filter Operator
+ predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
+ Union 2
+ Vertex: Union 2
Stage: Stage-3
Dependency Collection
@@ -1434,7 +1349,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -1447,31 +1362,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi2
+ Stats-Aggr Operator
PREHOOK: query: from (select * from src union all select * from src) s
insert overwrite table src_multi1 select * where key < 10
@@ -1559,72 +1451,83 @@ insert overwrite table src_multi2 select
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
Edges:
- Union 4 <- Map 3 (NONE, 0), Map 5 (NONE, 0)
+ Union 2 <- Map 1 (NONE, 0), Map 3 (NONE, 0)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Map 5
+ Filter Operator
+ predicate: (_col0 < 10) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi1
+ Filter Operator
+ predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
+ Map 3
Map Operator Tree:
TableScan
alias: src
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Union 4
- Vertex: Union 4
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: (_col0 < 10) (type: boolean)
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi1
+ Filter Operator
+ predicate: (_col0 < 10) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi1
+ Filter Operator
+ predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
+ Union 2
+ Vertex: Union 2
Stage: Stage-3
Dependency Collection
@@ -1639,7 +1542,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -1652,31 +1555,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi2
+ Stats-Aggr Operator
PREHOOK: query: from (select * from src union all select * from src) s
insert overwrite table src_multi1 select * where key < 10
@@ -1764,72 +1644,83 @@ insert overwrite table src_multi2 select
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
Edges:
- Union 4 <- Map 3 (NONE, 0), Map 5 (NONE, 0)
+ Union 2 <- Map 1 (NONE, 0), Map 3 (NONE, 0)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Map 5
+ Filter Operator
+ predicate: (_col0 < 10) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi1
+ Filter Operator
+ predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
+ Map 3
Map Operator Tree:
TableScan
alias: src
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Union 4
- Vertex: Union 4
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: (_col0 < 10) (type: boolean)
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi1
+ Filter Operator
+ predicate: (_col0 < 10) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi1
+ Filter Operator
+ predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
+ Union 2
+ Vertex: Union 2
Stage: Stage-3
Dependency Collection
@@ -1844,7 +1735,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -1857,31 +1748,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi2
+ Stats-Aggr Operator
PREHOOK: query: from (select * from src union all select * from src) s
insert overwrite table src_multi1 select * where key < 10
@@ -1969,72 +1837,83 @@ insert overwrite table src_multi2 select
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-2 is a root stage
- Stage-4 depends on stages: Stage-2
- Stage-3 depends on stages: Stage-4, Stage-5
+ Stage-3 depends on stages: Stage-2
Stage-0 depends on stages: Stage-3
- Stage-6 depends on stages: Stage-0
+ Stage-4 depends on stages: Stage-0
Stage-1 depends on stages: Stage-3
- Stage-7 depends on stages: Stage-1
- Stage-5 depends on stages: Stage-2
+ Stage-5 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-2
Spark
Edges:
- Union 4 <- Map 3 (NONE, 0), Map 5 (NONE, 0)
+ Union 2 <- Map 1 (NONE, 0), Map 3 (NONE, 0)
#### A masked pattern was here ####
Vertices:
- Map 3
+ Map 1
Map Operator Tree:
TableScan
alias: src
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Map 5
+ Filter Operator
+ predicate: (_col0 < 10) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi1
+ Filter Operator
+ predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
+ Map 3
Map Operator Tree:
TableScan
alias: src
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Union 4
- Vertex: Union 4
-
- Stage: Stage-4
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: (_col0 < 10) (type: boolean)
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi1
+ Filter Operator
+ predicate: (_col0 < 10) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi1
+ Filter Operator
+ predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.src_multi2
+ Union 2
+ Vertex: Union 2
Stage: Stage-3
Dependency Collection
@@ -2049,7 +1928,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi1
- Stage: Stage-6
+ Stage: Stage-4
Stats-Aggr Operator
Stage: Stage-1
@@ -2062,31 +1941,8 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.src_multi2
- Stage: Stage-7
- Stats-Aggr Operator
-
Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- Filter Operator
- predicate: ((_col0 > 10) and (_col0 < 20)) (type: boolean)
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 111 Data size: 1179 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.src_multi2
+ Stats-Aggr Operator
PREHOOK: query: from (select * from src union all select * from src) s
insert overwrite table src_multi1 select * where key < 10
@@ -2173,38 +2029,21 @@ from src
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-3 is a root stage
- Stage-5 depends on stages: Stage-3
- Stage-4 depends on stages: Stage-5, Stage-6, Stage-7
- Stage-0 depends on stages: Stage-5
- Stage-6 depends on stages: Stage-3
- Stage-1 depends on stages: Stage-6
- Stage-7 depends on stages: Stage-3
- Stage-2 depends on stages: Stage-7
+ Stage-4 depends on stages: Stage-3
+ Stage-0 depends on stages: Stage-3
+ Stage-1 depends on stages: Stage-3
+ Stage-2 depends on stages: Stage-3
STAGE PLANS:
Stage: Stage-3
Spark
#### A masked pattern was here ####
Vertices:
- Map 4
+ Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 0) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2219,23 +2058,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-4
- Dependency Collection
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: false
-#### A masked pattern was here ####
-
- Stage: Stage-6
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 2) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2250,20 +2072,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-1
- Move Operator
- files:
- hdfs directory: false
-#### A masked pattern was here ####
-
- Stage: Stage-7
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 3
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 4) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2279,6 +2087,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Stage: Stage-4
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ files:
+ hdfs directory: false
+#### A masked pattern was here ####
+
+ Stage: Stage-1
+ Move Operator
+ files:
+ hdfs directory: false
+#### A masked pattern was here ####
+
Stage: Stage-2
Move Operator
files:
@@ -2307,38 +2130,21 @@ from src
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-3 is a root stage
- Stage-5 depends on stages: Stage-3
- Stage-4 depends on stages: Stage-5, Stage-6, Stage-7
- Stage-0 depends on stages: Stage-5
- Stage-6 depends on stages: Stage-3
- Stage-1 depends on stages: Stage-6
- Stage-7 depends on stages: Stage-3
- Stage-2 depends on stages: Stage-7
+ Stage-4 depends on stages: Stage-3
+ Stage-0 depends on stages: Stage-3
+ Stage-1 depends on stages: Stage-3
+ Stage-2 depends on stages: Stage-3
STAGE PLANS:
Stage: Stage-3
Spark
#### A masked pattern was here ####
Vertices:
- Map 4
+ Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 0) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2353,23 +2159,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-4
- Dependency Collection
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: false
-#### A masked pattern was here ####
-
- Stage: Stage-6
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 2) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2384,20 +2173,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-1
- Move Operator
- files:
- hdfs directory: false
-#### A masked pattern was here ####
-
- Stage: Stage-7
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 3
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 4) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2413,6 +2188,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Stage: Stage-4
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ files:
+ hdfs directory: false
+#### A masked pattern was here ####
+
+ Stage: Stage-1
+ Move Operator
+ files:
+ hdfs directory: false
+#### A masked pattern was here ####
+
Stage: Stage-2
Move Operator
files:
@@ -2441,38 +2231,21 @@ from src
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-3 is a root stage
- Stage-5 depends on stages: Stage-3
- Stage-4 depends on stages: Stage-5, Stage-6, Stage-7
- Stage-0 depends on stages: Stage-5
- Stage-6 depends on stages: Stage-3
- Stage-1 depends on stages: Stage-6
- Stage-7 depends on stages: Stage-3
- Stage-2 depends on stages: Stage-7
+ Stage-4 depends on stages: Stage-3
+ Stage-0 depends on stages: Stage-3
+ Stage-1 depends on stages: Stage-3
+ Stage-2 depends on stages: Stage-3
STAGE PLANS:
Stage: Stage-3
Spark
#### A masked pattern was here ####
Vertices:
- Map 4
+ Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 0) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2487,23 +2260,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-4
- Dependency Collection
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: false
-#### A masked pattern was here ####
-
- Stage: Stage-6
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 2) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2518,20 +2274,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-1
- Move Operator
- files:
- hdfs directory: false
-#### A masked pattern was here ####
-
- Stage: Stage-7
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 3
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 4) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2547,6 +2289,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Stage: Stage-4
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ files:
+ hdfs directory: false
+#### A masked pattern was here ####
+
+ Stage: Stage-1
+ Move Operator
+ files:
+ hdfs directory: false
+#### A masked pattern was here ####
+
Stage: Stage-2
Move Operator
files:
@@ -2575,38 +2332,21 @@ from src
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
Stage-3 is a root stage
- Stage-5 depends on stages: Stage-3
- Stage-4 depends on stages: Stage-5, Stage-6, Stage-7
- Stage-0 depends on stages: Stage-5
- Stage-6 depends on stages: Stage-3
- Stage-1 depends on stages: Stage-6
- Stage-7 depends on stages: Stage-3
- Stage-2 depends on stages: Stage-7
+ Stage-4 depends on stages: Stage-3
+ Stage-0 depends on stages: Stage-3
+ Stage-1 depends on stages: Stage-3
+ Stage-2 depends on stages: Stage-3
STAGE PLANS:
Stage: Stage-3
Spark
#### A masked pattern was here ####
Vertices:
- Map 4
+ Map 1
Map Operator Tree:
TableScan
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-5
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 0) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2621,23 +2361,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-4
- Dependency Collection
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: false
-#### A masked pattern was here ####
-
- Stage: Stage-6
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 2) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2652,20 +2375,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-1
- Move Operator
- files:
- hdfs directory: false
-#### A masked pattern was here ####
-
- Stage: Stage-7
- Spark
-#### A masked pattern was here ####
- Vertices:
- Map 3
- Map Operator Tree:
- TableScan
Filter Operator
predicate: (key = 4) (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
@@ -2681,6 +2390,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Stage: Stage-4
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ files:
+ hdfs directory: false
+#### A masked pattern was here ####
+
+ Stage: Stage-1
+ Move Operator
+ files:
+ hdfs directory: false
+#### A masked pattern was here ####
+
Stage: Stage-2
Move Operator
files: