You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by se...@apache.org on 2017/10/13 00:07:07 UTC
[16/23] hive git commit: HIVE-17726: Using exists may lead to
incorrect results (Vineet Garg, reviewed by Ashutosh Chauhan)
http://git-wip-us.apache.org/repos/asf/hive/blob/9375cf3c/ql/src/test/results/clientpositive/subquery_in_having.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/subquery_in_having.q.out b/ql/src/test/results/clientpositive/subquery_in_having.q.out
index 8ccc185..6893442 100644
--- a/ql/src/test/results/clientpositive/subquery_in_having.q.out
+++ b/ql/src/test/results/clientpositive/subquery_in_having.q.out
@@ -1776,14 +1776,11 @@ POSTHOOK: type: QUERY
POSTHOOK: Output: default@src_null
POSTHOOK: Lineage: src_null.key SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
POSTHOOK: Lineage: src_null.value SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
-Warning: Map Join MAPJOIN[146][bigTable=?] in task 'Stage-18:MAPRED' is a cross product
-Warning: Shuffle Join JOIN[11][tables = [$hdt$_0, $hdt$_1]] in Stage 'Stage-1:MAPRED' is a cross product
-Warning: Map Join MAPJOIN[147][bigTable=?] in task 'Stage-7:MAPRED' is a cross product
-Warning: Map Join MAPJOIN[148][bigTable=?] in task 'Stage-21:MAPRED' is a cross product
-Warning: Shuffle Join JOIN[32][tables = [$hdt$_2, $hdt$_3]] in Stage 'Stage-8:MAPRED' is a cross product
-Warning: Map Join MAPJOIN[149][bigTable=?] in task 'Stage-12:MAPRED' is a cross product
-Warning: Map Join MAPJOIN[150][bigTable=?] in task 'Stage-24:MAPRED' is a cross product
-Warning: Shuffle Join JOIN[72][tables = [$hdt$_3, $hdt$_4]] in Stage 'Stage-13:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[185][bigTable=?] in task 'Stage-1:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[186][bigTable=?] in task 'Stage-8:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[187][bigTable=?] in task 'Stage-9:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[188][bigTable=?] in task 'Stage-14:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[189][bigTable=?] in task 'Stage-15:MAPRED' is a cross product
PREHOOK: query: explain
select key, value, count(*)
from src_null b
@@ -1799,44 +1796,39 @@ group by key, value
having count(*) not in (select count(*) from src_null s1 where s1.key > '9' and s1.value <> b.value group by s1.key )
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-5 is a root stage
- Stage-19 depends on stages: Stage-5 , consists of Stage-27, Stage-1
- Stage-27 has a backup stage: Stage-1
- Stage-18 depends on stages: Stage-27
- Stage-2 depends on stages: Stage-1, Stage-18
- Stage-17 depends on stages: Stage-2, Stage-7 , consists of Stage-26, Stage-3
+ Stage-6 is a root stage
+ Stage-27 depends on stages: Stage-6
+ Stage-1 depends on stages: Stage-27
+ Stage-2 depends on stages: Stage-1
+ Stage-20 depends on stages: Stage-2, Stage-8 , consists of Stage-26, Stage-3
Stage-26 has a backup stage: Stage-3
- Stage-16 depends on stages: Stage-26
- Stage-4 depends on stages: Stage-3, Stage-12, Stage-16
+ Stage-19 depends on stages: Stage-26
+ Stage-4 depends on stages: Stage-3, Stage-14, Stage-19
Stage-3
- Stage-1
- Stage-10 is a root stage
- Stage-22 depends on stages: Stage-10 , consists of Stage-29, Stage-8
- Stage-29 has a backup stage: Stage-8
- Stage-21 depends on stages: Stage-29
- Stage-9 depends on stages: Stage-8, Stage-21
- Stage-28 depends on stages: Stage-9
- Stage-7 depends on stages: Stage-28
- Stage-8
- Stage-15 is a root stage
- Stage-25 depends on stages: Stage-15 , consists of Stage-31, Stage-13
- Stage-31 has a backup stage: Stage-13
- Stage-24 depends on stages: Stage-31
- Stage-14 depends on stages: Stage-13, Stage-24
- Stage-30 depends on stages: Stage-14
- Stage-12 depends on stages: Stage-30
- Stage-13
+ Stage-12 is a root stage
+ Stage-29 depends on stages: Stage-12
+ Stage-9 depends on stages: Stage-29
+ Stage-10 depends on stages: Stage-9
+ Stage-28 depends on stages: Stage-10
+ Stage-8 depends on stages: Stage-28
+ Stage-18 is a root stage
+ Stage-31 depends on stages: Stage-18
+ Stage-15 depends on stages: Stage-31
+ Stage-16 depends on stages: Stage-15
+ Stage-30 depends on stages: Stage-16
+ Stage-14 depends on stages: Stage-30
Stage-0 depends on stages: Stage-4
STAGE PLANS:
- Stage: Stage-5
+ Stage: Stage-6
Map Reduce
Map Operator Tree:
TableScan
- alias: src_null
+ alias: b
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: value is not null (type: boolean)
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: value
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: value (type: string)
@@ -1854,35 +1846,34 @@ STAGE PLANS:
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), true (type: boolean)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-19
- Conditional Operator
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-27
Map Reduce Local Work
Alias -> Map Local Tables:
- $hdt$_0:$INTNAME
+ $hdt$_0:$hdt$_1:$hdt$_1:src_null
Fetch Operator
limit: -1
Alias -> Map Local Operator Tree:
- $hdt$_0:$INTNAME
+ $hdt$_0:$hdt$_1:$hdt$_1:src_null
TableScan
- HashTable Sink Operator
- keys:
- 0
- 1
+ alias: src_null
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ HashTable Sink Operator
+ keys:
+ 0
+ 1
- Stage: Stage-18
+ Stage: Stage-1
Map Reduce
Map Operator Tree:
TableScan
@@ -1892,36 +1883,110 @@ STAGE PLANS:
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Left Outer Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3
- residual filter predicates: {(_col1 <> _col2)}
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string)
+ TableScan
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (_col0 <> _col1) (type: boolean)
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col3 is null (type: boolean)
+ Select Operator
+ expressions: _col1 (type: string)
+ outputColumnNames: _col1
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
+ Group By Operator
+ keys: _col1 (type: string)
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- keys: _col1 (type: string), _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Reduce Operator Tree:
+ Demux Operator
+ Statistics: Num rows: 2 Data size: 17461 Basic stats: COMPLETE Column stats: NONE
+ Mux Operator
+ Statistics: Num rows: 3 Data size: 26191 Basic stats: COMPLETE Column stats: NONE
+ Join Operator
+ condition map:
+ Left Outer Join 0 to 1
+ keys:
+ 0 _col1 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Filter Operator
+ predicate: _col3 is null (type: boolean)
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col1 (type: string), _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ Group By Operator
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8730 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), true (type: boolean)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 8730 Basic stats: COMPLETE Column stats: NONE
+ Mux Operator
+ Statistics: Num rows: 3 Data size: 26191 Basic stats: COMPLETE Column stats: NONE
+ Join Operator
+ condition map:
+ Left Outer Join 0 to 1
+ keys:
+ 0 _col1 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Filter Operator
+ predicate: _col3 is null (type: boolean)
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col1 (type: string), _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-2
Map Reduce
@@ -1931,7 +1996,7 @@ STAGE PLANS:
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
value expressions: _col2 (type: bigint)
Reduce Operator Tree:
Group By Operator
@@ -1939,11 +2004,11 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col1 (type: string), _col0 (type: string), _col2 (type: bigint)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: false
table:
@@ -1951,7 +2016,7 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-17
+ Stage: Stage-20
Conditional Operator
Stage: Stage-26
@@ -1968,7 +2033,7 @@ STAGE PLANS:
0 _col1 (type: string)
1 _col0 (type: string)
- Stage: Stage-16
+ Stage: Stage-19
Map Reduce
Map Operator Tree:
TableScan
@@ -1979,7 +2044,7 @@ STAGE PLANS:
0 _col1 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4, _col5
- Statistics: Num rows: 1 Data size: 12805 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: false
table:
@@ -1997,19 +2062,19 @@ STAGE PLANS:
key expressions: _col1 (type: string), _col2 (type: bigint)
sort order: ++
Map-reduce partition columns: _col1 (type: string), _col2 (type: bigint)
- Statistics: Num rows: 1 Data size: 12805 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
value expressions: _col0 (type: string), _col4 (type: bigint), _col5 (type: bigint)
TableScan
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: bigint)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint)
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Reduce Operator Tree:
Demux Operator
- Statistics: Num rows: 2 Data size: 30267 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Mux Operator
- Statistics: Num rows: 3 Data size: 45400 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 8731 Basic stats: COMPLETE Column stats: NONE
Join Operator
condition map:
Left Outer Join 0 to 1
@@ -2036,20 +2101,20 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: bigint)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 15133 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 2910 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: bigint), _col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 15133 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 2910 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 15133 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 2910 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: bigint), _col1 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 15133 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 2910 Basic stats: COMPLETE Column stats: NONE
Mux Operator
- Statistics: Num rows: 3 Data size: 45400 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 8731 Basic stats: COMPLETE Column stats: NONE
Join Operator
condition map:
Left Outer Join 0 to 1
@@ -2081,14 +2146,14 @@ STAGE PLANS:
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
value expressions: _col0 (type: string), _col2 (type: bigint)
TableScan
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reduce Operator Tree:
Join Operator
@@ -2098,7 +2163,7 @@ STAGE PLANS:
0 _col1 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4, _col5
- Statistics: Num rows: 1 Data size: 12805 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: false
table:
@@ -2106,63 +2171,15 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-1
+ Stage: Stage-12
Map Reduce
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: string)
- TableScan
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: boolean)
- Reduce Operator Tree:
- Join Operator
- condition map:
- Left Outer Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3
- residual filter predicates: {(_col1 <> _col2)}
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col3 is null (type: boolean)
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- keys: _col1 (type: string), _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-10
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: src_null
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: value is not null (type: boolean)
+ expressions: value (type: string)
+ outputColumnNames: value
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: value (type: string)
@@ -2180,35 +2197,34 @@ STAGE PLANS:
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), true (type: boolean)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-22
- Conditional Operator
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-29
Map Reduce Local Work
Alias -> Map Local Tables:
- $hdt$_1:$hdt$_1:$hdt$_2:$hdt$_2:$INTNAME
+ $hdt$_1:$hdt$_1:$hdt$_2:$hdt$_2:$hdt$_3:$hdt$_3:src_null
Fetch Operator
limit: -1
Alias -> Map Local Operator Tree:
- $hdt$_1:$hdt$_1:$hdt$_2:$hdt$_2:$INTNAME
+ $hdt$_1:$hdt$_1:$hdt$_2:$hdt$_2:$hdt$_3:$hdt$_3:src_null
TableScan
- HashTable Sink Operator
- keys:
- 0
- 1
+ alias: src_null
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ HashTable Sink Operator
+ keys:
+ 0
+ 1
- Stage: Stage-21
+ Stage: Stage-9
Map Reduce
Map Operator Tree:
TableScan
@@ -2218,37 +2234,110 @@ STAGE PLANS:
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Left Outer Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3
- residual filter predicates: {(_col1 <> _col2)}
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string)
+ TableScan
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (_col0 <> _col1) (type: boolean)
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col3 is null (type: boolean)
+ Select Operator
+ expressions: _col1 (type: string)
+ outputColumnNames: _col1
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
+ Group By Operator
+ keys: _col1 (type: string)
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: _col1 (type: string), _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Reduce Operator Tree:
+ Demux Operator
+ Statistics: Num rows: 2 Data size: 17461 Basic stats: COMPLETE Column stats: NONE
+ Mux Operator
+ Statistics: Num rows: 3 Data size: 26191 Basic stats: COMPLETE Column stats: NONE
+ Join Operator
+ condition map:
+ Left Outer Join 0 to 1
+ keys:
+ 0 _col1 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Filter Operator
+ predicate: _col3 is null (type: boolean)
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Group By Operator
+ keys: _col1 (type: string), _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ Group By Operator
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8730 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), true (type: boolean)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 8730 Basic stats: COMPLETE Column stats: NONE
+ Mux Operator
+ Statistics: Num rows: 3 Data size: 26191 Basic stats: COMPLETE Column stats: NONE
+ Join Operator
+ condition map:
+ Left Outer Join 0 to 1
+ keys:
+ 0 _col1 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Filter Operator
+ predicate: _col3 is null (type: boolean)
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Group By Operator
+ keys: _col1 (type: string), _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-9
+ Stage: Stage-10
Map Reduce
Map Operator Tree:
TableScan
@@ -2256,22 +2345,22 @@ STAGE PLANS:
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Group By Operator
keys: _col1 (type: string)
mode: complete
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: false
table:
@@ -2302,7 +2391,7 @@ STAGE PLANS:
0
1
- Stage: Stage-7
+ Stage: Stage-8
Map Reduce
Map Operator Tree:
TableScan
@@ -2313,25 +2402,25 @@ STAGE PLANS:
0
1
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (_col1 <> _col2) (type: boolean)
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: _col2 (type: string), _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
value expressions: _col2 (type: bigint)
Local Work:
Map Reduce Local Work
@@ -2341,17 +2430,17 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col2 (type: bigint)
outputColumnNames: _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), count(_col2)
keys: _col1 (type: string)
mode: complete
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -2359,62 +2448,15 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-8
+ Stage: Stage-18
Map Reduce
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: string)
- TableScan
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: boolean)
- Reduce Operator Tree:
- Join Operator
- condition map:
- Left Outer Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3
- residual filter predicates: {(_col1 <> _col2)}
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col3 is null (type: boolean)
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: _col1 (type: string), _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-15
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: src_null
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: value is not null (type: boolean)
+ expressions: value (type: string)
+ outputColumnNames: value
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: value (type: string)
@@ -2432,35 +2474,34 @@ STAGE PLANS:
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), true (type: boolean)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
- Stage: Stage-25
- Conditional Operator
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-31
Map Reduce Local Work
Alias -> Map Local Tables:
- $hdt$_2:$hdt$_2:$hdt$_2:$hdt$_2:$hdt$_3:$hdt$_3:$INTNAME
+ $hdt$_2:$hdt$_2:$hdt$_2:$hdt$_2:$hdt$_3:$hdt$_3:$hdt$_4:$hdt$_4:src_null
Fetch Operator
limit: -1
Alias -> Map Local Operator Tree:
- $hdt$_2:$hdt$_2:$hdt$_2:$hdt$_2:$hdt$_3:$hdt$_3:$INTNAME
+ $hdt$_2:$hdt$_2:$hdt$_2:$hdt$_2:$hdt$_3:$hdt$_3:$hdt$_4:$hdt$_4:src_null
TableScan
- HashTable Sink Operator
- keys:
- 0
- 1
+ alias: src_null
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ HashTable Sink Operator
+ keys:
+ 0
+ 1
- Stage: Stage-24
+ Stage: Stage-15
Map Reduce
Map Operator Tree:
TableScan
@@ -2470,37 +2511,110 @@ STAGE PLANS:
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Left Outer Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3
- residual filter predicates: {(_col1 <> _col2)}
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string)
+ TableScan
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (_col0 <> _col1) (type: boolean)
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col3 is null (type: boolean)
+ Select Operator
+ expressions: _col1 (type: string)
+ outputColumnNames: _col1
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
+ Group By Operator
+ keys: _col1 (type: string)
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: _col1 (type: string), _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Local Work:
Map Reduce Local Work
+ Reduce Operator Tree:
+ Demux Operator
+ Statistics: Num rows: 2 Data size: 17461 Basic stats: COMPLETE Column stats: NONE
+ Mux Operator
+ Statistics: Num rows: 3 Data size: 26191 Basic stats: COMPLETE Column stats: NONE
+ Join Operator
+ condition map:
+ Left Outer Join 0 to 1
+ keys:
+ 0 _col1 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Filter Operator
+ predicate: _col3 is null (type: boolean)
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Group By Operator
+ keys: _col1 (type: string), _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ Group By Operator
+ keys: KEY._col0 (type: string)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8730 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), true (type: boolean)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 8730 Basic stats: COMPLETE Column stats: NONE
+ Mux Operator
+ Statistics: Num rows: 3 Data size: 26191 Basic stats: COMPLETE Column stats: NONE
+ Join Operator
+ condition map:
+ Left Outer Join 0 to 1
+ keys:
+ 0 _col1 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col3
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Filter Operator
+ predicate: _col3 is null (type: boolean)
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Group By Operator
+ keys: _col1 (type: string), _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-14
+ Stage: Stage-16
Map Reduce
Map Operator Tree:
TableScan
@@ -2508,22 +2622,22 @@ STAGE PLANS:
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Group By Operator
keys: _col1 (type: string)
mode: complete
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: false
table:
@@ -2554,7 +2668,7 @@ STAGE PLANS:
0
1
- Stage: Stage-12
+ Stage: Stage-14
Map Reduce
Map Operator Tree:
TableScan
@@ -2565,25 +2679,25 @@ STAGE PLANS:
0
1
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (_col1 <> _col2) (type: boolean)
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: _col2 (type: string), _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
value expressions: _col2 (type: bigint)
Local Work:
Map Reduce Local Work
@@ -2593,16 +2707,16 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col2 (type: bigint)
outputColumnNames: _col1, _col2
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col1 (type: string), _col2 (type: bigint)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 17462 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 5821 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -2610,68 +2724,17 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
- Stage: Stage-13
- Map Reduce
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: string)
- TableScan
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 5820 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: boolean)
- Reduce Operator Tree:
- Join Operator
- condition map:
- Left Outer Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col2, _col3
- residual filter predicates: {(_col1 <> _col2)}
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col3 is null (type: boolean)
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: _col1 (type: string), _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 11641 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
-Warning: Map Join MAPJOIN[146][bigTable=?] in task 'Stage-18:MAPRED' is a cross product
-Warning: Shuffle Join JOIN[11][tables = [$hdt$_0, $hdt$_1]] in Stage 'Stage-1:MAPRED' is a cross product
-Warning: Map Join MAPJOIN[147][bigTable=?] in task 'Stage-7:MAPRED' is a cross product
-Warning: Map Join MAPJOIN[148][bigTable=?] in task 'Stage-21:MAPRED' is a cross product
-Warning: Shuffle Join JOIN[32][tables = [$hdt$_2, $hdt$_3]] in Stage 'Stage-8:MAPRED' is a cross product
-Warning: Map Join MAPJOIN[149][bigTable=?] in task 'Stage-12:MAPRED' is a cross product
-Warning: Map Join MAPJOIN[150][bigTable=?] in task 'Stage-24:MAPRED' is a cross product
-Warning: Shuffle Join JOIN[72][tables = [$hdt$_3, $hdt$_4]] in Stage 'Stage-13:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[185][bigTable=?] in task 'Stage-1:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[186][bigTable=?] in task 'Stage-8:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[187][bigTable=?] in task 'Stage-9:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[188][bigTable=?] in task 'Stage-14:MAPRED' is a cross product
+Warning: Map Join MAPJOIN[189][bigTable=?] in task 'Stage-15:MAPRED' is a cross product
PREHOOK: query: select key, value, count(*)
from src_null b
where NOT EXISTS (select key from src_null where src_null.value <> b.value)
http://git-wip-us.apache.org/repos/asf/hive/blob/9375cf3c/ql/src/test/results/clientpositive/subquery_notexists.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/subquery_notexists.q.out b/ql/src/test/results/clientpositive/subquery_notexists.q.out
index a6175f8..329573e 100644
--- a/ql/src/test/results/clientpositive/subquery_notexists.q.out
+++ b/ql/src/test/results/clientpositive/subquery_notexists.q.out
@@ -29,28 +29,28 @@ STAGE PLANS:
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((value > 'val_2') and key is not null) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((key = key) and (value = value) and (value > 'val_2')) (type: boolean)
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: key (type: string), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -78,7 +78,7 @@ STAGE PLANS:
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
value expressions: _col2 (type: boolean)
Reduce Operator Tree:
Join Operator
@@ -280,37 +280,37 @@ STAGE PLANS:
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (value > 'val_2') (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((value = value) and (value > 'val_2')) (type: boolean)
+ Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: value (type: string), key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col1 (type: string)
mode: complete
outputColumnNames: _col0
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -339,7 +339,7 @@ STAGE PLANS:
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: boolean)
Reduce Operator Tree:
Join Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/9375cf3c/ql/src/test/results/clientpositive/subquery_notexists_having.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/subquery_notexists_having.q.out b/ql/src/test/results/clientpositive/subquery_notexists_having.q.out
index 0d90e1b..4d2b2fc 100644
--- a/ql/src/test/results/clientpositive/subquery_notexists_having.q.out
+++ b/ql/src/test/results/clientpositive/subquery_notexists_having.q.out
@@ -72,7 +72,7 @@ STAGE PLANS:
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
value expressions: _col2 (type: boolean)
Reduce Operator Tree:
Join Operator
@@ -105,28 +105,28 @@ STAGE PLANS:
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((value > 'val_12') and key is not null) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((key = key) and (value = value) and (value > 'val_12')) (type: boolean)
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: key (type: string), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -255,7 +255,7 @@ STAGE PLANS:
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: boolean)
Reduce Operator Tree:
Join Operator
@@ -288,37 +288,37 @@ STAGE PLANS:
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (value > 'val_12') (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((value = value) and (value > 'val_12')) (type: boolean)
+ Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: value (type: string), key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col1 (type: string)
mode: complete
outputColumnNames: _col0
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), true (type: boolean)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 212 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
http://git-wip-us.apache.org/repos/asf/hive/blob/9375cf3c/ql/src/test/results/clientpositive/subquery_notin_having.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/subquery_notin_having.q.out b/ql/src/test/results/clientpositive/subquery_notin_having.q.out
index 433609d..c321fe6 100644
--- a/ql/src/test/results/clientpositive/subquery_notin_having.q.out
+++ b/ql/src/test/results/clientpositive/subquery_notin_having.q.out
@@ -286,7 +286,7 @@ STAGE PLANS:
key expressions: _col0 (type: double)
sort order: +
Map-reduce partition columns: _col0 (type: double)
- Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 121 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reduce Operator Tree:
Join Operator
@@ -319,7 +319,7 @@ STAGE PLANS:
key expressions: _col0 (type: string), _col1 (type: double)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: double)
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
value expressions: _col2 (type: boolean)
Reduce Operator Tree:
Join Operator
@@ -375,18 +375,18 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((_col2 - _col1) > 600.0) and _col1 is not null) (type: boolean)
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ predicate: (((_col2 - _col1) > 600.0) and (_col1 = _col1)) (type: boolean)
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: double)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), count(_col0)
keys: _col1 (type: double)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -402,7 +402,7 @@ STAGE PLANS:
key expressions: _col0 (type: double)
sort order: +
Map-reduce partition columns: _col0 (type: double)
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: bigint)
Reduce Operator Tree:
Group By Operator
@@ -410,7 +410,7 @@ STAGE PLANS:
keys: KEY._col0 (type: double)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 121 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -448,19 +448,19 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((_col2 - _col1) > 600.0) and _col1 is not null) (type: boolean)
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ predicate: (((_col2 - _col1) > 600.0) and (_col1 = _col1)) (type: boolean)
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: double)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col1 (type: double), true (type: boolean)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table: