You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by jp...@apache.org on 2015/12/12 07:54:55 UTC
[06/51] [partial] hive git commit: HIVE-11110: Reorder
applyPreJoinOrderingTransforms, add NotNULL/FilterMerge rules,
improve Filter selectivity estimation (Laljo John Pullokkaran reviewed by
Jesus Camacho Rodriguez, Ashutosh Chauhan)
http://git-wip-us.apache.org/repos/asf/hive/blob/08f73adc/ql/src/test/results/clientpositive/spark/auto_sortmerge_join_9.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/auto_sortmerge_join_9.q.out b/ql/src/test/results/clientpositive/spark/auto_sortmerge_join_9.q.out
index 686d7ca..be42791 100644
--- a/ql/src/test/results/clientpositive/spark/auto_sortmerge_join_9.q.out
+++ b/ql/src/test/results/clientpositive/spark/auto_sortmerge_join_9.q.out
@@ -72,18 +72,18 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -163,11 +163,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
@@ -175,18 +175,18 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint)
Reducer 2
Reduce Operator Tree:
@@ -195,10 +195,10 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -275,11 +275,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
@@ -287,26 +287,26 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reducer 2
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Select Operator
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -419,11 +419,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
@@ -431,18 +431,18 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint)
Reducer 4
Local Work:
@@ -453,7 +453,7 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -472,11 +472,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
@@ -484,18 +484,18 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint)
Reducer 2
Local Work:
@@ -506,7 +506,7 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -516,14 +516,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
1 Reducer 4
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: bigint), _col3 (type: bigint)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1068,17 +1068,20 @@ STAGE PLANS:
TableScan
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: (key + 1) (type: int)
- outputColumnNames: _col0
+ Filter Operator
+ predicate: (key + 1) is not null (type: boolean)
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Spark HashTable Sink Operator
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
+ Select Operator
+ expressions: (key + 1) (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: _col0 is not null (type: boolean)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Spark HashTable Sink Operator
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
Local Work:
Map Reduce Local Work
@@ -1093,31 +1096,34 @@ STAGE PLANS:
TableScan
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: (key + 1) (type: int)
- outputColumnNames: _col0
+ Filter Operator
+ predicate: (key + 1) is not null (type: boolean)
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: (key + 1) (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: _col0 is not null (type: boolean)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Local Work:
Map Reduce Local Work
Reducer 2
@@ -1603,11 +1609,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -1628,11 +1634,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1641,7 +1647,7 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1722,11 +1728,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -1747,11 +1753,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1761,18 +1767,18 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint)
Local Work:
Map Reduce Local Work
@@ -1783,10 +1789,10 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1861,11 +1867,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -1887,11 +1893,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1901,17 +1907,17 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 4
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Local Work:
Map Reduce Local Work
Reducer 2
@@ -1920,9 +1926,9 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Select Operator
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2035,11 +2041,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -2060,11 +2066,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -2074,18 +2080,18 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 6
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint)
Local Work:
Map Reduce Local Work
@@ -2098,7 +2104,7 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -2115,11 +2121,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -2140,11 +2146,11 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -2154,18 +2160,18 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint)
Local Work:
Map Reduce Local Work
@@ -2178,7 +2184,7 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -2188,14 +2194,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
1 Reducer 5
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: bigint), _col3 (type: bigint)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/08f73adc/ql/src/test/results/clientpositive/spark/bucket_map_join_spark1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/bucket_map_join_spark1.q.out b/ql/src/test/results/clientpositive/spark/bucket_map_join_spark1.q.out
index fb8f8d6..e162f4a 100644
--- a/ql/src/test/results/clientpositive/spark/bucket_map_join_spark1.q.out
+++ b/ql/src/test/results/clientpositive/spark/bucket_map_join_spark1.q.out
@@ -188,11 +188,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -271,11 +271,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -286,18 +286,18 @@ STAGE PLANS:
input vertices:
1 Map 2
Position of Big Table: 0
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
BucketMapJoin: true
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
@@ -533,11 +533,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -611,11 +611,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -626,17 +626,17 @@ STAGE PLANS:
input vertices:
1 Map 2
Position of Big Table: 0
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/08f73adc/ql/src/test/results/clientpositive/spark/bucket_map_join_spark2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/bucket_map_join_spark2.q.out b/ql/src/test/results/clientpositive/spark/bucket_map_join_spark2.q.out
index 4dd1157..58903d7 100644
--- a/ql/src/test/results/clientpositive/spark/bucket_map_join_spark2.q.out
+++ b/ql/src/test/results/clientpositive/spark/bucket_map_join_spark2.q.out
@@ -172,11 +172,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 15 Data size: 1583 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 3062 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 15 Data size: 1583 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 3062 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -255,11 +255,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -270,18 +270,18 @@ STAGE PLANS:
input vertices:
1 Map 2
Position of Big Table: 0
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
BucketMapJoin: true
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
@@ -517,11 +517,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 15 Data size: 1583 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 3062 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 15 Data size: 1583 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 3062 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -600,11 +600,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -615,18 +615,18 @@ STAGE PLANS:
input vertices:
1 Map 2
Position of Big Table: 0
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
BucketMapJoin: true
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/08f73adc/ql/src/test/results/clientpositive/spark/bucket_map_join_spark3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/bucket_map_join_spark3.q.out b/ql/src/test/results/clientpositive/spark/bucket_map_join_spark3.q.out
index 4133fda..60ce126 100644
--- a/ql/src/test/results/clientpositive/spark/bucket_map_join_spark3.q.out
+++ b/ql/src/test/results/clientpositive/spark/bucket_map_join_spark3.q.out
@@ -172,11 +172,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 15 Data size: 1583 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 3062 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 15 Data size: 1583 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 3062 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -255,11 +255,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -270,18 +270,18 @@ STAGE PLANS:
input vertices:
0 Map 1
Position of Big Table: 1
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
BucketMapJoin: true
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
@@ -517,11 +517,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 15 Data size: 1583 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 3062 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 15 Data size: 1583 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 3062 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
@@ -595,11 +595,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 28 Data size: 2958 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -610,17 +610,17 @@ STAGE PLANS:
input vertices:
0 Map 1
Position of Big Table: 1
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 30 Data size: 3253 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/08f73adc/ql/src/test/results/clientpositive/spark/bucket_map_join_spark4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/bucket_map_join_spark4.q.out b/ql/src/test/results/clientpositive/spark/bucket_map_join_spark4.q.out
index 64fb785..338384f 100644
--- a/ql/src/test/results/clientpositive/spark/bucket_map_join_spark4.q.out
+++ b/ql/src/test/results/clientpositive/spark/bucket_map_join_spark4.q.out
@@ -147,25 +147,25 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
- alias: a
+ alias: b
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
- predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Position of Big Table: 1
+ Position of Big Table: 0
Local Work:
Map Reduce Local Work
Bucket Mapjoin Context:
@@ -178,7 +178,7 @@ STAGE PLANS:
Path -> Partition:
#### A masked pattern was here ####
Partition
- base file name: tbl1
+ base file name: tbl2
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
@@ -190,11 +190,11 @@ STAGE PLANS:
columns.comments
columns.types int:string
#### A masked pattern was here ####
- name default.tbl1
+ name default.tbl2
numFiles 2
numRows 10
rawDataSize 70
- serialization.ddl struct tbl1 { i32 key, string value}
+ serialization.ddl struct tbl2 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 80
@@ -212,20 +212,20 @@ STAGE PLANS:
columns.comments
columns.types int:string
#### A masked pattern was here ####
- name default.tbl1
+ name default.tbl2
numFiles 2
numRows 10
rawDataSize 70
- serialization.ddl struct tbl1 { i32 key, string value}
+ serialization.ddl struct tbl2 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 80
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.tbl1
- name: default.tbl1
+ name: default.tbl2
+ name: default.tbl2
Truncated Path -> Alias:
- /tbl1 [a]
+ /tbl2 [b]
Map 3
Map Operator Tree:
TableScan
@@ -235,11 +235,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: value is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: value (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col1 (type: string)
@@ -305,20 +305,20 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 1
Map Operator Tree:
TableScan
- alias: b
+ alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ predicate: (key is not null and value is not null) (type: boolean)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -327,9 +327,9 @@ STAGE PLANS:
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col3
input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ 1 Map 2
+ Position of Big Table: 0
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
BucketMapJoin: true
Map Join Operator
condition map:
@@ -341,17 +341,17 @@ STAGE PLANS:
input vertices:
1 Map 3
Position of Big Table: 0
- Statistics: Num rows: 5 Data size: 41 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 84 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string), _col5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 5 Data size: 41 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 84 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 5 Data size: 41 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 84 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
@@ -380,7 +380,7 @@ STAGE PLANS:
Path -> Partition:
#### A masked pattern was here ####
Partition
- base file name: tbl2
+ base file name: tbl1
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
@@ -392,11 +392,11 @@ STAGE PLANS:
columns.comments
columns.types int:string
#### A masked pattern was here ####
- name default.tbl2
+ name default.tbl1
numFiles 2
numRows 10
rawDataSize 70
- serialization.ddl struct tbl2 { i32 key, string value}
+ serialization.ddl struct tbl1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 80
@@ -414,20 +414,20 @@ STAGE PLANS:
columns.comments
columns.types int:string
#### A masked pattern was here ####
- name default.tbl2
+ name default.tbl1
numFiles 2
numRows 10
rawDataSize 70
- serialization.ddl struct tbl2 { i32 key, string value}
+ serialization.ddl struct tbl1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 80
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.tbl2
- name: default.tbl2
+ name: default.tbl1
+ name: default.tbl1
Truncated Path -> Alias:
- /tbl2 [b]
+ /tbl1 [a]
Stage: Stage-0
Fetch Operator
@@ -592,25 +592,25 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 1
+ Map 2
Map Operator Tree:
TableScan
- alias: a
+ alias: b
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
- predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Position of Big Table: 1
+ Position of Big Table: 0
Local Work:
Map Reduce Local Work
Path -> Alias:
@@ -618,7 +618,7 @@ STAGE PLANS:
Path -> Partition:
#### A masked pattern was here ####
Partition
- base file name: tbl1
+ base file name: tbl2
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
@@ -630,11 +630,11 @@ STAGE PLANS:
columns.comments
columns.types int:string
#### A masked pattern was here ####
- name default.tbl1
+ name default.tbl2
numFiles 2
numRows 10
rawDataSize 70
- serialization.ddl struct tbl1 { i32 key, string value}
+ serialization.ddl struct tbl2 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 80
@@ -652,20 +652,20 @@ STAGE PLANS:
columns.comments
columns.types int:string
#### A masked pattern was here ####
- name default.tbl1
+ name default.tbl2
numFiles 2
numRows 10
rawDataSize 70
- serialization.ddl struct tbl1 { i32 key, string value}
+ serialization.ddl struct tbl2 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 80
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.tbl1
- name: default.tbl1
+ name: default.tbl2
+ name: default.tbl2
Truncated Path -> Alias:
- /tbl1 [a]
+ /tbl2 [b]
Map 3
Map Operator Tree:
TableScan
@@ -675,11 +675,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: value is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: value (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Spark HashTable Sink Operator
keys:
0 _col1 (type: string)
@@ -745,20 +745,20 @@ STAGE PLANS:
Spark
#### A masked pattern was here ####
Vertices:
- Map 2
+ Map 1
Map Operator Tree:
TableScan
- alias: b
+ alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ predicate: (key is not null and value is not null) (type: boolean)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -767,9 +767,9 @@ STAGE PLANS:
1 _col0 (type: int)
outputColumnNames: _col0, _col1, _col3
input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ 1 Map 2
+ Position of Big Table: 0
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -780,17 +780,17 @@ STAGE PLANS:
input vertices:
1 Map 3
Position of Big Table: 0
- Statistics: Num rows: 5 Data size: 41 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 84 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string), _col5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 5 Data size: 41 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 84 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 5 Data size: 41 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 84 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
@@ -814,7 +814,7 @@ STAGE PLANS:
Path -> Partition:
#### A masked pattern was here ####
Partition
- base file name: tbl2
+ base file name: tbl1
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
@@ -826,11 +826,11 @@ STAGE PLANS:
columns.comments
columns.types int:string
#### A masked pattern was here ####
- name default.tbl2
+ name default.tbl1
numFiles 2
numRows 10
rawDataSize 70
- serialization.ddl struct tbl2 { i32 key, string value}
+ serialization.ddl struct tbl1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 80
@@ -848,20 +848,20 @@ STAGE PLANS:
columns.comments
columns.types int:string
#### A masked pattern was here ####
- name default.tbl2
+ name default.tbl1
numFiles 2
numRows 10
rawDataSize 70
- serialization.ddl struct tbl2 { i32 key, string value}
+ serialization.ddl struct tbl1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 80
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.tbl2
- name: default.tbl2
+ name: default.tbl1
+ name: default.tbl1
Truncated Path -> Alias:
- /tbl2 [b]
+ /tbl1 [a]
Stage: Stage-0
Fetch Operator