You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by xu...@apache.org on 2015/11/30 01:11:23 UTC
[26/91] [abbrv] [partial] hive git commit: HIVE-12017: Do not disable
CBO by default when number of joins in a query is equal or less than 1 (Jesus
Camacho Rodriguez, reviewed by Ashutosh Chauhan)
http://git-wip-us.apache.org/repos/asf/hive/blob/7dab21ac/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out b/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out
index 36a032a..3b634be 100644
--- a/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out
@@ -788,38 +788,46 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 289 Data size: 6872 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 289 Data size: 6872 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
Map 3
Map Operator Tree:
TableScan
- alias: p2
+ alias: p1
Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Reducer 2
Execution mode: uber
@@ -900,39 +908,47 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 289 Data size: 6872 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 289 Data size: 6872 Basic stats: COMPLETE Column stats: NONE
+ HybridGraceHashJoin: true
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
Map 3
Map Operator Tree:
TableScan
- alias: p2
+ alias: p1
Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Reducer 2
Execution mode: uber
@@ -1013,38 +1029,46 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 289 Data size: 6872 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 289 Data size: 6872 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
Map 3
Map Operator Tree:
TableScan
- alias: p2
+ alias: p1
Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Reducer 2
Execution mode: uber
@@ -1125,39 +1149,47 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 289 Data size: 6872 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 289 Data size: 6872 Basic stats: COMPLETE Column stats: NONE
+ HybridGraceHashJoin: true
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
Map 3
Map Operator Tree:
TableScan
- alias: p2
+ alias: p1
Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 263 Data size: 6248 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Reducer 2
Execution mode: uber
@@ -1232,6 +1264,7 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@alltypesorc
POSTHOOK: Output: database:default
POSTHOOK: Output: default@decimal_mapjoin
+Warning: Map Join MAPJOIN[15][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: EXPLAIN SELECT l.cint, r.cint, l.cdecimal1, r.cdecimal2
FROM decimal_mapjoin l
JOIN decimal_mapjoin r ON l.cint = r.cint
@@ -1261,42 +1294,48 @@ STAGE PLANS:
Filter Operator
predicate: (cint = 6981) (type: boolean)
Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 6981 (type: int)
- 1 6981 (type: int)
- outputColumnNames: _col1, _col9
- input vertices:
- 1 Map 2
- Statistics: Num rows: 6758 Data size: 1190783 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: 6981 (type: int), 6981 (type: int), _col1 (type: decimal(20,10)), _col9 (type: decimal(23,14))
- outputColumnNames: _col0, _col1, _col2, _col3
+ Select Operator
+ expressions: cdecimal1 (type: decimal(20,10))
+ outputColumnNames: _col0
+ Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
+ outputColumnNames: _col0, _col2
+ input vertices:
+ 1 Map 2
Statistics: Num rows: 6758 Data size: 1190783 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
+ Select Operator
+ expressions: 6981 (type: int), 6981 (type: int), _col0 (type: decimal(20,10)), _col2 (type: decimal(23,14))
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 6758 Data size: 1190783 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 6758 Data size: 1190783 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: vectorized, llap
Map 2
Map Operator Tree:
TableScan
- alias: r
+ alias: l
Statistics: Num rows: 12288 Data size: 2165060 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (cint = 6981) (type: boolean)
+ predicate: (6981 = cint) (type: boolean)
Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: 6981 (type: int)
- sort order: +
- Map-reduce partition columns: 6981 (type: int)
+ Select Operator
+ expressions: cdecimal2 (type: decimal(23,14))
+ outputColumnNames: _col0
Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
- value expressions: cdecimal2 (type: decimal(23,14))
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: decimal(23,14))
Execution mode: vectorized, llap
Stage: Stage-0
@@ -1305,6 +1344,7 @@ STAGE PLANS:
Processor Tree:
ListSink
+Warning: Map Join MAPJOIN[15][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT l.cint, r.cint, l.cdecimal1, r.cdecimal2
FROM decimal_mapjoin l
JOIN decimal_mapjoin r ON l.cint = r.cint
@@ -1419,6 +1459,7 @@ POSTHOOK: Input: default@decimal_mapjoin
6981 6981 -515.6210729730 NULL
6981 6981 -515.6210729730 NULL
6981 6981 -515.6210729730 NULL
+Warning: Map Join MAPJOIN[15][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: EXPLAIN SELECT l.cint, r.cint, l.cdecimal1, r.cdecimal2
FROM decimal_mapjoin l
JOIN decimal_mapjoin r ON l.cint = r.cint
@@ -1448,43 +1489,48 @@ STAGE PLANS:
Filter Operator
predicate: (cint = 6981) (type: boolean)
Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 6981 (type: int)
- 1 6981 (type: int)
- outputColumnNames: _col1, _col9
- input vertices:
- 1 Map 2
- Statistics: Num rows: 6758 Data size: 1190783 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Select Operator
- expressions: 6981 (type: int), 6981 (type: int), _col1 (type: decimal(20,10)), _col9 (type: decimal(23,14))
- outputColumnNames: _col0, _col1, _col2, _col3
+ Select Operator
+ expressions: cdecimal1 (type: decimal(20,10))
+ outputColumnNames: _col0
+ Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
+ outputColumnNames: _col0, _col2
+ input vertices:
+ 1 Map 2
Statistics: Num rows: 6758 Data size: 1190783 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
+ Select Operator
+ expressions: 6981 (type: int), 6981 (type: int), _col0 (type: decimal(20,10)), _col2 (type: decimal(23,14))
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 6758 Data size: 1190783 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 6758 Data size: 1190783 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: vectorized, llap
Map 2
Map Operator Tree:
TableScan
- alias: r
+ alias: l
Statistics: Num rows: 12288 Data size: 2165060 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (cint = 6981) (type: boolean)
+ predicate: (6981 = cint) (type: boolean)
Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: 6981 (type: int)
- sort order: +
- Map-reduce partition columns: 6981 (type: int)
+ Select Operator
+ expressions: cdecimal2 (type: decimal(23,14))
+ outputColumnNames: _col0
Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
- value expressions: cdecimal2 (type: decimal(23,14))
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 6144 Data size: 1082530 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: decimal(23,14))
Execution mode: vectorized, llap
Stage: Stage-0
@@ -1493,6 +1539,7 @@ STAGE PLANS:
Processor Tree:
ListSink
+Warning: Map Join MAPJOIN[15][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT l.cint, r.cint, l.cdecimal1, r.cdecimal2
FROM decimal_mapjoin l
JOIN decimal_mapjoin r ON l.cint = r.cint
http://git-wip-us.apache.org/repos/asf/hive/blob/7dab21ac/ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out b/ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out
index 4c8b295..7310ec3 100644
--- a/ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out
+++ b/ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out
@@ -102,21 +102,21 @@ STAGE PLANS:
Filter Operator
predicate: dec is not null (type: boolean)
Statistics: Num rows: 525 Data size: 58800 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 dec (type: decimal(6,2))
- 1 dec (type: decimal(6,2))
- outputColumnNames: _col0, _col4
- input vertices:
- 1 Map 3
- Statistics: Num rows: 577 Data size: 64680 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Select Operator
- expressions: _col0 (type: decimal(4,2)), _col4 (type: decimal(4,0))
+ Select Operator
+ expressions: dec (type: decimal(4,2))
+ outputColumnNames: _col0
+ Statistics: Num rows: 525 Data size: 58800 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: decimal(6,2))
+ 1 _col0 (type: decimal(6,2))
outputColumnNames: _col0, _col1
+ input vertices:
+ 1 Map 3
Statistics: Num rows: 577 Data size: 64680 Basic stats: COMPLETE Column stats: NONE
+ HybridGraceHashJoin: true
Reduce Output Operator
key expressions: _col0 (type: decimal(4,2))
sort order: +
@@ -131,12 +131,16 @@ STAGE PLANS:
Filter Operator
predicate: dec is not null (type: boolean)
Statistics: Num rows: 525 Data size: 58800 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: dec (type: decimal(6,2))
- sort order: +
- Map-reduce partition columns: dec (type: decimal(6,2))
+ Select Operator
+ expressions: dec (type: decimal(4,0))
+ outputColumnNames: _col0
Statistics: Num rows: 525 Data size: 58800 Basic stats: COMPLETE Column stats: NONE
- value expressions: dec (type: decimal(4,0))
+ Reduce Output Operator
+ key expressions: _col0 (type: decimal(6,2))
+ sort order: +
+ Map-reduce partition columns: _col0 (type: decimal(6,2))
+ Statistics: Num rows: 525 Data size: 58800 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: decimal(4,0))
Execution mode: llap
Reducer 2
Execution mode: uber
http://git-wip-us.apache.org/repos/asf/hive/blob/7dab21ac/ql/src/test/results/clientpositive/llap/mrr.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/mrr.q.out b/ql/src/test/results/clientpositive/llap/mrr.q.out
index 7026cd4..152266a 100644
--- a/ql/src/test/results/clientpositive/llap/mrr.q.out
+++ b/ql/src/test/results/clientpositive/llap/mrr.q.out
@@ -422,15 +422,14 @@ STAGE PLANS:
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: string)
Execution mode: llap
Map 5
Map Operator Tree:
@@ -441,14 +440,15 @@ STAGE PLANS:
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: string)
Execution mode: llap
Reducer 2
Execution mode: llap
@@ -459,10 +459,10 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col0, _col1
+ outputColumnNames: _col1, _col2
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- keys: _col0 (type: string), _col1 (type: string)
+ keys: _col1 (type: string), _col2 (type: string)
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
@@ -855,8 +855,8 @@ STAGE PLANS:
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
@@ -864,13 +864,13 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col0, _col1
+ outputColumnNames: _col1, _col2
input vertices:
1 Map 4
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
HybridGraceHashJoin: true
Group By Operator
- keys: _col0 (type: string), _col1 (type: string)
+ keys: _col1 (type: string), _col2 (type: string)
mode: hash
outputColumnNames: _col0, _col1
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
@@ -889,14 +889,15 @@ STAGE PLANS:
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: string)
Execution mode: llap
Reducer 2
Execution mode: llap
http://git-wip-us.apache.org/repos/asf/hive/blob/7dab21ac/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out b/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out
index 4d3b13d..da57522 100644
--- a/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out
@@ -90,22 +90,26 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 500 Data size: 26156 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
+ Select Operator
+ expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- input vertices:
- 1 Map 3
- Statistics: Num rows: 550 Data size: 28771 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
+ Statistics: Num rows: 500 Data size: 26156 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0, _col1
+ input vertices:
+ 1 Map 3
Statistics: Num rows: 550 Data size: 28771 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: string)
+ HybridGraceHashJoin: true
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Statistics: Num rows: 550 Data size: 28771 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: string)
Execution mode: llap
Map 3
Map Operator Tree:
@@ -115,11 +119,15 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Reducer 2
Execution mode: uber
http://git-wip-us.apache.org/repos/asf/hive/blob/7dab21ac/ql/src/test/results/clientpositive/llap/tez_join.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_join.q.out b/ql/src/test/results/clientpositive/llap/tez_join.q.out
index 881a721..41e2156 100644
--- a/ql/src/test/results/clientpositive/llap/tez_join.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_join.q.out
@@ -103,22 +103,15 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col0, _col1
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: (_col0 = _col1) (type: boolean)
+ File Output Operator
+ compressed: false
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/7dab21ac/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out b/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out
index 921397a..81e317c 100644
--- a/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out
@@ -46,11 +46,15 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 4
Map Operator Tree:
@@ -60,11 +64,15 @@ STAGE PLANS:
Filter Operator
predicate: key is not null (type: boolean)
Statistics: Num rows: 250 Data size: 44000 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 250 Data size: 44000 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 250 Data size: 44000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
Reducer 2
Execution mode: llap
@@ -73,24 +81,18 @@ STAGE PLANS:
condition map:
Inner Join 0 to 1
keys:
- 0 key (type: string)
- 1 key (type: string)
- outputColumnNames: _col0, _col5
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (_col0 = _col5) (type: boolean)
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Reducer 3
Execution mode: vectorized, uber
Reduce Operator Tree:
http://git-wip-us.apache.org/repos/asf/hive/blob/7dab21ac/ql/src/test/results/clientpositive/llap/tez_join_result_complex.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_join_result_complex.q.out b/ql/src/test/results/clientpositive/llap/tez_join_result_complex.q.out
index 0ad18ed..bc78b76 100644
--- a/ql/src/test/results/clientpositive/llap/tez_join_result_complex.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_join_result_complex.q.out
@@ -157,22 +157,144 @@ TOK_CREATETABLE
TOK_QUERY
TOK_FROM
TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- ct_events_clean
- a
- TOK_TABREF
- TOK_TABNAME
- service_request_clean
- b
+ TOK_SUBQUERY
+ TOK_QUERY
+ TOK_FROM
+ TOK_TABREF
+ TOK_TABNAME
+ default
+ ct_events_clean
+ a
+ TOK_INSERT
+ TOK_DESTINATION
+ TOK_DIR
+ TOK_TMP_FILE
+ TOK_SELECT
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ contact_event_id
+ contact_event_id
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ ce_create_dt
+ ce_create_dt
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ ce_end_dt
+ ce_end_dt
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ contact_type
+ contact_type
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ cnctevs_cd
+ cnctevs_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ contact_mode
+ contact_mode
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ cntvnst_stts_cd
+ cntvnst_stts_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ total_transfers
+ total_transfers
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ ce_notes
+ ce_notes
+ $hdt$_0
+ TOK_SUBQUERY
+ TOK_QUERY
+ TOK_FROM
+ TOK_TABREF
+ TOK_TABNAME
+ default
+ service_request_clean
+ b
+ TOK_INSERT
+ TOK_DESTINATION
+ TOK_DIR
+ TOK_TMP_FILE
+ TOK_SELECT
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ cnctevn_id
+ cnctevn_id
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ svcrqst_id
+ svcrqst_id
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ cnctmd_cd
+ cnctmd_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ svcrtyp_cd
+ svcrtyp_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ cmpltyp_cd
+ cmpltyp_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ sum_reason_cd
+ sum_reason_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ svcrqct_cds
+ svcrqct_cds
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ notes
+ notes
+ $hdt$_1
=
.
TOK_TABLE_OR_COL
- a
+ $hdt$_0
contact_event_id
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
cnctevn_id
TOK_INSERT
TOK_DESTINATION
@@ -180,45 +302,101 @@ TOK_CREATETABLE
TOK_TMP_FILE
TOK_SELECT
TOK_SELEXPR
- TOK_ALLCOLREF
- TOK_TABNAME
- a
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ contact_event_id
+ a.contact_event_id
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ ce_create_dt
+ a.ce_create_dt
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ ce_end_dt
+ a.ce_end_dt
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ contact_type
+ a.contact_type
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ cnctevs_cd
+ a.cnctevs_cd
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_0
+ contact_mode
+ a.contact_mode
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ cntvnst_stts_cd
+ a.cntvnst_stts_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ total_transfers
+ a.total_transfers
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ ce_notes
+ a.ce_notes
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_1
svcrqst_id
+ b.svcrqst_id
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
svcrqct_cds
+ b.svcrqct_cds
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
svcrtyp_cd
+ b.svcrtyp_cd
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
cmpltyp_cd
+ b.cmpltyp_cd
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
sum_reason_cd
src
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
cnctmd_cd
+ b.cnctmd_cd
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
notes
+ b.notes
STAGE DEPENDENCIES:
@@ -245,14 +423,18 @@ STAGE PLANS:
isSamplingPred: false
predicate: contact_event_id is not null (type: boolean)
Statistics: Num rows: 1 Data size: 3750 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: contact_event_id (type: string)
- sort order: +
- Map-reduce partition columns: contact_event_id (type: string)
+ Select Operator
+ expressions: contact_event_id (type: string), ce_create_dt (type: string), ce_end_dt (type: string), contact_type (type: string), cnctevs_cd (type: string), contact_mode (type: string), cntvnst_stts_cd (type: string), total_transfers (type: int), ce_notes (type: array<string>)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 1 Data size: 3750 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- value expressions: ce_create_dt (type: string), ce_end_dt (type: string), contact_type (type: string), cnctevs_cd (type: string), contact_mode (type: string), cntvnst_stts_cd (type: string), total_transfers (type: int), ce_notes (type: array<string>)
- auto parallelism: true
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 3750 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string), _col7 (type: int), _col8 (type: array<string>)
+ auto parallelism: true
Execution mode: llap
Path -> Alias:
#### A masked pattern was here ####
@@ -317,43 +499,47 @@ STAGE PLANS:
isSamplingPred: false
predicate: cnctevn_id is not null (type: boolean)
Statistics: Num rows: 2 Data size: 9081 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 1
- keys:
- 0 contact_event_id (type: string)
- 1 cnctevn_id (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col13, _col19, _col21, _col23, _col30, _col33, _col38
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 2 Data size: 9989 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string), _col7 (type: int), _col8 (type: array<string>), _col13 (type: string), _col33 (type: array<string>), _col21 (type: string), _col23 (type: string), _col30 (type: string), _col19 (type: string), _col38 (type: array<string>)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15
+ Select Operator
+ expressions: cnctevn_id (type: string), svcrqst_id (type: string), cnctmd_cd (type: string), svcrtyp_cd (type: string), cmpltyp_cd (type: string), sum_reason_cd (type: string), svcrqct_cds (type: array<string>), notes (type: array<string>)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
+ Statistics: Num rows: 2 Data size: 9081 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11, _col12, _col13, _col14, _col15, _col16
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
Statistics: Num rows: 2 Data size: 9989 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- GlobalTableId: 1
-#### A masked pattern was here ####
- NumFilesPerFileSink: 1
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string), _col7 (type: int), _col8 (type: array<string>), _col10 (type: string), _col15 (type: array<string>), _col12 (type: string), _col13 (type: string), _col14 (type: string), _col11 (type: string), _col16 (type: array<string>)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15
Statistics: Num rows: 2 Data size: 9989 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ GlobalTableId: 1
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 2 Data size: 9989 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- columns contact_event_id,ce_create_dt,ce_end_dt,contact_type,cnctevs_cd,contact_mode,cntvnst_stts_cd,total_transfers,ce_notes,svcrqst_id,svcrqct_cds,svcrtyp_cd,cmpltyp_cd,src,cnctmd_cd,notes
- columns.types string:string:string:string:string:string:string:int:array<string>:string:array<string>:string:string:string:string:array<string>
- name default.ct_events1_test
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.ct_events1_test
- TotalFiles: 1
- GatherStats: true
- MultiFileSpray: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ properties:
+ columns contact_event_id,ce_create_dt,ce_end_dt,contact_type,cnctevs_cd,contact_mode,cntvnst_stts_cd,total_transfers,ce_notes,svcrqst_id,svcrqct_cds,svcrtyp_cd,cmpltyp_cd,src,cnctmd_cd,notes
+ columns.types string:string:string:string:string:string:string:int:array<string>:string:array<string>:string:string:string:string:array<string>
+ name default.ct_events1_test
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.ct_events1_test
+ TotalFiles: 1
+ GatherStats: true
+ MultiFileSpray: false
Execution mode: llap
Path -> Alias:
#### A masked pattern was here ####
@@ -1182,22 +1368,144 @@ TOK_CREATETABLE
TOK_QUERY
TOK_FROM
TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- ct_events_clean
- a
- TOK_TABREF
- TOK_TABNAME
- service_request_clean
- b
+ TOK_SUBQUERY
+ TOK_QUERY
+ TOK_FROM
+ TOK_TABREF
+ TOK_TABNAME
+ default
+ ct_events_clean
+ a
+ TOK_INSERT
+ TOK_DESTINATION
+ TOK_DIR
+ TOK_TMP_FILE
+ TOK_SELECT
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ contact_event_id
+ contact_event_id
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ ce_create_dt
+ ce_create_dt
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ ce_end_dt
+ ce_end_dt
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ contact_type
+ contact_type
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ cnctevs_cd
+ cnctevs_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ contact_mode
+ contact_mode
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ cntvnst_stts_cd
+ cntvnst_stts_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ total_transfers
+ total_transfers
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ a
+ ce_notes
+ ce_notes
+ $hdt$_0
+ TOK_SUBQUERY
+ TOK_QUERY
+ TOK_FROM
+ TOK_TABREF
+ TOK_TABNAME
+ default
+ service_request_clean
+ b
+ TOK_INSERT
+ TOK_DESTINATION
+ TOK_DIR
+ TOK_TMP_FILE
+ TOK_SELECT
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ cnctevn_id
+ cnctevn_id
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ svcrqst_id
+ svcrqst_id
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ cnctmd_cd
+ cnctmd_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ svcrtyp_cd
+ svcrtyp_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ cmpltyp_cd
+ cmpltyp_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ sum_reason_cd
+ sum_reason_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ svcrqct_cds
+ svcrqct_cds
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ b
+ notes
+ notes
+ $hdt$_1
=
.
TOK_TABLE_OR_COL
- a
+ $hdt$_0
contact_event_id
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
cnctevn_id
TOK_INSERT
TOK_DESTINATION
@@ -1205,45 +1513,101 @@ TOK_CREATETABLE
TOK_TMP_FILE
TOK_SELECT
TOK_SELEXPR
- TOK_ALLCOLREF
- TOK_TABNAME
- a
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ contact_event_id
+ a.contact_event_id
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ ce_create_dt
+ a.ce_create_dt
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ ce_end_dt
+ a.ce_end_dt
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ contact_type
+ a.contact_type
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ cnctevs_cd
+ a.cnctevs_cd
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_0
+ contact_mode
+ a.contact_mode
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ cntvnst_stts_cd
+ a.cntvnst_stts_cd
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ total_transfers
+ a.total_transfers
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_0
+ ce_notes
+ a.ce_notes
+ TOK_SELEXPR
+ .
+ TOK_TABLE_OR_COL
+ $hdt$_1
svcrqst_id
+ b.svcrqst_id
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
svcrqct_cds
+ b.svcrqct_cds
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
svcrtyp_cd
+ b.svcrtyp_cd
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
cmpltyp_cd
+ b.cmpltyp_cd
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
sum_reason_cd
src
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
cnctmd_cd
+ b.cnctmd_cd
TOK_SELEXPR
.
TOK_TABLE_OR_COL
- b
+ $hdt$_1
notes
+ b.notes
STAGE DEPENDENCIES:
@@ -1270,14 +1634,18 @@ STAGE PLANS:
isSamplingPred: false
predicate: contact_event_id is not null (type: boolean)
Statistics: Num rows: 1 Data size: 3750 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: contact_event_id (type: string)
- sort order: +
- Map-reduce partition columns: contact_event_id (type: string)
+ Select Operator
+ expressions: contact_event_id (type: string), ce_create_dt (type: string), ce_end_dt (type: string), contact_type (type: string), cnctevs_cd (type: string), contact_mode (type: string), cntvnst_stts_cd (type: string), total_transfers (type: int), ce_notes (type: array<string>)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
Statistics: Num rows: 1 Data size: 3750 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- value expressions: ce_create_dt (type: string), ce_end_dt (type: string), contact_type (type: string), cnctevs_cd (type: string), contact_mode (type: string), cntvnst_stts_cd (type: string), total_transfers (type: int), ce_notes (type: array<string>)
- auto parallelism: true
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 3750 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string), _col7 (type: int), _col8 (type: array<string>)
+ auto parallelism: true
Execution mode: llap
Path -> Alias:
#### A masked pattern was here ####
@@ -1342,44 +1710,48 @@ STAGE PLANS:
isSamplingPred: false
predicate: cnctevn_id is not null (type: boolean)
Statistics: Num rows: 2 Data size: 9081 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 1
- keys:
- 0 contact_event_id (type: string)
- 1 cnctevn_id (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col13, _col19, _col21, _col23, _col30, _col33, _col38
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 2 Data size: 9989 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string), _col7 (type: int), _col8 (type: array<string>), _col13 (type: string), _col33 (type: array<string>), _col21 (type: string), _col23 (type: string), _col30 (type: string), _col19 (type: string), _col38 (type: array<string>)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15
+ Select Operator
+ expressions: cnctevn_id (type: string), svcrqst_id (type: string), cnctmd_cd (type: string), svcrtyp_cd (type: string), cmpltyp_cd (type: string), sum_reason_cd (type: string), svcrqct_cds (type: array<string>), notes (type: array<string>)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
+ Statistics: Num rows: 2 Data size: 9081 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col10, _col11, _col12, _col13, _col14, _col15, _col16
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
Statistics: Num rows: 2 Data size: 9989 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- GlobalTableId: 1
-#### A masked pattern was here ####
- NumFilesPerFileSink: 1
+ HybridGraceHashJoin: true
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string), _col7 (type: int), _col8 (type: array<string>), _col10 (type: string), _col15 (type: array<string>), _col12 (type: string), _col13 (type: string), _col14 (type: string), _col11 (type: string), _col16 (type: array<string>)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15
Statistics: Num rows: 2 Data size: 9989 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ GlobalTableId: 1
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 2 Data size: 9989 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- properties:
- columns contact_event_id,ce_create_dt,ce_end_dt,contact_type,cnctevs_cd,contact_mode,cntvnst_stts_cd,total_transfers,ce_notes,svcrqst_id,svcrqct_cds,svcrtyp_cd,cmpltyp_cd,src,cnctmd_cd,notes
- columns.types string:string:string:string:string:string:string:int:array<string>:string:array<string>:string:string:string:string:array<string>
- name default.ct_events1_test
- serialization.format 1
- serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.ct_events1_test
- TotalFiles: 1
- GatherStats: true
- MultiFileSpray: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ properties:
+ columns contact_event_id,ce_create_dt,ce_end_dt,contact_type,cnctevs_cd,contact_mode,cntvnst_stts_cd,total_transfers,ce_notes,svcrqst_id,svcrqct_cds,svcrtyp_cd,cmpltyp_cd,src,cnctmd_cd,notes
+ columns.types string:string:string:string:string:string:string:int:array<string>:string:array<string>:string:string:string:string:array<string>
+ name default.ct_events1_test
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.ct_events1_test
+ TotalFiles: 1
+ GatherStats: true
+ MultiFileSpray: false
Execution mode: llap
Path -> Alias:
#### A masked pattern was here ####
http://git-wip-us.apache.org/repos/asf/hive/blob/7dab21ac/ql/src/test/results/clientpositive/llap/tez_self_join.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_self_join.q.out b/ql/src/test/results/clientpositive/llap/tez_self_join.q.out
index f7922c9..2f38be1 100644
--- a/ql/src/test/results/clientpositive/llap/tez_self_join.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_self_join.q.out
@@ -42,6 +42,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@values__tmp__table__2
POSTHOOK: Output: default@tez_self_join2
POSTHOOK: Lineage: tez_self_join2.id1 EXPRESSION [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+Warning: Shuffle Join MERGEJOIN[27][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain
select s.id2, s.id3
from
@@ -82,28 +83,30 @@ STAGE PLANS:
alias: self1
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((id2 is not null and id1 is not null) and (id2 = 'ab')) (type: boolean)
+ predicate: ((id2 = 'ab') and id1 is not null) (type: boolean)
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: 'ab' (type: string)
- sort order: +
- Map-reduce partition columns: 'ab' (type: string)
+ Select Operator
+ expressions: id1 (type: int), id3 (type: string)
+ outputColumnNames: _col0, _col2
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- value expressions: id1 (type: int), id3 (type: string)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: int), _col2 (type: string)
Execution mode: llap
Map 4
Map Operator Tree:
TableScan
- alias: self2
+ alias: self1
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (id3 = 'ab') (type: boolean)
+ predicate: ('ab' = id3) (type: boolean)
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: 'ab' (type: string)
- sort order: +
- Map-reduce partition columns: 'ab' (type: string)
+ Select Operator
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
@@ -113,11 +116,15 @@ STAGE PLANS:
Filter Operator
predicate: id1 is not null (type: boolean)
Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: id1 (type: int)
- sort order: +
- Map-reduce partition columns: id1 (type: int)
+ Select Operator
+ expressions: id1 (type: int)
+ outputColumnNames: _col0
Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Reducer 2
Execution mode: llap
@@ -126,8 +133,8 @@ STAGE PLANS:
condition map:
Inner Join 0 to 1
keys:
- 0 'ab' (type: string)
- 1 'ab' (type: string)
+ 0
+ 1
outputColumnNames: _col0, _col2
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
@@ -144,7 +151,7 @@ STAGE PLANS:
Inner Join 0 to 1
keys:
0 _col0 (type: int)
- 1 id1 (type: int)
+ 1 _col0 (type: int)
outputColumnNames: _col2
Statistics: Num rows: 2 Data size: 2 Basic stats: COMPLETE Column stats: NONE
Select Operator
@@ -165,6 +172,7 @@ STAGE PLANS:
Processor Tree:
ListSink
+Warning: Shuffle Join MERGEJOIN[27][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select s.id2, s.id3
from
(