You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by pr...@apache.org on 2016/08/23 18:45:32 UTC
[01/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Repository: hive
Updated Branches:
refs/heads/master a4d8c6608 -> 5f01dc42a
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out b/ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out
index 7e5fe2d..52af8fd 100644
--- a/ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out
@@ -43,6 +43,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 10 <- Map 9 (SIMPLE_EDGE), Union 5 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -68,10 +69,11 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
- alias: s1
+ alias: s2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
@@ -85,10 +87,11 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 9
Map Operator Tree:
TableScan
- alias: s1
+ alias: s3
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
@@ -102,8 +105,9 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 10
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -125,7 +129,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -147,7 +151,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: bigint)
@@ -165,7 +169,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 6
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: bigint)
@@ -185,7 +189,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.tmptable
Reducer 8
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -291,6 +295,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 10 <- Map 9 (SIMPLE_EDGE), Union 5 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -316,10 +321,11 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
- alias: s1
+ alias: s2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
@@ -333,10 +339,11 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 9
Map Operator Tree:
TableScan
- alias: s1
+ alias: s3
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
@@ -350,8 +357,9 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 10
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -373,7 +381,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -395,7 +403,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: bigint)
@@ -413,7 +421,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 6
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: bigint)
@@ -434,11 +442,11 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 8
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -534,6 +542,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 10 <- Map 9 (SIMPLE_EDGE), Union 5 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -559,6 +568,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
@@ -576,6 +586,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 9
Map Operator Tree:
TableScan
@@ -593,8 +604,9 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 10
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -616,7 +628,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -638,7 +650,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: bigint)
@@ -656,7 +668,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string), _col1 (type: bigint)
Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 6
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: bigint)
@@ -676,7 +688,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.tmptable12
Reducer 8
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -778,6 +790,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Map 4 <- Union 2 (CONTAINS)
@@ -788,56 +801,58 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
- alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: s2
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 2
Vertex: Union 2
@@ -1198,6 +1213,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Reducer 3 <- Union 2 (SIMPLE_EDGE)
@@ -1208,22 +1224,23 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -1241,33 +1258,34 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 13 Data size: 1248 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 13 Data size: 1248 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1282,12 +1300,12 @@ STAGE PLANS:
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Union 2
Vertex: Union 2
@@ -1363,6 +1381,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 7 <- Union 3 (CONTAINS)
Map 8 <- Union 5 (CONTAINS)
@@ -1388,48 +1407,51 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 8
Map Operator Tree:
TableScan
- alias: s2
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: s3
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 19 Data size: 5168 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 19 Data size: 5168 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1444,54 +1466,54 @@ STAGE PLANS:
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 13 Data size: 3536 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 19 Data size: 5168 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 19 Data size: 5168 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 6
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 19 Data size: 5168 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 19 Data size: 5168 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 14 Data size: 1344 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 14 Data size: 1344 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 3
Vertex: Union 3
@@ -1616,6 +1638,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Map 51 <- Union 2 (CONTAINS)
@@ -1673,862 +1696,887 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 51
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 52
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 53
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 54
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 55
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 56
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 57
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 58
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 59
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 60
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 61
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 62
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 63
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 64
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 65
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 66
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 67
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 68
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 69
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 70
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 71
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 72
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 73
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 74
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 11
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 13
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 15
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 17
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 19
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 21
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 23
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 25
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 27
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 29
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 31
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 33
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
<TRUNCATED>
[27/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out b/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
index 57cd8ec..840b757 100644
--- a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/explainuser_2.q.out
@@ -185,81 +185,47 @@ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 3
- File Output Operator [FS_18]
- compressed:false
- Statistics:Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_17]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator [MERGEJOIN_28]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{"0":"_col3 (type: string)","1":"_col0 (type: string)"}
- | outputColumnNames:["_col0","_col3","_col6"]
- | Statistics:Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
- |<-Map 5 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_15]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col1 (type: string)
- | Select Operator [SEL_6]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_26]
- | predicate:key is not null (type: boolean)
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_5]
- | alias:y
- | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 2 [SIMPLE_EDGE]
- Reduce Output Operator [RS_13]
- key expressions:_col3 (type: string)
- Map-reduce partition columns:_col3 (type: string)
- sort order:+
- Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: string)
- Merge Join Operator [MERGEJOIN_27]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{"0":"_col0 (type: string)","1":"_col1 (type: string)"}
- | outputColumnNames:["_col0","_col3"]
- | Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_8]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_2]
- | outputColumnNames:["_col0"]
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_24]
- | predicate:value is not null (type: boolean)
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_0]
- | alias:z
- | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- |<-Map 4 [SIMPLE_EDGE]
- Reduce Output Operator [RS_10]
- key expressions:_col1 (type: string)
- Map-reduce partition columns:_col1 (type: string)
- sort order:+
- Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: string)
- Select Operator [SEL_4]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_25]
- predicate:(value is not null and key is not null) (type: boolean)
- Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_3]
- alias:x
- Statistics:Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_16]
+ Select Operator [SEL_15] (rows=141 width=268)
+ Output:["_col0","_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_26] (rows=141 width=268)
+ Conds:RS_12._col3=RS_13._col0(Inner),Output:["_col0","_col3","_col6"]
+ <-Map 5 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_13]
+ PartitionCols:_col0
+ Select Operator [SEL_8] (rows=500 width=178)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_24] (rows=500 width=178)
+ predicate:key is not null
+ TableScan [TS_6] (rows=500 width=178)
+ default@src,y,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_25] (rows=58 width=177)
+ Conds:RS_9._col0=RS_10._col1(Inner),Output:["_col0","_col3"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_9]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=500 width=369)
+ Output:["_col0"]
+ Filter Operator [FIL_22] (rows=500 width=91)
+ predicate:value is not null
+ TableScan [TS_0] (rows=500 width=91)
+ default@srcpart,z,Tbl:COMPLETE,Col:COMPLETE,Output:["value"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_10]
+ PartitionCols:_col1
+ Select Operator [SEL_5] (rows=25 width=175)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_23] (rows=25 width=175)
+ predicate:(key is not null and value is not null)
+ TableScan [TS_3] (rows=25 width=175)
+ default@src1,x,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
PREHOOK: query: EXPLAIN
select
@@ -331,252 +297,132 @@ Reducer 8 <- Map 12 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
Reducer 9 <- Map 13 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:100
- Stage-1
- Reducer 5
- File Output Operator [FS_68]
- compressed:false
- Statistics:Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Limit [LIM_67]
- Number of rows:100
- Statistics:Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_66]
- | outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
- | Statistics:Num rows: 402 Data size: 4276 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 4 [SIMPLE_EDGE]
- Reduce Output Operator [RS_65]
- key expressions:_col0 (type: string), _col1 (type: string), _col2 (type: string)
- sort order:+++
- Statistics:Num rows: 402 Data size: 4276 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col3 (type: bigint), _col4 (type: bigint), _col5 (type: bigint)
- Select Operator [SEL_64]
- outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Statistics:Num rows: 402 Data size: 4276 Basic stats: COMPLETE Column stats: NONE
- Group By Operator [GBY_63]
- | aggregations:["count(VALUE._col0)","count(VALUE._col1)","count(VALUE._col2)"]
- | keys:KEY._col0 (type: string), KEY._col1 (type: string), KEY._col2 (type: string)
- | outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
- | Statistics:Num rows: 402 Data size: 4276 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 3 [SIMPLE_EDGE]
- Reduce Output Operator [RS_62]
- key expressions:_col0 (type: string), _col1 (type: string), _col2 (type: string)
- Map-reduce partition columns:_col0 (type: string), _col1 (type: string), _col2 (type: string)
- sort order:+++
- Statistics:Num rows: 804 Data size: 8552 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col3 (type: bigint), _col4 (type: bigint), _col5 (type: bigint)
- Group By Operator [GBY_61]
- aggregations:["count(_col13)","count(_col21)","count(_col3)"]
- keys:_col2 (type: string), _col12 (type: string), _col20 (type: string)
- outputColumnNames:["_col0","_col1","_col2","_col3","_col4","_col5"]
- Statistics:Num rows: 804 Data size: 8552 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_60]
- outputColumnNames:["_col2","_col12","_col20","_col13","_col21","_col3"]
- Statistics:Num rows: 804 Data size: 8552 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator [MERGEJOIN_110]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{"0":"_col1 (type: string), _col3 (type: string)","1":"_col15 (type: string), _col17 (type: string)"}
- | outputColumnNames:["_col2","_col3","_col12","_col13","_col20","_col21"]
- | Statistics:Num rows: 804 Data size: 8552 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 11 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_58]
- | key expressions:_col15 (type: string), _col17 (type: string)
- | Map-reduce partition columns:_col15 (type: string), _col17 (type: string)
- | sort order:++
- | Statistics:Num rows: 731 Data size: 7775 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col6 (type: string), _col7 (type: string), _col14 (type: string)
- | Select Operator [SEL_49]
- | outputColumnNames:["_col14","_col15","_col17","_col6","_col7"]
- | Statistics:Num rows: 731 Data size: 7775 Basic stats: COMPLETE Column stats: NONE
- | Merge Join Operator [MERGEJOIN_109]
- | | condition map:[{"":"Inner Join 0 to 1"}]
- | | keys:{"0":"_col4 (type: string), _col6 (type: string)","1":"_col2 (type: string), _col4 (type: string)"}
- | | outputColumnNames:["_col2","_col3","_col14","_col15","_col17"]
- | | Statistics:Num rows: 731 Data size: 7775 Basic stats: COMPLETE Column stats: NONE
- | |<-Reducer 10 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_45]
- | | key expressions:_col4 (type: string), _col6 (type: string)
- | | Map-reduce partition columns:_col4 (type: string), _col6 (type: string)
- | | sort order:++
- | | Statistics:Num rows: 665 Data size: 7069 Basic stats: COMPLETE Column stats: NONE
- | | value expressions:_col2 (type: string), _col3 (type: string)
- | | Merge Join Operator [MERGEJOIN_107]
- | | | condition map:[{"":"Inner Join 0 to 1"}]
- | | | keys:{"0":"_col3 (type: string)","1":"_col1 (type: string)"}
- | | | outputColumnNames:["_col2","_col3","_col4","_col6"]
- | | | Statistics:Num rows: 665 Data size: 7069 Basic stats: COMPLETE Column stats: NONE
- | | |<-Map 14 [SIMPLE_EDGE]
- | | | Reduce Output Operator [RS_42]
- | | | key expressions:_col1 (type: string)
- | | | Map-reduce partition columns:_col1 (type: string)
- | | | sort order:+
- | | | Statistics:Num rows: 6 Data size: 45 Basic stats: COMPLETE Column stats: NONE
- | | | Select Operator [SEL_16]
- | | | outputColumnNames:["_col1"]
- | | | Statistics:Num rows: 6 Data size: 45 Basic stats: COMPLETE Column stats: NONE
- | | | Filter Operator [FIL_101]
- | | | predicate:((key = 'src1key') and value is not null) (type: boolean)
- | | | Statistics:Num rows: 6 Data size: 45 Basic stats: COMPLETE Column stats: NONE
- | | | TableScan [TS_14]
- | | | alias:src1
- | | | Statistics:Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- | | |<-Reducer 9 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_40]
- | | key expressions:_col3 (type: string)
- | | Map-reduce partition columns:_col3 (type: string)
- | | sort order:+
- | | Statistics:Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
- | | value expressions:_col2 (type: string), _col4 (type: string), _col6 (type: string)
- | | Merge Join Operator [MERGEJOIN_106]
- | | | condition map:[{"":"Inner Join 0 to 1"}]
- | | | keys:{"0":"_col2 (type: string)","1":"_col0 (type: string)"}
- | | | outputColumnNames:["_col2","_col3","_col4","_col6"]
- | | | Statistics:Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
- | | |<-Map 13 [SIMPLE_EDGE]
- | | | Reduce Output Operator [RS_37]
- | | | key expressions:_col0 (type: string)
- | | | Map-reduce partition columns:_col0 (type: string)
- | | | sort order:+
- | | | Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- | | | Select Operator [SEL_13]
- | | | outputColumnNames:["_col0"]
- | | | Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- | | | Filter Operator [FIL_100]
- | | | predicate:((value = 'd1value') and key is not null) (type: boolean)
- | | | Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- | | | TableScan [TS_11]
- | | | alias:d1
- | | | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- | | |<-Reducer 8 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_35]
- | | key expressions:_col2 (type: string)
- | | Map-reduce partition columns:_col2 (type: string)
- | | sort order:+
- | | Statistics:Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- | | value expressions:_col3 (type: string), _col4 (type: string), _col6 (type: string)
- | | Merge Join Operator [MERGEJOIN_105]
- | | | condition map:[{"":"Inner Join 0 to 1"}]
- | | | keys:{"0":"_col1 (type: string)","1":"_col3 (type: string)"}
- | | | outputColumnNames:["_col2","_col3","_col4","_col6"]
- | | | Statistics:Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- | | |<-Map 12 [SIMPLE_EDGE]
- | | | Reduce Output Operator [RS_32]
- | | | key expressions:_col3 (type: string)
- | | | Map-reduce partition columns:_col3 (type: string)
- | | | sort order:+
- | | | Statistics:Num rows: 2 Data size: 69 Basic stats: COMPLETE Column stats: NONE
- | | | value expressions:_col0 (type: string), _col1 (type: string), _col2 (type: string), _col4 (type: string)
- | | | Select Operator [SEL_10]
- | | | outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
- | | | Statistics:Num rows: 2 Data size: 69 Basic stats: COMPLETE Column stats: NONE
- | | | Filter Operator [FIL_99]
- | | | predicate:((((((v3 = 'ssv3') and v2 is not null) and k1 is not null) and v1 is not null) and k2 is not null) and k3 is not null) (type: boolean)
- | | | Statistics:Num rows: 2 Data size: 69 Basic stats: COMPLETE Column stats: NONE
- | | | TableScan [TS_8]
- | | | alias:ss
- | | | Statistics:Num rows: 85 Data size: 2945 Basic stats: COMPLETE Column stats: NONE
- | | |<-Map 7 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_30]
- | | key expressions:_col1 (type: string)
- | | Map-reduce partition columns:_col1 (type: string)
- | | sort order:+
- | | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- | | Select Operator [SEL_7]
- | | outputColumnNames:["_col1"]
- | | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_98]
- | | predicate:((key = 'srcpartkey') and value is not null) (type: boolean)
- | | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_5]
- | | alias:srcpart
- | | Statistics:Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
- | |<-Reducer 16 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_47]
- | key expressions:_col2 (type: string), _col4 (type: string)
- | Map-reduce partition columns:_col2 (type: string), _col4 (type: string)
- | sort order:++
- | Statistics:Num rows: 137 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col3 (type: string), _col5 (type: string)
- | Merge Join Operator [MERGEJOIN_108]
- | | condition map:[{"":"Inner Join 0 to 1"}]
- | | keys:{"0":"_col0 (type: string)","1":"_col0 (type: string)"}
- | | outputColumnNames:["_col2","_col3","_col4","_col5"]
- | | Statistics:Num rows: 137 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
- | |<-Map 15 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_24]
- | | key expressions:_col0 (type: string)
- | | Map-reduce partition columns:_col0 (type: string)
- | | sort order:+
- | | Statistics:Num rows: 2 Data size: 69 Basic stats: COMPLETE Column stats: NONE
- | | value expressions:_col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
- | | Select Operator [SEL_19]
- | | outputColumnNames:["_col0","_col2","_col3","_col4","_col5"]
- | | Statistics:Num rows: 2 Data size: 69 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_102]
- | | predicate:((((((v1 = 'srv1') and k1 is not null) and k2 is not null) and k3 is not null) and v2 is not null) and v3 is not null) (type: boolean)
- | | Statistics:Num rows: 2 Data size: 69 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_17]
- | | alias:sr
- | | Statistics:Num rows: 85 Data size: 2945 Basic stats: COMPLETE Column stats: NONE
- | |<-Map 17 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_26]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_22]
- | outputColumnNames:["_col0"]
- | Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_103]
- | predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null) (type: boolean)
- | Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_20]
- | alias:d1
- | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 2 [SIMPLE_EDGE]
- Reduce Output Operator [RS_56]
- key expressions:_col1 (type: string), _col3 (type: string)
- Map-reduce partition columns:_col1 (type: string), _col3 (type: string)
- sort order:++
- Statistics:Num rows: 137 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col2 (type: string)
- Merge Join Operator [MERGEJOIN_104]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{"0":"_col0 (type: string)","1":"_col0 (type: string)"}
- | outputColumnNames:["_col1","_col2","_col3"]
- | Statistics:Num rows: 137 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_51]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 22 Data size: 762 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col1 (type: string), _col2 (type: string), _col3 (type: string)
- | Select Operator [SEL_1]
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 22 Data size: 762 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_96]
- | predicate:((k1 is not null and v2 is not null) and v3 is not null) (type: boolean)
- | Statistics:Num rows: 22 Data size: 762 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_0]
- | alias:cs
- | Statistics:Num rows: 170 Data size: 5890 Basic stats: COMPLETE Column stats: NONE
- |<-Map 6 [SIMPLE_EDGE]
- Reduce Output Operator [RS_53]
- key expressions:_col0 (type: string)
- Map-reduce partition columns:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_4]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_97]
- predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null) (type: boolean)
- Statistics:Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_2]
- alias:d1
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:100
+ Stage-1
+ Reducer 5 llap
+ File Output Operator [FS_55]
+ Limit [LIM_54] (rows=24 width=285)
+ Number of rows:100
+ Select Operator [SEL_53] (rows=24 width=285)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
+ <-Reducer 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_52]
+ Group By Operator [GBY_50] (rows=24 width=285)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(VALUE._col0)","count(VALUE._col1)","count(VALUE._col2)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Reducer 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_49]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_48] (rows=24 width=285)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5"],aggregations:["count(_col13)","count(_col21)","count(_col3)"],keys:_col12, _col20, _col2
+ Select Operator [SEL_47] (rows=650 width=534)
+ Output:["_col12","_col20","_col2","_col13","_col21","_col3"]
+ Merge Join Operator [MERGEJOIN_97] (rows=650 width=534)
+ Conds:RS_44._col1, _col3=RS_45._col15, _col17(Inner),Output:["_col2","_col3","_col12","_col13","_col20","_col21"]
+ <-Reducer 11 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_45]
+ PartitionCols:_col15, _col17
+ Select Operator [SEL_40] (rows=190 width=447)
+ Output:["_col14","_col15","_col17","_col6","_col7"]
+ Merge Join Operator [MERGEJOIN_96] (rows=190 width=447)
+ Conds:RS_37._col6, _col4=RS_38._col4, _col2(Inner),Output:["_col2","_col3","_col14","_col15","_col17"]
+ <-Reducer 10 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_37]
+ PartitionCols:_col6, _col4
+ Merge Join Operator [MERGEJOIN_94] (rows=40 width=352)
+ Conds:RS_34._col3=RS_35._col1(Inner),Output:["_col2","_col3","_col4","_col6"]
+ <-Map 14 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_35]
+ PartitionCols:_col1
+ Select Operator [SEL_17] (rows=2 width=180)
+ Output:["_col1"]
+ Filter Operator [FIL_88] (rows=2 width=175)
+ predicate:((key = 'src1key') and value is not null)
+ TableScan [TS_15] (rows=25 width=175)
+ default@src1,src1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 9 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_34]
+ PartitionCols:_col3
+ Merge Join Operator [MERGEJOIN_93] (rows=40 width=352)
+ Conds:RS_31._col2=RS_32._col0(Inner),Output:["_col2","_col3","_col4","_col6"]
+ <-Map 13 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_32]
+ PartitionCols:_col0
+ Select Operator [SEL_14] (rows=2 width=178)
+ Output:["_col0"]
+ Filter Operator [FIL_87] (rows=2 width=178)
+ predicate:((value = 'd1value') and key is not null)
+ TableScan [TS_12] (rows=500 width=178)
+ default@src,d1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 8 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_31]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_92] (rows=40 width=352)
+ Conds:RS_28._col1=RS_29._col3(Inner),Output:["_col2","_col3","_col4","_col6"]
+ <-Map 12 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_29]
+ PartitionCols:_col3
+ Select Operator [SEL_11] (rows=8 width=531)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_86] (rows=8 width=534)
+ predicate:((v3 = 'ssv3') and k2 is not null and k3 is not null and k1 is not null and v1 is not null and v2 is not null)
+ TableScan [TS_9] (rows=85 width=534)
+ default@ss,ss,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v1","k2","v2","k3","v3"]
+ <-Map 7 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_28]
+ PartitionCols:_col1
+ Select Operator [SEL_8] (rows=10 width=185)
+ Output:["_col1"]
+ Filter Operator [FIL_85] (rows=10 width=178)
+ predicate:((key = 'srcpartkey') and value is not null)
+ TableScan [TS_6] (rows=2000 width=178)
+ default@srcpart,srcpart,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 16 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_38]
+ PartitionCols:_col4, _col2
+ Merge Join Operator [MERGEJOIN_95] (rows=19 width=356)
+ Conds:RS_24._col0=RS_25._col0(Inner),Output:["_col2","_col3","_col4","_col5"]
+ <-Map 15 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_24]
+ PartitionCols:_col0
+ Select Operator [SEL_20] (rows=8 width=531)
+ Output:["_col0","_col2","_col3","_col4","_col5"]
+ Filter Operator [FIL_89] (rows=8 width=534)
+ predicate:((v1 = 'srv1') and k2 is not null and k3 is not null and v2 is not null and v3 is not null and k1 is not null)
+ TableScan [TS_18] (rows=85 width=534)
+ default@sr,sr,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v1","k2","v2","k3","v3"]
+ <-Map 17 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_25]
+ PartitionCols:_col0
+ Select Operator [SEL_23] (rows=500 width=178)
+ Output:["_col0"]
+ Filter Operator [FIL_90] (rows=500 width=178)
+ predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
+ TableScan [TS_21] (rows=500 width=178)
+ default@src,d2,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_44]
+ PartitionCols:_col1, _col3
+ Merge Join Operator [MERGEJOIN_91] (rows=414 width=269)
+ Conds:RS_41._col0=RS_42._col0(Inner),Output:["_col1","_col2","_col3"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_41]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=170 width=356)
+ Output:["_col0","_col1","_col2","_col3"]
+ Filter Operator [FIL_83] (rows=170 width=356)
+ predicate:(v2 is not null and v3 is not null and k1 is not null)
+ TableScan [TS_0] (rows=170 width=356)
+ default@cs,cs,Tbl:COMPLETE,Col:COMPLETE,Output:["k1","v2","k3","v3"]
+ <-Map 6 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_42]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=500 width=178)
+ Output:["_col0"]
+ Filter Operator [FIL_84] (rows=500 width=178)
+ predicate:((value) IN ('2000Q1', '2000Q2', '2000Q3') and key is not null)
+ TableScan [TS_3] (rows=500 width=178)
+ default@src,d3,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
PREHOOK: query: explain
SELECT x.key, z.value, y.value
@@ -612,245 +458,138 @@ Reducer 5 <- Map 10 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE), Union 6 (CONTAINS)
Reducer 7 <- Union 6 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 7
- File Output Operator [FS_59]
- compressed:false
- Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [GBY_57]
- | keys:KEY._col0 (type: string), KEY._col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- |<-Union 6 [SIMPLE_EDGE]
- |<-Reducer 15 [CONTAINS]
- | Reduce Output Operator [RS_56]
- | key expressions:_col0 (type: string), _col1 (type: string)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | sort order:++
- | Statistics:Num rows: 550 Data size: 5842 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_55]
- | keys:_col0 (type: string), _col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 550 Data size: 5842 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_51]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- | Merge Join Operator [MERGEJOIN_85]
- | | condition map:[{"":"Inner Join 0 to 1"}]
- | | keys:{"0":"_col2 (type: string)","1":"_col0 (type: string)"}
- | | outputColumnNames:["_col1","_col2"]
- | | Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- | |<-Map 18 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_49]
- | | key expressions:_col0 (type: string)
- | | Map-reduce partition columns:_col0 (type: string)
- | | sort order:+
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | Select Operator [SEL_40]
- | | outputColumnNames:["_col0"]
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_81]
- | | predicate:key is not null (type: boolean)
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_39]
- | | alias:y
- | | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- | |<-Reducer 14 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_47]
- | key expressions:_col2 (type: string)
- | Map-reduce partition columns:_col2 (type: string)
- | sort order:+
- | Statistics:Num rows: 144 Data size: 1509 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col1 (type: string)
- | Merge Join Operator [MERGEJOIN_84]
- | | condition map:[{"":"Inner Join 0 to 1"}]
- | | keys:{"0":"_col1 (type: string)","1":"_col1 (type: string)"}
- | | outputColumnNames:["_col1","_col2"]
- | | Statistics:Num rows: 144 Data size: 1509 Basic stats: COMPLETE Column stats: NONE
- | |<-Map 17 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_44]
- | | key expressions:_col1 (type: string)
- | | Map-reduce partition columns:_col1 (type: string)
- | | sort order:+
- | | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | | value expressions:_col0 (type: string)
- | | Select Operator [SEL_38]
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_80]
- | | predicate:(value is not null and key is not null) (type: boolean)
- | | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_37]
- | | alias:x
- | | Statistics:Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- | |<-Reducer 13 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_42]
- | key expressions:_col1 (type: string)
- | Map-reduce partition columns:_col1 (type: string)
- | sort order:+
- | Statistics:Num rows: 131 Data size: 1372 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_36]
- | outputColumnNames:["_col1"]
- | Statistics:Num rows: 131 Data size: 1372 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_35]
- | | keys:KEY._col0 (type: string), KEY._col1 (type: string)
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 131 Data size: 1372 Basic stats: COMPLETE Column stats: NONE
- | |<-Union 12 [SIMPLE_EDGE]
- | |<-Map 11 [CONTAINS]
- | | Reduce Output Operator [RS_34]
- | | key expressions:_col0 (type: string), _col1 (type: string)
- | | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | | sort order:++
- | | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | | Group By Operator [GBY_33]
- | | keys:_col0 (type: string), _col1 (type: string)
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | | Select Operator [SEL_27]
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_78]
- | | predicate:value is not null (type: boolean)
- | | Statistics:Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_26]
- | | alias:x
- | | Statistics:Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- | |<-Map 16 [CONTAINS]
- | Reduce Output Operator [RS_34]
- | key expressions:_col0 (type: string), _col1 (type: string)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | sort order:++
- | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_33]
- | keys:_col0 (type: string), _col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_29]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_79]
- | predicate:value is not null (type: boolean)
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_28]
- | alias:y
- | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 5 [CONTAINS]
- Reduce Output Operator [RS_56]
- key expressions:_col0 (type: string), _col1 (type: string)
- Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- sort order:++
- Statistics:Num rows: 550 Data size: 5842 Basic stats: COMPLETE Column stats: NONE
- Group By Operator [GBY_55]
- keys:_col0 (type: string), _col1 (type: string)
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 550 Data size: 5842 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_25]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator [MERGEJOIN_83]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{"0":"_col2 (type: string)","1":"_col0 (type: string)"}
- | outputColumnNames:["_col1","_col2"]
- | Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- |<-Map 10 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_23]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_14]
- | outputColumnNames:["_col0"]
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_77]
- | predicate:key is not null (type: boolean)
- | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_13]
- | alias:y
- | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 4 [SIMPLE_EDGE]
- Reduce Output Operator [RS_21]
- key expressions:_col2 (type: string)
- Map-reduce partition columns:_col2 (type: string)
- sort order:+
- Statistics:Num rows: 144 Data size: 1509 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col1 (type: string)
- Merge Join Operator [MERGEJOIN_82]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{"0":"_col1 (type: string)","1":"_col1 (type: string)"}
- | outputColumnNames:["_col1","_col2"]
- | Statistics:Num rows: 144 Data size: 1509 Basic stats: COMPLETE Column stats: NONE
- |<-Map 9 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_18]
- | key expressions:_col1 (type: string)
- | Map-reduce partition columns:_col1 (type: string)
- | sort order:+
- | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col0 (type: string)
- | Select Operator [SEL_12]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_76]
- | predicate:(value is not null and key is not null) (type: boolean)
- | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_11]
- | alias:x
- | Statistics:Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 3 [SIMPLE_EDGE]
- Reduce Output Operator [RS_16]
- key expressions:_col1 (type: string)
- Map-reduce partition columns:_col1 (type: string)
- sort order:+
- Statistics:Num rows: 131 Data size: 1372 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_10]
- outputColumnNames:["_col1"]
- Statistics:Num rows: 131 Data size: 1372 Basic stats: COMPLETE Column stats: NONE
- Group By Operator [GBY_9]
- | keys:KEY._col0 (type: string), KEY._col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 131 Data size: 1372 Basic stats: COMPLETE Column stats: NONE
- |<-Union 2 [SIMPLE_EDGE]
- |<-Map 1 [CONTAINS]
- | Reduce Output Operator [RS_8]
- | key expressions:_col0 (type: string), _col1 (type: string)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | sort order:++
- | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_7]
- | keys:_col0 (type: string), _col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_1]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_74]
- | predicate:value is not null (type: boolean)
- | Statistics:Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_0]
- | alias:x
- | Statistics:Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- |<-Map 8 [CONTAINS]
- Reduce Output Operator [RS_8]
- key expressions:_col0 (type: string), _col1 (type: string)
- Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- sort order:++
- Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- Group By Operator [GBY_7]
- keys:_col0 (type: string), _col1 (type: string)
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_3]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_75]
- predicate:value is not null (type: boolean)
- Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_2]
- alias:y
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 7 llap
+ File Output Operator [FS_59]
+ Group By Operator [GBY_57] (rows=28 width=177)
+ Output:["_col0","_col1"],keys:KEY._col0, KEY._col1
+ <-Union 6 [SIMPLE_EDGE]
+ <-Reducer 15 [CONTAINS] llap
+ Reduce Output Operator [RS_56]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_55] (rows=28 width=177)
+ Output:["_col0","_col1"],keys:_col0, _col1
+ Select Operator [SEL_51] (rows=73 width=177)
+ Output:["_col0","_col1"]
+ Merge Join Operator [MERGEJOIN_85] (rows=73 width=177)
+ Conds:RS_48._col2=RS_49._col0(Inner),Output:["_col1","_col2"]
+ <-Map 18 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_49]
+ PartitionCols:_col0
+ Select Operator [SEL_44] (rows=500 width=87)
+ Output:["_col0"]
+ Filter Operator [FIL_81] (rows=500 width=87)
+ predicate:key is not null
+ TableScan [TS_42] (rows=500 width=87)
+ default@src,y,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ <-Reducer 14 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_48]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_84] (rows=30 width=177)
+ Conds:RS_45._col1=RS_46._col1(Inner),Output:["_col1","_col2"]
+ <-Map 17 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_46]
+ PartitionCols:_col1
+ Select Operator [SEL_41] (rows=25 width=175)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_80] (rows=25 width=175)
+ predicate:(key is not null and value is not null)
+ TableScan [TS_39] (rows=25 width=175)
+ default@src1,x,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 13 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_45]
+ PartitionCols:_col1
+ Select Operator [SEL_38] (rows=262 width=178)
+ Output:["_col1"]
+ Group By Operator [GBY_37] (rows=262 width=178)
+ Output:["_col0","_col1"],keys:KEY._col0, KEY._col1
+ <-Union 12 [SIMPLE_EDGE]
+ <-Map 11 [CONTAINS] llap
+ Reduce Output Operator [RS_36]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_35] (rows=262 width=178)
+ Output:["_col0","_col1"],keys:_col1, _col0
+ Select Operator [SEL_28] (rows=25 width=175)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_78] (rows=25 width=175)
+ predicate:value is not null
+ TableScan [TS_26] (rows=25 width=175)
+ Output:["key","value"]
+ <-Map 16 [CONTAINS] llap
+ Reduce Output Operator [RS_36]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_35] (rows=262 width=178)
+ Output:["_col0","_col1"],keys:_col1, _col0
+ Select Operator [SEL_31] (rows=500 width=178)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_79] (rows=500 width=178)
+ predicate:value is not null
+ TableScan [TS_29] (rows=500 width=178)
+ Output:["key","value"]
+ <-Reducer 5 [CONTAINS] llap
+ Reduce Output Operator [RS_56]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_55] (rows=28 width=177)
+ Output:["_col0","_col1"],keys:_col0, _col1
+ Select Operator [SEL_25] (rows=73 width=177)
+ Output:["_col0","_col1"]
+ Merge Join Operator [MERGEJOIN_83] (rows=73 width=177)
+ Conds:RS_22._col2=RS_23._col0(Inner),Output:["_col1","_col2"]
+ <-Map 10 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_23]
+ PartitionCols:_col0
+ Select Operator [SEL_18] (rows=500 width=87)
+ Output:["_col0"]
+ Filter Operator [FIL_77] (rows=500 width=87)
+ predicate:key is not null
+ TableScan [TS_16] (rows=500 width=87)
+ default@src,y,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ <-Reducer 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_22]
+ PartitionCols:_col2
+ Merge Join Operator [MERGEJOIN_82] (rows=30 width=177)
+ Conds:RS_19._col1=RS_20._col1(Inner),Output:["_col1","_col2"]
+ <-Map 9 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_20]
+ PartitionCols:_col1
+ Select Operator [SEL_15] (rows=25 width=175)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_76] (rows=25 width=175)
+ predicate:(key is not null and value is not null)
+ TableScan [TS_13] (rows=25 width=175)
+ default@src1,x,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Reducer 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_19]
+ PartitionCols:_col1
+ Select Operator [SEL_12] (rows=262 width=178)
+ Output:["_col1"]
+ Group By Operator [GBY_11] (rows=262 width=178)
+ Output:["_col0","_col1"],keys:KEY._col0, KEY._col1
+ <-Union 2 [SIMPLE_EDGE]
+ <-Map 1 [CONTAINS] llap
+ Reduce Output Operator [RS_10]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_9] (rows=262 width=178)
+ Output:["_col0","_col1"],keys:_col1, _col0
+ Select Operator [SEL_2] (rows=25 width=175)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_74] (rows=25 width=175)
+ predicate:value is not null
+ TableScan [TS_0] (rows=25 width=175)
+ Output:["key","value"]
+ <-Map 8 [CONTAINS] llap
+ Reduce Output Operator [RS_10]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_9] (rows=262 width=178)
+ Output:["_col0","_col1"],keys:_col1, _col0
+ Select Operator [SEL_5] (rows=500 width=178)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_75] (rows=500 width=178)
+ predicate:value is not null
+ TableScan [TS_3] (rows=500 width=178)
+ Output:["key","value"]
PREHOOK: query: explain
SELECT x.key, y.value
@@ -906,475 +645,271 @@ Reducer 7 <- Union 6 (SIMPLE_EDGE), Union 8 (CONTAINS)
Reducer 9 <- Union 8 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 9
- File Output Operator [FS_119]
- compressed:false
- Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [GBY_117]
- | keys:KEY._col0 (type: string), KEY._col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- |<-Union 8 [SIMPLE_EDGE]
- |<-Reducer 32 [CONTAINS]
- | Reduce Output Operator [RS_116]
- | key expressions:_col0 (type: string), _col1 (type: string)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | sort order:++
- | Statistics:Num rows: 550 Data size: 5842 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_115]
- | keys:_col0 (type: string), _col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 550 Data size: 5842 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_111]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- | Merge Join Operator [MERGEJOIN_167]
- | | condition map:[{"":"Inner Join 0 to 1"}]
- | | keys:{"0":"_col2 (type: string)","1":"_col0 (type: string)"}
- | | outputColumnNames:["_col2","_col5"]
- | | Statistics:Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- | |<-Map 37 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_109]
- | | key expressions:_col0 (type: string)
- | | Map-reduce partition columns:_col0 (type: string)
- | | sort order:+
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | value expressions:_col1 (type: string)
- | | Select Operator [SEL_100]
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_161]
- | | predicate:key is not null (type: boolean)
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_99]
- | | alias:y
- | | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- | |<-Reducer 31 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_107]
- | key expressions:_col2 (type: string)
- | Map-reduce partition columns:_col2 (type: string)
- | sort order:+
- | Statistics:Num rows: 242 Data size: 2565 Basic stats: COMPLETE Column stats: NONE
- | Merge Join Operator [MERGEJOIN_166]
- | | condition map:[{"":"Inner Join 0 to 1"}]
- | | keys:{"0":"_col1 (type: string)","1":"_col1 (type: string)"}
- | | outputColumnNames:["_col2"]
- | | Statistics:Num rows: 242 Data size: 2565 Basic stats: COMPLETE Column stats: NONE
- | |<-Map 36 [SIMPLE_EDGE]
- | | Reduce Output Operator [RS_104]
- | | key expressions:_col1 (type: string)
- | | Map-reduce partition columns:_col1 (type: string)
- | | sort order:+
- | | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | | value expressions:_col0 (type: string)
- | | Select Operator [SEL_98]
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_160]
- | | predicate:(value is not null and key is not null) (type: boolean)
- | | Statistics:Num rows: 7 Data size: 53 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_97]
- | | alias:x
- | | Statistics:Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- | |<-Reducer 30 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_102]
- | key expressions:_col1 (type: string)
- | Map-reduce partition columns:_col1 (type: string)
- | sort order:+
- | Statistics:Num rows: 220 Data size: 2332 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_96]
- | outputColumnNames:["_col1"]
- | Statistics:Num rows: 220 Data size: 2332 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_95]
- | | keys:KEY._col0 (type: string), KEY._col1 (type: string)
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 220 Data size: 2332 Basic stats: COMPLETE Column stats: NONE
- | |<-Union 29 [SIMPLE_EDGE]
- | |<-Map 35 [CONTAINS]
- | | Reduce Output Operator [RS_94]
- | | key expressions:_col0 (type: string), _col1 (type: string)
- | | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | | sort order:++
- | | Statistics:Num rows: 440 Data size: 4664 Basic stats: COMPLETE Column stats: NONE
- | | Group By Operator [GBY_93]
- | | keys:_col0 (type: string), _col1 (type: string)
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 440 Data size: 4664 Basic stats: COMPLETE Column stats: NONE
- | | Select Operator [SEL_89]
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_159]
- | | predicate:value is not null (type: boolean)
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_88]
- | | alias:y
- | | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- | |<-Reducer 28 [CONTAINS]
- | Reduce Output Operator [RS_94]
- | key expressions:_col0 (type: string), _col1 (type: string)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | sort order:++
- | Statistics:Num rows: 440 Data size: 4664 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_93]
- | keys:_col0 (type: string), _col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 440 Data size: 4664 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_86]
- | | keys:KEY._col0 (type: string), KEY._col1 (type: string)
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 190 Data size: 2008 Basic stats: COMPLETE Column stats: NONE
- | |<-Union 27 [SIMPLE_EDGE]
- | |<-Map 34 [CONTAINS]
- | | Reduce Output Operator [RS_85]
- | | key expressions:_col0 (type: string), _col1 (type: string)
- | | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | | sort order:++
- | | Statistics:Num rows: 381 Data size: 4028 Basic stats: COMPLETE Column stats: NONE
- | | Group By Operator [GBY_84]
- | | keys:_col0 (type: string), _col1 (type: string)
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 381 Data size: 4028 Basic stats: COMPLETE Column stats: NONE
- | | Select Operator [SEL_80]
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_158]
- | | predicate:value is not null (type: boolean)
- | | Statistics:Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_79]
- | | alias:y
- | | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- | |<-Reducer 26 [CONTAINS]
- | Reduce Output Operator [RS_85]
- | key expressions:_col0 (type: string), _col1 (type: string)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | sort order:++
- | Statistics:Num rows: 381 Data size: 4028 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_84]
- | keys:_col0 (type: string), _col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 381 Data size: 4028 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_77]
- | | keys:KEY._col0 (type: string), KEY._col1 (type: string)
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 131 Data size: 1372 Basic stats: COMPLETE Column stats: NONE
- | |<-Union 25 [SIMPLE_EDGE]
- | |<-Map 24 [CONTAINS]
- | | Reduce Output Operator [RS_76]
- | | key expressions:_col0 (type: string), _col1 (type: string)
- | | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | | sort order:++
- | | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | | Group By Operator [GBY_75]
- | | keys:_col0 (type: string), _col1 (type: string)
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | | Select Operator [SEL_69]
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
- | | Filter Operator [FIL_156]
- | | predicate:value is not null (type: boolean)
- | | Statistics:Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
- | | TableScan [TS_68]
- | | alias:x
- | | Statistics:Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
- | |<-Map 33 [CONTAINS]
- | Reduce Output Operator [RS_76]
- | key expressions:_col0 (type: string), _col1 (type: string)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: string)
- | sort order:++
- | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | Group By Operator [GBY_75]
- | keys:_col0 (type: string), _col1 (type: string)
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 263 Data size: 2755 Basic stats: COMPLETE Column stats: NONE
- | Select Operator [SEL_71]
- | outputColumnNames:["_col0","_col
<TRUNCATED>
[20/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/metadata_only_queries.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/metadata_only_queries.q.out b/ql/src/test/results/clientpositive/llap/metadata_only_queries.q.out
index ed3179d..04b556e 100644
--- a/ql/src/test/results/clientpositive/llap/metadata_only_queries.q.out
+++ b/ql/src/test/results/clientpositive/llap/metadata_only_queries.q.out
@@ -189,6 +189,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -206,26 +207,27 @@ STAGE PLANS:
aggregations: count(), sum(1), sum(0.2), count(1), count(_col2), count(_col3), count(_col4), count(_col5), max(_col6), min(_col7)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
- Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: double), _col3 (type: bigint), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: int), _col9 (type: bigint)
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: decimal(11,1)), _col3 (type: bigint), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: int), _col9 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), sum(VALUE._col1), sum(VALUE._col2), count(VALUE._col3), count(VALUE._col4), count(VALUE._col5), count(VALUE._col6), count(VALUE._col7), max(VALUE._col8), min(VALUE._col9)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
- Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -247,6 +249,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -255,35 +258,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: stats_tbl_part
- Statistics: Num rows: 9489 Data size: 978785 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9489 Data size: 1054697 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: s (type: string), bo (type: boolean), bin (type: binary), si (type: smallint), i (type: int), b (type: bigint)
outputColumnNames: _col2, _col3, _col4, _col5, _col6, _col7
- Statistics: Num rows: 9489 Data size: 978785 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9489 Data size: 1054697 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), sum(1), sum(0.2), count(1), count(_col2), count(_col3), count(_col4), count(_col5), max(_col6), min(_col7)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
- Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: double), _col3 (type: bigint), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: int), _col9 (type: bigint)
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: decimal(11,1)), _col3 (type: bigint), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: int), _col9 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), sum(VALUE._col1), sum(VALUE._col2), count(VALUE._col3), count(VALUE._col4), count(VALUE._col5), count(VALUE._col6), count(VALUE._col7), max(VALUE._col8), min(VALUE._col9)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
- Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -292,6 +296,134 @@ STAGE PLANS:
Processor Tree:
ListSink
+PREHOOK: query: explain
+select count(*), '1' as one, sum(1), sum(0.2), 2 as two, count(1), count(s), 3+4.0 as three, count(bo), count(bin), count(si), max(i), min(b) from stats_tbl
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select count(*), '1' as one, sum(1), sum(0.2), 2 as two, count(1), count(s), 3+4.0 as three, count(bo), count(bin), count(si), max(i), min(b) from stats_tbl
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: stats_tbl
+ Statistics: Num rows: 9999 Data size: 1030908 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: s (type: string), bo (type: boolean), bin (type: binary), si (type: smallint), i (type: int), b (type: bigint)
+ outputColumnNames: _col2, _col3, _col4, _col5, _col6, _col7
+ Statistics: Num rows: 9999 Data size: 1030908 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count(), sum(1), sum(0.2), count(1), count(_col2), count(_col3), count(_col4), count(_col5), max(_col6), min(_col7)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: decimal(11,1)), _col3 (type: bigint), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: int), _col9 (type: bigint)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0), sum(VALUE._col1), sum(VALUE._col2), count(VALUE._col3), count(VALUE._col4), count(VALUE._col5), count(VALUE._col6), count(VALUE._col7), max(VALUE._col8), min(VALUE._col9)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: bigint), '1' (type: string), _col1 (type: bigint), _col2 (type: decimal(11,1)), 2 (type: int), _col3 (type: bigint), _col4 (type: bigint), 7 (type: decimal(11,0)), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: int), _col9 (type: bigint)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain
+select count(*), '1' as one, sum(1), sum(0.2), 2 as two, count(1), count(s), 3+4.0 as three, count(bo), count(bin), count(si), max(i), min(b) from stats_tbl_part
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select count(*), '1' as one, sum(1), sum(0.2), 2 as two, count(1), count(s), 3+4.0 as three, count(bo), count(bin), count(si), max(i), min(b) from stats_tbl_part
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: stats_tbl_part
+ Statistics: Num rows: 9489 Data size: 1054697 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: s (type: string), bo (type: boolean), bin (type: binary), si (type: smallint), i (type: int), b (type: bigint)
+ outputColumnNames: _col2, _col3, _col4, _col5, _col6, _col7
+ Statistics: Num rows: 9489 Data size: 1054697 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count(), sum(1), sum(0.2), count(1), count(_col2), count(_col3), count(_col4), count(_col5), max(_col6), min(_col7)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: decimal(11,1)), _col3 (type: bigint), _col4 (type: bigint), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: int), _col9 (type: bigint)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0), sum(VALUE._col1), sum(VALUE._col2), count(VALUE._col3), count(VALUE._col4), count(VALUE._col5), count(VALUE._col6), count(VALUE._col7), max(VALUE._col8), min(VALUE._col9)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: bigint), '1' (type: string), _col1 (type: bigint), _col2 (type: decimal(11,1)), 2 (type: int), _col3 (type: bigint), _col4 (type: bigint), 7 (type: decimal(11,0)), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: int), _col9 (type: bigint)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 180 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
PREHOOK: query: analyze table stats_tbl compute statistics for columns t,si,i,b,f,d,bo,s,bin
PREHOOK: type: QUERY
PREHOOK: Input: default@stats_tbl
@@ -354,7 +486,7 @@ POSTHOOK: query: select count(*), sum(1), sum(0.2), count(1), count(s), count(bo
POSTHOOK: type: QUERY
POSTHOOK: Input: default@stats_tbl
#### A masked pattern was here ####
-9999 9999 1999.8000000000002 9999 9999 9999 9999 9999
+9999 9999 1999.8 9999 9999 9999 9999 9999
PREHOOK: query: explain
select min(i), max(i), min(b), max(b), min(f), max(f), min(d), max(d) from stats_tbl
PREHOOK: type: QUERY
@@ -380,6 +512,31 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@stats_tbl
#### A masked pattern was here ####
65536 65791 4294967296 4294967551 0.01 99.98 0.01 50.0
+PREHOOK: query: explain
+select min(i), '1' as one, max(i), min(b), max(b), min(f), max(f), 3+4.0 as three, min(d), max(d) from stats_tbl
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select min(i), '1' as one, max(i), min(b), max(b), min(f), max(f), 3+4.0 as three, min(d), max(d) from stats_tbl
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-0 is a root stage
+
+STAGE PLANS:
+ Stage: Stage-0
+ Fetch Operator
+ limit: 1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select min(i), '1' as one, max(i), min(b), max(b), min(f), max(f), 3+4.0 as three, min(d), max(d) from stats_tbl
+PREHOOK: type: QUERY
+PREHOOK: Input: default@stats_tbl
+#### A masked pattern was here ####
+POSTHOOK: query: select min(i), '1' as one, max(i), min(b), max(b), min(f), max(f), 3+4.0 as three, min(d), max(d) from stats_tbl
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@stats_tbl
+#### A masked pattern was here ####
+65536 1 65791 4294967296 4294967551 0.01 99.98 7 0.01 50.0
PREHOOK: query: explain
select count(*), sum(1), sum(0.2), count(1), count(s), count(bo), count(bin), count(si) from stats_tbl_part
PREHOOK: type: QUERY
@@ -404,7 +561,7 @@ POSTHOOK: query: select count(*), sum(1), sum(0.2), count(1), count(s), count(bo
POSTHOOK: type: QUERY
POSTHOOK: Input: default@stats_tbl_part
#### A masked pattern was here ####
-9489 9489 1897.8000000000002 9489 9489 9489 9489 9489
+9489 9489 1897.8 9489 9489 9489 9489 9489
PREHOOK: query: explain
select min(i), max(i), min(b), max(b), min(f), max(f), min(d), max(d) from stats_tbl_part
PREHOOK: type: QUERY
@@ -430,6 +587,31 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@stats_tbl_part
#### A masked pattern was here ####
65536 65791 4294967296 4294967551 0.01 99.98 0.01 50.0
+PREHOOK: query: explain
+select min(i), '1' as one, max(i), min(b), max(b), min(f), max(f), 3+4.0 as three, min(d), max(d) from stats_tbl_part
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select min(i), '1' as one, max(i), min(b), max(b), min(f), max(f), 3+4.0 as three, min(d), max(d) from stats_tbl_part
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-0 is a root stage
+
+STAGE PLANS:
+ Stage: Stage-0
+ Fetch Operator
+ limit: 1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select min(i), '1' as one, max(i), min(b), max(b), min(f), max(f), 3+4.0 as three, min(d), max(d) from stats_tbl_part
+PREHOOK: type: QUERY
+PREHOOK: Input: default@stats_tbl_part
+#### A masked pattern was here ####
+POSTHOOK: query: select min(i), '1' as one, max(i), min(b), max(b), min(f), max(f), 3+4.0 as three, min(d), max(d) from stats_tbl_part
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@stats_tbl_part
+#### A masked pattern was here ####
+65536 1 65791 4294967296 4294967551 0.01 99.98 7 0.01 50.0
PREHOOK: query: explain select count(ts) from stats_tbl_part
PREHOOK: type: QUERY
POSTHOOK: query: explain select count(ts) from stats_tbl_part
@@ -441,6 +623,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -449,11 +632,11 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: stats_tbl_part
- Statistics: Num rows: 9489 Data size: 978785 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9489 Data size: 1054697 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: ts (type: timestamp)
outputColumnNames: ts
- Statistics: Num rows: 9489 Data size: 978785 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9489 Data size: 1054697 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(ts)
mode: hash
@@ -464,8 +647,9 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -476,8 +660,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -486,6 +670,52 @@ STAGE PLANS:
Processor Tree:
ListSink
+PREHOOK: query: explain select count('1') from stats_tbl group by '1'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select count('1') from stats_tbl group by '1'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-0 is a root stage
+
+STAGE PLANS:
+ Stage: Stage-0
+ Fetch Operator
+ limit: 1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count('1') from stats_tbl group by '1'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@stats_tbl
+#### A masked pattern was here ####
+POSTHOOK: query: select count('1') from stats_tbl group by '1'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@stats_tbl
+#### A masked pattern was here ####
+9999
+PREHOOK: query: explain select count('1') from stats_tbl_part group by '1'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select count('1') from stats_tbl_part group by '1'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-0 is a root stage
+
+STAGE PLANS:
+ Stage: Stage-0
+ Fetch Operator
+ limit: 1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count('1') from stats_tbl_part group by '1'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@stats_tbl_part
+#### A masked pattern was here ####
+POSTHOOK: query: select count('1') from stats_tbl_part group by '1'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@stats_tbl_part
+#### A masked pattern was here ####
+9489
PREHOOK: query: drop table stats_tbl
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@stats_tbl
[44/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out
index c237025..3e4f408 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out
@@ -122,38 +122,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- Since size is being used to find the big table, the order of the tables in the join does not matter
explain extended select count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -161,6 +129,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -170,20 +139,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 4 Data size: 452 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -195,7 +170,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -243,7 +217,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -290,36 +263,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 54 Data size: 5500 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 2
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 29 Data size: 3025 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 4
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
+ Statistics: Num rows: 59 Data size: 6525 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -331,7 +309,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -379,7 +356,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -423,7 +399,7 @@ STAGE PLANS:
/bucket_big/ds=2008-04-08 [b]
/bucket_big/ds=2008-04-09 [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -439,13 +415,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -482,38 +459,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -521,6 +466,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -530,36 +476,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 54 Data size: 5500 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 2
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 29 Data size: 3025 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 4
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 59 Data size: 6525 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -571,7 +522,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -619,7 +569,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -666,20 +615,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 4 Data size: 452 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -691,7 +646,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -739,7 +693,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -783,7 +736,7 @@ STAGE PLANS:
/bucket_small/ds=2008-04-08 [b]
/bucket_small/ds=2008-04-09 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -799,13 +752,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -842,38 +796,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -881,6 +803,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -890,36 +813,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 54 Data size: 5500 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 2
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 29 Data size: 3025 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 4
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 59 Data size: 6525 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -931,7 +859,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -979,7 +906,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -1026,20 +952,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 4 Data size: 452 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -1051,7 +983,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -1099,7 +1030,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -1143,7 +1073,7 @@ STAGE PLANS:
/bucket_small/ds=2008-04-08 [b]
/bucket_small/ds=2008-04-09 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -1159,13 +1089,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_8.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_8.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_8.q.out
index 7c51746..dae32a5 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_8.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_8.q.out
@@ -122,38 +122,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- Since size is being used to find the big table, the order of the tables in the join does not matter
explain extended select count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -161,6 +129,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -170,20 +139,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 2 Data size: 228 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -195,7 +170,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -243,7 +217,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -290,36 +263,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -331,7 +309,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -379,7 +356,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -423,7 +399,7 @@ STAGE PLANS:
/bucket_big/ds=2008-04-08 [b]
/bucket_big/ds=2008-04-09 [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -439,13 +415,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -482,38 +459,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -521,6 +466,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -530,36 +476,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -571,7 +522,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -619,7 +569,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -666,20 +615,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 2 Data size: 228 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -691,7 +646,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -739,7 +693,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -783,7 +736,7 @@ STAGE PLANS:
/bucket_small/ds=2008-04-08 [b]
/bucket_small/ds=2008-04-09 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -799,13 +752,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -844,38 +798,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- The mapjoin should fail resulting in the sort-merge join
explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -883,6 +805,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -892,36 +815,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -933,7 +861,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -981,7 +908,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -1028,20 +954,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 2 Data size: 228 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -1053,7 +985,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -1101,7 +1032,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -1145,7 +1075,7 @@ STAGE PLANS:
/bucket_small/ds=2008-04-08 [b]
/bucket_small/ds=2008-04-09 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -1161,13 +1091,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
[33/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/disable_merge_for_bucketing.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/disable_merge_for_bucketing.q.out b/ql/src/test/results/clientpositive/llap/disable_merge_for_bucketing.q.out
index 6de3e3a..600a266 100644
--- a/ql/src/test/results/clientpositive/llap/disable_merge_for_bucketing.q.out
+++ b/ql/src/test/results/clientpositive/llap/disable_merge_for_bucketing.q.out
@@ -14,23 +14,6 @@ POSTHOOK: query: explain extended
insert overwrite table bucket2_1
select * from src
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_TAB
- TOK_TABNAME
- bucket2_1
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-2 depends on stages: Stage-1
@@ -40,6 +23,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -48,20 +32,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string), _col1 (type: string)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -71,7 +57,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -91,7 +77,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -112,24 +98,25 @@ STAGE PLANS:
Truncated Path -> Alias:
/src [src]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Select Operator
expressions: UDFToInteger(VALUE._col0) (type: int), VALUE._col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count 2
bucket_field_name key
columns key,value
@@ -137,9 +124,13 @@ STAGE PLANS:
columns.types int:string
#### A masked pattern was here ####
name default.bucket2_1
+ numFiles 0
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket2_1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.bucket2_1
@@ -159,6 +150,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count 2
bucket_field_name key
columns key,value
@@ -166,9 +158,13 @@ STAGE PLANS:
columns.types int:string
#### A masked pattern was here ####
name default.bucket2_1
+ numFiles 0
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket2_1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.bucket2_1
@@ -202,6 +198,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -224,8 +221,9 @@ STAGE PLANS:
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: int), VALUE._col0 (type: string)
@@ -235,8 +233,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
[41/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out b/ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out
index 1fff22b..20702f9 100644
--- a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out
+++ b/ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out
@@ -132,19 +132,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -152,14 +152,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -169,14 +169,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -217,39 +217,39 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab_part
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: key (type: int), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -259,11 +259,11 @@ STAGE PLANS:
keys: KEY._col0 (type: int), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 4656 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 4656 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -272,7 +272,7 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 4
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 275 Data size: 5121 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -354,33 +354,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -390,7 +390,7 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -399,7 +399,7 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 4
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -415,19 +415,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: d
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -506,52 +506,52 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: d
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -561,7 +561,7 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
0 Map 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -570,7 +570,7 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
0 Map 1
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -658,25 +658,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcbucket_mapjoin
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), substr(value, 5) (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(_col1)
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: double)
Execution mode: llap
LLAP IO: no inputs
@@ -684,14 +684,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -701,14 +701,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
0 Reducer 2
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: int), _col0 (type: double), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -723,16 +723,16 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: double), _col0 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: int)
sort order: +
Map-reduce partition columns: _col1 (type: int)
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: double)
Stage: Stage-0
@@ -768,14 +768,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab_part
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -785,22 +785,22 @@ STAGE PLANS:
outputColumnNames: _col1, _col2
input vertices:
1 Map 3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: int), substr(_col2, 5) (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(_col1)
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: double)
Execution mode: llap
LLAP IO: no inputs
@@ -808,19 +808,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -828,19 +828,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -852,11 +852,11 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 275 Data size: 5121 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: double), _col0 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 275 Data size: 5121 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -866,14 +866,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
1 Map 4
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 302 Data size: 5633 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: int), _col0 (type: double), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 302 Data size: 5633 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 302 Data size: 5633 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -912,14 +912,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -929,22 +929,22 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 3
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), substr(_col1, 5) (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(_col1)
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: double)
Execution mode: llap
LLAP IO: no inputs
@@ -952,38 +952,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -995,11 +995,11 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 133 Data size: 1411 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 133 Data size: 2476 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: double), _col0 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 133 Data size: 1411 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 133 Data size: 2476 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1009,14 +1009,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
1 Map 4
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: int), _col0 (type: double), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1054,14 +1054,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1074,14 +1074,14 @@ STAGE PLANS:
input vertices:
1 Map 2
2 Map 3
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 20486 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 20486 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 20486 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1092,19 +1092,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -1112,19 +1112,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
@@ -1158,14 +1158,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1175,7 +1175,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1185,14 +1185,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
1 Map 3
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1203,38 +1203,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -1274,25 +1274,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcbucket_mapjoin
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), substr(value, 5) (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(_col1)
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: double)
Execution mode: llap
LLAP IO: no inputs
@@ -1300,14 +1300,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1317,14 +1317,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
0 Reducer 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: double), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1339,12 +1339,12 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: double)
Stage: Stage-0
@@ -1380,19 +1380,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcbucket_mapjoin
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), substr(value, 5) (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 27 Data size: 2808 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 3024 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -1400,14 +1400,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1417,14 +1417,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
0 Reducer 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: double), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1439,12 +1439,12 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 1352 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1456 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: double)
Stage: Stage-0
@@ -1479,19 +1479,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
@@ -1499,14 +1499,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: value (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1516,10 +1516,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2
input vertices:
0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1599,14 +1599,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1616,14 +1616,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1657,14 +1657,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1674,7 +1674,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1684,14 +1684,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col3
input vertices:
1 Map 3
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col3 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1702,38 +1702,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
@@ -1768,34 +1768,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 47094 Basic stats: COMPLETE Column stats: PARTIAL
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 44528 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: key (type: int), value (type: string), ds (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 44528 Basic stats: COMPLETE Column stats: PARTIAL
Reduce Output Operator
key expressions: _col0 (type: int), _col2 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: int), _col2 (type: string)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 44528 Basic stats: COMPLETE Column stats: PARTIAL
value expressions: _col1 (type: string)
Select Operator
expressions: _col2 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 44528 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: b
Partition key expr: ds
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Target Vertex: Map 2
Execution mode: llap
LLAP IO: no inputs
@@ -1803,14 +1803,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 97312 Basic stats: COMPLETE Column stats: PARTIAL
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 92000 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: key (type: int), value (type: string), ds (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 92000 Basic stats: COMPLETE Column stats: PARTIAL
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1820,14 +1820,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col4
input vertices:
0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60500 Data size: 12826000 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col4 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60500 Data size: 484000 Basic stats: COMPLETE Column stats: PARTIAL
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60500 Data size: 484000 Basic stats: COMPLETE Column stats: PARTIAL
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1880,14 +1880,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1896,29 +1896,29 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 19 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 19 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 19 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 19 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -1999,14 +1999,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -2015,29 +2015,29 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 19 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 19 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 19 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 19 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
[31/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out
index 5ec1430..21d82d2 100644
--- a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out
@@ -166,11 +166,11 @@ STAGE PLANS:
TableScan
alias: agg
filterExpr: dim_shops_id is not null (type: boolean)
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 63 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: amount (type: decimal(10,0)), dim_shops_id (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column stats: PARTIAL
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -180,22 +180,22 @@ STAGE PLANS:
outputColumnNames: _col0, _col3
input vertices:
1 Map 4
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col3 (type: string), _col0 (type: decimal(10,0))
outputColumnNames: _col3, _col0
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), sum(_col0)
keys: _col3 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Execution mode: llap
LLAP IO: no inputs
@@ -243,11 +243,11 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Reducer 3
Execution mode: llap
@@ -255,10 +255,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint), VALUE._col1 (type: decimal(20,0))
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -339,11 +339,11 @@ STAGE PLANS:
TableScan
alias: agg
filterExpr: dim_shops_id is not null (type: boolean)
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 63 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: amount (type: decimal(10,0)), dim_shops_id (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column stats: PARTIAL
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -353,22 +353,22 @@ STAGE PLANS:
outputColumnNames: _col0, _col3
input vertices:
1 Map 4
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col3 (type: string), _col0 (type: decimal(10,0))
outputColumnNames: _col3, _col0
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), sum(_col0)
keys: _col3 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Execution mode: llap
LLAP IO: no inputs
@@ -401,11 +401,11 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Reducer 3
Execution mode: llap
@@ -413,10 +413,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint), VALUE._col1 (type: decimal(20,0))
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -487,11 +487,11 @@ STAGE PLANS:
TableScan
alias: agg
filterExpr: dim_shops_id is not null (type: boolean)
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 63 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: dim_shops_id (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -501,14 +501,14 @@ STAGE PLANS:
outputColumnNames: _col2
input vertices:
1 Map 2
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col2 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -604,11 +604,11 @@ STAGE PLANS:
TableScan
alias: agg
filterExpr: (dim_shops_id = 1) (type: boolean)
- Statistics: Num rows: 3 Data size: 9 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 33 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: amount (type: decimal(10,0))
outputColumnNames: _col0
- Statistics: Num rows: 3 Data size: 9 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 33 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -618,10 +618,10 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 2
- Statistics: Num rows: 3 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 51 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 3 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 51 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -712,11 +712,11 @@ STAGE PLANS:
TableScan
alias: agg
filterExpr: dim_shops_id is not null (type: boolean)
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 63 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: amount (type: decimal(10,0)), dim_shops_id (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column stats: PARTIAL
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -726,22 +726,22 @@ STAGE PLANS:
outputColumnNames: _col0, _col3
input vertices:
1 Map 4
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col3 (type: string), _col0 (type: decimal(10,0))
outputColumnNames: _col3, _col0
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), sum(_col0)
keys: _col3 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Execution mode: llap
LLAP IO: no inputs
@@ -789,11 +789,11 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Reducer 3
Execution mode: llap
@@ -801,10 +801,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint), VALUE._col1 (type: decimal(20,0))
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 17 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -876,11 +876,11 @@ STAGE PLANS:
TableScan
alias: agg_01
filterExpr: dim_shops_id is not null (type: boolean)
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 63 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: amount (type: decimal(10,0)), dim_shops_id (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column stats: PARTIAL
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -890,10 +890,10 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 3
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 18 Data size: 58 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 18 Data size: 78 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -940,11 +940,11 @@ STAGE PLANS:
TableScan
alias: agg_01
filterExpr: dim_shops_id is not null (type: boolean)
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 63 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: amount (type: decimal(10,0)), dim_shops_id (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 9 Data size: 27 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column stats: PARTIAL
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -954,10 +954,10 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 5
- Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 18 Data size: 58 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 18 Data size: 78 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1068,11 +1068,11 @@ STAGE PLANS:
TableScan
alias: s1
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1081,15 +1081,15 @@ STAGE PLANS:
1 _col0 (type: string)
input vertices:
1 Map 3
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000000 Data size: 16000000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -1098,16 +1098,16 @@ STAGE PLANS:
TableScan
alias: s2
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -1117,10 +1117,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out b/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
index 2f70b40..2248a35 100644
--- a/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out
@@ -166,9 +166,9 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -184,29 +184,18 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: _col0 (type: smallint)
- sort order: +
+ key expressions: _col4 (type: tinyint), _col0 (type: smallint)
+ sort order: ++
+ Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: KEY.reducesinkkey0 (type: smallint), VALUE._col0 (type: int), VALUE._col1 (type: bigint), VALUE._col2 (type: float), VALUE._col3 (type: tinyint)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col4 (type: tinyint), _col0 (type: smallint)
- sort order: ++
- Map-reduce partition columns: _col4 (type: tinyint)
- Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
- Reducer 3
- Execution mode: vectorized, uber
- Reduce Operator Tree:
- Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: KEY._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -250,6 +239,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -273,8 +263,10 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 10 Data size: 2960 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -290,12 +282,12 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 10 Data size: 2960 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 10 Data size: 2960 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -339,6 +331,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -360,14 +353,15 @@ STAGE PLANS:
sort order: ++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -409,6 +403,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -430,14 +425,15 @@ STAGE PLANS:
sort order: +++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), KEY._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -549,9 +545,9 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -567,29 +563,18 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: _col0 (type: smallint)
- sort order: +
+ key expressions: _col4 (type: tinyint), _col0 (type: smallint)
+ sort order: ++
+ Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: KEY.reducesinkkey0 (type: smallint), VALUE._col0 (type: int), VALUE._col1 (type: bigint), VALUE._col2 (type: float), VALUE._col3 (type: tinyint)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col4 (type: tinyint), _col0 (type: smallint)
- sort order: ++
- Map-reduce partition columns: _col4 (type: tinyint)
- Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
- Reducer 3
- Execution mode: vectorized, uber
- Reduce Operator Tree:
- Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: KEY._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -633,6 +618,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -656,8 +642,10 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 10 Data size: 2960 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -673,12 +661,12 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 10 Data size: 2960 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 10 Data size: 2960 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -722,6 +710,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -743,14 +732,15 @@ STAGE PLANS:
sort order: ++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -792,6 +782,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -813,14 +804,15 @@ STAGE PLANS:
sort order: +++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), KEY._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -942,11 +934,11 @@ Database: default
Table: over1k_part_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 2
numRows 32
rawDataSize 640
- totalSize 1400
+ totalSize 1392
#### A masked pattern was here ####
# Storage Information
@@ -984,11 +976,11 @@ Database: default
Table: over1k_part_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 2
numRows 6
rawDataSize 120
- totalSize 1102
+ totalSize 1096
#### A masked pattern was here ####
# Storage Information
@@ -1026,11 +1018,11 @@ Database: default
Table: over1k_part_limit_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 2
numRows 14
rawDataSize 280
- totalSize 1216
+ totalSize 1210
#### A masked pattern was here ####
# Storage Information
@@ -1068,11 +1060,11 @@ Database: default
Table: over1k_part_limit_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 2
numRows 6
rawDataSize 120
- totalSize 1102
+ totalSize 1096
#### A masked pattern was here ####
# Storage Information
@@ -1109,11 +1101,11 @@ Database: default
Table: over1k_part_buck_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 8
numRows 32
rawDataSize 640
- totalSize 4548
+ totalSize 4524
#### A masked pattern was here ####
# Storage Information
@@ -1150,11 +1142,11 @@ Database: default
Table: over1k_part_buck_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 8
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 6
rawDataSize 120
- totalSize 2212
+ totalSize 2004
#### A masked pattern was here ####
# Storage Information
@@ -1191,11 +1183,11 @@ Database: default
Table: over1k_part_buck_sort_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 8
numRows 32
rawDataSize 640
- totalSize 4534
+ totalSize 4510
#### A masked pattern was here ####
# Storage Information
@@ -1232,11 +1224,11 @@ Database: default
Table: over1k_part_buck_sort_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 8
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 6
rawDataSize 120
- totalSize 2212
+ totalSize 2004
#### A masked pattern was here ####
# Storage Information
@@ -1334,6 +1326,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1356,8 +1349,9 @@ STAGE PLANS:
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: smallint), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: smallint), KEY.reducesinkkey0 (type: int), VALUE._col1 (type: bigint), VALUE._col2 (type: float), VALUE._col3 (type: tinyint)
@@ -1404,9 +1398,9 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1422,29 +1416,18 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: _col1 (type: int)
- sort order: +
+ key expressions: _col4 (type: tinyint), _col1 (type: int)
+ sort order: ++
+ Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col2 (type: bigint), _col3 (type: float)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), KEY.reducesinkkey0 (type: int), VALUE._col1 (type: bigint), VALUE._col2 (type: float), VALUE._col3 (type: tinyint)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col4 (type: tinyint), _col1 (type: int)
- sort order: ++
- Map-reduce partition columns: _col4 (type: tinyint)
- Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
- Reducer 3
- Execution mode: vectorized, uber
- Reduce Operator Tree:
- Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), KEY._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -1488,6 +1471,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1506,8 +1490,10 @@ STAGE PLANS:
key expressions: _col2 (type: int)
sort order: +
Statistics: Num rows: 1049 Data size: 311170 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
@@ -1530,12 +1516,12 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 10 Data size: 2960 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 10 Data size: 2960 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -1579,6 +1565,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1602,8 +1589,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: tinyint), KEY._col1 (type: smallint), KEY._col2 (type: int), KEY._col3 (type: bigint), KEY._col4 (type: float)
@@ -1657,6 +1645,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1680,8 +1669,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: tinyint), KEY._col1 (type: smallint), KEY._col2 (type: int), KEY._col3 (type: bigint), KEY._col4 (type: float)
@@ -1762,7 +1752,7 @@ Database: default
Table: over1k_part2_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 16
rawDataSize 415
@@ -1804,7 +1794,7 @@ Database: default
Table: over1k_part2_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 3
rawDataSize 78
@@ -1909,7 +1899,7 @@ Database: default
Table: over1k_part2_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 16
rawDataSize 415
@@ -1951,7 +1941,7 @@ Database: default
Table: over1k_part2_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 3
rawDataSize 78
@@ -2055,6 +2045,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -2078,8 +2069,9 @@ STAGE PLANS:
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col4 (type: tinyint)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), KEY.reducesinkkey0 (type: float), VALUE._col3 (type: tinyint)
@@ -2125,6 +2117,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -2146,14 +2139,15 @@ STAGE PLANS:
sort order: +++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), KEY._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 1048 Data size: 310873 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -2223,7 +2217,7 @@ Database: default
Table: over1k_part_buck_sort2_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 16
rawDataSize 415
@@ -2264,7 +2258,7 @@ Database: default
Table: over1k_part_buck_sort2_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 3
rawDataSize 78
@@ -2342,6 +2336,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -2350,33 +2345,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: over1k_part_buck_sort2_orc
- Statistics: Num rows: 19 Data size: 493 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 19 Data size: 645 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 19 Data size: 493 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 19 Data size: 645 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2439,7 +2435,7 @@ Database: default
Table: over1k_part_buck_sort2_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 16
rawDataSize 415
@@ -2480,7 +2476,7 @@ Database: default
Table: over1k_part_buck_sort2_orc
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 3
rawDataSize 78
@@ -2558,6 +2554,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -2566,33 +2563,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: over1k_part_buck_sort2_orc
- Statistics: Num rows: 19 Data size: 493 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 19 Data size: 645 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 19 Data size: 493 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 19 Data size: 645 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
[17/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/optimize_nullscan.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/optimize_nullscan.q.out b/ql/src/test/results/clientpositive/llap/optimize_nullscan.q.out
index d58d033..d9775b5 100644
--- a/ql/src/test/results/clientpositive/llap/optimize_nullscan.q.out
+++ b/ql/src/test/results/clientpositive/llap/optimize_nullscan.q.out
@@ -14,25 +14,6 @@ POSTHOOK: query: -- SORT_QUERY_RESULTS
explain extended
select key from src where false
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- key
- TOK_WHERE
- false
-
-
STAGE DEPENDENCIES:
Stage-0 is a root stage
@@ -66,32 +47,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain extended
select count(key) from srcpart where 1=2 group by key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- srcpart
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- count
- TOK_TABLE_OR_COL
- key
- TOK_WHERE
- =
- 1
- 2
- TOK_GROUPBY
- TOK_TABLE_OR_COL
- key
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -99,6 +54,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -107,29 +63,31 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(key)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
tag: -1
value expressions: _col1 (type: bigint)
auto parallelism: true
Execution mode: llap
+ LLAP IO: unknown
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -137,26 +95,27 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col1 (type: bigint)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -184,63 +143,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain extended
select * from (select key from src where false) a left outer join (select key from srcpart limit 0) b on a.key=b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_LEFTOUTERJOIN
- TOK_SUBQUERY
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- key
- TOK_WHERE
- false
- a
- TOK_SUBQUERY
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- srcpart
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- key
- TOK_LIMIT
- 0
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -248,6 +150,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
@@ -257,32 +160,35 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
tag: 0
auto parallelism: true
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Path -> Partition:
- -mr-10003default.src{}
+ nullscan://null/default.src/part_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -302,7 +208,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -321,33 +227,36 @@ STAGE PLANS:
name: default.src
name: default.src
Truncated Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Map 3
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 0
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string)
auto parallelism: false
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10004default.srcpart{ds=2008-04-08, hr=11} [srcpart]
- -mr-10005default.srcpart{ds=2008-04-08, hr=12} [srcpart]
- -mr-10006default.srcpart{ds=2008-04-09, hr=11} [srcpart]
- -mr-10007default.srcpart{ds=2008-04-09, hr=12} [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=11_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=12_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=11_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=12_ [srcpart]
Path -> Partition:
- -mr-10004default.srcpart{ds=2008-04-08, hr=11}
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=11_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -355,7 +264,7 @@ STAGE PLANS:
ds 2008-04-08
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -392,7 +301,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.srcpart
name: default.srcpart
- -mr-10005default.srcpart{ds=2008-04-08, hr=12}
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=12_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -400,7 +309,7 @@ STAGE PLANS:
ds 2008-04-08
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -437,7 +346,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.srcpart
name: default.srcpart
- -mr-10006default.srcpart{ds=2008-04-09, hr=11}
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=11_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -445,7 +354,7 @@ STAGE PLANS:
ds 2008-04-09
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -482,7 +391,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.srcpart
name: default.srcpart
- -mr-10007default.srcpart{ds=2008-04-09, hr=12}
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=12_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -490,7 +399,7 @@ STAGE PLANS:
ds 2008-04-09
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -528,10 +437,10 @@ STAGE PLANS:
name: default.srcpart
name: default.srcpart
Truncated Path -> Alias:
- -mr-10004default.srcpart{ds=2008-04-08, hr=11} [srcpart]
- -mr-10005default.srcpart{ds=2008-04-08, hr=12} [srcpart]
- -mr-10006default.srcpart{ds=2008-04-09, hr=11} [srcpart]
- -mr-10007default.srcpart{ds=2008-04-09, hr=12} [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=11_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=12_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=11_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=12_ [srcpart]
Reducer 2
Execution mode: llap
Needs Tagging: false
@@ -544,22 +453,23 @@ STAGE PLANS:
1 _col0 (type: string)
outputColumnNames: _col0, _col1
Position of Big Table: 0
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1
columns.types string:string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -573,15 +483,16 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 0
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
tag: 1
auto parallelism: true
@@ -615,54 +526,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain extended
select count(key) from src where false union all select count(key) from srcpart
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_SUBQUERY
- TOK_UNIONALL
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- count
- TOK_TABLE_OR_COL
- key
- TOK_WHERE
- false
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- srcpart
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- count
- TOK_TABLE_OR_COL
- key
- _u1
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -670,6 +533,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
Reducer 5 <- Map 4 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -679,32 +543,35 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(key)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: bigint)
auto parallelism: false
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Path -> Partition:
- -mr-10003default.src{}
+ nullscan://null/default.src/part_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -724,7 +591,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -743,29 +610,31 @@ STAGE PLANS:
name: default.src
name: default.src
Truncated Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Map 4
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: key (type: string)
outputColumnNames: key
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(key)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: bigint)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -778,7 +647,7 @@ STAGE PLANS:
ds 2008-04-08
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -824,7 +693,7 @@ STAGE PLANS:
ds 2008-04-08
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -870,7 +739,7 @@ STAGE PLANS:
ds 2008-04-09
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -916,7 +785,7 @@ STAGE PLANS:
ds 2008-04-09
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -959,29 +828,30 @@ STAGE PLANS:
/srcpart/ds=2008-04-09/hr=11 [srcpart]
/srcpart/ds=2008-04-09/hr=12 [srcpart]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -989,29 +859,30 @@ STAGE PLANS:
GatherStats: false
MultiFileSpray: false
Reducer 5
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1054,54 +925,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain extended
select * from (select key from src where false) a left outer join (select value from srcpart limit 0) b
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_LEFTOUTERJOIN
- TOK_SUBQUERY
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- key
- TOK_WHERE
- false
- a
- TOK_SUBQUERY
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- srcpart
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- value
- TOK_LIMIT
- 0
- b
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -1109,6 +932,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
@@ -1118,31 +942,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
tag: 0
value expressions: _col0 (type: string)
auto parallelism: false
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Path -> Partition:
- -mr-10003default.src{}
+ nullscan://null/default.src/part_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1162,7 +989,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1181,33 +1008,36 @@ STAGE PLANS:
name: default.src
name: default.src
Truncated Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Map 3
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 182000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: value (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 182000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 0
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 91 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 91 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string)
auto parallelism: false
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10004default.srcpart{ds=2008-04-08, hr=11} [srcpart]
- -mr-10005default.srcpart{ds=2008-04-08, hr=12} [srcpart]
- -mr-10006default.srcpart{ds=2008-04-09, hr=11} [srcpart]
- -mr-10007default.srcpart{ds=2008-04-09, hr=12} [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=11_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=12_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=11_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=12_ [srcpart]
Path -> Partition:
- -mr-10004default.srcpart{ds=2008-04-08, hr=11}
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=11_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1215,7 +1045,7 @@ STAGE PLANS:
ds 2008-04-08
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1252,7 +1082,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.srcpart
name: default.srcpart
- -mr-10005default.srcpart{ds=2008-04-08, hr=12}
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=12_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1260,7 +1090,7 @@ STAGE PLANS:
ds 2008-04-08
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1297,7 +1127,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.srcpart
name: default.srcpart
- -mr-10006default.srcpart{ds=2008-04-09, hr=11}
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=11_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1305,7 +1135,7 @@ STAGE PLANS:
ds 2008-04-09
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1342,7 +1172,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.srcpart
name: default.srcpart
- -mr-10007default.srcpart{ds=2008-04-09, hr=12}
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=12_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1350,7 +1180,7 @@ STAGE PLANS:
ds 2008-04-09
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1388,10 +1218,10 @@ STAGE PLANS:
name: default.srcpart
name: default.srcpart
Truncated Path -> Alias:
- -mr-10004default.srcpart{ds=2008-04-08, hr=11} [srcpart]
- -mr-10005default.srcpart{ds=2008-04-08, hr=12} [srcpart]
- -mr-10006default.srcpart{ds=2008-04-09, hr=11} [srcpart]
- -mr-10007default.srcpart{ds=2008-04-09, hr=12} [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=11_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-08_hr=12_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=11_ [srcpart]
+ nullscan://null/default.srcpart/part_ds=2008-04-09_hr=12_ [srcpart]
Reducer 2
Execution mode: llap
Needs Tagging: false
@@ -1404,22 +1234,23 @@ STAGE PLANS:
1
outputColumnNames: _col0, _col1
Position of Big Table: 0
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1
columns.types string:string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1433,13 +1264,14 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 91 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 0
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 91 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 91 Basic stats: COMPLETE Column stats: COMPLETE
tag: 1
value expressions: _col0 (type: string)
auto parallelism: false
@@ -1475,65 +1307,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain extended
select * from (select key from src union all select src.key from src left outer join srcpart on src.key = srcpart.key) a where false
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_SUBQUERY
- TOK_UNIONALL
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- key
- TOK_QUERY
- TOK_FROM
- TOK_LEFTOUTERJOIN
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_TABREF
- TOK_TABNAME
- srcpart
- =
- .
- TOK_TABLE_OR_COL
- src
- key
- .
- TOK_TABLE_OR_COL
- srcpart
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- .
- TOK_TABLE_OR_COL
- src
- key
- a
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
- TOK_WHERE
- false
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -1541,6 +1314,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Reducer 4 <- Map 3 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE), Union 2 (CONTAINS)
@@ -1550,31 +1324,32 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 2 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 182 Basic stats: COMPLETE Column stats: PARTIAL
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1582,6 +1357,7 @@ STAGE PLANS:
GatherStats: false
MultiFileSpray: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -1591,7 +1367,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1611,7 +1387,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1635,28 +1411,31 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 87 Basic stats: COMPLETE Column stats: COMPLETE
tag: 0
auto parallelism: true
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Path -> Partition:
- -mr-10003default.src{}
+ nullscan://null/default.src/part_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1676,7 +1455,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1695,25 +1474,27 @@ STAGE PLANS:
name: default.src
name: default.src
Truncated Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Map 5
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
tag: 1
auto parallelism: true
Execution mode: llap
+ LLAP IO: unknown
Reducer 4
Execution mode: llap
Needs Tagging: false
@@ -1726,22 +1507,23 @@ STAGE PLANS:
1 key (type: string)
outputColumnNames: _col0
Position of Big Table: 0
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 95 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 2 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 182 Basic stats: COMPLETE Column stats: PARTIAL
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1773,40 +1555,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain extended
select * from src s1, src s2 where false and s1.value = s2.value
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- src
- s1
- TOK_TABREF
- TOK_TABNAME
- src
- s2
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
- TOK_WHERE
- and
- false
- =
- .
- TOK_TABLE_OR_COL
- s1
- value
- .
- TOK_TABLE_OR_COL
- s2
- value
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -1814,6 +1562,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1822,29 +1571,32 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
tag: 0
value expressions: key (type: string)
auto parallelism: true
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10003default.src{} [s1]
+ nullscan://null/default.src/part_ [s1]
Path -> Partition:
- -mr-10003default.src{}
+ nullscan://null/default.src/part_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1864,7 +1616,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1883,34 +1635,37 @@ STAGE PLANS:
name: default.src
name: default.src
Truncated Path -> Alias:
- -mr-10003default.src{} [s1]
+ nullscan://null/default.src/part_ [s1]
Map 3
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
tag: 1
value expressions: key (type: string)
auto parallelism: true
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10004default.src{} [s2]
+ nullscan://null/default.src/part_ [s2]
Path -> Partition:
- -mr-10004default.src{}
+ nullscan://null/default.src/part_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1930,7 +1685,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -1949,7 +1704,7 @@ STAGE PLANS:
name: default.src
name: default.src
Truncated Path -> Alias:
- -mr-10004default.src{} [s2]
+ nullscan://null/default.src/part_ [s2]
Reducer 2
Execution mode: llap
Needs Tagging: false
@@ -1962,26 +1717,27 @@ STAGE PLANS:
1 value (type: string)
outputColumnNames: _col0, _col1, _col5, _col6
Position of Big Table: 0
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 11 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1,_col2,_col3
columns.types string:string:string:string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -2009,28 +1765,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain extended
select count(1) from src where null = 1
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- count
- 1
- TOK_WHERE
- =
- TOK_NULL
- 1
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -2038,6 +1772,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -2051,27 +1786,30 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: false (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: bigint)
auto parallelism: false
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Path -> Partition:
- -mr-10003default.src{}
+ nullscan://null/default.src/part_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -2091,7 +1829,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -2110,9 +1848,9 @@ STAGE PLANS:
name: default.src
name: default.src
Truncated Path -> Alias:
- -mr-10003default.src{} [src]
+ nullscan://null/default.src/part_ [src]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -2128,13 +1866,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
[32/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out
index 01b4402..f993cf0 100644
--- a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out
@@ -57,21 +57,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: ds (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -81,14 +81,14 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 736 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 736 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -222,16 +222,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -278,7 +278,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -351,16 +351,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -392,7 +392,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -480,16 +480,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: day(_col0) (type: int)
sort order: +
Map-reduce partition columns: day(_col0) (type: int)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -536,7 +536,7 @@ STAGE PLANS:
keys:
0 day(_col0) (type: int)
1 day(_col0) (type: int)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -609,16 +609,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: day(_col0) (type: int)
sort order: +
Map-reduce partition columns: day(_col0) (type: int)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -650,7 +650,7 @@ STAGE PLANS:
keys:
0 day(_col0) (type: int)
1 day(_col0) (type: int)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -727,16 +727,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 757248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -820,12 +820,12 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 809600 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 809600 Basic stats: COMPLETE Column stats: NONE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -835,7 +835,7 @@ STAGE PLANS:
keys:
0 _col1 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2420 Data size: 25709 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2420 Data size: 890560 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -915,16 +915,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: (ds is not null and hr is not null) (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 757248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -978,12 +978,12 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 809600 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 809600 Basic stats: COMPLETE Column stats: NONE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -993,7 +993,7 @@ STAGE PLANS:
keys:
0 _col1 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2420 Data size: 25709 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2420 Data size: 890560 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1082,16 +1082,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 757248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -1153,7 +1153,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col2 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 809600 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1226,16 +1226,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: (ds is not null and hr is not null) (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 757248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -1267,7 +1267,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col2 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 809600 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1353,16 +1353,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -1409,7 +1409,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1482,16 +1482,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -1523,7 +1523,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1607,16 +1607,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: hr is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hr (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToDouble(_col0) (type: double)
sort order: +
Map-reduce partition columns: UDFToDouble(_col0) (type: double)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -1663,7 +1663,7 @@ STAGE PLANS:
keys:
0 UDFToDouble(_col0) (type: double)
1 UDFToDouble(UDFToInteger((_col0 / 2.0))) (type: double)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1736,16 +1736,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: hr is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hr (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (UDFToDouble(_col0) * 2.0) (type: double)
sort order: +
Map-reduce partition columns: (UDFToDouble(_col0) * 2.0) (type: double)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -1792,7 +1792,7 @@ STAGE PLANS:
keys:
0 (UDFToDouble(_col0) * 2.0) (type: double)
1 _col0 (type: double)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1865,16 +1865,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: hr is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hr (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToDouble(_col0) (type: double)
sort order: +
Map-reduce partition columns: UDFToDouble(_col0) (type: double)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -1906,7 +1906,7 @@ STAGE PLANS:
keys:
0 UDFToDouble(_col0) (type: double)
1 UDFToDouble(UDFToInteger((_col0 / 2.0))) (type: double)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1979,16 +1979,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: hr is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hr (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (UDFToDouble(_col0) * 2.0) (type: double)
sort order: +
Map-reduce partition columns: (UDFToDouble(_col0) * 2.0) (type: double)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -2020,7 +2020,7 @@ STAGE PLANS:
keys:
0 (UDFToDouble(_col0) * 2.0) (type: double)
1 _col0 (type: double)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2106,16 +2106,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: hr is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hr (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToString((UDFToDouble(_col0) * 2.0)) (type: string)
sort order: +
Map-reduce partition columns: UDFToString((UDFToDouble(_col0) * 2.0)) (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -2162,7 +2162,7 @@ STAGE PLANS:
keys:
0 UDFToString((UDFToDouble(_col0) * 2.0)) (type: string)
1 UDFToString(_col0) (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2252,12 +2252,12 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: (ds = '2008-04-08') (type: boolean)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 94000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 94000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -2265,19 +2265,19 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: (ds = '2008-04-08') (type: boolean)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: '2008-04-08' (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -2289,15 +2289,15 @@ STAGE PLANS:
keys:
0
1
- Statistics: Num rows: 500000 Data size: 11124000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 8000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 3
Execution mode: llap
@@ -2306,10 +2306,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -2321,12 +2321,12 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
@@ -2385,14 +2385,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 757248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -2425,12 +2425,12 @@ STAGE PLANS:
0
1
outputColumnNames: _col0, _col1, _col2, _col4
- Statistics: Num rows: 2000 Data size: 77248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 792000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: ((_col0 = _col2) or (_col1 = _col4)) (type: boolean)
- Statistics: Num rows: 2000 Data size: 77248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 792000 Basic stats: COMPLETE Column stats: NONE
Select Operator
- Statistics: Num rows: 2000 Data size: 77248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 792000 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2505,16 +2505,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 757248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 736000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -2576,7 +2576,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col2 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 809600 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2651,16 +2651,16 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -2707,7 +2707,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2795,16 +2795,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -2816,7 +2816,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2871,16 +2871,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -2927,7 +2927,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2200 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2985,16 +2985,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 378624 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -3063,12 +3063,12 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 404800 Basic stats: COMPLETE Column stats: NONE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -3078,7 +3078,7 @@ STAGE PLANS:
keys:
0 _col1 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 1210 Data size: 12854 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1210 Data size: 445280 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -3154,19 +3154,19 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ((UDFToDouble(hr) = 13.0) and ds is not null) (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((UDFToDouble(hr) = 13.0) and ds is not null) (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: unknown
@@ -3220,12 +3220,12 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 404 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 404 Basic stats: COMPLETE Column stats: NONE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -3235,7 +3235,7 @@ STAGE PLANS:
keys:
0 _col1 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 444 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -3308,35 +3308,35 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -3344,19 +3344,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -3369,15 +3369,15 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 8000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 3
Execution mode: llap
@@ -3386,10 +3386,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -3401,34 +3401,34 @@ STAGE PLANS:
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: srcpart
Partition key expr: ds
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Target Vertex: Map 1
Reducer 8
Execution mode: llap
@@ -3437,34 +3437,34 @@ STAGE PLANS:
aggregations: min(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: srcpart
Partition key expr: ds
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Target Vertex: Map 1
Union 6
Vertex: Union 6
@@ -3516,35 +3516,35 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 368000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -3552,19 +3552,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -3578,17 +3578,17 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 184000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -3596,10 +3596,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -3611,34 +3611,34 @@ STAGE PLANS:
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: srcpart
Partition key expr: ds
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Target Vertex: Map 1
Reducer 8
Execution mode: llap
@@ -3647,34 +3647,34 @@ STAGE PLANS:
aggregations: min(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: srcpart
Partition key expr: ds
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Target Vertex: Map 1
Union 6
Vertex: Union 6
@@ -3728,36 +3728,36 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: ds (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 10
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -3766,36 +3766,36 @@ STAGE PLANS:
TableScan
alias: srcpart
filterExpr: ds is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: ds (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 389248 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -3806,49 +3806,49 @@ STAGE PLANS:
aggregations: min(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: srcpart
Partition key expr: ds
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Target Vertex: Map 1
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: srcpart
Partition key expr: ds
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Target Vertex: Map 5
Reducer 2
Execution mode: llap
@@ -3857,12 +3857,12 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 736 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: llap
Reduce Operator Tree:
@@ -3873,10 +3873,10 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -3888,12 +3888,12 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 736 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 8
Execution mode: llap
Reduce Operator Tree:
@@ -3901,49 +3901,49 @@ STAGE PLANS:
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: srcpart
Partition key expr: ds
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Target Vertex: Map 1
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 368 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data
<TRUNCATED>
[35/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cross_product_check_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cross_product_check_1.q.out b/ql/src/test/results/clientpositive/llap/cross_product_check_1.q.out
index 5556478..be303c4 100644
--- a/ql/src/test/results/clientpositive/llap/cross_product_check_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/cross_product_check_1.q.out
@@ -14,6 +14,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@A
+POSTHOOK: Lineage: a.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: a.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: create table B as
select * from src
limit 10
@@ -28,7 +30,9 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@B
-Warning: Shuffle Join MERGEJOIN[11][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+POSTHOOK: Lineage: b.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: b.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+Warning: Shuffle Join MERGEJOIN[9][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select * from A join B
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from A join B
@@ -40,6 +44,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -58,6 +63,7 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -72,6 +78,7 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -82,13 +89,13 @@ STAGE PLANS:
0
1
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5000 Data size: 110120 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5000 Data size: 110120 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -97,7 +104,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[18][tables = [d1, d2, a]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[21][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from B d1 join B d2 on d1.key = d2.key join A
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from B d1 join B d2 on d1.key = d2.key join A
@@ -109,6 +116,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
@@ -121,14 +129,19 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: string)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -136,24 +149,34 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: string)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: string), value (type: string)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -161,14 +184,14 @@ STAGE PLANS:
condition map:
Inner Join 0 to 1
keys:
- 0 key (type: string)
- 1 key (type: string)
- outputColumnNames: _col0, _col1, _col5, _col6
- Statistics: Num rows: 5 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 5 Data size: 57 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string)
+ Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -178,19 +201,15 @@ STAGE PLANS:
keys:
0
1
- outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 5500 Data size: 120932 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 5500 Data size: 120932 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
@@ -198,7 +217,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[24][tables = [a, od1]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[26][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select * from A join
(select d1.key
from B d1 join B d2 on d1.key = d2.key
@@ -216,50 +235,66 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
- Reducer 4 <- Map 6 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE)
+ Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
- alias: d1
- Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
+ alias: a
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
+ sort order:
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
- Map 5
+ LLAP IO: no inputs
+ Map 3
Map Operator Tree:
TableScan
- alias: d2
+ alias: d1
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 6
Map Operator Tree:
TableScan
- alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: string), value (type: string)
+ alias: d2
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -267,54 +302,50 @@ STAGE PLANS:
condition map:
Inner Join 0 to 1
keys:
- 0 key (type: string)
- 1 key (type: string)
+ 0
+ 1
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 2500 Data size: 54560 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 2500 Data size: 54560 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 4
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 5 Data size: 57 Basic stats: COMPLETE Column stats: NONE
- Reducer 3
+ Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Reducer 5
Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 22 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 51 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 2 Data size: 22 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 51 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string)
- Reducer 4
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
@@ -322,8 +353,8 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[17][tables = [d1, d2]] in Stage 'Reducer 2' is a cross product
-Warning: Shuffle Join MERGEJOIN[18][tables = [a, od1]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[19][tables = [$hdt$_1, $hdt$_2]] in Stage 'Reducer 4' is a cross product
+Warning: Shuffle Join MERGEJOIN[20][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: explain select * from A join (select d1.key from B d1 join B d2 where 1 = 1 group by d1.key) od1
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from A join (select d1.key from B d1 join B d2 where 1 = 1 group by d1.key) od1
@@ -335,41 +366,55 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
- Reducer 4 <- Map 6 (SIMPLE_EDGE), Reducer 3 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE)
+ Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
+ alias: a
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string), _col1 (type: string)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 3
+ Map Operator Tree:
+ TableScan
alias: d1
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: string)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string)
Execution mode: llap
- Map 5
+ LLAP IO: no inputs
+ Map 6
Map Operator Tree:
TableScan
alias: d2
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- Map 6
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions: key (type: string), value (type: string)
+ Select Operator
+ Statistics: Num rows: 10 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 10 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -379,52 +424,48 @@ STAGE PLANS:
keys:
0
1
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 25000 Data size: 675600 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 25000 Data size: 675600 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 4
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
outputColumnNames: _col0
- Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 1540 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 1540 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reducer 3
+ Statistics: Num rows: 100 Data size: 1540 Basic stats: COMPLETE Column stats: NONE
+ Reducer 5
Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 51 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 770 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 5 Data size: 51 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 770 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string)
- Reducer 4
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
@@ -432,7 +473,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[29][tables = [ss, od1]] in Stage 'Reducer 3' is a cross product
+Warning: Shuffle Join MERGEJOIN[30][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: explain select * from
(select A.key from A group by key) ss join
(select d1.key from B d1 join B d2 on d1.key = d2.key where 1 = 1 group by d1.key) od1
@@ -448,6 +489,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
@@ -475,6 +517,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -482,13 +525,18 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
@@ -496,13 +544,18 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 104 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -525,13 +578,13 @@ STAGE PLANS:
0
1
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1250 Data size: 27280 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1250 Data size: 27280 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: llap
@@ -540,20 +593,20 @@ STAGE PLANS:
condition map:
Inner Join 0 to 1
keys:
- 0 key (type: string)
- 1 key (type: string)
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 5 Data size: 57 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 114 Basic stats: COMPLETE Column stats: NONE
Reducer 6
Execution mode: llap
Reduce Operator Tree:
@@ -561,10 +614,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 22 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 51 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 2 Data size: 22 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 51 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string)
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cross_product_check_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cross_product_check_2.q.out b/ql/src/test/results/clientpositive/llap/cross_product_check_2.q.out
index c2a6b94..3b14b84 100644
--- a/ql/src/test/results/clientpositive/llap/cross_product_check_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/cross_product_check_2.q.out
@@ -14,6 +14,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@A
+POSTHOOK: Lineage: a.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: a.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: create table B as
select * from src order by key
limit 10
@@ -28,7 +30,9 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@B
-Warning: Map Join MAPJOIN[11][bigTable=?] in task 'Map 1' is a cross product
+POSTHOOK: Lineage: b.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: b.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+Warning: Map Join MAPJOIN[9][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: explain select * from A join B
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from A join B
@@ -40,6 +44,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 2 (BROADCAST_EDGE)
#### A masked pattern was here ####
@@ -62,16 +67,16 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3
input vertices:
1 Map 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 5000 Data size: 106120 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5000 Data size: 106120 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: llap
+ LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
@@ -86,6 +91,7 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-0
Fetch Operator
@@ -93,7 +99,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Map Join MAPJOIN[18][bigTable=a] in task 'Map 3' is a cross product
+Warning: Map Join MAPJOIN[21][bigTable=?] in task 'Map 3' is a cross product
PREHOOK: query: explain select * from B d1 join B d2 on d1.key = d2.key join A
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from B d1 join B d2 on d1.key = d2.key join A
@@ -105,6 +111,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 2 (BROADCAST_EDGE)
Map 3 <- Map 1 (BROADCAST_EDGE)
@@ -117,23 +124,27 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- outputColumnNames: _col0, _col1, _col5, _col6
- input vertices:
- 1 Map 2
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string)
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3
+ input vertices:
+ 1 Map 2
+ Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
@@ -141,42 +152,47 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
- value expressions: value (type: string)
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- input vertices:
- 0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ input vertices:
+ 0 Map 1
+ Statistics: Num rows: 5500 Data size: 116432 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5500 Data size: 116432 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-0
Fetch Operator
@@ -184,7 +200,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Map Join MAPJOIN[24][bigTable=a] in task 'Map 4' is a cross product
+Warning: Map Join MAPJOIN[26][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: explain select * from A join
(select d1.key
from B d1 join B d2 on d1.key = d2.key
@@ -202,95 +218,105 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 1 <- Map 3 (BROADCAST_EDGE)
- Map 4 <- Reducer 2 (BROADCAST_EDGE)
- Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Map 1 <- Reducer 3 (BROADCAST_EDGE)
+ Map 2 <- Map 4 (BROADCAST_EDGE)
+ Reducer 3 <- Map 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
- alias: d1
- Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
+ alias: a
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
- 0 key (type: string)
- 1 key (type: string)
- outputColumnNames: _col0
+ 0
+ 1
+ outputColumnNames: _col0, _col1, _col2
input vertices:
- 1 Map 3
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
+ 1 Reducer 3
+ Statistics: Num rows: 2500 Data size: 52560 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 2500 Data size: 52560 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: llap
- Map 3
+ LLAP IO: no inputs
+ Map 2
Map Operator Tree:
TableScan
- alias: d2
+ alias: d1
Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 4
+ Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
- alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col5
- input vertices:
- 1 Reducer 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ alias: d2
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
- Reducer 2
+ LLAP IO: no inputs
+ Reducer 3
Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 20 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 47 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 2 Data size: 20 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 47 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string)
Stage: Stage-0
@@ -299,8 +325,8 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Map Join MAPJOIN[17][bigTable=d1] in task 'Map 1' is a cross product
-Warning: Map Join MAPJOIN[18][bigTable=a] in task 'Map 4' is a cross product
+Warning: Map Join MAPJOIN[19][bigTable=?] in task 'Map 2' is a cross product
+Warning: Map Join MAPJOIN[20][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: explain select * from A join (select d1.key from B d1 join B d2 where 1 = 1 group by d1.key) od1
PREHOOK: type: QUERY
POSTHOOK: query: explain select * from A join (select d1.key from B d1 join B d2 where 1 = 1 group by d1.key) od1
@@ -312,87 +338,95 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 1 <- Map 3 (BROADCAST_EDGE)
- Map 4 <- Reducer 2 (BROADCAST_EDGE)
- Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Map 1 <- Reducer 3 (BROADCAST_EDGE)
+ Map 2 <- Map 4 (BROADCAST_EDGE)
+ Reducer 3 <- Map 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
+ alias: a
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
+ outputColumnNames: _col0, _col1, _col2
+ input vertices:
+ 1 Reducer 3
+ Statistics: Num rows: 25000 Data size: 655600 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 25000 Data size: 655600 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 2
+ Map Operator Tree:
+ TableScan
alias: d1
Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
+ Select Operator
+ expressions: key (type: string)
outputColumnNames: _col0
- input vertices:
- 1 Map 3
- Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- keys: _col0 (type: string)
- mode: hash
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
outputColumnNames: _col0
- Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
+ input vertices:
+ 1 Map 4
+ Statistics: Num rows: 100 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 100 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 100 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
- Map 3
+ LLAP IO: no inputs
+ Map 4
Map Operator Tree:
TableScan
alias: d2
Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- Map 4
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- outputColumnNames: _col0, _col1, _col5
- input vertices:
- 1 Reducer 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Select Operator
- expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Select Operator
+ Statistics: Num rows: 10 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 10 Data size: 40 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
- Reducer 2
+ LLAP IO: no inputs
+ Reducer 3
Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 47 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 730 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 5 Data size: 47 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 730 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string)
Stage: Stage-0
@@ -401,7 +435,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Map Join MAPJOIN[29][bigTable=?] in task 'Reducer 2' is a cross product
+Warning: Map Join MAPJOIN[30][bigTable=?] in task 'Reducer 2' is a cross product
PREHOOK: query: explain select * from
(select A.key from A group by key) ss join
(select d1.key from B d1 join B d2 on d1.key = d2.key where 1 = 1 group by d1.key) od1
@@ -417,6 +451,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 3 <- Map 5 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (BROADCAST_EDGE)
@@ -443,6 +478,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -450,29 +486,33 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
outputColumnNames: _col0
- input vertices:
- 1 Map 5
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- keys: _col0 (type: string)
- mode: hash
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 5 Data size: 52 Basic stats: COMPLETE Column stats: NONE
+ input vertices:
+ 1 Map 5
+ Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 11 Data size: 105 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
@@ -480,15 +520,20 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 5 Data size: 48 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 10 Data size: 96 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string)
@@ -504,14 +549,13 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Reducer 4
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 1250 Data size: 26280 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1250 Data size: 26280 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: llap
@@ -520,10 +564,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 20 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 47 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 2 Data size: 20 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 47 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string)
Stage: Stage-0
[02/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/transform_ppr1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/transform_ppr1.q.out b/ql/src/test/results/clientpositive/llap/transform_ppr1.q.out
index 2000ef1..5d992bf 100644
--- a/ql/src/test/results/clientpositive/llap/transform_ppr1.q.out
+++ b/ql/src/test/results/clientpositive/llap/transform_ppr1.q.out
@@ -20,81 +20,6 @@ FROM (
) tmap
SELECT tmap.tkey, tmap.tvalue WHERE tmap.tkey < 100 AND tmap.ds = '2008-04-08'
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_SUBQUERY
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- srcpart
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TRANSFORM
- TOK_EXPLIST
- .
- TOK_TABLE_OR_COL
- src
- ds
- .
- TOK_TABLE_OR_COL
- src
- key
- .
- TOK_TABLE_OR_COL
- src
- value
- TOK_SERDE
- TOK_RECORDWRITER
- 'cat'
- TOK_SERDE
- TOK_RECORDREADER
- TOK_ALIASLIST
- ds
- tkey
- tvalue
- TOK_CLUSTERBY
- TOK_TABLE_OR_COL
- tkey
- tmap
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- .
- TOK_TABLE_OR_COL
- tmap
- tkey
- TOK_SELEXPR
- .
- TOK_TABLE_OR_COL
- tmap
- tvalue
- TOK_WHERE
- AND
- <
- .
- TOK_TABLE_OR_COL
- tmap
- tkey
- 100
- =
- .
- TOK_TABLE_OR_COL
- tmap
- ds
- '2008-04-08'
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -102,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -110,12 +36,12 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 724000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string), key (type: string), value (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 724000 Basic stats: COMPLETE Column stats: COMPLETE
Transform Operator
command: cat
output info:
@@ -128,16 +54,17 @@ STAGE PLANS:
serialization.format 9
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 724000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
isSamplingPred: false
predicate: ((_col1 < 100) and (_col0 = '2008-04-08')) (type: boolean)
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 333 Data size: 2664 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 333 Data size: 2664 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: '2008-04-08' (type: string), _col1 (type: string), _col2 (type: string)
auto parallelism: true
@@ -153,7 +80,7 @@ STAGE PLANS:
ds 2008-04-08
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -199,7 +126,7 @@ STAGE PLANS:
ds 2008-04-08
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -245,7 +172,7 @@ STAGE PLANS:
ds 2008-04-09
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -291,7 +218,7 @@ STAGE PLANS:
ds 2008-04-09
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -334,28 +261,29 @@ STAGE PLANS:
/srcpart/ds=2008-04-09/hr=11 [src]
/srcpart/ds=2008-04-09/hr=12 [src]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Select Operator
expressions: VALUE._col1 (type: string), VALUE._col2 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 333 Data size: 2664 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 333 Data size: 2664 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1
columns.types string:string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/transform_ppr2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/transform_ppr2.q.out b/ql/src/test/results/clientpositive/llap/transform_ppr2.q.out
index f62b657..51dfee7 100644
--- a/ql/src/test/results/clientpositive/llap/transform_ppr2.q.out
+++ b/ql/src/test/results/clientpositive/llap/transform_ppr2.q.out
@@ -22,81 +22,6 @@ FROM (
) tmap
SELECT tmap.tkey, tmap.tvalue WHERE tmap.tkey < 100
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_SUBQUERY
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- srcpart
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TRANSFORM
- TOK_EXPLIST
- .
- TOK_TABLE_OR_COL
- src
- ds
- .
- TOK_TABLE_OR_COL
- src
- key
- .
- TOK_TABLE_OR_COL
- src
- value
- TOK_SERDE
- TOK_RECORDWRITER
- 'cat'
- TOK_SERDE
- TOK_RECORDREADER
- TOK_ALIASLIST
- ds
- tkey
- tvalue
- TOK_WHERE
- =
- .
- TOK_TABLE_OR_COL
- src
- ds
- '2008-04-08'
- TOK_CLUSTERBY
- TOK_TABLE_OR_COL
- tkey
- tmap
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- .
- TOK_TABLE_OR_COL
- tmap
- tkey
- TOK_SELEXPR
- .
- TOK_TABLE_OR_COL
- tmap
- tvalue
- TOK_WHERE
- <
- .
- TOK_TABLE_OR_COL
- tmap
- tkey
- 100
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -104,6 +29,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -112,12 +38,12 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: '2008-04-08' (type: string), key (type: string), value (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 272000 Basic stats: COMPLETE Column stats: COMPLETE
Transform Operator
command: cat
output info:
@@ -130,16 +56,17 @@ STAGE PLANS:
serialization.format 9
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 272000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
isSamplingPred: false
predicate: (_col1 < 100) (type: boolean)
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 333 Data size: 2664 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 333 Data size: 2664 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
auto parallelism: true
@@ -155,7 +82,7 @@ STAGE PLANS:
ds 2008-04-08
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -201,7 +128,7 @@ STAGE PLANS:
ds 2008-04-08
hr 12
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -242,28 +169,29 @@ STAGE PLANS:
/srcpart/ds=2008-04-08/hr=11 [src]
/srcpart/ds=2008-04-08/hr=12 [src]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Select Operator
expressions: VALUE._col1 (type: string), VALUE._col2 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 333 Data size: 2664 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 333 Data size: 3537 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 333 Data size: 2664 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1
columns.types string:string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/union2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/union2.q.out b/ql/src/test/results/clientpositive/llap/union2.q.out
index f0dcbf3..c1f6837 100644
--- a/ql/src/test/results/clientpositive/llap/union2.q.out
+++ b/ql/src/test/results/clientpositive/llap/union2.q.out
@@ -19,6 +19,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Map 4 <- Union 2 (CONTAINS)
@@ -29,58 +30,56 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: value (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 8000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
- alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: s2
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: value (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 8000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 2
Vertex: Union 2
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/union3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/union3.q.out b/ql/src/test/results/clientpositive/llap/union3.q.out
index 8333ed2..6e28152 100644
--- a/ql/src/test/results/clientpositive/llap/union3.q.out
+++ b/ql/src/test/results/clientpositive/llap/union3.q.out
@@ -43,6 +43,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 10 <- Map 9 (SIMPLE_EDGE), Union 3 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -60,11 +61,13 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
@@ -74,11 +77,13 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
@@ -88,11 +93,13 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Map 9
Map Operator Tree:
TableScan
@@ -102,85 +109,87 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 10
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: 4 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 4 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: 1 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 4 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 4 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 4 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: 2 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 4 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 8
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: 3 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 4 Data size: 0 Basic stats: PARTIAL Column stats: COMPLETE
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Union 3
Vertex: Union 3
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/union4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/union4.q.out b/ql/src/test/results/clientpositive/llap/union4.q.out
index ff55172..85d60bf 100644
--- a/ql/src/test/results/clientpositive/llap/union4.q.out
+++ b/ql/src/test/results/clientpositive/llap/union4.q.out
@@ -33,11 +33,11 @@ STAGE DEPENDENCIES:
Stage-2 depends on stages: Stage-1
Stage-0 depends on stages: Stage-2
Stage-3 depends on stages: Stage-0
- Stage-4 depends on stages: Stage-0
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
Reducer 5 <- Map 4 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -60,10 +60,11 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
- alias: s1
+ alias: s2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
@@ -77,8 +78,9 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -102,7 +104,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.tmptable
Reducer 5
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -144,9 +146,6 @@ STAGE PLANS:
Stage: Stage-3
Stats-Aggr Operator
- Stage: Stage-4
- Stats-Aggr Operator
-
PREHOOK: query: insert overwrite table tmptable
select unionsrc.key, unionsrc.value FROM (select 'tst1' as key, count(1) as value from src s1
UNION ALL
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/union5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/union5.q.out b/ql/src/test/results/clientpositive/llap/union5.q.out
index bc3821b..856f9f6 100644
--- a/ql/src/test/results/clientpositive/llap/union5.q.out
+++ b/ql/src/test/results/clientpositive/llap/union5.q.out
@@ -21,6 +21,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
Reducer 4 <- Union 3 (SIMPLE_EDGE)
@@ -31,103 +32,105 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: key
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(key)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: s2
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: key
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(key)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: 'tst1' (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: 'tst2' (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Union 3
Vertex: Union 3
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/union6.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/union6.q.out b/ql/src/test/results/clientpositive/llap/union6.q.out
index cb5fa80..1b1edf8 100644
--- a/ql/src/test/results/clientpositive/llap/union6.q.out
+++ b/ql/src/test/results/clientpositive/llap/union6.q.out
@@ -31,11 +31,11 @@ STAGE DEPENDENCIES:
Stage-2 depends on stages: Stage-1
Stage-0 depends on stages: Stage-2
Stage-3 depends on stages: Stage-0
- Stage-4 depends on stages: Stage-0
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 4 <- Union 3 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -58,26 +58,28 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 463 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 26 Data size: 4647 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.tmptable
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -90,7 +92,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 463 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 26 Data size: 4647 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -115,9 +117,6 @@ STAGE PLANS:
Stage: Stage-3
Stats-Aggr Operator
- Stage: Stage-4
- Stats-Aggr Operator
-
PREHOOK: query: insert overwrite table tmptable
select unionsrc.key, unionsrc.value FROM (select 'tst1' as key, cast(count(1) as string) as value from src s1
UNION ALL
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/union7.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/union7.q.out b/ql/src/test/results/clientpositive/llap/union7.q.out
index a301cad..4039b7d 100644
--- a/ql/src/test/results/clientpositive/llap/union7.q.out
+++ b/ql/src/test/results/clientpositive/llap/union7.q.out
@@ -21,6 +21,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 5 <- Union 3 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -31,82 +32,84 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: key
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(key)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 26 Data size: 199 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1248 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 26 Data size: 199 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1248 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: 'tst1' (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 88 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 26 Data size: 199 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1248 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 26 Data size: 199 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1248 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1248 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 1248 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 3
Vertex: Union 3
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/union8.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/union8.q.out b/ql/src/test/results/clientpositive/llap/union8.q.out
index 972694d..1e54073 100644
--- a/ql/src/test/results/clientpositive/llap/union8.q.out
+++ b/ql/src/test/results/clientpositive/llap/union8.q.out
@@ -21,6 +21,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Map 3 <- Union 2 (CONTAINS)
@@ -31,53 +32,56 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1500 Data size: 15936 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1500 Data size: 267000 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: s2
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1500 Data size: 15936 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1500 Data size: 267000 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
- alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: s3
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1500 Data size: 15936 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1500 Data size: 267000 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: llap
+ LLAP IO: no inputs
Union 2
Vertex: Union 2
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/union9.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/union9.q.out b/ql/src/test/results/clientpositive/llap/union9.q.out
index 3867806..bb70d74 100644
--- a/ql/src/test/results/clientpositive/llap/union9.q.out
+++ b/ql/src/test/results/clientpositive/llap/union9.q.out
@@ -21,6 +21,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Map 4 <- Union 2 (CONTAINS)
@@ -32,79 +33,76 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: value (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 1500 Data size: 15936 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1500 Data size: 12000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
- alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: s2
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: value (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 1500 Data size: 15936 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1500 Data size: 12000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: s3
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: value (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 1500 Data size: 15936 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1500 Data size: 12000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 2
Vertex: Union 2
[38/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/constprog_semijoin.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/constprog_semijoin.q.out b/ql/src/test/results/clientpositive/llap/constprog_semijoin.q.out
new file mode 100644
index 0000000..e9f8a8f
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/constprog_semijoin.q.out
@@ -0,0 +1,494 @@
+PREHOOK: query: create table table1 (id int, val string, val1 string, dimid int)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@table1
+POSTHOOK: query: create table table1 (id int, val string, val1 string, dimid int)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@table1
+PREHOOK: query: insert into table1 (id, val, val1, dimid) values (1, 't1val01', 'val101', 100), (2, 't1val02', 'val102', 200), (3, 't1val03', 'val103', 103), (3, 't1val01', 'val104', 100), (2, 't1val05', 'val105', 200), (3, 't1val01', 'val106', 103), (1, 't1val07', 'val107', 200), (2, 't1val01', 'val108', 200), (3, 't1val09', 'val109', 103), (4,'t1val01', 'val110', 200)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__1
+PREHOOK: Output: default@table1
+POSTHOOK: query: insert into table1 (id, val, val1, dimid) values (1, 't1val01', 'val101', 100), (2, 't1val02', 'val102', 200), (3, 't1val03', 'val103', 103), (3, 't1val01', 'val104', 100), (2, 't1val05', 'val105', 200), (3, 't1val01', 'val106', 103), (1, 't1val07', 'val107', 200), (2, 't1val01', 'val108', 200), (3, 't1val09', 'val109', 103), (4,'t1val01', 'val110', 200)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__1
+POSTHOOK: Output: default@table1
+POSTHOOK: Lineage: table1.dimid EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col4, type:string, comment:), ]
+POSTHOOK: Lineage: table1.id EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: table1.val SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+POSTHOOK: Lineage: table1.val1 SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col3, type:string, comment:), ]
+PREHOOK: query: create table table2 (id int, val2 string)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@table2
+POSTHOOK: query: create table table2 (id int, val2 string)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@table2
+PREHOOK: query: insert into table2 (id, val2) values (1, 't2val201'), (2, 't2val202'), (3, 't2val203')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__2
+PREHOOK: Output: default@table2
+POSTHOOK: query: insert into table2 (id, val2) values (1, 't2val201'), (2, 't2val202'), (3, 't2val203')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__2
+POSTHOOK: Output: default@table2
+POSTHOOK: Lineage: table2.id EXPRESSION [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: table2.val2 SIMPLE [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: create table table3 (id int)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@table3
+POSTHOOK: query: create table table3 (id int)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@table3
+PREHOOK: query: insert into table3 (id) values (100), (100), (101), (102), (103)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__3
+PREHOOK: Output: default@table3
+POSTHOOK: query: insert into table3 (id) values (100), (100), (101), (102), (103)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__3
+POSTHOOK: Output: default@table3
+POSTHOOK: Lineage: table3.id EXPRESSION [(values__tmp__table__3)values__tmp__table__3.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+PREHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id where table1.val = 't1val01'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id where table1.val = 't1val01'
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_12]
+ Select Operator [SEL_11] (rows=5 width=22)
+ Output:["_col0","_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_17] (rows=5 width=22)
+ Conds:RS_8._col3=RS_9._col0(Left Semi),Output:["_col0","_col2"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_8]
+ PartitionCols:_col3
+ Select Operator [SEL_2] (rows=5 width=20)
+ Output:["_col0","_col2","_col3"]
+ Filter Operator [FIL_15] (rows=5 width=20)
+ predicate:((val = 't1val01') and dimid is not null)
+ TableScan [TS_0] (rows=10 width=20)
+ default@table1,table1,Tbl:COMPLETE,Col:NONE,Output:["id","val","val1","dimid"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_9]
+ PartitionCols:_col0
+ Group By Operator [GBY_7] (rows=5 width=3)
+ Output:["_col0"],keys:_col0
+ Select Operator [SEL_5] (rows=5 width=3)
+ Output:["_col0"]
+ Filter Operator [FIL_16] (rows=5 width=3)
+ predicate:id is not null
+ TableScan [TS_3] (rows=5 width=3)
+ default@table3,table3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id where table1.val = 't1val01'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@table1
+PREHOOK: Input: default@table3
+#### A masked pattern was here ####
+POSTHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id where table1.val = 't1val01'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@table1
+POSTHOOK: Input: default@table3
+#### A masked pattern was here ####
+1 t1val01 val101
+3 t1val01 val104
+3 t1val01 val106
+PREHOOK: query: explain select table1.id, table1.val, table2.val2 from table1 inner join table2 on table1.val = 't1val01' and table1.id = table2.id left semi join table3 on table1.dimid = table3.id
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select table1.id, table1.val, table2.val2 from table1 inner join table2 on table1.val = 't1val01' and table1.id = table2.id left semi join table3 on table1.dimid = table3.id
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_18]
+ Select Operator [SEL_17] (rows=5 width=24)
+ Output:["_col0","_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_28] (rows=5 width=24)
+ Conds:RS_14._col0=RS_15._col0(Inner),Output:["_col0","_col4"]
+ <-Map 5 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_15]
+ PartitionCols:_col0
+ Select Operator [SEL_8] (rows=3 width=10)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_26] (rows=3 width=10)
+ predicate:id is not null
+ TableScan [TS_6] (rows=3 width=10)
+ default@table2,table2,Tbl:COMPLETE,Col:NONE,Output:["id","val2"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_14]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_27] (rows=5 width=22)
+ Conds:RS_11._col2=RS_12._col0(Left Semi),Output:["_col0"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=5 width=20)
+ Output:["_col0","_col2"]
+ Filter Operator [FIL_24] (rows=5 width=20)
+ predicate:((val = 't1val01') and id is not null and dimid is not null)
+ TableScan [TS_0] (rows=10 width=20)
+ default@table1,table1,Tbl:COMPLETE,Col:NONE,Output:["id","val","dimid"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col0
+ Group By Operator [GBY_10] (rows=5 width=3)
+ Output:["_col0"],keys:_col0
+ Select Operator [SEL_5] (rows=5 width=3)
+ Output:["_col0"]
+ Filter Operator [FIL_25] (rows=5 width=3)
+ predicate:id is not null
+ TableScan [TS_3] (rows=5 width=3)
+ default@table3,table3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: select table1.id, table1.val, table2.val2 from table1 inner join table2 on table1.val = 't1val01' and table1.id = table2.id left semi join table3 on table1.dimid = table3.id
+PREHOOK: type: QUERY
+PREHOOK: Input: default@table1
+PREHOOK: Input: default@table2
+PREHOOK: Input: default@table3
+#### A masked pattern was here ####
+POSTHOOK: query: select table1.id, table1.val, table2.val2 from table1 inner join table2 on table1.val = 't1val01' and table1.id = table2.id left semi join table3 on table1.dimid = table3.id
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@table1
+POSTHOOK: Input: default@table2
+POSTHOOK: Input: default@table3
+#### A masked pattern was here ####
+1 t1val01 t2val201
+3 t1val01 t2val203
+3 t1val01 t2val203
+PREHOOK: query: explain select table1.id, table1.val, table2.val2 from table1 left semi join table3 on table1.dimid = table3.id inner join table2 on table1.val = 't1val01' and table1.id = table2.id
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select table1.id, table1.val, table2.val2 from table1 left semi join table3 on table1.dimid = table3.id inner join table2 on table1.val = 't1val01' and table1.id = table2.id
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_18]
+ Select Operator [SEL_17] (rows=5 width=24)
+ Output:["_col0","_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_28] (rows=5 width=24)
+ Conds:RS_14._col0=RS_15._col0(Inner),Output:["_col0","_col4"]
+ <-Map 5 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_15]
+ PartitionCols:_col0
+ Select Operator [SEL_8] (rows=3 width=10)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_26] (rows=3 width=10)
+ predicate:id is not null
+ TableScan [TS_6] (rows=3 width=10)
+ default@table2,table2,Tbl:COMPLETE,Col:NONE,Output:["id","val2"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_14]
+ PartitionCols:_col0
+ Merge Join Operator [MERGEJOIN_27] (rows=5 width=22)
+ Conds:RS_11._col2=RS_12._col0(Left Semi),Output:["_col0"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=5 width=20)
+ Output:["_col0","_col2"]
+ Filter Operator [FIL_24] (rows=5 width=20)
+ predicate:((val = 't1val01') and dimid is not null and id is not null)
+ TableScan [TS_0] (rows=10 width=20)
+ default@table1,table1,Tbl:COMPLETE,Col:NONE,Output:["id","val","dimid"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col0
+ Group By Operator [GBY_10] (rows=5 width=3)
+ Output:["_col0"],keys:_col0
+ Select Operator [SEL_5] (rows=5 width=3)
+ Output:["_col0"]
+ Filter Operator [FIL_25] (rows=5 width=3)
+ predicate:id is not null
+ TableScan [TS_3] (rows=5 width=3)
+ default@table3,table3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: select table1.id, table1.val, table2.val2 from table1 left semi join table3 on table1.dimid = table3.id inner join table2 on table1.val = 't1val01' and table1.id = table2.id
+PREHOOK: type: QUERY
+PREHOOK: Input: default@table1
+PREHOOK: Input: default@table2
+PREHOOK: Input: default@table3
+#### A masked pattern was here ####
+POSTHOOK: query: select table1.id, table1.val, table2.val2 from table1 left semi join table3 on table1.dimid = table3.id inner join table2 on table1.val = 't1val01' and table1.id = table2.id
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@table1
+POSTHOOK: Input: default@table2
+POSTHOOK: Input: default@table3
+#### A masked pattern was here ####
+1 t1val01 t2val201
+3 t1val01 t2val203
+3 t1val01 t2val203
+PREHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid <> 100
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid <> 100
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_12]
+ Merge Join Operator [MERGEJOIN_17] (rows=2 width=3)
+ Conds:RS_8.100, true=RS_9._col0, _col1(Left Semi),Output:["_col0","_col1","_col2"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_8]
+ PartitionCols:100, true
+ Select Operator [SEL_2] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_15] (rows=1 width=20)
+ predicate:false
+ TableScan [TS_0] (rows=10 width=20)
+ default@table1,table1,Tbl:COMPLETE,Col:NONE,Output:["id","val","val1"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_9]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_7] (rows=2 width=3)
+ Output:["_col0","_col1"],keys:_col0, _col1
+ Select Operator [SEL_5] (rows=2 width=3)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_16] (rows=2 width=3)
+ predicate:((id = 100) and (id = 100) is not null)
+ TableScan [TS_3] (rows=5 width=3)
+ default@table3,table3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid <> 100
+PREHOOK: type: QUERY
+PREHOOK: Input: default@table1
+PREHOOK: Input: default@table3
+#### A masked pattern was here ####
+POSTHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid <> 100
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@table1
+POSTHOOK: Input: default@table3
+#### A masked pattern was here ####
+PREHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid IN (100,200)
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid IN (100,200)
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_12]
+ Merge Join Operator [MERGEJOIN_17] (rows=5 width=22)
+ Conds:RS_8.100, true=RS_9._col0, _col1(Left Semi),Output:["_col0","_col1","_col2"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_8]
+ PartitionCols:100, true
+ Select Operator [SEL_2] (rows=5 width=20)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_15] (rows=5 width=20)
+ predicate:((dimid = 100) and (dimid = 100) is not null)
+ TableScan [TS_0] (rows=10 width=20)
+ default@table1,table1,Tbl:COMPLETE,Col:NONE,Output:["id","val","val1","dimid"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_9]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_7] (rows=2 width=3)
+ Output:["_col0","_col1"],keys:_col0, _col1
+ Select Operator [SEL_5] (rows=2 width=3)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_16] (rows=2 width=3)
+ predicate:((id = 100) and (id = 100) is not null)
+ TableScan [TS_3] (rows=5 width=3)
+ default@table3,table3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid IN (100,200)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@table1
+PREHOOK: Input: default@table3
+#### A masked pattern was here ####
+POSTHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid IN (100,200)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@table1
+POSTHOOK: Input: default@table3
+#### A masked pattern was here ####
+1 t1val01 val101
+3 t1val01 val104
+PREHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid = 200
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid = 200
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_12]
+ Merge Join Operator [MERGEJOIN_17] (rows=2 width=3)
+ Conds:RS_8.100, true=RS_9._col0, _col1(Left Semi),Output:["_col0","_col1","_col2"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_8]
+ PartitionCols:100, true
+ Select Operator [SEL_2] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_15] (rows=1 width=20)
+ predicate:false
+ TableScan [TS_0] (rows=10 width=20)
+ default@table1,table1,Tbl:COMPLETE,Col:NONE,Output:["id","val","val1"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_9]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_7] (rows=2 width=3)
+ Output:["_col0","_col1"],keys:_col0, _col1
+ Select Operator [SEL_5] (rows=2 width=3)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_16] (rows=2 width=3)
+ predicate:((id = 100) and (id = 100) is not null)
+ TableScan [TS_3] (rows=5 width=3)
+ default@table3,table3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid = 200
+PREHOOK: type: QUERY
+PREHOOK: Input: default@table1
+PREHOOK: Input: default@table3
+#### A masked pattern was here ####
+POSTHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid = 200
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@table1
+POSTHOOK: Input: default@table3
+#### A masked pattern was here ####
+PREHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid = 100
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid = 100
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_12]
+ Merge Join Operator [MERGEJOIN_17] (rows=5 width=22)
+ Conds:RS_8.100, true=RS_9._col0, _col1(Left Semi),Output:["_col0","_col1","_col2"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_8]
+ PartitionCols:100, true
+ Select Operator [SEL_2] (rows=5 width=20)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_15] (rows=5 width=20)
+ predicate:((dimid = 100) and (dimid = 100) is not null)
+ TableScan [TS_0] (rows=10 width=20)
+ default@table1,table1,Tbl:COMPLETE,Col:NONE,Output:["id","val","val1","dimid"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_9]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_7] (rows=2 width=3)
+ Output:["_col0","_col1"],keys:_col0, _col1
+ Select Operator [SEL_5] (rows=2 width=3)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_16] (rows=2 width=3)
+ predicate:((id = 100) and (id = 100) is not null)
+ TableScan [TS_3] (rows=5 width=3)
+ default@table3,table3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid = 100
+PREHOOK: type: QUERY
+PREHOOK: Input: default@table1
+PREHOOK: Input: default@table3
+#### A masked pattern was here ####
+POSTHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100 where table1.dimid = 100
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@table1
+POSTHOOK: Input: default@table3
+#### A masked pattern was here ####
+1 t1val01 val101
+3 t1val01 val104
+PREHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_12]
+ Merge Join Operator [MERGEJOIN_17] (rows=5 width=22)
+ Conds:RS_8.100, true=RS_9._col0, _col1(Left Semi),Output:["_col0","_col1","_col2"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_8]
+ PartitionCols:100, true
+ Select Operator [SEL_2] (rows=5 width=20)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_15] (rows=5 width=20)
+ predicate:((dimid = 100) and (dimid = 100) is not null)
+ TableScan [TS_0] (rows=10 width=20)
+ default@table1,table1,Tbl:COMPLETE,Col:NONE,Output:["id","val","val1","dimid"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_9]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_7] (rows=2 width=3)
+ Output:["_col0","_col1"],keys:_col0, _col1
+ Select Operator [SEL_5] (rows=2 width=3)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_16] (rows=2 width=3)
+ predicate:((id = 100) and (id = 100) is not null)
+ TableScan [TS_3] (rows=5 width=3)
+ default@table3,table3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100
+PREHOOK: type: QUERY
+PREHOOK: Input: default@table1
+PREHOOK: Input: default@table3
+#### A masked pattern was here ####
+POSTHOOK: query: select table1.id, table1.val, table1.val1 from table1 left semi join table3 on table1.dimid = table3.id and table3.id = 100
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@table1
+POSTHOOK: Input: default@table3
+#### A masked pattern was here ####
+1 t1val01 val101
+3 t1val01 val104
[16/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_analyze.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_analyze.q.out b/ql/src/test/results/clientpositive/llap/orc_analyze.q.out
index 6eb9a93..87855fa 100644
--- a/ql/src/test/results/clientpositive/llap/orc_analyze.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_analyze.q.out
@@ -102,11 +102,11 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 100
rawDataSize 52600
- totalSize 3174
+ totalSize 3202
#### A masked pattern was here ####
# Storage Information
@@ -150,11 +150,11 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 100
rawDataSize 52600
- totalSize 3174
+ totalSize 3202
#### A masked pattern was here ####
# Storage Information
@@ -198,11 +198,11 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 100
rawDataSize 52600
- totalSize 3174
+ totalSize 3202
#### A masked pattern was here ####
# Storage Information
@@ -287,11 +287,11 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 100
rawDataSize 52600
- totalSize 3174
+ totalSize 3202
#### A masked pattern was here ####
# Storage Information
@@ -401,11 +401,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 21950
- totalSize 2073
+ totalSize 2102
#### A masked pattern was here ####
# Storage Information
@@ -444,11 +444,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 22050
- totalSize 2088
+ totalSize 2118
#### A masked pattern was here ####
# Storage Information
@@ -499,11 +499,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 21950
- totalSize 2073
+ totalSize 2102
#### A masked pattern was here ####
# Storage Information
@@ -542,11 +542,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 22050
- totalSize 2088
+ totalSize 2118
#### A masked pattern was here ####
# Storage Information
@@ -597,11 +597,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 21950
- totalSize 2073
+ totalSize 2102
#### A masked pattern was here ####
# Storage Information
@@ -640,11 +640,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 22050
- totalSize 2088
+ totalSize 2118
#### A masked pattern was here ####
# Storage Information
@@ -740,11 +740,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 21950
- totalSize 2073
+ totalSize 2102
#### A masked pattern was here ####
# Storage Information
@@ -783,11 +783,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 22050
- totalSize 2088
+ totalSize 2118
#### A masked pattern was here ####
# Storage Information
@@ -903,11 +903,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 1
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 50
- rawDataSize 21950
- totalSize 2073
+ rawDataSize 21975
+ totalSize 5263
#### A masked pattern was here ####
# Storage Information
@@ -946,11 +946,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 1
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 50
- rawDataSize 22050
- totalSize 2088
+ rawDataSize 22043
+ totalSize 5336
#### A masked pattern was here ####
# Storage Information
@@ -1001,11 +1001,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 1
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 50
- rawDataSize 21950
- totalSize 2073
+ rawDataSize 21975
+ totalSize 5263
#### A masked pattern was here ####
# Storage Information
@@ -1044,11 +1044,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 1
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 50
- rawDataSize 22050
- totalSize 2088
+ rawDataSize 22043
+ totalSize 5336
#### A masked pattern was here ####
# Storage Information
@@ -1099,11 +1099,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 1
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 50
- rawDataSize 21950
- totalSize 2073
+ rawDataSize 21975
+ totalSize 5263
#### A masked pattern was here ####
# Storage Information
@@ -1142,11 +1142,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 1
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 50
- rawDataSize 22050
- totalSize 2088
+ rawDataSize 22043
+ totalSize 5336
#### A masked pattern was here ####
# Storage Information
@@ -1248,11 +1248,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 1
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 50
- rawDataSize 21950
- totalSize 2073
+ rawDataSize 21975
+ totalSize 5263
#### A masked pattern was here ####
# Storage Information
@@ -1291,11 +1291,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 1
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 50
- rawDataSize 22050
- totalSize 2088
+ rawDataSize 22043
+ totalSize 5336
#### A masked pattern was here ####
# Storage Information
@@ -1369,66 +1369,17 @@ POSTHOOK: Lineage: orc_create_people PARTITION(state=Or).id SIMPLE [(orc_create_
POSTHOOK: Lineage: orc_create_people PARTITION(state=Or).last_name SIMPLE [(orc_create_people_staging)orc_create_people_staging.FieldSchema(name:last_name, type:string, comment:null), ]
POSTHOOK: Lineage: orc_create_people PARTITION(state=Or).salary SIMPLE [(orc_create_people_staging)orc_create_people_staging.FieldSchema(name:salary, type:decimal(10,0), comment:null), ]
POSTHOOK: Lineage: orc_create_people PARTITION(state=Or).start_date SIMPLE [(orc_create_people_staging)orc_create_people_staging.FieldSchema(name:start_date, type:timestamp, comment:null), ]
-PREHOOK: query: -- set the table to text format
-ALTER TABLE orc_create_people SET SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
-PREHOOK: type: ALTERTABLE_SERIALIZER
-PREHOOK: Input: default@orc_create_people
-PREHOOK: Output: default@orc_create_people
-POSTHOOK: query: -- set the table to text format
-ALTER TABLE orc_create_people SET SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
-POSTHOOK: type: ALTERTABLE_SERIALIZER
-POSTHOOK: Input: default@orc_create_people
-POSTHOOK: Output: default@orc_create_people
-PREHOOK: query: ALTER TABLE orc_create_people SET FILEFORMAT TEXTFILE
-PREHOOK: type: ALTERTABLE_FILEFORMAT
-PREHOOK: Input: default@orc_create_people
-PREHOOK: Output: default@orc_create_people
-POSTHOOK: query: ALTER TABLE orc_create_people SET FILEFORMAT TEXTFILE
-POSTHOOK: type: ALTERTABLE_FILEFORMAT
-POSTHOOK: Input: default@orc_create_people
-POSTHOOK: Output: default@orc_create_people
-PREHOOK: query: -- load the text data into a new partition
-LOAD DATA LOCAL INPATH '../../data/files/kv1.txt' OVERWRITE INTO TABLE orc_create_people PARTITION(state="OH")
-PREHOOK: type: LOAD
-#### A masked pattern was here ####
-PREHOOK: Output: default@orc_create_people
-POSTHOOK: query: -- load the text data into a new partition
-LOAD DATA LOCAL INPATH '../../data/files/kv1.txt' OVERWRITE INTO TABLE orc_create_people PARTITION(state="OH")
-POSTHOOK: type: LOAD
-#### A masked pattern was here ####
-POSTHOOK: Output: default@orc_create_people
-POSTHOOK: Output: default@orc_create_people@state=OH
-PREHOOK: query: -- set the table back to orc
-ALTER TABLE orc_create_people SET SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
-PREHOOK: type: ALTERTABLE_SERIALIZER
-PREHOOK: Input: default@orc_create_people
-PREHOOK: Output: default@orc_create_people
-POSTHOOK: query: -- set the table back to orc
-ALTER TABLE orc_create_people SET SERDE 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
-POSTHOOK: type: ALTERTABLE_SERIALIZER
-POSTHOOK: Input: default@orc_create_people
-POSTHOOK: Output: default@orc_create_people
-PREHOOK: query: ALTER TABLE orc_create_people SET FILEFORMAT ORC
-PREHOOK: type: ALTERTABLE_FILEFORMAT
-PREHOOK: Input: default@orc_create_people
-PREHOOK: Output: default@orc_create_people
-POSTHOOK: query: ALTER TABLE orc_create_people SET FILEFORMAT ORC
-POSTHOOK: type: ALTERTABLE_FILEFORMAT
-POSTHOOK: Input: default@orc_create_people
-POSTHOOK: Output: default@orc_create_people
PREHOOK: query: analyze table orc_create_people partition(state) compute statistics
PREHOOK: type: QUERY
PREHOOK: Input: default@orc_create_people
PREHOOK: Output: default@orc_create_people
PREHOOK: Output: default@orc_create_people@state=Ca
-PREHOOK: Output: default@orc_create_people@state=OH
PREHOOK: Output: default@orc_create_people@state=Or
POSTHOOK: query: analyze table orc_create_people partition(state) compute statistics
POSTHOOK: type: QUERY
POSTHOOK: Input: default@orc_create_people
POSTHOOK: Output: default@orc_create_people
POSTHOOK: Output: default@orc_create_people@state=Ca
-POSTHOOK: Output: default@orc_create_people@state=OH
POSTHOOK: Output: default@orc_create_people@state=Or
PREHOOK: query: desc formatted orc_create_people partition(state="Ca")
PREHOOK: type: DESCTABLE
@@ -1456,11 +1407,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 21950
- totalSize 2073
+ totalSize 2102
#### A masked pattern was here ####
# Storage Information
@@ -1473,62 +1424,17 @@ Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
-PREHOOK: query: desc formatted orc_create_people partition(state="OH")
-PREHOOK: type: DESCTABLE
-PREHOOK: Input: default@orc_create_people
-POSTHOOK: query: desc formatted orc_create_people partition(state="OH")
-POSTHOOK: type: DESCTABLE
-POSTHOOK: Input: default@orc_create_people
-# col_name data_type comment
-
-id int
-first_name string
-last_name string
-address string
-salary decimal(10,0)
-start_date timestamp
-
-# Partition Information
-# col_name data_type comment
-
-state string
-
-# Detailed Partition Information
-Partition Value: [OH]
-Database: default
-Table: orc_create_people
-#### A masked pattern was here ####
-Partition Parameters:
- COLUMN_STATS_ACCURATE false
- numFiles 1
- numRows -1
- rawDataSize -1
- totalSize 5812
-#### A masked pattern was here ####
-
-# Storage Information
-SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-InputFormat: org.apache.hadoop.mapred.TextInputFormat
-OutputFormat: org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat
-Compressed: No
-Num Buckets: -1
-Bucket Columns: []
-Sort Columns: []
-Storage Desc Params:
- serialization.format 1
PREHOOK: query: analyze table orc_create_people partition(state) compute statistics partialscan
PREHOOK: type: QUERY
PREHOOK: Input: default@orc_create_people
PREHOOK: Output: default@orc_create_people
PREHOOK: Output: default@orc_create_people@state=Ca
-PREHOOK: Output: default@orc_create_people@state=OH
PREHOOK: Output: default@orc_create_people@state=Or
POSTHOOK: query: analyze table orc_create_people partition(state) compute statistics partialscan
POSTHOOK: type: QUERY
POSTHOOK: Input: default@orc_create_people
POSTHOOK: Output: default@orc_create_people
POSTHOOK: Output: default@orc_create_people@state=Ca
-POSTHOOK: Output: default@orc_create_people@state=OH
POSTHOOK: Output: default@orc_create_people@state=Or
PREHOOK: query: desc formatted orc_create_people partition(state="Ca")
PREHOOK: type: DESCTABLE
@@ -1556,11 +1462,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 21950
- totalSize 2073
+ totalSize 2102
#### A masked pattern was here ####
# Storage Information
@@ -1573,62 +1479,17 @@ Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
-PREHOOK: query: desc formatted orc_create_people partition(state="OH")
-PREHOOK: type: DESCTABLE
-PREHOOK: Input: default@orc_create_people
-POSTHOOK: query: desc formatted orc_create_people partition(state="OH")
-POSTHOOK: type: DESCTABLE
-POSTHOOK: Input: default@orc_create_people
-# col_name data_type comment
-
-id int
-first_name string
-last_name string
-address string
-salary decimal(10,0)
-start_date timestamp
-
-# Partition Information
-# col_name data_type comment
-
-state string
-
-# Detailed Partition Information
-Partition Value: [OH]
-Database: default
-Table: orc_create_people
-#### A masked pattern was here ####
-Partition Parameters:
- COLUMN_STATS_ACCURATE false
- numFiles 1
- numRows -1
- rawDataSize -1
- totalSize 5812
-#### A masked pattern was here ####
-
-# Storage Information
-SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-InputFormat: org.apache.hadoop.mapred.TextInputFormat
-OutputFormat: org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat
-Compressed: No
-Num Buckets: -1
-Bucket Columns: []
-Sort Columns: []
-Storage Desc Params:
- serialization.format 1
PREHOOK: query: analyze table orc_create_people partition(state) compute statistics noscan
PREHOOK: type: QUERY
PREHOOK: Input: default@orc_create_people
PREHOOK: Output: default@orc_create_people
PREHOOK: Output: default@orc_create_people@state=Ca
-PREHOOK: Output: default@orc_create_people@state=OH
PREHOOK: Output: default@orc_create_people@state=Or
POSTHOOK: query: analyze table orc_create_people partition(state) compute statistics noscan
POSTHOOK: type: QUERY
POSTHOOK: Input: default@orc_create_people
POSTHOOK: Output: default@orc_create_people
POSTHOOK: Output: default@orc_create_people@state=Ca
-POSTHOOK: Output: default@orc_create_people@state=OH
POSTHOOK: Output: default@orc_create_people@state=Or
PREHOOK: query: desc formatted orc_create_people partition(state="Ca")
PREHOOK: type: DESCTABLE
@@ -1656,11 +1517,11 @@ Database: default
Table: orc_create_people
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 50
rawDataSize 21950
- totalSize 2073
+ totalSize 2102
#### A masked pattern was here ####
# Storage Information
@@ -1673,49 +1534,6 @@ Bucket Columns: []
Sort Columns: []
Storage Desc Params:
serialization.format 1
-PREHOOK: query: desc formatted orc_create_people partition(state="OH")
-PREHOOK: type: DESCTABLE
-PREHOOK: Input: default@orc_create_people
-POSTHOOK: query: desc formatted orc_create_people partition(state="OH")
-POSTHOOK: type: DESCTABLE
-POSTHOOK: Input: default@orc_create_people
-# col_name data_type comment
-
-id int
-first_name string
-last_name string
-address string
-salary decimal(10,0)
-start_date timestamp
-
-# Partition Information
-# col_name data_type comment
-
-state string
-
-# Detailed Partition Information
-Partition Value: [OH]
-Database: default
-Table: orc_create_people
-#### A masked pattern was here ####
-Partition Parameters:
- COLUMN_STATS_ACCURATE false
- numFiles 1
- numRows -1
- rawDataSize -1
- totalSize 5812
-#### A masked pattern was here ####
-
-# Storage Information
-SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-InputFormat: org.apache.hadoop.mapred.TextInputFormat
-OutputFormat: org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat
-Compressed: No
-Num Buckets: -1
-Bucket Columns: []
-Sort Columns: []
-Storage Desc Params:
- serialization.format 1
PREHOOK: query: drop table orc_create_people
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@orc_create_people
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge1.q.out b/ql/src/test/results/clientpositive/llap/orc_merge1.q.out
index 241fb05..973d5a1 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge1.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge1.q.out
@@ -71,20 +71,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orcfile_merge1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -156,20 +157,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orcfile_merge1b
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-8
Conditional Operator
@@ -286,20 +288,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orcfile_merge1c
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-8
Conditional Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge10.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge10.q.out b/ql/src/test/results/clientpositive/llap/orc_merge10.q.out
new file mode 100644
index 0000000..164edda
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/orc_merge10.q.out
@@ -0,0 +1,743 @@
+PREHOOK: query: -- SORT_QUERY_RESULTS
+
+DROP TABLE orcfile_merge1
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: -- SORT_QUERY_RESULTS
+
+DROP TABLE orcfile_merge1
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: DROP TABLE orcfile_merge1b
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE orcfile_merge1b
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: DROP TABLE orcfile_merge1c
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE orcfile_merge1c
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: CREATE TABLE orcfile_merge1 (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC tblproperties("orc.compress"="SNAPPY","orc.compress.size"="4096")
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@orcfile_merge1
+POSTHOOK: query: CREATE TABLE orcfile_merge1 (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC tblproperties("orc.compress"="SNAPPY","orc.compress.size"="4096")
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@orcfile_merge1
+PREHOOK: query: CREATE TABLE orcfile_merge1b (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC tblproperties("orc.compress"="SNAPPY","orc.compress.size"="4096")
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@orcfile_merge1b
+POSTHOOK: query: CREATE TABLE orcfile_merge1b (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC tblproperties("orc.compress"="SNAPPY","orc.compress.size"="4096")
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@orcfile_merge1b
+PREHOOK: query: CREATE TABLE orcfile_merge1c (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC tblproperties("orc.compress"="SNAPPY","orc.compress.size"="4096")
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@orcfile_merge1c
+POSTHOOK: query: CREATE TABLE orcfile_merge1c (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC tblproperties("orc.compress"="SNAPPY","orc.compress.size"="4096")
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@orcfile_merge1c
+PREHOOK: query: -- merge disabled
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1 PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- merge disabled
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1 PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ ds 1
+ part
+ replace: true
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1 PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: default@orcfile_merge1@ds=1
+POSTHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1 PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: Output: default@orcfile_merge1@ds=1/part=1
+POSTHOOK: Lineage: orcfile_merge1 PARTITION(ds=1,part=0).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1 PARTITION(ds=1,part=0).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1 PARTITION(ds=1,part=1).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1 PARTITION(ds=1,part=1).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+Found 3 items
+#### A masked pattern was here ####
+PREHOOK: query: -- auto-merge slow way
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1b PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- auto-merge slow way
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1b PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-8 depends on stages: Stage-1 , consists of Stage-5, Stage-4, Stage-6
+ Stage-5
+ Stage-2 depends on stages: Stage-5, Stage-4, Stage-7
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+ Stage-4
+ Stage-6
+ Stage-7 depends on stages: Stage-6
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1b
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-8
+ Conditional Operator
+
+ Stage: Stage-5
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ ds 1
+ part
+ replace: true
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1b
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+ Stage: Stage-4
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ File Merge
+ Map Operator Tree:
+ TableScan
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1b
+
+ Stage: Stage-6
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ File Merge
+ Map Operator Tree:
+ TableScan
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1b
+
+ Stage: Stage-7
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+PREHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1b PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: default@orcfile_merge1b@ds=1
+POSTHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1b PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: default@orcfile_merge1b@ds=1/part=0
+POSTHOOK: Output: default@orcfile_merge1b@ds=1/part=1
+POSTHOOK: Lineage: orcfile_merge1b PARTITION(ds=1,part=0).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1b PARTITION(ds=1,part=0).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1b PARTITION(ds=1,part=1).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1b PARTITION(ds=1,part=1).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+Found 1 items
+#### A masked pattern was here ####
+PREHOOK: query: -- auto-merge fast way
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1c PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- auto-merge fast way
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1c PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-8 depends on stages: Stage-1 , consists of Stage-5, Stage-4, Stage-6
+ Stage-5
+ Stage-2 depends on stages: Stage-5, Stage-4, Stage-7
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+ Stage-4
+ Stage-6
+ Stage-7 depends on stages: Stage-6
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1c
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-8
+ Conditional Operator
+
+ Stage: Stage-5
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ ds 1
+ part
+ replace: true
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1c
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+ Stage: Stage-4
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ File Merge
+ Merge File Operator
+ Map Operator Tree:
+ ORC File Merge Operator
+ merge level: stripe
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+
+ Stage: Stage-6
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ File Merge
+ Merge File Operator
+ Map Operator Tree:
+ ORC File Merge Operator
+ merge level: stripe
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+
+ Stage: Stage-7
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+PREHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1c PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: default@orcfile_merge1c@ds=1
+POSTHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1c PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: default@orcfile_merge1c@ds=1/part=0
+POSTHOOK: Output: default@orcfile_merge1c@ds=1/part=1
+POSTHOOK: Lineage: orcfile_merge1c PARTITION(ds=1,part=0).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1c PARTITION(ds=1,part=0).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1c PARTITION(ds=1,part=1).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1c PARTITION(ds=1,part=1).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+Found 1 items
+#### A masked pattern was here ####
+PREHOOK: query: -- Verify
+SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1 WHERE ds='1'
+) t
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: -- Verify
+SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1 WHERE ds='1'
+) t
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+-21975308766
+PREHOOK: query: SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1b WHERE ds='1'
+) t
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1b
+PREHOOK: Input: default@orcfile_merge1b@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1b@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1b WHERE ds='1'
+) t
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1b
+POSTHOOK: Input: default@orcfile_merge1b@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1b@ds=1/part=1
+#### A masked pattern was here ####
+-21975308766
+PREHOOK: query: select count(*) from orcfile_merge1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from orcfile_merge1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+500
+PREHOOK: query: select count(*) from orcfile_merge1b
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1b
+PREHOOK: Input: default@orcfile_merge1b@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1b@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from orcfile_merge1b
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1b
+POSTHOOK: Input: default@orcfile_merge1b@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1b@ds=1/part=1
+#### A masked pattern was here ####
+500
+PREHOOK: query: -- concatenate
+explain ALTER TABLE orcfile_merge1 PARTITION (ds='1', part='0') CONCATENATE
+PREHOOK: type: ALTER_PARTITION_MERGE
+POSTHOOK: query: -- concatenate
+explain ALTER TABLE orcfile_merge1 PARTITION (ds='1', part='0') CONCATENATE
+POSTHOOK: type: ALTER_PARTITION_MERGE
+STAGE DEPENDENCIES:
+ Stage-0 is a root stage
+ Stage-1 depends on stages: Stage-0
+ Stage-2 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-0
+
+ Stage: Stage-1
+ Move Operator
+ tables:
+ partition:
+ ds 1
+ part 0
+ replace: true
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1
+
+ Stage: Stage-2
+ Stats-Aggr Operator
+
+PREHOOK: query: ALTER TABLE orcfile_merge1 PARTITION (ds='1', part='0') CONCATENATE
+PREHOOK: type: ALTER_PARTITION_MERGE
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Output: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: query: ALTER TABLE orcfile_merge1 PARTITION (ds='1', part='0') CONCATENATE
+POSTHOOK: type: ALTER_PARTITION_MERGE
+POSTHOOK: Input: default@orcfile_merge1
+POSTHOOK: Output: default@orcfile_merge1@ds=1/part=0
+Found 1 items
+#### A masked pattern was here ####
+PREHOOK: query: -- Verify
+SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1c WHERE ds='1'
+) t
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1c
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: -- Verify
+SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1c WHERE ds='1'
+) t
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1c
+POSTHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1c@ds=1/part=1
+#### A masked pattern was here ####
+-21975308766
+PREHOOK: query: SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1 WHERE ds='1'
+) t
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1 WHERE ds='1'
+) t
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+-21975308766
+PREHOOK: query: select count(*) from orcfile_merge1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from orcfile_merge1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+500
+PREHOOK: query: select count(*) from orcfile_merge1c
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1c
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from orcfile_merge1c
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1c
+POSTHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1c@ds=1/part=1
+#### A masked pattern was here ####
+500
+PREHOOK: query: select * from orcfile_merge1 where ds='1' and part='0' limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=0
+#### A masked pattern was here ####
+-- BEGIN ORC FILE DUMP --
+#### A masked pattern was here ####
+File Version: 0.12 with HIVE_13083
+Rows: 242
+Compression: SNAPPY
+Compression size: 4096
+Type: struct<key:int,value:string>
+
+Stripe Statistics:
+ Stripe 1:
+ Column 0: count: 90 hasNull: false
+ Column 1: count: 90 hasNull: false min: 0 max: 495 sum: 22736
+ Column 2: count: 90 hasNull: false min: val_0 max: val_86 sum: 612
+ Stripe 2:
+ Column 0: count: 78 hasNull: false
+ Column 1: count: 78 hasNull: false min: 0 max: 497 sum: 18371
+ Column 2: count: 78 hasNull: false min: val_0 max: val_95 sum: 529
+ Stripe 3:
+ Column 0: count: 74 hasNull: false
+ Column 1: count: 74 hasNull: false min: 2 max: 493 sum: 19663
+ Column 2: count: 74 hasNull: false min: val_105 max: val_97 sum: 505
+
+File Statistics:
+ Column 0: count: 242 hasNull: false
+ Column 1: count: 242 hasNull: false min: 0 max: 497 sum: 60770
+ Column 2: count: 242 hasNull: false min: val_0 max: val_97 sum: 1646
+
+Stripes:
+ Stripe: offset: 3 data: 616 rows: 90 tail: 61 index: 76
+ Stream: column 0 section ROW_INDEX start: 3 length 11
+ Stream: column 1 section ROW_INDEX start: 14 length 27
+ Stream: column 2 section ROW_INDEX start: 41 length 38
+ Stream: column 1 section DATA start: 79 length 185
+ Stream: column 2 section DATA start: 264 length 377
+ Stream: column 2 section LENGTH start: 641 length 54
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 90 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 90 hasNull: false min: 0 max: 495 sum: 22736 positions: 0,0,0
+ Row group indices for column 2:
+ Entry 0: count: 90 hasNull: false min: val_0 max: val_86 sum: 612 positions: 0,0,0,0,0
+ Stripe: offset: 756 data: 544 rows: 78 tail: 61 index: 76
+ Stream: column 0 section ROW_INDEX start: 756 length 11
+ Stream: column 1 section ROW_INDEX start: 767 length 27
+ Stream: column 2 section ROW_INDEX start: 794 length 38
+ Stream: column 1 section DATA start: 832 length 161
+ Stream: column 2 section DATA start: 993 length 332
+ Stream: column 2 section LENGTH start: 1325 length 51
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 78 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 78 hasNull: false min: 0 max: 497 sum: 18371 positions: 0,0,0
+ Row group indices for column 2:
+ Entry 0: count: 78 hasNull: false min: val_0 max: val_95 sum: 529 positions: 0,0,0,0,0
+ Stripe: offset: 1437 data: 519 rows: 74 tail: 61 index: 78
+ Stream: column 0 section ROW_INDEX start: 1437 length 11
+ Stream: column 1 section ROW_INDEX start: 1448 length 27
+ Stream: column 2 section ROW_INDEX start: 1475 length 40
+ Stream: column 1 section DATA start: 1515 length 153
+ Stream: column 2 section DATA start: 1668 length 331
+ Stream: column 2 section LENGTH start: 1999 length 35
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 74 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 74 hasNull: false min: 2 max: 493 sum: 19663 positions: 0,0,0
+ Row group indices for column 2:
+ Entry 0: count: 74 hasNull: false min: val_105 max: val_97 sum: 505 positions: 0,0,0,0,0
+
+File length: 2393 bytes
+Padding length: 0 bytes
+Padding ratio: 0%
+________________________________________________________________________________________________________________________
+
+-- END ORC FILE DUMP --
+86 val_86 1 0
+PREHOOK: query: select * from orcfile_merge1c where ds='1' and part='0' limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1c
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+#### A masked pattern was here ####
+-- BEGIN ORC FILE DUMP --
+#### A masked pattern was here ####
+File Version: 0.12 with HIVE_13083
+Rows: 242
+Compression: SNAPPY
+Compression size: 4096
+Type: struct<key:int,value:string>
+
+Stripe Statistics:
+ Stripe 1:
+ Column 0: count: 90 hasNull: false
+ Column 1: count: 90 hasNull: false min: 0 max: 495 sum: 22736
+ Column 2: count: 90 hasNull: false min: val_0 max: val_86 sum: 612
+ Stripe 2:
+ Column 0: count: 78 hasNull: false
+ Column 1: count: 78 hasNull: false min: 0 max: 497 sum: 18371
+ Column 2: count: 78 hasNull: false min: val_0 max: val_95 sum: 529
+ Stripe 3:
+ Column 0: count: 74 hasNull: false
+ Column 1: count: 74 hasNull: false min: 2 max: 493 sum: 19663
+ Column 2: count: 74 hasNull: false min: val_105 max: val_97 sum: 505
+
+File Statistics:
+ Column 0: count: 242 hasNull: false
+ Column 1: count: 242 hasNull: false min: 0 max: 497 sum: 60770
+ Column 2: count: 242 hasNull: false min: val_0 max: val_97 sum: 1646
+
+Stripes:
+ Stripe: offset: 3 data: 616 rows: 90 tail: 61 index: 76
+ Stream: column 0 section ROW_INDEX start: 3 length 11
+ Stream: column 1 section ROW_INDEX start: 14 length 27
+ Stream: column 2 section ROW_INDEX start: 41 length 38
+ Stream: column 1 section DATA start: 79 length 185
+ Stream: column 2 section DATA start: 264 length 377
+ Stream: column 2 section LENGTH start: 641 length 54
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 90 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 90 hasNull: false min: 0 max: 495 sum: 22736 positions: 0,0,0
+ Row group indices for column 2:
+ Entry 0: count: 90 hasNull: false min: val_0 max: val_86 sum: 612 positions: 0,0,0,0,0
+ Stripe: offset: 756 data: 544 rows: 78 tail: 61 index: 76
+ Stream: column 0 section ROW_INDEX start: 756 length 11
+ Stream: column 1 section ROW_INDEX start: 767 length 27
+ Stream: column 2 section ROW_INDEX start: 794 length 38
+ Stream: column 1 section DATA start: 832 length 161
+ Stream: column 2 section DATA start: 993 length 332
+ Stream: column 2 section LENGTH start: 1325 length 51
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 78 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 78 hasNull: false min: 0 max: 497 sum: 18371 positions: 0,0,0
+ Row group indices for column 2:
+ Entry 0: count: 78 hasNull: false min: val_0 max: val_95 sum: 529 positions: 0,0,0,0,0
+ Stripe: offset: 1437 data: 519 rows: 74 tail: 61 index: 78
+ Stream: column 0 section ROW_INDEX start: 1437 length 11
+ Stream: column 1 section ROW_INDEX start: 1448 length 27
+ Stream: column 2 section ROW_INDEX start: 1475 length 40
+ Stream: column 1 section DATA start: 1515 length 153
+ Stream: column 2 section DATA start: 1668 length 331
+ Stream: column 2 section LENGTH start: 1999 length 35
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 74 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 74 hasNull: false min: 2 max: 493 sum: 19663 positions: 0,0,0
+ Row group indices for column 2:
+ Entry 0: count: 74 hasNull: false min: val_105 max: val_97 sum: 505 positions: 0,0,0,0,0
+
+File length: 2393 bytes
+Padding length: 0 bytes
+Padding ratio: 0%
+________________________________________________________________________________________________________________________
+
+-- END ORC FILE DUMP --
+86 val_86 1 0
+PREHOOK: query: DROP TABLE orcfile_merge1
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Output: default@orcfile_merge1
+PREHOOK: query: DROP TABLE orcfile_merge1b
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@orcfile_merge1b
+PREHOOK: Output: default@orcfile_merge1b
+PREHOOK: query: DROP TABLE orcfile_merge1c
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@orcfile_merge1c
+PREHOOK: Output: default@orcfile_merge1c
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge11.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge11.q.out b/ql/src/test/results/clientpositive/llap/orc_merge11.q.out
new file mode 100644
index 0000000..5143a6a
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/orc_merge11.q.out
@@ -0,0 +1,442 @@
+PREHOOK: query: DROP TABLE orcfile_merge1
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE orcfile_merge1
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: DROP TABLE orc_split_elim
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE orc_split_elim
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: create table orc_split_elim (userid bigint, string1 string, subtype double, decimal1 decimal, ts timestamp) stored as orc
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@orc_split_elim
+POSTHOOK: query: create table orc_split_elim (userid bigint, string1 string, subtype double, decimal1 decimal, ts timestamp) stored as orc
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@orc_split_elim
+PREHOOK: query: load data local inpath '../../data/files/orc_split_elim.orc' into table orc_split_elim
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@orc_split_elim
+POSTHOOK: query: load data local inpath '../../data/files/orc_split_elim.orc' into table orc_split_elim
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@orc_split_elim
+PREHOOK: query: load data local inpath '../../data/files/orc_split_elim.orc' into table orc_split_elim
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@orc_split_elim
+POSTHOOK: query: load data local inpath '../../data/files/orc_split_elim.orc' into table orc_split_elim
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@orc_split_elim
+PREHOOK: query: create table orcfile_merge1 (userid bigint, string1 string, subtype double, decimal1 decimal, ts timestamp) stored as orc tblproperties("orc.compress.size"="4096")
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@orcfile_merge1
+POSTHOOK: query: create table orcfile_merge1 (userid bigint, string1 string, subtype double, decimal1 decimal, ts timestamp) stored as orc tblproperties("orc.compress.size"="4096")
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@orcfile_merge1
+PREHOOK: query: insert overwrite table orcfile_merge1 select * from orc_split_elim
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orc_split_elim
+PREHOOK: Output: default@orcfile_merge1
+POSTHOOK: query: insert overwrite table orcfile_merge1 select * from orc_split_elim
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orc_split_elim
+POSTHOOK: Output: default@orcfile_merge1
+POSTHOOK: Lineage: orcfile_merge1.decimal1 SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:decimal1, type:decimal(10,0), comment:null), ]
+POSTHOOK: Lineage: orcfile_merge1.string1 SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:string1, type:string, comment:null), ]
+POSTHOOK: Lineage: orcfile_merge1.subtype SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:subtype, type:double, comment:null), ]
+POSTHOOK: Lineage: orcfile_merge1.ts SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:ts, type:timestamp, comment:null), ]
+POSTHOOK: Lineage: orcfile_merge1.userid SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:userid, type:bigint, comment:null), ]
+PREHOOK: query: insert into table orcfile_merge1 select * from orc_split_elim
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orc_split_elim
+PREHOOK: Output: default@orcfile_merge1
+POSTHOOK: query: insert into table orcfile_merge1 select * from orc_split_elim
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orc_split_elim
+POSTHOOK: Output: default@orcfile_merge1
+POSTHOOK: Lineage: orcfile_merge1.decimal1 SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:decimal1, type:decimal(10,0), comment:null), ]
+POSTHOOK: Lineage: orcfile_merge1.string1 SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:string1, type:string, comment:null), ]
+POSTHOOK: Lineage: orcfile_merge1.subtype SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:subtype, type:double, comment:null), ]
+POSTHOOK: Lineage: orcfile_merge1.ts SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:ts, type:timestamp, comment:null), ]
+POSTHOOK: Lineage: orcfile_merge1.userid SIMPLE [(orc_split_elim)orc_split_elim.FieldSchema(name:userid, type:bigint, comment:null), ]
+Found 2 items
+#### A masked pattern was here ####
+PREHOOK: query: select * from orcfile_merge1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+#### A masked pattern was here ####
+-- BEGIN ORC FILE DUMP --
+#### A masked pattern was here ####
+File Version: 0.12 with HIVE_13083
+Rows: 50000
+Compression: ZLIB
+Compression size: 4096
+Type: struct<userid:bigint,string1:string,subtype:double,decimal1:decimal(10,0),ts:timestamp>
+
+Stripe Statistics:
+ Stripe 1:
+ Column 0: count: 50000 hasNull: false
+ Column 1: count: 50000 hasNull: false min: 2 max: 100 sum: 4999238
+ Column 2: count: 50000 hasNull: false min: bar max: zebra sum: 249980
+ Column 3: count: 50000 hasNull: false min: 0.8 max: 80.0 sum: 400102.80000000005
+ Column 4: count: 50000 hasNull: false min: 0 max: 6 sum: 32
+ Column 5: count: 50000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0
+
+File Statistics:
+ Column 0: count: 50000 hasNull: false
+ Column 1: count: 50000 hasNull: false min: 2 max: 100 sum: 4999238
+ Column 2: count: 50000 hasNull: false min: bar max: zebra sum: 249980
+ Column 3: count: 50000 hasNull: false min: 0.8 max: 80.0 sum: 400102.80000000005
+ Column 4: count: 50000 hasNull: false min: 0 max: 6 sum: 32
+ Column 5: count: 50000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0
+
+Stripes:
+ Stripe: offset: 3 data: 5897 rows: 50000 tail: 113 index: 498
+ Stream: column 0 section ROW_INDEX start: 3 length 17
+ Stream: column 1 section ROW_INDEX start: 20 length 83
+ Stream: column 2 section ROW_INDEX start: 103 length 81
+ Stream: column 3 section ROW_INDEX start: 184 length 111
+ Stream: column 4 section ROW_INDEX start: 295 length 110
+ Stream: column 5 section ROW_INDEX start: 405 length 96
+ Stream: column 1 section DATA start: 501 length 45
+ Stream: column 2 section DATA start: 546 length 41
+ Stream: column 2 section LENGTH start: 587 length 8
+ Stream: column 2 section DICTIONARY_DATA start: 595 length 23
+ Stream: column 3 section DATA start: 618 length 5167
+ Stream: column 4 section DATA start: 5785 length 524
+ Stream: column 4 section SECONDARY start: 6309 length 18
+ Stream: column 5 section DATA start: 6327 length 53
+ Stream: column 5 section SECONDARY start: 6380 length 18
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DICTIONARY_V2[6]
+ Encoding column 3: DIRECT
+ Encoding column 4: DIRECT_V2
+ Encoding column 5: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 10000 hasNull: false positions:
+ Entry 1: count: 10000 hasNull: false positions:
+ Entry 2: count: 10000 hasNull: false positions:
+ Entry 3: count: 10000 hasNull: false positions:
+ Entry 4: count: 10000 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 10000 hasNull: false min: 2 max: 100 sum: 999815 positions: 0,0,0
+ Entry 1: count: 10000 hasNull: false min: 29 max: 100 sum: 999899 positions: 0,101,391
+ Entry 2: count: 10000 hasNull: false min: 2 max: 100 sum: 999807 positions: 0,207,391
+ Entry 3: count: 10000 hasNull: false min: 13 max: 100 sum: 999842 positions: 0,313,391
+ Entry 4: count: 10000 hasNull: false min: 5 max: 100 sum: 999875 positions: 0,419,391
+ Row group indices for column 2:
+ Entry 0: count: 10000 hasNull: false min: bar max: zebra sum: 49996 positions: 0,0,0
+ Entry 1: count: 10000 hasNull: false min: cat max: zebra sum: 49996 positions: 0,82,391
+ Entry 2: count: 10000 hasNull: false min: eat max: zebra sum: 49996 positions: 0,168,391
+ Entry 3: count: 10000 hasNull: false min: bar max: zebra sum: 49996 positions: 0,254,391
+ Entry 4: count: 10000 hasNull: false min: dog max: zebra sum: 49996 positions: 0,340,391
+ Row group indices for column 3:
+ Entry 0: count: 10000 hasNull: false min: 0.8 max: 80.0 sum: 80064.8 positions: 0,0
+ Entry 1: count: 10000 hasNull: false min: 1.8 max: 8.0 sum: 79993.8 positions: 1002,2176
+ Entry 2: count: 10000 hasNull: false min: 0.8 max: 8.0 sum: 79985.6 positions: 2053,256
+ Entry 3: count: 10000 hasNull: false min: 8.0 max: 80.0 sum: 80072.0 positions: 3067,2432
+ Entry 4: count: 10000 hasNull: false min: 0.8 max: 8.0 sum: 79986.6 positions: 4117,512
+ Row group indices for column 4:
+ Entry 0: count: 10000 hasNull: false min: 0 max: 2 sum: 3 positions: 0,0,0,0,0
+ Entry 1: count: 10000 hasNull: false min: 0 max: 4 sum: 7 positions: 83,1808,0,76,272
+ Entry 2: count: 10000 hasNull: false min: 0 max: 6 sum: 7 positions: 167,3616,0,156,32
+ Entry 3: count: 10000 hasNull: false min: 0 max: 3 sum: 5 positions: 290,1328,0,232,304
+ Entry 4: count: 10000 hasNull: false min: 0 max: 6 sum: 10 positions: 380,3136,0,312,64
+ Row group indices for column 5:
+ Entry 0: count: 10000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0 positions: 0,0,0,0,0,0
+ Entry 1: count: 10000 hasNull: false min: 1969-12-31 16:00:10.0 max: 1969-12-31 16:04:10.0 positions: 0,164,391,0,76,272
+ Entry 2: count: 10000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0 positions: 0,336,391,0,156,32
+ Entry 3: count: 10000 hasNull: false min: 1969-12-31 16:00:05.0 max: 1969-12-31 16:04:10.0 positions: 0,508,391,0,232,304
+ Entry 4: count: 10000 hasNull: false min: 1969-12-31 16:00:15.0 max: 1969-12-31 16:04:10.0 positions: 0,680,391,0,312,64
+
+File length: 6849 bytes
+Padding length: 0 bytes
+Padding ratio: 0%
+________________________________________________________________________________________________________________________
+
+-- END ORC FILE DUMP --
+-- BEGIN ORC FILE DUMP --
+#### A masked pattern was here ####
+File Version: 0.12 with HIVE_13083
+Rows: 50000
+Compression: ZLIB
+Compression size: 4096
+Type: struct<userid:bigint,string1:string,subtype:double,decimal1:decimal(10,0),ts:timestamp>
+
+Stripe Statistics:
+ Stripe 1:
+ Column 0: count: 50000 hasNull: false
+ Column 1: count: 50000 hasNull: false min: 2 max: 100 sum: 4999238
+ Column 2: count: 50000 hasNull: false min: bar max: zebra sum: 249980
+ Column 3: count: 50000 hasNull: false min: 0.8 max: 80.0 sum: 400102.80000000005
+ Column 4: count: 50000 hasNull: false min: 0 max: 6 sum: 32
+ Column 5: count: 50000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0
+
+File Statistics:
+ Column 0: count: 50000 hasNull: false
+ Column 1: count: 50000 hasNull: false min: 2 max: 100 sum: 4999238
+ Column 2: count: 50000 hasNull: false min: bar max: zebra sum: 249980
+ Column 3: count: 50000 hasNull: false min: 0.8 max: 80.0 sum: 400102.80000000005
+ Column 4: count: 50000 hasNull: false min: 0 max: 6 sum: 32
+ Column 5: count: 50000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0
+
+Stripes:
+ Stripe: offset: 3 data: 5897 rows: 50000 tail: 113 index: 498
+ Stream: column 0 section ROW_INDEX start: 3 length 17
+ Stream: column 1 section ROW_INDEX start: 20 length 83
+ Stream: column 2 section ROW_INDEX start: 103 length 81
+ Stream: column 3 section ROW_INDEX start: 184 length 111
+ Stream: column 4 section ROW_INDEX start: 295 length 110
+ Stream: column 5 section ROW_INDEX start: 405 length 96
+ Stream: column 1 section DATA start: 501 length 45
+ Stream: column 2 section DATA start: 546 length 41
+ Stream: column 2 section LENGTH start: 587 length 8
+ Stream: column 2 section DICTIONARY_DATA start: 595 length 23
+ Stream: column 3 section DATA start: 618 length 5167
+ Stream: column 4 section DATA start: 5785 length 524
+ Stream: column 4 section SECONDARY start: 6309 length 18
+ Stream: column 5 section DATA start: 6327 length 53
+ Stream: column 5 section SECONDARY start: 6380 length 18
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DICTIONARY_V2[6]
+ Encoding column 3: DIRECT
+ Encoding column 4: DIRECT_V2
+ Encoding column 5: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 10000 hasNull: false positions:
+ Entry 1: count: 10000 hasNull: false positions:
+ Entry 2: count: 10000 hasNull: false positions:
+ Entry 3: count: 10000 hasNull: false positions:
+ Entry 4: count: 10000 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 10000 hasNull: false min: 2 max: 100 sum: 999815 positions: 0,0,0
+ Entry 1: count: 10000 hasNull: false min: 29 max: 100 sum: 999899 positions: 0,101,391
+ Entry 2: count: 10000 hasNull: false min: 2 max: 100 sum: 999807 positions: 0,207,391
+ Entry 3: count: 10000 hasNull: false min: 13 max: 100 sum: 999842 positions: 0,313,391
+ Entry 4: count: 10000 hasNull: false min: 5 max: 100 sum: 999875 positions: 0,419,391
+ Row group indices for column 2:
+ Entry 0: count: 10000 hasNull: false min: bar max: zebra sum: 49996 positions: 0,0,0
+ Entry 1: count: 10000 hasNull: false min: cat max: zebra sum: 49996 positions: 0,82,391
+ Entry 2: count: 10000 hasNull: false min: eat max: zebra sum: 49996 positions: 0,168,391
+ Entry 3: count: 10000 hasNull: false min: bar max: zebra sum: 49996 positions: 0,254,391
+ Entry 4: count: 10000 hasNull: false min: dog max: zebra sum: 49996 positions: 0,340,391
+ Row group indices for column 3:
+ Entry 0: count: 10000 hasNull: false min: 0.8 max: 80.0 sum: 80064.8 positions: 0,0
+ Entry 1: count: 10000 hasNull: false min: 1.8 max: 8.0 sum: 79993.8 positions: 1002,2176
+ Entry 2: count: 10000 hasNull: false min: 0.8 max: 8.0 sum: 79985.6 positions: 2053,256
+ Entry 3: count: 10000 hasNull: false min: 8.0 max: 80.0 sum: 80072.0 positions: 3067,2432
+ Entry 4: count: 10000 hasNull: false min: 0.8 max: 8.0 sum: 79986.6 positions: 4117,512
+ Row group indices for column 4:
+ Entry 0: count: 10000 hasNull: false min: 0 max: 2 sum: 3 positions: 0,0,0,0,0
+ Entry 1: count: 10000 hasNull: false min: 0 max: 4 sum: 7 positions: 83,1808,0,76,272
+ Entry 2: count: 10000 hasNull: false min: 0 max: 6 sum: 7 positions: 167,3616,0,156,32
+ Entry 3: count: 10000 hasNull: false min: 0 max: 3 sum: 5 positions: 290,1328,0,232,304
+ Entry 4: count: 10000 hasNull: false min: 0 max: 6 sum: 10 positions: 380,3136,0,312,64
+ Row group indices for column 5:
+ Entry 0: count: 10000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0 positions: 0,0,0,0,0,0
+ Entry 1: count: 10000 hasNull: false min: 1969-12-31 16:00:10.0 max: 1969-12-31 16:04:10.0 positions: 0,164,391,0,76,272
+ Entry 2: count: 10000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0 positions: 0,336,391,0,156,32
+ Entry 3: count: 10000 hasNull: false min: 1969-12-31 16:00:05.0 max: 1969-12-31 16:04:10.0 positions: 0,508,391,0,232,304
+ Entry 4: count: 10000 hasNull: false min: 1969-12-31 16:00:15.0 max: 1969-12-31 16:04:10.0 positions: 0,680,391,0,312,64
+
+File length: 6849 bytes
+Padding length: 0 bytes
+Padding ratio: 0%
+________________________________________________________________________________________________________________________
+
+-- END ORC FILE DUMP --
+2 foo 0.8 1 1969-12-31 16:00:00
+PREHOOK: query: -- concatenate
+ALTER TABLE orcfile_merge1 CONCATENATE
+PREHOOK: type: ALTER_TABLE_MERGE
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Output: default@orcfile_merge1
+Found 1 items
+#### A masked pattern was here ####
+PREHOOK: query: select count(*) from orc_split_elim
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orc_split_elim
+#### A masked pattern was here ####
+50000
+PREHOOK: query: -- will have double the number of rows
+select count(*) from orcfile_merge1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+#### A masked pattern was here ####
+100000
+PREHOOK: query: select * from orcfile_merge1 limit 1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+#### A masked pattern was here ####
+-- BEGIN ORC FILE DUMP --
+#### A masked pattern was here ####
+File Version: 0.12 with HIVE_13083
+Rows: 100000
+Compression: ZLIB
+Compression size: 4096
+Type: struct<userid:bigint,string1:string,subtype:double,decimal1:decimal(10,0),ts:timestamp>
+
+Stripe Statistics:
+ Stripe 1:
+ Column 0: count: 50000 hasNull: false
+ Column 1: count: 50000 hasNull: false min: 2 max: 100 sum: 4999238
+ Column 2: count: 50000 hasNull: false min: bar max: zebra sum: 249980
+ Column 3: count: 50000 hasNull: false min: 0.8 max: 80.0 sum: 400102.80000000005
+ Column 4: count: 50000 hasNull: false min: 0 max: 6 sum: 32
+ Column 5: count: 50000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0
+ Stripe 2:
+ Column 0: count: 50000 hasNull: false
+ Column 1: count: 50000 hasNull: false min: 2 max: 100 sum: 4999238
+ Column 2: count: 50000 hasNull: false min: bar max: zebra sum: 249980
+ Column 3: count: 50000 hasNull: false min: 0.8 max: 80.0 sum: 400102.80000000005
+ Column 4: count: 50000 hasNull: false min: 0 max: 6 sum: 32
+ Column 5: count: 50000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0
+
+File Statistics:
+ Column 0: count: 100000 hasNull: false
+ Column 1: count: 100000 hasNull: false min: 2 max: 100 sum: 9998476
+ Column 2: count: 100000 hasNull: false min: bar max: zebra sum: 499960
+ Column 3: count: 100000 hasNull: false min: 0.8 max: 80.0 sum: 800205.6000000001
+ Column 4: count: 100000 hasNull: false min: 0 max: 6 sum: 64
+ Column 5: count: 100000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0
+
+Stripes:
+ Stripe: offset: 3 data: 5897 rows: 50000 tail: 113 index: 498
+ Stream: column 0 section ROW_INDEX start: 3 length 17
+ Stream: column 1 section ROW_INDEX start: 20 length 83
+ Stream: column 2 section ROW_INDEX start: 103 length 81
+ Stream: column 3 section ROW_INDEX start: 184 length 111
+ Stream: column 4 section ROW_INDEX start: 295 length 110
+ Stream: column 5 section ROW_INDEX start: 405 length 96
+ Stream: column 1 section DATA start: 501 length 45
+ Stream: column 2 section DATA start: 546 length 41
+ Stream: column 2 section LENGTH start: 587 length 8
+ Stream: column 2 section DICTIONARY_DATA start: 595 length 23
+ Stream: column 3 section DATA start: 618 length 5167
+ Stream: column 4 section DATA start: 5785 length 524
+ Stream: column 4 section SECONDARY start: 6309 length 18
+ Stream: column 5 section DATA start: 6327 length 53
+ Stream: column 5 section SECONDARY start: 6380 length 18
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DICTIONARY_V2[6]
+ Encoding column 3: DIRECT
+ Encoding column 4: DIRECT_V2
+ Encoding column 5: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 10000 hasNull: false positions:
+ Entry 1: count: 10000 hasNull: false positions:
+ Entry 2: count: 10000 hasNull: false positions:
+ Entry 3: count: 10000 hasNull: false positions:
+ Entry 4: count: 10000 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 10000 hasNull: false min: 2 max: 100 sum: 999815 positions: 0,0,0
+ Entry 1: count: 10000 hasNull: false min: 29 max: 100 sum: 999899 positions: 0,101,391
+ Entry 2: count: 10000 hasNull: false min: 2 max: 100 sum: 999807 positions: 0,207,391
+ Entry 3: count: 10000 hasNull: false min: 13 max: 100 sum: 999842 positions: 0,313,391
+ Entry 4: count: 10000 hasNull: false min: 5 max: 100 sum: 999875 positions: 0,419,391
+ Row group indices for column 2:
+ Entry 0: count: 10000 hasNull: false min: bar max: zebra sum: 49996 positions: 0,0,0
+ Entry 1: count: 10000 hasNull: false min: cat max: zebra sum: 49996 positions: 0,82,391
+ Entry 2: count: 10000 hasNull: false min: eat max: zebra sum: 49996 positions: 0,168,391
+ Entry 3: count: 10000 hasNull: false min: bar max: zebra sum: 49996 positions: 0,254,391
+ Entry 4: count: 10000 hasNull: false min: dog max: zebra sum: 49996 positions: 0,340,391
+ Row group indices for column 3:
+ Entry 0: count: 10000 hasNull: false min: 0.8 max: 80.0 sum: 80064.8 positions: 0,0
+ Entry 1: count: 10000 hasNull: false min: 1.8 max: 8.0 sum: 79993.8 positions: 1002,2176
+ Entry 2: count: 10000 hasNull: false min: 0.8 max: 8.0 sum: 79985.6 positions: 2053,256
+ Entry 3: count: 10000 hasNull: false min: 8.0 max: 80.0 sum: 80072.0 positions: 3067,2432
+ Entry 4: count: 10000 hasNull: false min: 0.8 max: 8.0 sum: 79986.6 positions: 4117,512
+ Row group indices for column 4:
+ Entry 0: count: 10000 hasNull: false min: 0 max: 2 sum: 3 positions: 0,0,0,0,0
+ Entry 1: count: 10000 hasNull: false min: 0 max: 4 sum: 7 positions: 83,1808,0,76,272
+ Entry 2: count: 10000 hasNull: false min: 0 max: 6 sum: 7 positions: 167,3616,0,156,32
+ Entry 3: count: 10000 hasNull: false min: 0 max: 3 sum: 5 positions: 290,1328,0,232,304
+ Entry 4: count: 10000 hasNull: false min: 0 max: 6 sum: 10 positions: 380,3136,0,312,64
+ Row group indices for column 5:
+ Entry 0: count: 10000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0 positions: 0,0,0,0,0,0
+ Entry 1: count: 10000 hasNull: false min: 1969-12-31 16:00:10.0 max: 1969-12-31 16:04:10.0 positions: 0,164,391,0,76,272
+ Entry 2: count: 10000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0 positions: 0,336,391,0,156,32
+ Entry 3: count: 10000 hasNull: false min: 1969-12-31 16:00:05.0 max: 1969-12-31 16:04:10.0 positions: 0,508,391,0,232,304
+ Entry 4: count: 10000 hasNull: false min: 1969-12-31 16:00:15.0 max: 1969-12-31 16:04:10.0 positions: 0,680,391,0,312,64
+ Stripe: offset: 6511 data: 5897 rows: 50000 tail: 113 index: 498
+ Stream: column 0 section ROW_INDEX start: 6511 length 17
+ Stream: column 1 section ROW_INDEX start: 6528 length 83
+ Stream: column 2 section ROW_INDEX start: 6611 length 81
+ Stream: column 3 section ROW_INDEX start: 6692 length 111
+ Stream: column 4 section ROW_INDEX start: 6803 length 110
+ Stream: column 5 section ROW_INDEX start: 6913 length 96
+ Stream: column 1 section DATA start: 7009 length 45
+ Stream: column 2 section DATA start: 7054 length 41
+ Stream: column 2 section LENGTH start: 7095 length 8
+ Stream: column 2 section DICTIONARY_DATA start: 7103 length 23
+ Stream: column 3 section DATA start: 7126 length 5167
+ Stream: column 4 section DATA start: 12293 length 524
+ Stream: column 4 section SECONDARY start: 12817 length 18
+ Stream: column 5 section DATA start: 12835 length 53
+ Stream: column 5 section SECONDARY start: 12888 length 18
+ Encoding column 0: DIRECT
+ Encoding column 1: DIRECT_V2
+ Encoding column 2: DICTIONARY_V2[6]
+ Encoding column 3: DIRECT
+ Encoding column 4: DIRECT_V2
+ Encoding column 5: DIRECT_V2
+ Row group indices for column 0:
+ Entry 0: count: 10000 hasNull: false positions:
+ Entry 1: count: 10000 hasNull: false positions:
+ Entry 2: count: 10000 hasNull: false positions:
+ Entry 3: count: 10000 hasNull: false positions:
+ Entry 4: count: 10000 hasNull: false positions:
+ Row group indices for column 1:
+ Entry 0: count: 10000 hasNull: false min: 2 max: 100 sum: 999815 positions: 0,0,0
+ Entry 1: count: 10000 hasNull: false min: 29 max: 100 sum: 999899 positions: 0,101,391
+ Entry 2: count: 10000 hasNull: false min: 2 max: 100 sum: 999807 positions: 0,207,391
+ Entry 3: count: 10000 hasNull: false min: 13 max: 100 sum: 999842 positions: 0,313,391
+ Entry 4: count: 10000 hasNull: false min: 5 max: 100 sum: 999875 positions: 0,419,391
+ Row group indices for column 2:
+ Entry 0: count: 10000 hasNull: false min: bar max: zebra sum: 49996 positions: 0,0,0
+ Entry 1: count: 10000 hasNull: false min: cat max: zebra sum: 49996 positions: 0,82,391
+ Entry 2: count: 10000 hasNull: false min: eat max: zebra sum: 49996 positions: 0,168,391
+ Entry 3: count: 10000 hasNull: false min: bar max: zebra sum: 49996 positions: 0,254,391
+ Entry 4: count: 10000 hasNull: false min: dog max: zebra sum: 49996 positions: 0,340,391
+ Row group indices for column 3:
+ Entry 0: count: 10000 hasNull: false min: 0.8 max: 80.0 sum: 80064.8 positions: 0,0
+ Entry 1: count: 10000 hasNull: false min: 1.8 max: 8.0 sum: 79993.8 positions: 1002,2176
+ Entry 2: count: 10000 hasNull: false min: 0.8 max: 8.0 sum: 79985.6 positions: 2053,256
+ Entry 3: count: 10000 hasNull: false min: 8.0 max: 80.0 sum: 80072.0 positions: 3067,2432
+ Entry 4: count: 10000 hasNull: false min: 0.8 max: 8.0 sum: 79986.6 positions: 4117,512
+ Row group indices for column 4:
+ Entry 0: count: 10000 hasNull: false min: 0 max: 2 sum: 3 positions: 0,0,0,0,0
+ Entry 1: count: 10000 hasNull: false min: 0 max: 4 sum: 7 positions: 83,1808,0,76,272
+ Entry 2: count: 10000 hasNull: false min: 0 max: 6 sum: 7 positions: 167,3616,0,156,32
+ Entry 3: count: 10000 hasNull: false min: 0 max: 3 sum: 5 positions: 290,1328,0,232,304
+ Entry 4: count: 10000 hasNull: false min: 0 max: 6 sum: 10 positions: 380,3136,0,312,64
+ Row group indices for column 5:
+ Entry 0: count: 10000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0 positions: 0,0,0,0,0,0
+ Entry 1: count: 10000 hasNull: false min: 1969-12-31 16:00:10.0 max: 1969-12-31 16:04:10.0 positions: 0,164,391,0,76,272
+ Entry 2: count: 10000 hasNull: false min: 1969-12-31 16:00:00.0 max: 1969-12-31 16:04:10.0 positions: 0,336,391,0,156,32
+ Entry 3: count: 10000 hasNull: false min: 1969-12-31 16:00:05.0 max: 1969-12-31 16:04:10.0 positions: 0,508,391,0,232,304
+ Entry 4: count: 10000 hasNull: false min: 1969-12-31 16:00:15.0 max: 1969-12-31 16:04:10.0 positions: 0,680,391,0,312,64
+
+File length: 13369 bytes
+Padding length: 0 bytes
+Padding ratio: 0%
+________________________________________________________________________________________________________________________
+
+-- END ORC FILE DUMP --
+2 foo 0.8 1 1969-12-31 16:00:00
+PREHOOK: query: DROP TABLE orc_split_elim
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@orc_split_elim
+PREHOOK: Output: default@orc_split_elim
+PREHOOK: query: DROP TABLE orcfile_merge1
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Output: default@orcfile_merge1
[42/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/bucket2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/bucket2.q.out b/ql/src/test/results/clientpositive/llap/bucket2.q.out
index a2cfc0a..244f247 100644
--- a/ql/src/test/results/clientpositive/llap/bucket2.q.out
+++ b/ql/src/test/results/clientpositive/llap/bucket2.q.out
@@ -18,23 +18,6 @@ POSTHOOK: query: explain extended
insert overwrite table bucket2_1
select * from src
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_TAB
- TOK_TABNAME
- bucket2_1
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-2 depends on stages: Stage-1
@@ -44,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -52,20 +36,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string), _col1 (type: string)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -75,7 +61,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -95,7 +81,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -116,24 +102,25 @@ STAGE PLANS:
Truncated Path -> Alias:
/src [src]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Select Operator
expressions: UDFToInteger(VALUE._col0) (type: int), VALUE._col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count 2
bucket_field_name key
columns key,value
@@ -141,9 +128,13 @@ STAGE PLANS:
columns.types int:string
#### A masked pattern was here ####
name default.bucket2_1
+ numFiles 0
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket2_1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.bucket2_1
@@ -163,6 +154,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count 2
bucket_field_name key
columns key,value
@@ -170,9 +162,13 @@ STAGE PLANS:
columns.types int:string
#### A masked pattern was here ####
name default.bucket2_1
+ numFiles 0
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket2_1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.bucket2_1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/bucket3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/bucket3.q.out b/ql/src/test/results/clientpositive/llap/bucket3.q.out
index 8c25962..dcf731d 100644
--- a/ql/src/test/results/clientpositive/llap/bucket3.q.out
+++ b/ql/src/test/results/clientpositive/llap/bucket3.q.out
@@ -18,27 +18,6 @@ POSTHOOK: query: explain extended
insert overwrite table bucket3_1 partition (ds='1')
select * from src
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_TAB
- TOK_TABNAME
- bucket3_1
- TOK_PARTSPEC
- TOK_PARTVAL
- ds
- '1'
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-2 depends on stages: Stage-1
@@ -48,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -56,20 +36,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string), _col1 (type: string)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -79,7 +61,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -99,7 +81,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -120,20 +102,20 @@ STAGE PLANS:
Truncated Path -> Alias:
/src [src]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Select Operator
expressions: UDFToInteger(VALUE._col0) (type: int), VALUE._col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 2
Static Partition Specification: ds=1/
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/bucket4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/bucket4.q.out b/ql/src/test/results/clientpositive/llap/bucket4.q.out
index c681933..4504177 100644
--- a/ql/src/test/results/clientpositive/llap/bucket4.q.out
+++ b/ql/src/test/results/clientpositive/llap/bucket4.q.out
@@ -14,23 +14,6 @@ POSTHOOK: query: explain extended
insert overwrite table bucket4_1
select * from src
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_TAB
- TOK_TABNAME
- bucket4_1
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-2 depends on stages: Stage-1
@@ -40,6 +23,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -48,21 +32,23 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col0) (type: int)
+ null sort order: a
sort order: +
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string), _col1 (type: string)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -72,7 +58,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -92,7 +78,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -113,24 +99,25 @@ STAGE PLANS:
Truncated Path -> Alias:
/src [src]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Select Operator
expressions: UDFToInteger(VALUE._col0) (type: int), VALUE._col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
SORTBUCKETCOLSPREFIX TRUE
bucket_count 2
bucket_field_name key
@@ -139,9 +126,13 @@ STAGE PLANS:
columns.types int:string
#### A masked pattern was here ####
name default.bucket4_1
+ numFiles 0
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket4_1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.bucket4_1
@@ -161,6 +152,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
SORTBUCKETCOLSPREFIX TRUE
bucket_count 2
bucket_field_name key
@@ -169,9 +161,13 @@ STAGE PLANS:
columns.types int:string
#### A masked pattern was here ####
name default.bucket4_1
+ numFiles 0
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket4_1 { i32 key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.bucket4_1
[15/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge2.q.out b/ql/src/test/results/clientpositive/llap/orc_merge2.q.out
index a399be9..c38852a 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge2.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge2.q.out
@@ -39,20 +39,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 10) (type: int), (hash(value) pmod 10) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orcfile_merge2a
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge3.q.out b/ql/src/test/results/clientpositive/llap/orc_merge3.q.out
index 893b6d2..fd79d9d 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge3.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge3.q.out
@@ -69,20 +69,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: orcfile_merge3a
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orcfile_merge3b
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge4.q.out b/ql/src/test/results/clientpositive/llap/orc_merge4.q.out
index 1018bcf..6e2617b 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge4.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge4.q.out
@@ -87,20 +87,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: orcfile_merge3a
- Statistics: Num rows: 1000 Data size: 94000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 102000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 94000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 102000 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1000 Data size: 94000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 102000 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.orcfile_merge3b
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-2
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge5.q.out b/ql/src/test/results/clientpositive/llap/orc_merge5.q.out
index 0837007..18ccba0 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge5.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge5.q.out
@@ -65,6 +65,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_merge5b
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-2
Dependency Collection
@@ -105,7 +106,7 @@ analyze table orc_merge5b compute statistics noscan
POSTHOOK: type: QUERY
POSTHOOK: Input: default@orc_merge5b
POSTHOOK: Output: default@orc_merge5b
-Found 3 items
+Found 2 items
#### A masked pattern was here ####
PREHOOK: query: select * from orc_merge5b
PREHOOK: type: QUERY
@@ -162,6 +163,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_merge5b
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-8
Conditional Operator
@@ -273,7 +275,7 @@ POSTHOOK: query: analyze table orc_merge5b compute statistics noscan
POSTHOOK: type: QUERY
POSTHOOK: Input: default@orc_merge5b
POSTHOOK: Output: default@orc_merge5b
-Found 3 items
+Found 2 items
#### A masked pattern was here ####
PREHOOK: query: select * from orc_merge5b
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge6.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge6.q.out b/ql/src/test/results/clientpositive/llap/orc_merge6.q.out
index 847b22d..b38bffc 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge6.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge6.q.out
@@ -67,6 +67,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_merge5a
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-2
Dependency Collection
@@ -135,9 +136,9 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@orc_merge5a
POSTHOOK: Output: default@orc_merge5a
POSTHOOK: Output: default@orc_merge5a@year=2001/hour=24
-Found 3 items
+Found 2 items
#### A masked pattern was here ####
-Found 3 items
+Found 2 items
#### A masked pattern was here ####
PREHOOK: query: show partitions orc_merge5a
PREHOOK: type: SHOWPARTITIONS
@@ -209,6 +210,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_merge5a
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-8
Conditional Operator
@@ -390,9 +392,9 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@orc_merge5a
POSTHOOK: Output: default@orc_merge5a
POSTHOOK: Output: default@orc_merge5a@year=2001/hour=24
-Found 3 items
+Found 2 items
#### A masked pattern was here ####
-Found 3 items
+Found 2 items
#### A masked pattern was here ####
PREHOOK: query: show partitions orc_merge5a
PREHOOK: type: SHOWPARTITIONS
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge7.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge7.q.out b/ql/src/test/results/clientpositive/llap/orc_merge7.q.out
index eee65c0..595061e 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge7.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge7.q.out
@@ -65,6 +65,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_merge5a
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-2
Dependency Collection
@@ -241,6 +242,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_merge5a
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-8
Conditional Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge_diff_fs.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge_diff_fs.q.out b/ql/src/test/results/clientpositive/llap/orc_merge_diff_fs.q.out
new file mode 100644
index 0000000..973d5a1
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/orc_merge_diff_fs.q.out
@@ -0,0 +1,503 @@
+PREHOOK: query: -- SORT_QUERY_RESULTS
+
+DROP TABLE orcfile_merge1
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: -- SORT_QUERY_RESULTS
+
+DROP TABLE orcfile_merge1
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: DROP TABLE orcfile_merge1b
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE orcfile_merge1b
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: DROP TABLE orcfile_merge1c
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE orcfile_merge1c
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: CREATE TABLE orcfile_merge1 (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@orcfile_merge1
+POSTHOOK: query: CREATE TABLE orcfile_merge1 (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@orcfile_merge1
+PREHOOK: query: CREATE TABLE orcfile_merge1b (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@orcfile_merge1b
+POSTHOOK: query: CREATE TABLE orcfile_merge1b (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@orcfile_merge1b
+PREHOOK: query: CREATE TABLE orcfile_merge1c (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@orcfile_merge1c
+POSTHOOK: query: CREATE TABLE orcfile_merge1c (key INT, value STRING)
+ PARTITIONED BY (ds STRING, part STRING) STORED AS ORC
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@orcfile_merge1c
+PREHOOK: query: -- merge disabled
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1 PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- merge disabled
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1 PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ ds 1
+ part
+ replace: true
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1 PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: default@orcfile_merge1@ds=1
+POSTHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1 PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: Output: default@orcfile_merge1@ds=1/part=1
+POSTHOOK: Lineage: orcfile_merge1 PARTITION(ds=1,part=0).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1 PARTITION(ds=1,part=0).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1 PARTITION(ds=1,part=1).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1 PARTITION(ds=1,part=1).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+Found 6 items
+#### A masked pattern was here ####
+PREHOOK: query: -- auto-merge slow way
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1b PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- auto-merge slow way
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1b PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-8 depends on stages: Stage-1 , consists of Stage-5, Stage-4, Stage-6
+ Stage-5
+ Stage-2 depends on stages: Stage-5, Stage-4, Stage-7
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+ Stage-4
+ Stage-6
+ Stage-7 depends on stages: Stage-6
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1b
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-8
+ Conditional Operator
+
+ Stage: Stage-5
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ ds 1
+ part
+ replace: true
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1b
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+ Stage: Stage-4
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ File Merge
+ Map Operator Tree:
+ TableScan
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1b
+
+ Stage: Stage-6
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ File Merge
+ Map Operator Tree:
+ TableScan
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1b
+
+ Stage: Stage-7
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+PREHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1b PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: default@orcfile_merge1b@ds=1
+POSTHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1b PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: default@orcfile_merge1b@ds=1/part=0
+POSTHOOK: Output: default@orcfile_merge1b@ds=1/part=1
+POSTHOOK: Lineage: orcfile_merge1b PARTITION(ds=1,part=0).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1b PARTITION(ds=1,part=0).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1b PARTITION(ds=1,part=1).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1b PARTITION(ds=1,part=1).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+Found 1 items
+#### A masked pattern was here ####
+PREHOOK: query: -- auto-merge fast way
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1c PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- auto-merge fast way
+EXPLAIN
+ INSERT OVERWRITE TABLE orcfile_merge1c PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-8 depends on stages: Stage-1 , consists of Stage-5, Stage-4, Stage-6
+ Stage-5
+ Stage-2 depends on stages: Stage-5, Stage-4, Stage-7
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+ Stage-4
+ Stage-6
+ Stage-7 depends on stages: Stage-6
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: UDFToInteger(key) (type: int), value (type: string), (hash(key) pmod 2) (type: int)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 500 Data size: 49500 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1c
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-8
+ Conditional Operator
+
+ Stage: Stage-5
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ ds 1
+ part
+ replace: true
+ table:
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.orcfile_merge1c
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+ Stage: Stage-4
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ File Merge
+ Merge File Operator
+ Map Operator Tree:
+ ORC File Merge Operator
+ merge level: stripe
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+
+ Stage: Stage-6
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ File Merge
+ Merge File Operator
+ Map Operator Tree:
+ ORC File Merge Operator
+ merge level: stripe
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+
+ Stage: Stage-7
+ Move Operator
+ files:
+ hdfs directory: true
+#### A masked pattern was here ####
+
+PREHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1c PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Output: default@orcfile_merge1c@ds=1
+POSTHOOK: query: INSERT OVERWRITE TABLE orcfile_merge1c PARTITION (ds='1', part)
+ SELECT key, value, PMOD(HASH(key), 2) as part
+ FROM src
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Output: default@orcfile_merge1c@ds=1/part=0
+POSTHOOK: Output: default@orcfile_merge1c@ds=1/part=1
+POSTHOOK: Lineage: orcfile_merge1c PARTITION(ds=1,part=0).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1c PARTITION(ds=1,part=0).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1c PARTITION(ds=1,part=1).key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: orcfile_merge1c PARTITION(ds=1,part=1).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+Found 1 items
+#### A masked pattern was here ####
+PREHOOK: query: -- Verify
+SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1 WHERE ds='1'
+) t
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: -- Verify
+SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1 WHERE ds='1'
+) t
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+-21975308766
+PREHOOK: query: SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1b WHERE ds='1'
+) t
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1b
+PREHOOK: Input: default@orcfile_merge1b@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1b@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1b WHERE ds='1'
+) t
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1b
+POSTHOOK: Input: default@orcfile_merge1b@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1b@ds=1/part=1
+#### A masked pattern was here ####
+-21975308766
+PREHOOK: query: SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1c WHERE ds='1'
+) t
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1c
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT SUM(HASH(c)) FROM (
+ SELECT TRANSFORM(*) USING 'tr \t _' AS (c)
+ FROM orcfile_merge1c WHERE ds='1'
+) t
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1c
+POSTHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1c@ds=1/part=1
+#### A masked pattern was here ####
+-21975308766
+PREHOOK: query: select count(*) from orcfile_merge1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from orcfile_merge1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1@ds=1/part=1
+#### A masked pattern was here ####
+500
+PREHOOK: query: select count(*) from orcfile_merge1b
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1b
+PREHOOK: Input: default@orcfile_merge1b@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1b@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from orcfile_merge1b
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1b
+POSTHOOK: Input: default@orcfile_merge1b@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1b@ds=1/part=1
+#### A masked pattern was here ####
+500
+PREHOOK: query: select count(*) from orcfile_merge1c
+PREHOOK: type: QUERY
+PREHOOK: Input: default@orcfile_merge1c
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+PREHOOK: Input: default@orcfile_merge1c@ds=1/part=1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from orcfile_merge1c
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@orcfile_merge1c
+POSTHOOK: Input: default@orcfile_merge1c@ds=1/part=0
+POSTHOOK: Input: default@orcfile_merge1c@ds=1/part=1
+#### A masked pattern was here ####
+500
+PREHOOK: query: DROP TABLE orcfile_merge1
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@orcfile_merge1
+PREHOOK: Output: default@orcfile_merge1
+POSTHOOK: query: DROP TABLE orcfile_merge1
+POSTHOOK: type: DROPTABLE
+POSTHOOK: Input: default@orcfile_merge1
+POSTHOOK: Output: default@orcfile_merge1
+PREHOOK: query: DROP TABLE orcfile_merge1b
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@orcfile_merge1b
+PREHOOK: Output: default@orcfile_merge1b
+POSTHOOK: query: DROP TABLE orcfile_merge1b
+POSTHOOK: type: DROPTABLE
+POSTHOOK: Input: default@orcfile_merge1b
+POSTHOOK: Output: default@orcfile_merge1b
+PREHOOK: query: DROP TABLE orcfile_merge1c
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@orcfile_merge1c
+PREHOOK: Output: default@orcfile_merge1c
+POSTHOOK: query: DROP TABLE orcfile_merge1c
+POSTHOOK: type: DROPTABLE
+POSTHOOK: Input: default@orcfile_merge1c
+POSTHOOK: Output: default@orcfile_merge1c
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge_incompat1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge_incompat1.q.out b/ql/src/test/results/clientpositive/llap/orc_merge_incompat1.q.out
index 5d7a985..3f8d559 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge_incompat1.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge_incompat1.q.out
@@ -64,6 +64,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_merge5b
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-2
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge_incompat2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge_incompat2.q.out b/ql/src/test/results/clientpositive/llap/orc_merge_incompat2.q.out
index 83543d5..e6bfd33 100644
--- a/ql/src/test/results/clientpositive/llap/orc_merge_incompat2.q.out
+++ b/ql/src/test/results/clientpositive/llap/orc_merge_incompat2.q.out
@@ -63,6 +63,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_merge5a
Execution mode: llap
+ LLAP IO: all inputs
Stage: Stage-2
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/orc_merge_incompat3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/orc_merge_incompat3.q.out b/ql/src/test/results/clientpositive/llap/orc_merge_incompat3.q.out
new file mode 100644
index 0000000..e34492b
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/orc_merge_incompat3.q.out
@@ -0,0 +1,70 @@
+PREHOOK: query: create table concat_incompat like alltypesorc
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@concat_incompat
+POSTHOOK: query: create table concat_incompat like alltypesorc
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@concat_incompat
+PREHOOK: query: load data local inpath '../../data/files/alltypesorc' into table concat_incompat
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@concat_incompat
+POSTHOOK: query: load data local inpath '../../data/files/alltypesorc' into table concat_incompat
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@concat_incompat
+PREHOOK: query: load data local inpath '../../data/files/alltypesorc' into table concat_incompat
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@concat_incompat
+POSTHOOK: query: load data local inpath '../../data/files/alltypesorc' into table concat_incompat
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@concat_incompat
+PREHOOK: query: load data local inpath '../../data/files/alltypesorc' into table concat_incompat
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@concat_incompat
+POSTHOOK: query: load data local inpath '../../data/files/alltypesorc' into table concat_incompat
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@concat_incompat
+PREHOOK: query: load data local inpath '../../data/files/alltypesorc' into table concat_incompat
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@concat_incompat
+POSTHOOK: query: load data local inpath '../../data/files/alltypesorc' into table concat_incompat
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@concat_incompat
+Found 4 items
+#### A masked pattern was here ####
+PREHOOK: query: select count(*) from concat_incompat
+PREHOOK: type: QUERY
+PREHOOK: Input: default@concat_incompat
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from concat_incompat
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@concat_incompat
+#### A masked pattern was here ####
+49152
+PREHOOK: query: ALTER TABLE concat_incompat CONCATENATE
+PREHOOK: type: ALTER_TABLE_MERGE
+PREHOOK: Input: default@concat_incompat
+PREHOOK: Output: default@concat_incompat
+POSTHOOK: query: ALTER TABLE concat_incompat CONCATENATE
+POSTHOOK: type: ALTER_TABLE_MERGE
+POSTHOOK: Input: default@concat_incompat
+POSTHOOK: Output: default@concat_incompat
+Found 4 items
+#### A masked pattern was here ####
+PREHOOK: query: select count(*) from concat_incompat
+PREHOOK: type: QUERY
+PREHOOK: Input: default@concat_incompat
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from concat_incompat
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@concat_incompat
+#### A masked pattern was here ####
+49152
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/order_null.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/order_null.q.out b/ql/src/test/results/clientpositive/llap/order_null.q.out
new file mode 100644
index 0000000..252e348
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/order_null.q.out
@@ -0,0 +1,222 @@
+PREHOOK: query: create table src_null (a int, b string)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@src_null
+POSTHOOK: query: create table src_null (a int, b string)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@src_null
+PREHOOK: query: insert into src_null values (1, 'A')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__1
+PREHOOK: Output: default@src_null
+POSTHOOK: query: insert into src_null values (1, 'A')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__1
+POSTHOOK: Output: default@src_null
+POSTHOOK: Lineage: src_null.a EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: src_null.b SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: insert into src_null values (null, null)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__2
+PREHOOK: Output: default@src_null
+POSTHOOK: query: insert into src_null values (null, null)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__2
+POSTHOOK: Output: default@src_null
+POSTHOOK: Lineage: src_null.a EXPRESSION [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: src_null.b SIMPLE [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: insert into src_null values (3, null)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__3
+PREHOOK: Output: default@src_null
+POSTHOOK: query: insert into src_null values (3, null)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__3
+POSTHOOK: Output: default@src_null
+POSTHOOK: Lineage: src_null.a EXPRESSION [(values__tmp__table__3)values__tmp__table__3.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: src_null.b SIMPLE [(values__tmp__table__3)values__tmp__table__3.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: insert into src_null values (2, null)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__4
+PREHOOK: Output: default@src_null
+POSTHOOK: query: insert into src_null values (2, null)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__4
+POSTHOOK: Output: default@src_null
+POSTHOOK: Lineage: src_null.a EXPRESSION [(values__tmp__table__4)values__tmp__table__4.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: src_null.b SIMPLE [(values__tmp__table__4)values__tmp__table__4.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: insert into src_null values (2, 'A')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__5
+PREHOOK: Output: default@src_null
+POSTHOOK: query: insert into src_null values (2, 'A')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__5
+POSTHOOK: Output: default@src_null
+POSTHOOK: Lineage: src_null.a EXPRESSION [(values__tmp__table__5)values__tmp__table__5.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: src_null.b SIMPLE [(values__tmp__table__5)values__tmp__table__5.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: insert into src_null values (2, 'B')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__6
+PREHOOK: Output: default@src_null
+POSTHOOK: query: insert into src_null values (2, 'B')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__6
+POSTHOOK: Output: default@src_null
+POSTHOOK: Lineage: src_null.a EXPRESSION [(values__tmp__table__6)values__tmp__table__6.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: src_null.b SIMPLE [(values__tmp__table__6)values__tmp__table__6.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY a asc
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY a asc
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+NULL NULL
+1 A
+2 NULL
+2 A
+2 B
+3 NULL
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY a desc
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY a desc
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+3 NULL
+2 NULL
+2 A
+2 B
+1 A
+NULL NULL
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY b asc, a asc nulls last
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY b asc, a asc nulls last
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+2 NULL
+3 NULL
+NULL NULL
+1 A
+2 A
+2 B
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY b desc, a asc
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY b desc, a asc
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+2 B
+1 A
+2 A
+NULL NULL
+2 NULL
+3 NULL
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY a asc nulls first
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY a asc nulls first
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+NULL NULL
+1 A
+2 NULL
+2 A
+2 B
+3 NULL
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY a desc nulls first
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY a desc nulls first
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+NULL NULL
+3 NULL
+2 NULL
+2 A
+2 B
+1 A
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY b asc nulls last, a
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY b asc nulls last, a
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+1 A
+2 A
+2 B
+NULL NULL
+2 NULL
+3 NULL
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY b desc nulls last, a
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY b desc nulls last, a
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+2 B
+1 A
+2 A
+NULL NULL
+2 NULL
+3 NULL
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY a asc nulls last, b desc
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY a asc nulls last, b desc
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+1 A
+2 B
+2 A
+2 NULL
+3 NULL
+NULL NULL
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY b desc nulls last, a desc nulls last
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY b desc nulls last, a desc nulls last
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+2 B
+2 A
+1 A
+3 NULL
+2 NULL
+NULL NULL
+PREHOOK: query: SELECT x.* FROM src_null x ORDER BY b asc nulls first, a asc nulls last
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src_null
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT x.* FROM src_null x ORDER BY b asc nulls first, a asc nulls last
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src_null
+#### A masked pattern was here ####
+2 NULL
+3 NULL
+NULL NULL
+1 A
+2 A
+2 B
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/parallel.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/parallel.q.out b/ql/src/test/results/clientpositive/llap/parallel.q.out
index eb00a72..05220db 100644
--- a/ql/src/test/results/clientpositive/llap/parallel.q.out
+++ b/ql/src/test/results/clientpositive/llap/parallel.q.out
@@ -39,6 +39,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-2
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -48,22 +49,23 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -71,25 +73,25 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Forward
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -99,10 +101,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
[29/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out b/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out
index 06e94c3..ba56486 100644
--- a/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out
@@ -85,6 +85,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -108,8 +109,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 5 Data size: 60 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: float), KEY._col2 (type: float)
@@ -325,6 +327,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -348,8 +351,9 @@ STAGE PLANS:
Statistics: Num rows: 5 Data size: 60 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: float), _col1 (type: float), _col2 (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: float), VALUE._col1 (type: float), VALUE._col2 (type: int)
@@ -564,6 +568,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -587,8 +592,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 5 Data size: 60 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: float), KEY._col2 (type: float)
@@ -804,6 +810,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -826,8 +833,9 @@ STAGE PLANS:
Statistics: Num rows: 5 Data size: 60 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: float), _col1 (type: float), _col2 (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: float), VALUE._col1 (type: float), VALUE._col2 (type: int)
@@ -1097,6 +1105,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1120,8 +1129,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 2 Data size: 24 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: int), KEY._col1 (type: float), KEY._col2 (type: float)
@@ -1337,6 +1347,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1359,8 +1370,9 @@ STAGE PLANS:
Statistics: Num rows: 2 Data size: 24 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: float), _col1 (type: float), _col2 (type: int)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: float), VALUE._col1 (type: float), VALUE._col2 (type: int)
@@ -1603,6 +1615,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1611,40 +1624,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
- outputColumnNames: _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: key, value
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
- aggregations: count(_col2)
- keys: 'day' (type: string), _col1 (type: string)
+ aggregations: count(value)
+ keys: key (type: string)
mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
- key expressions: 'day' (type: string), _col1 (type: string)
- sort order: ++
- Map-reduce partition columns: 'day' (type: string), _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col2 (type: bigint)
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
- keys: 'day' (type: string), KEY._col1 (type: string)
+ keys: KEY._col0 (type: string)
mode: mergepartial
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: UDFToInteger(_col1) (type: int), UDFToInteger(_col2) (type: int), 'day' (type: string)
+ expressions: UDFToInteger(_col0) (type: int), UDFToInteger(_col1) (type: int), 'day' (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
@@ -1731,6 +1745,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1739,40 +1754,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
- outputColumnNames: _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: key, value
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
- aggregations: count(_col2)
- keys: 'day' (type: string), _col1 (type: string)
+ aggregations: count(value)
+ keys: key (type: string)
mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
- key expressions: 'day' (type: string), _col1 (type: string)
- sort order: ++
- Map-reduce partition columns: 'day' (type: string), _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col2 (type: bigint)
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
- keys: 'day' (type: string), KEY._col1 (type: string)
+ keys: KEY._col0 (type: string)
mode: mergepartial
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: UDFToInteger(_col1) (type: int), UDFToInteger(_col2) (type: int), 'day' (type: string)
+ expressions: UDFToInteger(_col0) (type: int), UDFToInteger(_col1) (type: int), 'day' (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/empty_join.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/empty_join.q.out b/ql/src/test/results/clientpositive/llap/empty_join.q.out
new file mode 100644
index 0000000..19aa89f
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/empty_join.q.out
@@ -0,0 +1,111 @@
+PREHOOK: query: DROP TABLE IF EXISTS test_1
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE IF EXISTS test_1
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: CREATE TABLE test_1 AS SELECT 1 AS id
+PREHOOK: type: CREATETABLE_AS_SELECT
+PREHOOK: Input: _dummy_database@_dummy_table
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_1
+POSTHOOK: query: CREATE TABLE test_1 AS SELECT 1 AS id
+POSTHOOK: type: CREATETABLE_AS_SELECT
+POSTHOOK: Input: _dummy_database@_dummy_table
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_1
+POSTHOOK: Lineage: test_1.id SIMPLE []
+PREHOOK: query: DROP TABLE IF EXISTS test_2
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE IF EXISTS test_2
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: CREATE TABLE test_2 (id INT)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_2
+POSTHOOK: query: CREATE TABLE test_2 (id INT)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_2
+PREHOOK: query: DROP TABLE IF EXISTS test_3
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: DROP TABLE IF EXISTS test_3
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: CREATE TABLE test_3 AS SELECT 1 AS id
+PREHOOK: type: CREATETABLE_AS_SELECT
+PREHOOK: Input: _dummy_database@_dummy_table
+PREHOOK: Output: database:default
+PREHOOK: Output: default@test_3
+POSTHOOK: query: CREATE TABLE test_3 AS SELECT 1 AS id
+POSTHOOK: type: CREATETABLE_AS_SELECT
+POSTHOOK: Input: _dummy_database@_dummy_table
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@test_3
+POSTHOOK: Lineage: test_3.id SIMPLE []
+PREHOOK: query: explain
+SELECT t1.id, t2.id, t3.id
+FROM test_1 t1
+LEFT JOIN test_2 t2 ON t1.id = t2.id
+INNER JOIN test_3 t3 ON t1.id = t3.id
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+SELECT t1.id, t2.id, t3.id
+FROM test_1 t1
+LEFT JOIN test_2 t2 ON t1.id = t2.id
+INNER JOIN test_3 t3 ON t1.id = t3.id
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Map 1 <- Map 2 (BROADCAST_EDGE), Map 3 (BROADCAST_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_14]
+ Map Join Operator [MAPJOIN_22] (rows=2 width=1)
+ Conds:SEL_2._col0=RS_10._col0(Left Outer),SEL_2._col0=RS_11._col0(Inner),Output:["_col0","_col1","_col2"]
+ <-Map 2 [BROADCAST_EDGE] llap
+ BROADCAST [RS_10]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_20] (rows=1 width=0)
+ predicate:id is not null
+ TableScan [TS_3] (rows=1 width=0)
+ default@test_2,t2,Tbl:PARTIAL,Col:NONE,Output:["id"]
+ <-Map 3 [BROADCAST_EDGE] llap
+ BROADCAST [RS_11]
+ PartitionCols:_col0
+ Select Operator [SEL_8] (rows=1 width=1)
+ Output:["_col0"]
+ Filter Operator [FIL_21] (rows=1 width=1)
+ predicate:id is not null
+ TableScan [TS_6] (rows=1 width=1)
+ default@test_3,t3,Tbl:COMPLETE,Col:NONE,Output:["id"]
+ <-Select Operator [SEL_2] (rows=1 width=1)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=1)
+ predicate:id is not null
+ TableScan [TS_0] (rows=1 width=1)
+ default@test_1,t1,Tbl:COMPLETE,Col:NONE,Output:["id"]
+
+PREHOOK: query: SELECT t1.id, t2.id, t3.id
+FROM test_1 t1
+LEFT JOIN test_2 t2 ON t1.id = t2.id
+INNER JOIN test_3 t3 ON t1.id = t3.id
+PREHOOK: type: QUERY
+PREHOOK: Input: default@test_1
+PREHOOK: Input: default@test_2
+PREHOOK: Input: default@test_3
+#### A masked pattern was here ####
+POSTHOOK: query: SELECT t1.id, t2.id, t3.id
+FROM test_1 t1
+LEFT JOIN test_2 t2 ON t1.id = t2.id
+INNER JOIN test_3 t3 ON t1.id = t3.id
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@test_1
+POSTHOOK: Input: default@test_2
+POSTHOOK: Input: default@test_3
+#### A masked pattern was here ####
+1 NULL 1
[06/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out b/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out
index be207d4..b44ab7d 100644
--- a/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out
@@ -43,19 +43,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (csmallint < 100) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col1) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col1) (type: int)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: llap
LLAP IO: all inputs
@@ -63,38 +63,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col0) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (UDFToInteger(_col0) + 0) (type: int)
sort order: +
Map-reduce partition columns: (UDFToInteger(_col0) + 0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -109,11 +109,11 @@ STAGE PLANS:
1 UDFToInteger(_col0) (type: int)
2 (UDFToInteger(_col0) + 0) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: smallint), _col0 (type: tinyint), _col2 (type: int)
sort order: +++
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Reducer 3
Execution mode: llap
@@ -121,10 +121,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: tinyint), KEY.reducesinkkey0 (type: smallint), KEY.reducesinkkey2 (type: int), VALUE._col0 (type: bigint), VALUE._col1 (type: float), VALUE._col2 (type: double), VALUE._col3 (type: string), VALUE._col4 (type: string), VALUE._col5 (type: timestamp), VALUE._col6 (type: timestamp), VALUE._col7 (type: boolean), VALUE._col8 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -238,19 +238,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (csmallint < 100) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col1) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col1) (type: int)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: llap
LLAP IO: all inputs
@@ -258,38 +258,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col0) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (UDFToInteger(_col0) + 0) (type: int)
sort order: +
Map-reduce partition columns: (UDFToInteger(_col0) + 0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -304,11 +304,11 @@ STAGE PLANS:
1 UDFToInteger(_col0) (type: int)
2 (UDFToInteger(_col0) + 0) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: smallint), _col0 (type: tinyint), _col2 (type: int)
sort order: +++
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Reducer 3
Execution mode: llap
@@ -316,10 +316,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: tinyint), KEY.reducesinkkey0 (type: smallint), KEY.reducesinkkey2 (type: int), VALUE._col0 (type: bigint), VALUE._col1 (type: float), VALUE._col2 (type: double), VALUE._col3 (type: string), VALUE._col4 (type: string), VALUE._col5 (type: timestamp), VALUE._col6 (type: timestamp), VALUE._col7 (type: boolean), VALUE._col8 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -431,19 +431,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (csmallint < 100) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col1) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col1) (type: int)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: llap
LLAP IO: all inputs
@@ -451,38 +451,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col0) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (UDFToInteger(_col0) + 0) (type: int)
sort order: +
Map-reduce partition columns: (UDFToInteger(_col0) + 0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -497,11 +497,11 @@ STAGE PLANS:
1 UDFToInteger(_col0) (type: int)
2 (UDFToInteger(_col0) + 0) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: smallint), _col0 (type: tinyint), _col2 (type: int)
sort order: +++
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Reducer 3
Execution mode: llap
@@ -509,10 +509,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: tinyint), KEY.reducesinkkey0 (type: smallint), KEY.reducesinkkey2 (type: int), VALUE._col0 (type: bigint), VALUE._col1 (type: float), VALUE._col2 (type: double), VALUE._col3 (type: string), VALUE._col4 (type: string), VALUE._col5 (type: timestamp), VALUE._col6 (type: timestamp), VALUE._col7 (type: boolean), VALUE._col8 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_3.q.out b/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_3.q.out
new file mode 100644
index 0000000..1909c30
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_3.q.out
@@ -0,0 +1,206 @@
+PREHOOK: query: explain
+select a.*
+from alltypesorc a left outer join src b
+on a.cint = cast(b.key as int) and (a.cint < 100)
+limit 1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select a.*
+from alltypesorc a left outer join src b
+on a.cint = cast(b.key as int) and (a.cint < 100)
+limit 1
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Map 4 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 1
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
+ Execution mode: llap
+ LLAP IO: all inputs
+ Map 4
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: UDFToInteger(_col0) (type: int)
+ sort order: +
+ Map-reduce partition columns: UDFToInteger(_col0) (type: int)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: tinyint), VALUE._col1 (type: smallint), VALUE._col2 (type: int), VALUE._col3 (type: bigint), VALUE._col4 (type: float), VALUE._col5 (type: double), VALUE._col6 (type: string), VALUE._col7 (type: string), VALUE._col8 (type: timestamp), VALUE._col9 (type: timestamp), VALUE._col10 (type: boolean), VALUE._col11 (type: boolean)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 1
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col2 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col2 (type: int)
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
+ Reducer 3
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ filter predicates:
+ 0 {(KEY.reducesinkkey0 < 100)}
+ 1
+ keys:
+ 0 _col2 (type: int)
+ 1 UDFToInteger(_col0) (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
+ Statistics: Num rows: 2 Data size: 620 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 1
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain
+select a.*
+from alltypesorc a left outer join src b
+on a.cint = cast(b.key as int) and (a.cint < 100)
+limit 1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select a.*
+from alltypesorc a left outer join src b
+on a.cint = cast(b.key as int) and (a.cint < 100)
+limit 1
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (BROADCAST_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 1
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
+ value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
+ Execution mode: llap
+ LLAP IO: all inputs
+ Map 3
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: UDFToInteger(_col0) (type: int)
+ sort order: +
+ Map-reduce partition columns: UDFToInteger(_col0) (type: int)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: tinyint), VALUE._col1 (type: smallint), VALUE._col2 (type: int), VALUE._col3 (type: bigint), VALUE._col4 (type: float), VALUE._col5 (type: double), VALUE._col6 (type: string), VALUE._col7 (type: string), VALUE._col8 (type: timestamp), VALUE._col9 (type: timestamp), VALUE._col10 (type: boolean), VALUE._col11 (type: boolean)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 1
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ Map Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ filter predicates:
+ 0 {(_col2 < 100)}
+ 1
+ keys:
+ 0 _col2 (type: int)
+ 1 UDFToInteger(_col0) (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 2 Data size: 620 Basic stats: COMPLETE Column stats: COMPLETE
+ Limit
+ Number of rows: 1
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 310 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: 1
+ Processor Tree:
+ ListSink
+
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out b/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out
index f14d9b0..2859946 100644
--- a/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_join_hash.q.out
@@ -43,19 +43,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -86,7 +86,7 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 47850 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -160,19 +160,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -180,38 +180,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 4200 Data size: 44620 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21512 Data size: 3829136 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -219,19 +219,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: value (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -244,16 +244,16 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col1, _col2
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 19512 Data size: 3473136 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 19512 Data size: 3473136 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 4200 Data size: 44620 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21512 Data size: 3829136 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 4
Execution mode: llap
@@ -265,18 +265,18 @@ STAGE PLANS:
0 _col1 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 4620 Data size: 49082 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50261 Data size: 8946458 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 4620 Data size: 49082 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25130 Data size: 4674180 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 4620 Data size: 49082 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25130 Data size: 4674180 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: bigint)
Reducer 5
Execution mode: vectorized, llap
@@ -286,14 +286,14 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2310 Data size: 24541 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25130 Data size: 4674180 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col2 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2310 Data size: 24541 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25130 Data size: 2387350 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 2310 Data size: 24541 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25130 Data size: 2387350 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_join_tests.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_join_tests.q.out b/ql/src/test/results/clientpositive/llap/tez_join_tests.q.out
index bb0a259..e223666 100644
--- a/ql/src/test/results/clientpositive/llap/tez_join_tests.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_join_tests.q.out
@@ -27,16 +27,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -44,32 +44,32 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -83,11 +83,11 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 10680 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 10680 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Reducer 3
Execution mode: llap
@@ -95,12 +95,12 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 10680 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 10680 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 4
Execution mode: llap
@@ -112,11 +112,11 @@ STAGE PLANS:
0 _col1 (type: string)
1 _col1 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string)
Reducer 5
Execution mode: llap
@@ -124,10 +124,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string), VALUE._col1 (type: string), VALUE._col2 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_joins_explain.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_joins_explain.q.out b/ql/src/test/results/clientpositive/llap/tez_joins_explain.q.out
index 2179600..18b0a5b 100644
--- a/ql/src/test/results/clientpositive/llap/tez_joins_explain.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_joins_explain.q.out
@@ -25,16 +25,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -42,32 +42,32 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -81,11 +81,11 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 10680 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 10680 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Reducer 3
Execution mode: llap
@@ -93,12 +93,12 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 10680 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 10680 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 4
Execution mode: llap
@@ -110,11 +110,11 @@ STAGE PLANS:
0 _col1 (type: string)
1 _col1 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string)
Reducer 5
Execution mode: llap
@@ -122,10 +122,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string), VALUE._col1 (type: string), VALUE._col2 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 178000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_smb_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_smb_1.q.out b/ql/src/test/results/clientpositive/llap/tez_smb_1.q.out
index 5d9e881..4782c18 100644
--- a/ql/src/test/results/clientpositive/llap/tez_smb_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_smb_1.q.out
@@ -130,32 +130,32 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s3
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -222,37 +222,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: vt1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: t2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -264,7 +264,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -295,12 +295,12 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Stage: Stage-0
Fetch Operator
@@ -367,37 +367,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: t2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: vt1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -406,12 +406,12 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -421,7 +421,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -513,36 +513,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: t1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: t2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 4
@@ -550,20 +550,20 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Merge Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
[34/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/ctas.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/ctas.q.out b/ql/src/test/results/clientpositive/llap/ctas.q.out
index 93fc978..f48d431 100644
--- a/ql/src/test/results/clientpositive/llap/ctas.q.out
+++ b/ql/src/test/results/clientpositive/llap/ctas.q.out
@@ -34,6 +34,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -43,43 +44,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -117,6 +121,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_CTAS1
+POSTHOOK: Lineage: nzhang_ctas1.k SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas1.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: select * from nzhang_CTAS1
PREHOOK: type: QUERY
PREHOOK: Input: default@nzhang_ctas1
@@ -153,7 +159,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 96
@@ -184,6 +190,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -193,43 +200,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -267,6 +277,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas2
+POSTHOOK: Lineage: nzhang_ctas2.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas2.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: select * from nzhang_ctas2
PREHOOK: type: QUERY
PREHOOK: Input: default@nzhang_ctas2
@@ -303,7 +315,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 96
@@ -334,6 +346,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -343,43 +356,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: (key / 2) (type: double), concat(value, '_con') (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 96000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 96000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: double), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 96000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: double), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat
output format: org.apache.hadoop.hive.ql.io.RCFileOutputFormat
@@ -417,6 +433,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas3
+POSTHOOK: Lineage: nzhang_ctas3.conb EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas3.half_key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
PREHOOK: query: select * from nzhang_ctas3
PREHOOK: type: QUERY
PREHOOK: Input: default@nzhang_ctas3
@@ -453,7 +471,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 120
@@ -517,7 +535,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 120
@@ -548,6 +566,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -557,43 +576,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -632,6 +654,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas4
+POSTHOOK: Lineage: nzhang_ctas4.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas4.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: select * from nzhang_ctas4
PREHOOK: type: QUERY
PREHOOK: Input: default@nzhang_ctas4
@@ -668,7 +692,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 96
@@ -690,47 +714,6 @@ PREHOOK: query: explain extended create table nzhang_ctas5 row format delimited
PREHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: query: explain extended create table nzhang_ctas5 row format delimited fields terminated by ',' lines terminated by '\012' stored as textfile as select key, value from src sort by key, value limit 10
POSTHOOK: type: CREATETABLE_AS_SELECT
-ABSTRACT SYNTAX TREE:
-
-TOK_CREATETABLE
- TOK_TABNAME
- nzhang_ctas5
- TOK_LIKETABLE
- TOK_TABLEROWFORMAT
- TOK_SERDEPROPS
- TOK_TABLEROWFORMATFIELD
- ','
- TOK_TABLEROWFORMATLINES
- '\012'
- TOK_FILEFORMAT_GENERIC
- textfile
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- key
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- value
- TOK_SORTBY
- TOK_TABSORTCOLNAMEASC
- TOK_TABLE_OR_COL
- key
- TOK_TABSORTCOLNAMEASC
- TOK_TABLE_OR_COL
- value
- TOK_LIMIT
- 10
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-2 depends on stages: Stage-1
@@ -741,6 +724,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -750,19 +734,23 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
+ null sort order: aa
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
+ TopN: 10
+ TopN Hash Memory Usage: 0.1
auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -772,7 +760,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -792,7 +780,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -819,33 +807,36 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
+ null sort order: aa
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
+ TopN: 10
+ TopN Hash Memory Usage: 0.1
auto parallelism: false
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
@@ -900,6 +891,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas5
+POSTHOOK: Lineage: nzhang_ctas5.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas5.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: create table nzhang_ctas6 (key string, `to` string)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
@@ -928,3 +921,5 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@nzhang_ctas6
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas7
+POSTHOOK: Lineage: nzhang_ctas7.key SIMPLE [(nzhang_ctas6)nzhang_ctas6.FieldSchema(name:key, type:string, comment:null), ]
+POSTHOOK: Lineage: nzhang_ctas7.to SIMPLE [(nzhang_ctas6)nzhang_ctas6.FieldSchema(name:to, type:string, comment:null), ]
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_3.q.out b/ql/src/test/results/clientpositive/llap/cte_3.q.out
index 82f6414..27f0ab1 100644
--- a/ql/src/test/results/clientpositive/llap/cte_3.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_3.q.out
@@ -8,59 +8,30 @@ with q1 as ( select key from src where key = '5')
select *
from q1
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-0 depends on stages: Stage-1
- Stage-3 is a root stage
-
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- TableScan
- alias: q1
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- ListSink
+Plan optimized by CBO.
+
+Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=85)
+ Output:["_col0"]
+ Filter Operator [FIL_4] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
+Stage-3
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_6]
+ Output:["_col0"]
+ TableScan [TS_5]
+ Output:["key"]
PREHOOK: query: with q1 as ( select key from src where key = '5')
select *
@@ -93,59 +64,30 @@ explain
with q1 as ( select key from src where key = '5')
select * from (select key from q1) a
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-0 depends on stages: Stage-1
- Stage-3 is a root stage
-
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- TableScan
- alias: q1
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- ListSink
+Plan optimized by CBO.
+
+Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=85)
+ Output:["_col0"]
+ Filter Operator [FIL_4] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
+Stage-3
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_6]
+ Output:["_col0"]
+ TableScan [TS_5]
+ Output:["key"]
PREHOOK: query: with q1 as ( select key from src where key = '5')
select * from (select key from q1) a
@@ -178,98 +120,45 @@ with q1 as ( select key from q2 where key = '5'),
q2 as ( select key from src where key = '5')
select * from (select key from q1) a
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-5 depends on stages: Stage-4
- Stage-3 depends on stages: Stage-4
- Stage-0 depends on stages: Stage-1
- Stage-6 is a root stage
-
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q2
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: q2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-5
- Dependency Collection
-
- Stage: Stage-3
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-6
- Fetch Operator
- limit: -1
- Processor Tree:
- TableScan
- alias: q1
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- ListSink
+Plan optimized by CBO.
+
+Stage-5
+ Dependency Collection{}
+ Stage-4
+ Map 2 llap
+ File Output Operator [FS_8]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_7] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_9] (rows=1 width=0)
+ predicate:(key = '5')
+ TableScan [TS_5] (rows=1 width=0)
+ default@q2,q2,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q2"}
+ Select Operator [SEL_2] (rows=2 width=85)
+ Output:["_col0"]
+ Filter Operator [FIL_4] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
+Stage-3
+ Move Operator
+ Please refer to the previous Stage-4
+Stage-6
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_11]
+ Output:["_col0"]
+ TableScan [TS_10]
+ Output:["key"]
PREHOOK: query: with q1 as ( select key from q2 where key = '5'),
q2 as ( select key from src where key = '5')
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_5.q.out b/ql/src/test/results/clientpositive/llap/cte_5.q.out
index b69f73d..48e7aef 100644
--- a/ql/src/test/results/clientpositive/llap/cte_5.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_5.q.out
@@ -76,75 +76,35 @@ select a.colnum
from mydb.q1 as a join q1 as b
on a.colnum=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (UDFToDouble(colnum) = 5.0) (type: boolean)
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 3
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 250 Data size: 3656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: 5 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 3656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 3656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
- Stage: Stage-0
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=2 width=89)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_13] (rows=2 width=89)
+ Conds:(Inner)
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ Select Operator [SEL_2] (rows=1 width=3)
+ Filter Operator [FIL_11] (rows=1 width=3)
+ predicate:(UDFToDouble(colnum) = 5.0)
+ TableScan [TS_0] (rows=1 width=3)
+ mydb@q1,a,Tbl:COMPLETE,Col:NONE,Output:["colnum"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ Select Operator [SEL_5] (rows=2 width=85)
+ Filter Operator [FIL_12] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_3] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: with q1 as (select * from src where key= '5')
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out
index 7e20214..081a138 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out
@@ -11,73 +11,33 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 3
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
- Stage: Stage-0
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=4 width=85)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_13] (rows=4 width=8)
+ Conds:(Inner)
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ Select Operator [SEL_2] (rows=2 width=85)
+ Filter Operator [FIL_11] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ Select Operator [SEL_5] (rows=2 width=85)
+ Filter Operator [FIL_12] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_3] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out
index 7e20214..081a138 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out
@@ -11,73 +11,33 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 3
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
- Stage: Stage-0
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=4 width=85)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_13] (rows=4 width=8)
+ Conds:(Inner)
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ Select Operator [SEL_2] (rows=2 width=85)
+ Filter Operator [FIL_11] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ Select Operator [SEL_5] (rows=2 width=85)
+ Filter Operator [FIL_12] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_3] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out
index 415492a..cb88f0c 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out
@@ -10,117 +10,50 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-4
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
+Vertex dependency in root stage
+Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: string)
- 1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-3
+ Fetch Operator
+ limit:-1
+ Stage-4
+ Reducer 3 llap
+ File Output Operator [FS_15]
+ Merge Join Operator [MERGEJOIN_20] (rows=1 width=0)
+ Conds:RS_11._col0=RS_12._col0(Inner),Output:["_col0"]
+ <-Map 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Select Operator [SEL_7] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_5] (rows=1 width=0)
+ default@q1,a,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col0
+ Select Operator [SEL_10] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_8] (rows=1 width=0)
+ default@q1,b,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=176)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_4] (rows=2 width=178)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out
index 5c2fa59..526d80f 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out
@@ -51,119 +51,52 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-4
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: string)
- 1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
+Vertex dependency in root stage
+Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-3
+ Fetch Operator
+ limit:-1
+ Stage-4
+ Reducer 3 llap
+ File Output Operator [FS_15]
+ Merge Join Operator [MERGEJOIN_20] (rows=1 width=0)
+ Conds:RS_11._col0=RS_12._col0(Inner),Output:["_col0"]
+ <-Map 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Select Operator [SEL_7] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_5] (rows=1 width=0)
+ default@q1,a,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col0
+ Select Operator [SEL_10] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_8] (rows=1 width=0)
+ default@q1,b,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=176)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_4] (rows=2 width=178)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
PREHOOK: query: with q1 as (select * from src where key= '5')
select a.key
@@ -268,119 +201,52 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-4
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: string)
- 1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
+Vertex dependency in root stage
+Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-3
+ Fetch Operator
+ limit:-1
+ Stage-4
+ Reducer 3 llap
+ File Output Operator [FS_15]
+ Merge Join Operator [MERGEJOIN_20] (rows=1 width=0)
+ Conds:RS_11._col0=RS_12._col0(Inner),Output:["_col0"]
+ <-Map 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Select Operator [SEL_7] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_5] (rows=1 width=0)
+ default@q1,a,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col0
+ Select Operator [SEL_10] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_8] (rows=1 width=0)
+ default@q1,b,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=176)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_4] (rows=2 width=178)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
PREHOOK: query: with q1 as (select * from src where key= '5')
select a.key
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out
index ee5bac1..95c65e4 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out
@@ -75,120 +75,52 @@ select a.colnum
from mydb.q1 as a join q1 as b
on a.colnum=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-4
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: colnum is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: colnum (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(_col0) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(_col0) (type: double)
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: int)
- Execution mode: llap
- LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(_col0) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(_col0) (type: double)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 UDFToDouble(_col0) (type: double)
- 1 UDFToDouble(_col0) (type: double)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
+Vertex dependency in root stage
+Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-3
+ Fetch Operator
+ limit:-1
+ Stage-4
+ Reducer 3 llap
+ File Output Operator [FS_15]
+ Merge Join Operator [MERGEJOIN_20] (rows=1 width=3)
+ Conds:RS_11.UDFToDouble(_col0)=RS_12.UDFToDouble(_col0)(Inner),Output:["_col0"]
+ <-Map 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:UDFToDouble(_col0)
+ Select Operator [SEL_7] (rows=1 width=3)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=1 width=3)
+ predicate:colnum is not null
+ TableScan [TS_5] (rows=1 width=3)
+ mydb@q1,a,Tbl:COMPLETE,Col:NONE,Output:["colnum"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:UDFToDouble(_col0)
+ Select Operator [SEL_10] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_8] (rows=1 width=0)
+ default@q1,b,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=176)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_4] (rows=2 width=178)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
PREHOOK: query: with q1 as (select * from src where key= '5')
select a.colnum
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out b/ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out
new file mode 100644
index 0000000..9b52994
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out
@@ -0,0 +1,140 @@
+PREHOOK: query: create table testdeci2(
+id int,
+amount decimal(10,3),
+sales_tax decimal(10,3),
+item string)
+#### A masked pattern was here ####
+PREHOOK: type: CREATETABLE
+#### A masked pattern was here ####
+PREHOOK: Output: database:default
+PREHOOK: Output: default@testdeci2
+POSTHOOK: query: create table testdeci2(
+id int,
+amount decimal(10,3),
+sales_tax decimal(10,3),
+item string)
+#### A masked pattern was here ####
+POSTHOOK: type: CREATETABLE
+#### A masked pattern was here ####
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@testdeci2
+PREHOOK: query: insert into table testdeci2 values(1,12.123,12345.123,'desk1'),(2,123.123,1234.123,'desk2')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__1
+PREHOOK: Output: default@testdeci2
+POSTHOOK: query: insert into table testdeci2 values(1,12.123,12345.123,'desk1'),(2,123.123,1234.123,'desk2')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__1
+POSTHOOK: Output: default@testdeci2
+POSTHOOK: Lineage: testdeci2.amount EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+POSTHOOK: Lineage: testdeci2.id EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: testdeci2.item SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col4, type:string, comment:), ]
+POSTHOOK: Lineage: testdeci2.sales_tax EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col3, type:string, comment:), ]
+PREHOOK: query: describe formatted testdeci2
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@testdeci2
+POSTHOOK: query: describe formatted testdeci2
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@testdeci2
+# col_name data_type comment
+
+id int
+amount decimal(10,3)
+sales_tax decimal(10,3)
+item string
+
+# Detailed Table Information
+Database: default
+#### A masked pattern was here ####
+Retention: 0
+#### A masked pattern was here ####
+Table Type: MANAGED_TABLE
+Table Parameters:
+ numFiles 1
+ totalSize 578
+#### A masked pattern was here ####
+
+# Storage Information
+SerDe Library: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+InputFormat: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+OutputFormat: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+Compressed: No
+Num Buckets: -1
+Bucket Columns: []
+Sort Columns: []
+Storage Desc Params:
+ serialization.format 1
+#### A masked pattern was here ####
+PREHOOK: query: describe formatted testdeci2 amount
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@testdeci2
+POSTHOOK: query: describe formatted testdeci2 amount
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@testdeci2
+# col_name data_type min max num_nulls distinct_count avg_col_len max_col_len num_trues num_falses comment
+
+amount decimal(10,3) from deserializer
+PREHOOK: query: analyze table testdeci2 compute statistics for columns
+PREHOOK: type: QUERY
+PREHOOK: Input: default@testdeci2
+#### A masked pattern was here ####
+POSTHOOK: query: analyze table testdeci2 compute statistics for columns
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@testdeci2
+#### A masked pattern was here ####
+PREHOOK: query: analyze table testdeci2 compute statistics for columns
+PREHOOK: type: QUERY
+PREHOOK: Input: default@testdeci2
+#### A masked pattern was here ####
+POSTHOOK: query: analyze table testdeci2 compute statistics for columns
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@testdeci2
+#### A masked pattern was here ####
+PREHOOK: query: explain
+select s.id,
+coalesce(d.amount,0) as sales,
+coalesce(d.sales_tax,0) as tax
+from testdeci2 s join testdeci2 d
+on s.item=d.item and d.id=2
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select s.id,
+coalesce(d.amount,0) as sales,
+coalesce(d.sales_tax,0) as tax
+from testdeci2 s join testdeci2 d
+on s.item=d.item and d.id=2
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=5 width=228)
+ Output:["_col0","_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_15] (rows=5 width=228)
+ Conds:RS_6._col1=RS_7._col3(Inner),Output:["_col0","_col3","_col4"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ PartitionCols:_col1
+ Select Operator [SEL_2] (rows=5 width=88)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_13] (rows=5 width=88)
+ predicate:item is not null
+ TableScan [TS_0] (rows=5 width=88)
+ default@testdeci2,s,Tbl:COMPLETE,Col:COMPLETE,Output:["id","item"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ PartitionCols:_col3
+ Select Operator [SEL_5] (rows=1 width=312)
+ Output:["_col1","_col2","_col3"]
+ Filter Operator [FIL_14] (rows=1 width=312)
+ predicate:((id = 2) and item is not null)
+ TableScan [TS_3] (rows=1 width=312)
+ default@testdeci2,d,Tbl:COMPLETE,Col:COMPLETE,Output:["id","amount","sales_tax","item"]
+
[14/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/ptf.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/ptf.q.out b/ql/src/test/results/clientpositive/llap/ptf.q.out
index 59c5def..aabe694 100644
--- a/ql/src/test/results/clientpositive/llap/ptf.q.out
+++ b/ql/src/test/results/clientpositive/llap/ptf.q.out
@@ -31,6 +31,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -40,21 +41,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -64,24 +66,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -91,7 +93,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -115,17 +117,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -205,6 +207,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -215,31 +218,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_name (type: string), p_mfgr (type: string), p_size (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: p2
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -250,12 +255,12 @@ STAGE PLANS:
0 p_partkey (type: int)
1 p_partkey (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 3
Execution mode: llap
@@ -263,7 +268,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -273,24 +278,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -300,7 +305,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -311,17 +316,17 @@ STAGE PLANS:
window function: GenericUDAFLagEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), (_col5 - lag_window_0) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6583 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6583 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -395,6 +400,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -403,21 +409,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -427,21 +434,21 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -519,6 +526,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -528,21 +536,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -552,24 +561,24 @@ STAGE PLANS:
Partition table definition
input alias: abc
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -579,7 +588,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -603,17 +612,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -699,6 +708,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -708,21 +718,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -732,24 +743,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -759,7 +770,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -784,17 +795,17 @@ STAGE PLANS:
window function: GenericUDAFLagEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), _col5 (type: int), (_col5 - lag_window_2) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -882,6 +893,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -891,21 +903,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -915,33 +928,33 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int)
outputColumnNames: _col2, _col1, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col2 (type: string), _col1 (type: string), _col5 (type: int)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 2899 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: int)
sort order: +++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 2899 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string), KEY._col2 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 2899 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -951,7 +964,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col0
raw input shape:
window functions:
@@ -976,17 +989,17 @@ STAGE PLANS:
window function: GenericUDAFLagEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 2899 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col2 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), _col2 (type: int), (_col2 - lag_window_2) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 3107 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 3107 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1067,6 +1080,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Map 4 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
@@ -1076,35 +1090,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_partkey (type: int), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: int), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col1 (type: string), VALUE._col2 (type: string), VALUE._col3 (type: int), VALUE._col4 (type: string), VALUE._col5 (type: double), VALUE._col6 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1114,19 +1130,19 @@ STAGE PLANS:
Partition table definition
input alias: abc
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col0: int, _col1: string, _col2: string, _col3: string, _col4: string, _col5: int, _col6: string, _col7: double, _col8: string
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Reducer 3
Execution mode: llap
@@ -1138,13 +1154,13 @@ STAGE PLANS:
0 _col0 (type: int)
1 p_partkey (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1220,6 +1236,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
@@ -1229,28 +1246,30 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_partkey (type: int), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1261,17 +1280,17 @@ STAGE PLANS:
0 p_partkey (type: int)
1 _col0 (type: int)
outputColumnNames: _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: llap
@@ -1279,7 +1298,7 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: int), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col1 (type: string), VALUE._col2 (type: string), VALUE._col3 (type: int), VALUE._col4 (type: string), VALUE._col5 (type: double), VALUE._col6 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1289,19 +1308,19 @@ STAGE PLANS:
Partition table definition
input alias: abc
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col0: int, _col1: string, _col2: string, _col3: string, _col4: string, _col5: int, _col6: string, _col7: double, _col8: string
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Stage: Stage-0
@@ -1377,6 +1396,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1386,7 +1406,7 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1396,26 +1416,27 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: p_name, p_size(DESC)
+ order by: p_name ASC NULLS FIRST, p_size DESC NULLS LAST
output shape: p_name: string, p_mfgr: string, p_size: int
partition by: p_mfgr
raw input shape:
transforms raw input: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string), p_size (type: int)
sort order: ++-
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), KEY.reducesinkkey2 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1425,24 +1446,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: _col1, _col5(DESC)
+ order by: _col1 ASC NULLS FIRST, _col5 DESC NULLS LAST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
transforms raw input: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int)
sort order: ++-
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), KEY.reducesinkkey2 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1452,7 +1473,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1, _col5(DESC)
+ order by: _col1 ASC NULLS FIRST, _col5 DESC NULLS LAST
partition by: _col2
raw input shape:
window functions:
@@ -1463,17 +1484,17 @@ STAGE PLANS:
window function: GenericUDAFRankEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1551,6 +1572,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1560,7 +1582,7 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1570,27 +1592,28 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: p_name
+ order by: p_name ASC NULLS FIRST
output shape: p_name: string, p_mfgr: string, p_size: int, p_retailprice: double
partition by: p_mfgr
raw input shape:
transforms raw input: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1600,25 +1623,25 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
transforms raw input: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1628,7 +1651,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -1652,17 +1675,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1744,6 +1767,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1753,21 +1777,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1777,24 +1802,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1804,7 +1829,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -1828,17 +1853,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1922,6 +1947,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1932,21 +1958,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: -+
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1956,11 +1983,11 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col2(DESC), _col1
+ order by: _col2 DESC NULLS LAST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1970,7 +1997,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: _col2(DESC), _col1
+ order by: _col2 DESC NULLS LAST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
@@ -1978,17 +2005,17 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noop
- order by: _col2(DESC), _col1
+ order by: _col2 DESC NULLS LAST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: -+
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
Execution mode: llap
@@ -1996,7 +2023,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2006,7 +2033,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: _col2(DESC), _col1
+ order by: _col2 DESC NULLS LAST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
@@ -2014,24 +2041,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noop
- order by: _col2(DESC), _col1
+ order by: _col2 DESC NULLS LAST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2041,7 +2068,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -2065,17 +2092,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2167,6 +2194,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -2176,21 +2204,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2200,24 +2229,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2227,7 +2256,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -2243,17 +2272,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(2)~FOLLOWING(2)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), count_window_0 (type: bigint), sum_window_1 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6110 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6110 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2349,6 +2378,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
@@ -2359,35 +2389,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6110 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6110 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_partkey (type: int), p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: int), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col0, _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 13078 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2397,19 +2429,19 @@ STAGE PLANS:
Partition table definition
input alias: abc
name: noop
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col0: int, _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 13078 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 13078 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 13078 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col5 (type: int), _col7 (type: double)
Reducer 3
Execution mode: llap
@@ -2421,20 +2453,20 @@ STAGE PLANS:
0 _col0 (type: int)
1 p_partkey (type: int)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6699 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6699 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 22243 Basic stats: COMPLETE Column stats: COMPLETE
<TRUNCATED>
[51/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
HIVE-14502: Convert MiniTez tests to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/5f01dc42
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/5f01dc42
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/5f01dc42
Branch: refs/heads/master
Commit: 5f01dc42a518044975cb54da2966cdd1ba1ac126
Parents: a4d8c66
Author: Prasanth Jayachandran <pr...@apache.org>
Authored: Tue Aug 23 11:45:10 2016 -0700
Committer: Prasanth Jayachandran <pr...@apache.org>
Committed: Tue Aug 23 11:45:10 2016 -0700
----------------------------------------------------------------------
data/conf/llap/hive-site.xml | 6 +
.../test/resources/testconfiguration.properties | 311 +-
.../queries/clientpositive/constprog_semijoin.q | 1 +
ql/src/test/queries/clientpositive/cte_3.q | 1 +
ql/src/test/queries/clientpositive/cte_5.q | 1 +
ql/src/test/queries/clientpositive/cte_mat_1.q | 1 +
ql/src/test/queries/clientpositive/cte_mat_2.q | 1 +
ql/src/test/queries/clientpositive/cte_mat_3.q | 1 +
ql/src/test/queries/clientpositive/cte_mat_4.q | 1 +
ql/src/test/queries/clientpositive/cte_mat_5.q | 1 +
.../test/queries/clientpositive/deleteAnalyze.q | 1 +
ql/src/test/queries/clientpositive/empty_join.q | 1 +
.../clientpositive/nonmr_fetch_threshold.q | 1 +
ql/src/test/queries/clientpositive/smb_cache.q | 1 +
.../vector_aggregate_without_gby.q | 3 +-
.../clientpositive/vector_auto_smb_mapjoin_14.q | 1 +
.../test/queries/clientpositive/vector_bround.q | 3 +-
.../clientpositive/vector_groupby_mapjoin.q | 2 +-
.../clientpositive/vector_join_part_col_char.q | 2 +
.../queries/clientpositive/vector_struct_in.q | 3 +-
.../clientpositive/vectorized_parquet_types.q | 3 +-
.../clientpositive/vectorized_timestamp.q | 3 +-
.../test/queries/clientpositive/windowing_gby.q | 1 +
.../llap/acid_bucket_pruning.q.out | 153 +
.../clientpositive/llap/acid_globallimit.q.out | 52 +
.../llap/acid_vectorization_missing_cols.q.out | 56 +
.../clientpositive/llap/alter_merge_2_orc.q.out | 2 -
.../llap/alter_merge_stats_orc.q.out | 10 +-
.../clientpositive/llap/auto_join0.q.out | 46 +-
.../clientpositive/llap/auto_join1.q.out | 62 +-
.../clientpositive/llap/auto_join21.q.out | 34 +-
.../clientpositive/llap/auto_join29.q.out | 323 +-
.../clientpositive/llap/auto_join30.q.out | 439 +-
.../clientpositive/llap/auto_join_filters.q.out | 48 +-
.../clientpositive/llap/auto_join_nulls.q.out | 6 +-
.../llap/auto_sortmerge_join_1.q.out | 360 +-
.../llap/auto_sortmerge_join_10.q.out | 136 +-
.../llap/auto_sortmerge_join_11.q.out | 397 +-
.../llap/auto_sortmerge_join_12.q.out | 222 +-
.../llap/auto_sortmerge_join_13.q.out | 66 +-
.../llap/auto_sortmerge_join_14.q.out | 120 +-
.../llap/auto_sortmerge_join_15.q.out | 120 +-
.../llap/auto_sortmerge_join_2.q.out | 240 +-
.../llap/auto_sortmerge_join_3.q.out | 360 +-
.../llap/auto_sortmerge_join_4.q.out | 360 +-
.../llap/auto_sortmerge_join_5.q.out | 327 +-
.../llap/auto_sortmerge_join_6.q.out | 688 +-
.../llap/auto_sortmerge_join_7.q.out | 363 +-
.../llap/auto_sortmerge_join_8.q.out | 363 +-
.../llap/auto_sortmerge_join_9.q.out | 1474 +--
.../results/clientpositive/llap/bucket2.q.out | 46 +-
.../results/clientpositive/llap/bucket3.q.out | 40 +-
.../results/clientpositive/llap/bucket4.q.out | 46 +-
.../llap/bucket_map_join_tez1.q.out | 432 +-
.../llap/bucket_map_join_tez2.q.out | 142 +-
.../clientpositive/llap/bucketpruning1.q.out | 1697 +++
...names_with_leading_and_trailing_spaces.q.out | 85 +
.../llap/constprog_semijoin.q.out | 494 +
.../llap/correlationoptimizer1.q.out | 1450 +--
.../results/clientpositive/llap/count.q.out | 642 +-
.../clientpositive/llap/cross_join.q.out | 272 +-
.../llap/cross_product_check_1.q.out | 399 +-
.../llap/cross_product_check_2.q.out | 444 +-
.../test/results/clientpositive/llap/ctas.q.out | 191 +-
.../results/clientpositive/llap/cte_3.q.out | 285 +-
.../results/clientpositive/llap/cte_5.q.out | 94 +-
.../results/clientpositive/llap/cte_mat_1.q.out | 94 +-
.../results/clientpositive/llap/cte_mat_2.q.out | 94 +-
.../results/clientpositive/llap/cte_mat_3.q.out | 155 +-
.../results/clientpositive/llap/cte_mat_4.q.out | 310 +-
.../results/clientpositive/llap/cte_mat_5.q.out | 156 +-
.../clientpositive/llap/deleteAnalyze.q.out | 140 +
.../llap/disable_merge_for_bucketing.q.out | 54 +-
.../llap/dynamic_partition_pruning.q.out | 486 +-
.../llap/dynamic_partition_pruning_2.q.out | 114 +-
.../llap/dynpart_sort_opt_vectorization.q.out | 260 +-
.../llap/dynpart_sort_optimization.q.out | 1022 +-
.../llap/dynpart_sort_optimization2.q.out | 104 +-
.../clientpositive/llap/empty_join.q.out | 111 +
.../clientpositive/llap/explainuser_1.q.out | 9948 ++++++------------
.../clientpositive/llap/explainuser_2.q.out | 7627 ++++++--------
.../clientpositive/llap/explainuser_4.q.out | 482 +
.../llap/filter_join_breaktask.q.out | 221 +-
.../results/clientpositive/llap/groupby1.q.out | 20 +-
.../results/clientpositive/llap/groupby2.q.out | 20 +-
.../results/clientpositive/llap/groupby3.q.out | 20 +-
.../results/clientpositive/llap/having.q.out | 138 +-
.../llap/hybridgrace_hashjoin_1.q.out | 200 +-
.../llap/hybridgrace_hashjoin_2.q.out | 316 +-
.../llap/identity_project_remove_skip.q.out | 60 +-
.../results/clientpositive/llap/insert1.q.out | 7 +-
.../clientpositive/llap/insert_into1.q.out | 317 +-
.../clientpositive/llap/insert_into2.q.out | 136 +-
.../results/clientpositive/llap/join0.q.out | 24 +-
.../results/clientpositive/llap/join1.q.out | 43 +-
.../clientpositive/llap/join_nullsafe.q.out | 82 +-
.../clientpositive/llap/limit_pushdown.q.out | 319 +-
.../clientpositive/llap/load_dyn_part1.q.out | 15 +-
.../clientpositive/llap/load_dyn_part2.q.out | 12 +-
.../clientpositive/llap/load_dyn_part3.q.out | 7 +-
.../results/clientpositive/llap/mapjoin2.q.out | 95 +
.../clientpositive/llap/mapreduce1.q.out | 15 +-
.../clientpositive/llap/mapreduce2.q.out | 15 +-
.../results/clientpositive/llap/merge1.q.out | 32 +-
.../results/clientpositive/llap/merge2.q.out | 32 +-
.../results/clientpositive/llap/mergejoin.q.out | 414 +-
.../clientpositive/llap/mergejoin_3way.q.out | 30 +
.../llap/metadata_only_queries.q.out | 280 +-
.../clientpositive/llap/metadataonly1.q.out | 627 +-
.../test/results/clientpositive/llap/mrr.q.out | 181 +-
.../llap/nonmr_fetch_threshold.q.out | 205 +
.../clientpositive/llap/optimize_nullscan.q.out | 689 +-
.../clientpositive/llap/orc_analyze.q.out | 306 +-
.../clientpositive/llap/orc_merge1.q.out | 21 +-
.../clientpositive/llap/orc_merge10.q.out | 743 ++
.../clientpositive/llap/orc_merge11.q.out | 442 +
.../clientpositive/llap/orc_merge2.q.out | 7 +-
.../clientpositive/llap/orc_merge3.q.out | 7 +-
.../clientpositive/llap/orc_merge4.q.out | 7 +-
.../clientpositive/llap/orc_merge5.q.out | 6 +-
.../clientpositive/llap/orc_merge6.q.out | 10 +-
.../clientpositive/llap/orc_merge7.q.out | 2 +
.../clientpositive/llap/orc_merge_diff_fs.q.out | 503 +
.../llap/orc_merge_incompat1.q.out | 1 +
.../llap/orc_merge_incompat2.q.out | 1 +
.../llap/orc_merge_incompat3.q.out | 70 +
.../clientpositive/llap/order_null.q.out | 222 +
.../results/clientpositive/llap/parallel.q.out | 26 +-
.../test/results/clientpositive/llap/ptf.q.out | 862 +-
.../clientpositive/llap/ptf_matchpath.q.out | 58 +-
.../clientpositive/llap/ptf_streaming.q.out | 493 +-
.../results/clientpositive/llap/sample1.q.out | 60 +-
.../schema_evol_orc_nonvec_mapwork_part.q.out | 144 +-
...ol_orc_nonvec_mapwork_part_all_complex.q.out | 18 +-
..._orc_nonvec_mapwork_part_all_primitive.q.out | 96 +-
.../llap/schema_evol_orc_vec_mapwork_part.q.out | 144 +-
..._evol_orc_vec_mapwork_part_all_complex.q.out | 18 +-
...vol_orc_vec_mapwork_part_all_primitive.q.out | 96 +-
.../schema_evol_text_nonvec_mapwork_part.q.out | 144 +-
...l_text_nonvec_mapwork_part_all_complex.q.out | 18 +-
...text_nonvec_mapwork_part_all_primitive.q.out | 96 +-
.../schema_evol_text_vec_mapwork_part.q.out | 144 +-
...evol_text_vec_mapwork_part_all_complex.q.out | 18 +-
...ol_text_vec_mapwork_part_all_primitive.q.out | 96 +-
.../schema_evol_text_vecrow_mapwork_part.q.out | 144 +-
...l_text_vecrow_mapwork_part_all_complex.q.out | 18 +-
...text_vecrow_mapwork_part_all_primitive.q.out | 96 +-
.../clientpositive/llap/script_pipe.q.out | 35 +-
.../clientpositive/llap/scriptfile1.q.out | 6 +
.../llap/selectDistinctStar.q.out | 138 +-
.../results/clientpositive/llap/skewjoin.q.out | 257 +-
.../results/clientpositive/llap/smb_cache.q.out | 347 +
.../clientpositive/llap/stats_noscan_1.q.out | 28 +-
.../clientpositive/llap/stats_only_null.q.out | 28 +-
.../clientpositive/llap/subquery_exists.q.out | 140 +-
.../clientpositive/llap/subquery_in.q.out | 300 +-
.../clientpositive/llap/temp_table.q.out | 147 +-
.../llap/tez_bmj_schema_evolution.q.out | 14 +-
.../results/clientpositive/llap/tez_dml.q.out | 30 +-
.../llap/tez_dynpart_hashjoin_1.q.out | 80 +-
.../llap/tez_dynpart_hashjoin_2.q.out | 96 +-
.../llap/tez_dynpart_hashjoin_3.q.out | 206 +
.../clientpositive/llap/tez_join_hash.q.out | 60 +-
.../clientpositive/llap/tez_join_tests.q.out | 34 +-
.../clientpositive/llap/tez_joins_explain.q.out | 34 +-
.../results/clientpositive/llap/tez_smb_1.q.out | 80 +-
.../clientpositive/llap/tez_smb_empty.q.out | 918 ++
.../clientpositive/llap/tez_smb_main.q.out | 52 +-
.../results/clientpositive/llap/tez_union.q.out | 270 +-
.../clientpositive/llap/tez_union2.q.out | 40 +-
.../llap/tez_union_group_by.q.out | 60 +-
.../llap/tez_union_multiinsert.q.out | 184 +-
.../llap/tez_vector_dynpart_hashjoin_1.q.out | 80 +-
.../llap/tez_vector_dynpart_hashjoin_2.q.out | 96 +-
.../clientpositive/llap/transform1.q.out | 8 +-
.../clientpositive/llap/transform_ppr1.q.out | 106 +-
.../clientpositive/llap/transform_ppr2.q.out | 102 +-
.../results/clientpositive/llap/union2.q.out | 39 +-
.../results/clientpositive/llap/union3.q.out | 67 +-
.../results/clientpositive/llap/union4.q.out | 13 +-
.../results/clientpositive/llap/union5.q.out | 51 +-
.../results/clientpositive/llap/union6.q.out | 17 +-
.../results/clientpositive/llap/union7.q.out | 39 +-
.../results/clientpositive/llap/union8.q.out | 38 +-
.../results/clientpositive/llap/union9.q.out | 54 +-
.../clientpositive/llap/unionDistinct_1.q.out | 3311 +++---
.../clientpositive/llap/unionDistinct_2.q.out | 6 +
.../clientpositive/llap/union_fast_stats.q.out | 154 +-
.../clientpositive/llap/union_stats.q.out | 484 +
.../clientpositive/llap/union_type_chk.q.out | 30 +
.../clientpositive/llap/update_all_types.q.out | 34 +-
.../llap/vector_aggregate_9.q.out | 18 +-
.../llap/vector_aggregate_without_gby.q.out | 72 +
.../llap/vector_auto_smb_mapjoin_14.q.out | 1521 +--
.../llap/vector_between_columns.q.out | 274 +
.../clientpositive/llap/vector_between_in.q.out | 422 +-
.../llap/vector_binary_join_groupby.q.out | 177 +-
.../clientpositive/llap/vector_bround.q.out | 66 +
.../clientpositive/llap/vector_bucket.q.out | 2 +
.../llap/vector_cast_constant.q.out | 1 +
.../clientpositive/llap/vector_char_2.q.out | 22 +-
.../clientpositive/llap/vector_char_4.q.out | 1 +
.../llap/vector_char_mapjoin1.q.out | 194 +-
.../llap/vector_char_simple.q.out | 39 +-
.../clientpositive/llap/vector_coalesce.q.out | 250 +-
.../clientpositive/llap/vector_coalesce_2.q.out | 26 +-
.../llap/vector_complex_join.q.out | 10 +-
.../clientpositive/llap/vector_count.q.out | 322 +
.../llap/vector_count_distinct.q.out | 18 +-
.../clientpositive/llap/vector_data_types.q.out | 22 +-
.../clientpositive/llap/vector_date_1.q.out | 55 +-
.../clientpositive/llap/vector_decimal_1.q.out | 72 +-
.../llap/vector_decimal_10_0.q.out | 9 +-
.../clientpositive/llap/vector_decimal_2.q.out | 253 +-
.../clientpositive/llap/vector_decimal_3.q.out | 516 +-
.../clientpositive/llap/vector_decimal_4.q.out | 288 +-
.../clientpositive/llap/vector_decimal_5.q.out | 180 +-
.../clientpositive/llap/vector_decimal_6.q.out | 216 +-
.../llap/vector_decimal_aggregate.q.out | 52 +-
.../llap/vector_decimal_cast.q.out | 22 +-
.../llap/vector_decimal_expressions.q.out | 48 +-
.../llap/vector_decimal_mapjoin.q.out | 267 +-
.../llap/vector_decimal_math_funcs.q.out | 6 +-
.../llap/vector_decimal_precision.q.out | 276 +-
.../llap/vector_decimal_round.q.out | 54 +-
.../llap/vector_decimal_round_2.q.out | 46 +-
.../llap/vector_decimal_trailing.q.out | 42 +-
.../llap/vector_decimal_udf.q.out | 1184 ++-
.../llap/vector_decimal_udf2.q.out | 12 +-
.../clientpositive/llap/vector_distinct_2.q.out | 8 +-
.../clientpositive/llap/vector_groupby4.q.out | 138 +
.../clientpositive/llap/vector_groupby6.q.out | 138 +
.../clientpositive/llap/vector_groupby_3.q.out | 8 +-
.../llap/vector_groupby_mapjoin.q.out | 87 +
.../llap/vector_groupby_reduce.q.out | 97 +-
.../llap/vector_grouping_sets.q.out | 45 +-
.../clientpositive/llap/vector_if_expr.q.out | 22 +-
.../clientpositive/llap/vector_inner_join.q.out | 508 +-
.../clientpositive/llap/vector_interval_1.q.out | 68 +-
.../clientpositive/llap/vector_interval_2.q.out | 92 +-
.../llap/vector_interval_arithmetic.q.out | 1093 ++
.../llap/vector_interval_mapjoin.q.out | 98 +-
.../clientpositive/llap/vector_join30.q.out | 133 +-
.../llap/vector_join_filters.q.out | 8 +-
.../clientpositive/llap/vector_join_nulls.q.out | 8 +-
.../llap/vector_join_part_col_char.q.out | 114 +-
.../llap/vector_left_outer_join.q.out | 42 +-
.../llap/vector_left_outer_join2.q.out | 393 +-
.../llap/vector_leftsemi_mapjoin.q.out | 3189 +++---
.../llap/vector_mapjoin_reduce.q.out | 171 +-
.../llap/vector_mr_diff_schema_alias.q.out | 123 +-
.../llap/vector_multi_insert.q.out | 2 +
.../llap/vector_null_projection.q.out | 54 +-
.../llap/vector_nullsafe_join.q.out | 154 +-
.../clientpositive/llap/vector_nvl.q.out | 194 +
.../clientpositive/llap/vector_orderby_5.q.out | 8 +-
.../llap/vector_outer_join0.q.out | 92 +-
.../llap/vector_outer_join1.q.out | 152 +-
.../llap/vector_outer_join2.q.out | 104 +-
.../llap/vector_outer_join3.q.out | 208 +-
.../llap/vector_outer_join4.q.out | 152 +-
.../llap/vector_outer_join5.q.out | 346 +-
.../llap/vector_outer_join6.q.out | 373 +
.../llap/vector_partition_diff_num_cols.q.out | 68 +-
.../llap/vector_partitioned_date_time.q.out | 120 +-
.../clientpositive/llap/vector_reduce1.q.out | 2169 ++++
.../clientpositive/llap/vector_reduce2.q.out | 2169 ++++
.../clientpositive/llap/vector_reduce3.q.out | 2169 ++++
.../llap/vector_reduce_groupby_decimal.q.out | 124 +-
.../llap/vector_string_concat.q.out | 10 +-
.../clientpositive/llap/vector_struct_in.q.out | 645 ++
.../clientpositive/llap/vector_varchar_4.q.out | 1 +
.../llap/vector_varchar_mapjoin1.q.out | 194 +-
.../llap/vector_varchar_simple.q.out | 39 +-
.../llap/vector_when_case_null.q.out | 99 +
.../clientpositive/llap/vectorization_0.q.out | 242 +-
.../clientpositive/llap/vectorization_1.q.out | 2 +-
.../clientpositive/llap/vectorization_12.q.out | 1064 +-
.../clientpositive/llap/vectorization_13.q.out | 118 +-
.../clientpositive/llap/vectorization_14.q.out | 634 +-
.../clientpositive/llap/vectorization_15.q.out | 122 +-
.../clientpositive/llap/vectorization_16.q.out | 748 +-
.../clientpositive/llap/vectorization_17.q.out | 38 +-
.../clientpositive/llap/vectorization_3.q.out | 1 -
.../clientpositive/llap/vectorization_7.q.out | 50 +-
.../clientpositive/llap/vectorization_8.q.out | 62 +-
.../clientpositive/llap/vectorization_9.q.out | 748 +-
.../llap/vectorization_decimal_date.q.out | 4 +
.../llap/vectorization_part_project.q.out | 21 +-
.../llap/vectorization_part_varchar.q.out | 72 +
.../llap/vectorization_pushdown.q.out | 22 +-
.../llap/vectorization_short_regress.q.out | 204 +-
.../llap/vectorized_bucketmapjoin1.q.out | 63 +-
.../clientpositive/llap/vectorized_case.q.out | 111 +-
.../clientpositive/llap/vectorized_casts.q.out | 51 +-
.../llap/vectorized_context.q.out | 102 +-
.../llap/vectorized_date_funcs.q.out | 34 +-
.../llap/vectorized_distinct_gby.q.out | 38 +-
.../vectorized_dynamic_partition_pruning.q.out | 464 +-
.../llap/vectorized_mapjoin.q.out | 38 +-
.../llap/vectorized_math_funcs.q.out | 2 +-
.../llap/vectorized_nested_mapjoin.q.out | 52 +-
.../llap/vectorized_parquet.q.out | 8 +-
.../llap/vectorized_parquet_types.q.out | 133 +-
.../clientpositive/llap/vectorized_ptf.q.out | 2745 +----
.../llap/vectorized_shufflejoin.q.out | 43 +-
.../llap/vectorized_string_funcs.q.out | 2 +-
.../llap/vectorized_timestamp.q.out | 157 +
.../llap/vectorized_timestamp_funcs.q.out | 66 +-
.../clientpositive/llap/windowing_gby.q.out | 66 +
310 files changed, 49490 insertions(+), 35236 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/data/conf/llap/hive-site.xml
----------------------------------------------------------------------
diff --git a/data/conf/llap/hive-site.xml b/data/conf/llap/hive-site.xml
index 8b6dc95..d93f8b7 100644
--- a/data/conf/llap/hive-site.xml
+++ b/data/conf/llap/hive-site.xml
@@ -45,6 +45,12 @@
</property>
<property>
+ <name>hive.stats.fetch.column.stats</name>
+ <value>true</value>
+ <description>Use column stats to annotate stats for physical optimization phase</description>
+</property>
+
+<property>
<name>hive.merge.tezfiles</name>
<value>false</value>
<description>Merge small files at the end of a Tez DAG</description>
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/itests/src/test/resources/testconfiguration.properties
----------------------------------------------------------------------
diff --git a/itests/src/test/resources/testconfiguration.properties b/itests/src/test/resources/testconfiguration.properties
index 662ae3e..a1bb37c 100644
--- a/itests/src/test/resources/testconfiguration.properties
+++ b/itests/src/test/resources/testconfiguration.properties
@@ -68,17 +68,59 @@ disabled.query.files=ql_rewrite_gbtoidx.q,\
rcfile_merge1.q,\
smb_mapjoin_8.q
-minitez.query.files.shared=acid_globallimit.q,\
- deleteAnalyze.q,\
- empty_join.q,\
+# NOTE: Add tests to minitez only if it is very
+# specific to tez and cannot be added to minillap.
+minitez.query.files.shared=delete_orig_table.q,\
+ orc_merge12.q,\
+ orc_vectorization_ppd.q,\
+ unionDistinct_2.q,\
+ update_orig_table.q,\
+ vector_join_part_col_char.q,\
+ vector_non_string_partition.q,\
+ vectorization_div0.q,\
+ vectorization_limit.q
+
+# NOTE: Add tests to minitez only if it is very
+# specific to tez and cannot be added to minillap.
+minitez.query.files=explainuser_3.q,\
+ hybridgrace_hashjoin_1.q,\
+ hybridgrace_hashjoin_2.q,\
+ partition_column_names_with_leading_and_trailing_spaces.q,\
+ stats_filemetadata.q,\
+ tez_union_with_udf.q
+
+minillap.shared.query.files=acid_globallimit.q,\
alter_merge_2_orc.q,\
alter_merge_orc.q,\
alter_merge_stats_orc.q,\
auto_join0.q,\
auto_join1.q,\
+ auto_join21.q,\
+ auto_join29.q,\
+ auto_join30.q,\
+ auto_join_filters.q,\
+ auto_join_nulls.q,\
+ auto_sortmerge_join_1.q,\
+ auto_sortmerge_join_10.q,\
+ auto_sortmerge_join_11.q,\
+ auto_sortmerge_join_12.q,\
+ auto_sortmerge_join_13.q,\
+ auto_sortmerge_join_14.q,\
+ auto_sortmerge_join_15.q,\
+ auto_sortmerge_join_16.q,\
+ auto_sortmerge_join_2.q,\
+ auto_sortmerge_join_3.q,\
+ auto_sortmerge_join_4.q,\
+ auto_sortmerge_join_5.q,\
+ auto_sortmerge_join_6.q,\
+ auto_sortmerge_join_7.q,\
+ auto_sortmerge_join_8.q,\
+ auto_sortmerge_join_9.q,\
bucket2.q,\
bucket3.q,\
bucket4.q,\
+ bucket_map_join_tez1.q,\
+ bucket_map_join_tez2.q,\
cbo_gby.q,\
cbo_gby_empty.q,\
cbo_join.q,\
@@ -94,6 +136,7 @@ minitez.query.files.shared=acid_globallimit.q,\
cbo_views.q,\
cbo_windowing.q,\
column_names_with_leading_and_trailing_spaces.q,\
+ constprog_dpp.q,\
constprog_semijoin.q,\
correlationoptimizer1.q,\
count.q,\
@@ -113,18 +156,21 @@ minitez.query.files.shared=acid_globallimit.q,\
cte_mat_4.q,\
cte_mat_5.q,\
custom_input_output_format.q,\
+ deleteAnalyze.q,\
delete_all_non_partitioned.q,\
delete_all_partitioned.q,\
- delete_orig_table.q,\
delete_tmp_table.q,\
delete_where_no_match.q,\
delete_where_non_partitioned.q,\
delete_where_partitioned.q,\
delete_whole_partition.q,\
disable_merge_for_bucketing.q,\
+ dynamic_partition_pruning.q,\
+ dynamic_partition_pruning_2.q,\
dynpart_sort_opt_vectorization.q,\
dynpart_sort_optimization.q,\
dynpart_sort_optimization2.q,\
+ empty_join.q,\
enforce_order.q,\
filter_join_breaktask.q,\
filter_join_breaktask2.q,\
@@ -132,40 +178,46 @@ minitez.query.files.shared=acid_globallimit.q,\
groupby2.q,\
groupby3.q,\
having.q,\
- identity_project_remove_skip.q\
+ identity_project_remove_skip.q,\
insert1.q,\
insert_into1.q,\
insert_into2.q,\
insert_orig_table.q,\
+ insert_update_delete.q,\
insert_values_dynamic_partitioned.q,\
insert_values_non_partitioned.q,\
- insert_values_orig_table.q\
+ insert_values_orig_table.,\
insert_values_partitioned.q,\
insert_values_tmp_table.q,\
- insert_update_delete.q,\
join0.q,\
join1.q,\
join_nullsafe.q,\
leftsemijoin.q,\
limit_pushdown.q,\
+ llap_nullscan.q,\
+ llapdecider.q,\
load_dyn_part1.q,\
load_dyn_part2.q,\
load_dyn_part3.q,\
+ lvj_mapjoin.q,\
mapjoin2.q,\
+ mapjoin_decimal.q,\
mapjoin_mapjoin.q,\
mapreduce1.q,\
mapreduce2.q,\
merge1.q,\
merge2.q,\
mergejoin.q,\
- metadataonly1.q,\
metadata_only_queries.q,\
metadata_only_queries_with_filters.q,\
+ metadataonly1.q,\
+ mrr.q,\
nonmr_fetch_threshold.q,\
- order_null.q,\
optimize_nullscan.q,\
orc_analyze.q,\
orc_merge1.q,\
+ orc_merge10.q,\
+ orc_merge11.q,\
orc_merge2.q,\
orc_merge3.q,\
orc_merge4.q,\
@@ -174,28 +226,53 @@ minitez.query.files.shared=acid_globallimit.q,\
orc_merge7.q,\
orc_merge8.q,\
orc_merge9.q,\
- orc_merge10.q,\
- orc_merge11.q,\
- orc_merge12.q,\
orc_merge_incompat1.q,\
orc_merge_incompat2.q,\
orc_merge_incompat3.q,\
+ orc_ppd_basic.q,\
orc_ppd_schema_evol_1a.q,\
orc_ppd_schema_evol_1b.q,\
orc_ppd_schema_evol_2a.q,\
orc_ppd_schema_evol_2b.q,\
orc_ppd_schema_evol_3a.q,\
- orc_vectorization_ppd.q,\
+ order_null.q,\
parallel.q,\
ptf.q,\
ptf_matchpath.q,\
ptf_streaming.q,\
sample1.q,\
- selectDistinctStar.q,\
+ schema_evol_orc_acid_mapwork_part.q,\
+ schema_evol_orc_acid_mapwork_table.q,\
+ schema_evol_orc_acidvec_mapwork_part.q,\
+ schema_evol_orc_acidvec_mapwork_table.q,\
+ schema_evol_orc_nonvec_fetchwork_part.q,\
+ schema_evol_orc_nonvec_fetchwork_table.q,\
+ schema_evol_orc_nonvec_mapwork_part.q,\
+ schema_evol_orc_nonvec_mapwork_part_all_complex.q,\
+ schema_evol_orc_nonvec_mapwork_part_all_primitive.q,\
+ schema_evol_orc_nonvec_mapwork_table.q,\
+ schema_evol_orc_vec_mapwork_part.q,\
+ schema_evol_orc_vec_mapwork_part_all_complex.q,\
+ schema_evol_orc_vec_mapwork_part_all_primitive.q,\
+ schema_evol_orc_vec_mapwork_table.q,\
+ schema_evol_stats.q,\
+ schema_evol_text_nonvec_mapwork_part.q,\
+ schema_evol_text_nonvec_mapwork_part_all_complex.q,\
+ schema_evol_text_nonvec_mapwork_part_all_primitive.q,\
+ schema_evol_text_nonvec_mapwork_table.q,\
+ schema_evol_text_vec_mapwork_part.q,\
+ schema_evol_text_vec_mapwork_part_all_complex.q,\
+ schema_evol_text_vec_mapwork_part_all_primitive.q,\
+ schema_evol_text_vec_mapwork_table.q,\
+ schema_evol_text_vecrow_mapwork_part.q,\
+ schema_evol_text_vecrow_mapwork_part_all_complex.q,\
+ schema_evol_text_vecrow_mapwork_part_all_primitive.q,\
+ schema_evol_text_vecrow_mapwork_table.q,\
script_env_var1.q,\
script_env_var2.q,\
script_pipe.q,\
scriptfile1.q,\
+ selectDistinctStar.q,\
select_dummy_source.q,\
skewjoin.q,\
stats_counter.q,\
@@ -204,8 +281,32 @@ minitez.query.files.shared=acid_globallimit.q,\
stats_only_null.q,\
subquery_exists.q,\
subquery_in.q,\
- tez_join_hash.q,\
temp_table.q,\
+ tez_bmj_schema_evolution.q,\
+ tez_dml.q,\
+ tez_dynpart_hashjoin_1.q,\
+ tez_dynpart_hashjoin_2.q,\
+ tez_fsstat.q,\
+ tez_insert_overwrite_local_directory_1.q,\
+ tez_join.q,\
+ tez_join_hash.q,\
+ tez_join_result_complex.q,\
+ tez_join_tests.q,\
+ tez_joins_explain.q,\
+ tez_multi_union.q,\
+ tez_schema_evolution.q,\
+ tez_self_join.q,\
+ tez_smb_1.q,\
+ tez_smb_main.q,\
+ tez_union.q,\
+ tez_union2.q,\
+ tez_union_decimal.q,\
+ tez_union_dynamic_partition.q,\
+ tez_union_group_by.q,\
+ tez_union_multiinsert.q,\
+ tez_union_view.q,\
+ tez_vector_dynpart_hashjoin_1.q,\
+ tez_vector_dynpart_hashjoin_2.q,\
transform1.q,\
transform2.q,\
transform_ppr1.q,\
@@ -219,32 +320,31 @@ minitez.query.files.shared=acid_globallimit.q,\
union8.q,\
union9.q,\
unionDistinct_1.q,\
- unionDistinct_2.q,\
union_fast_stats.q,\
union_stats.q,\
+ union_type_chk.q,\
update_after_multiple_inserts.q,\
update_all_non_partitioned.q,\
update_all_partitioned.q,\
update_all_types.q,\
- update_orig_table.q,\
update_tmp_table.q,\
+ update_two_cols.q,\
update_where_no_match.q,\
update_where_non_partitioned.q,\
update_where_partitioned.q,\
- update_two_cols.q,\
vector_acid3.q,\
vector_aggregate_9.q,\
vector_aggregate_without_gby.q,\
vector_auto_smb_mapjoin_14.q,\
- vector_between_in.q,\
vector_between_columns.q,\
+ vector_between_in.q,\
vector_binary_join_groupby.q,\
vector_bround.q,\
vector_bucket.q,\
- vector_char_cast.q,\
vector_cast_constant.q,\
vector_char_2.q,\
vector_char_4.q,\
+ vector_char_cast.q,\
vector_char_mapjoin1.q,\
vector_char_simple.q,\
vector_coalesce.q,\
@@ -297,9 +397,8 @@ minitez.query.files.shared=acid_globallimit.q,\
vector_mapjoin_reduce.q,\
vector_mr_diff_schema_alias.q,\
vector_multi_insert.q,\
- vector_non_string_partition.q,\
- vector_nullsafe_join.q,\
vector_null_projection.q,\
+ vector_nullsafe_join.q,\
vector_nvl.q,\
vector_orderby_5.q,\
vector_outer_join0.q,\
@@ -311,13 +410,12 @@ minitez.query.files.shared=acid_globallimit.q,\
vector_outer_join6.q,\
vector_partition_diff_num_cols.q,\
vector_partitioned_date_time.q,\
- vector_reduce_groupby_decimal.q,\
vector_reduce1.q,\
vector_reduce2.q,\
vector_reduce3.q,\
+ vector_reduce_groupby_decimal.q,\
vector_string_concat.q,\
vector_struct_in.q,\
- vectorized_timestamp.q,\
vector_varchar_4.q,\
vector_varchar_mapjoin1.q,\
vector_varchar_simple.q,\
@@ -341,8 +439,6 @@ minitez.query.files.shared=acid_globallimit.q,\
vectorization_8.q,\
vectorization_9.q,\
vectorization_decimal_date.q,\
- vectorization_div0.q,\
- vectorization_limit.q,\
vectorization_nested_udf.q,\
vectorization_not.q,\
vectorization_part.q,\
@@ -356,6 +452,7 @@ minitez.query.files.shared=acid_globallimit.q,\
vectorized_context.q,\
vectorized_date_funcs.q,\
vectorized_distinct_gby.q,\
+ vectorized_dynamic_partition_pruning.q,\
vectorized_mapjoin.q,\
vectorized_math_funcs.q,\
vectorized_nested_mapjoin.q,\
@@ -365,186 +462,66 @@ minitez.query.files.shared=acid_globallimit.q,\
vectorized_rcfile_columnar.q,\
vectorized_shufflejoin.q,\
vectorized_string_funcs.q,\
+ vectorized_timestamp.q,\
vectorized_timestamp_funcs.q,\
- vectorized_timestamp_ints_casts.q,\
- auto_sortmerge_join_1.q,\
- auto_sortmerge_join_10.q,\
- auto_sortmerge_join_11.q,\
- auto_sortmerge_join_12.q,\
- auto_sortmerge_join_13.q,\
- auto_sortmerge_join_14.q,\
- auto_sortmerge_join_15.q,\
- auto_sortmerge_join_16.q,\
- auto_sortmerge_join_2.q,\
- auto_sortmerge_join_3.q,\
- auto_sortmerge_join_4.q,\
- auto_sortmerge_join_5.q,\
- auto_sortmerge_join_6.q,\
- auto_sortmerge_join_7.q,\
- auto_sortmerge_join_8.q,\
- auto_sortmerge_join_9.q,\
- auto_join30.q,\
- auto_join21.q,\
- auto_join29.q,\
- auto_join_filters.q,\
- auto_join_nulls.q,\
- union_type_chk.q
-
+ vectorized_timestamp_ints_casts.q
-minitez.query.files=acid_bucket_pruning.q,\
+minillap.query.files=acid_bucket_pruning.q,\
acid_vectorization_missing_cols.q,\
bucket_map_join_tez1.q,\
- smb_cache.q,\
bucket_map_join_tez2.q,\
+ bucketpruning1.q,\
constprog_dpp.q,\
dynamic_partition_pruning.q,\
dynamic_partition_pruning_2.q,\
- bucketpruning1.q,\
explainuser_1.q,\
explainuser_2.q,\
- explainuser_3.q,\
explainuser_4.q,\
hybridgrace_hashjoin_1.q,\
hybridgrace_hashjoin_2.q,\
llap_nullscan.q,\
+ llap_udf.q,\
llapdecider.q,\
lvj_mapjoin.q,\
mapjoin_decimal.q,\
mergejoin_3way.q,\
mrr.q,\
- orc_ppd_basic.q,\
+ orc_llap.q,\
+ orc_llap_counters.q,\
+ orc_llap_counters1.q,\
+ orc_llap_nonvector.q,\
orc_merge_diff_fs.q,\
- partition_column_names_with_leading_and_trailing_spaces.q,\
- stats_filemetadata.q,\
+ orc_ppd_basic.q,\
+ smb_cache.q,\
tez_aggr_part_stats.q,\
tez_bmj_schema_evolution.q,\
tez_dml.q,\
- tez_fsstat.q,\
- tez_insert_overwrite_local_directory_1.q,\
tez_dynpart_hashjoin_1.q,\
tez_dynpart_hashjoin_2.q,\
tez_dynpart_hashjoin_3.q,\
- tez_vector_dynpart_hashjoin_1.q,\
- tez_vector_dynpart_hashjoin_2.q,\
+ tez_fsstat.q,\
+ tez_insert_overwrite_local_directory_1.q,\
+ tez_join.q,\
tez_join_result_complex.q,\
tez_join_tests.q,\
tez_joins_explain.q,\
+ tez_multi_union.q,\
tez_schema_evolution.q,\
tez_self_join.q,\
+ tez_smb_1.q,\
+ tez_smb_empty.q,\
+ tez_smb_main.q,\
tez_union.q,\
tez_union2.q,\
- tez_union_dynamic_partition.q,\
- tez_union_view.q,\
- tez_union_with_udf.q,\
tez_union_decimal.q,\
+ tez_union_dynamic_partition.q,\
tez_union_group_by.q,\
- tez_smb_main.q,\
- tez_smb_1.q,\
- tez_smb_empty.q,\
- vector_join_part_col_char.q,\
- vectorized_dynamic_partition_pruning.q,\
- tez_multi_union.q,\
- tez_join.q,\
tez_union_multiinsert.q,\
- windowing_gby.q
-
-
-
-
-minillap.shared.query.files=bucket_map_join_tez1.q,\
- bucket_map_join_tez2.q,\
- constprog_dpp.q,\
- cte_1.q,\
- cte_2.q,\
- cte_3.q,\
- cte_4.q,\
- cte_5.q,\
- cte_mat_1.q,\
- cte_mat_2.q,\
- cte_mat_3.q,\
- cte_mat_4.q,\
- cte_mat_5.q,\
- dynamic_partition_pruning.q,\
- dynamic_partition_pruning_2.q,\
- hybridgrace_hashjoin_1.q,\
- hybridgrace_hashjoin_2.q,\
- mapjoin_decimal.q,\
- lvj_mapjoin.q,\
- llapdecider.q,\
- llap_nullscan.q,\
- mrr.q,\
- orc_merge12.q,\
- orc_ppd_basic.q,\
- orc_ppd_schema_evol_1a.q,\
- orc_ppd_schema_evol_1b.q,\
- orc_ppd_schema_evol_2a.q,\
- orc_ppd_schema_evol_2b.q,\
- orc_ppd_schema_evol_3a.q,\
- schema_evol_stats.q,\
- schema_evol_orc_acid_mapwork_part.q,\
- schema_evol_orc_acid_mapwork_table.q,\
- schema_evol_orc_acidvec_mapwork_part.q,\
- schema_evol_orc_acidvec_mapwork_table.q,\
- schema_evol_orc_nonvec_fetchwork_part.q,\
- schema_evol_orc_nonvec_fetchwork_table.q,\
- schema_evol_orc_nonvec_mapwork_part.q,\
- schema_evol_orc_nonvec_mapwork_part_all_complex.q,\
- schema_evol_orc_nonvec_mapwork_part_all_primitive.q,\
- schema_evol_orc_nonvec_mapwork_table.q,\
- schema_evol_orc_vec_mapwork_part.q,\
- schema_evol_orc_vec_mapwork_part_all_complex.q,\
- schema_evol_orc_vec_mapwork_part_all_primitive.q,\
- schema_evol_orc_vec_mapwork_table.q,\
- schema_evol_text_nonvec_mapwork_part.q,\
- schema_evol_text_nonvec_mapwork_part_all_complex.q,\
- schema_evol_text_nonvec_mapwork_part_all_primitive.q,\
- schema_evol_text_nonvec_mapwork_table.q,\
- schema_evol_text_vec_mapwork_part.q,\
- schema_evol_text_vec_mapwork_part_all_complex.q,\
- schema_evol_text_vec_mapwork_part_all_primitive.q,\
- schema_evol_text_vec_mapwork_table.q,\
- schema_evol_text_vecrow_mapwork_part.q,\
- schema_evol_text_vecrow_mapwork_part_all_complex.q,\
- schema_evol_text_vecrow_mapwork_part_all_primitive.q,\
- schema_evol_text_vecrow_mapwork_table.q,\
- tez_bmj_schema_evolution.q,\
- tez_dml.q,\
- tez_fsstat.q,\
- tez_insert_overwrite_local_directory_1.q,\
- tez_dynpart_hashjoin_1.q,\
- tez_dynpart_hashjoin_2.q,\
+ tez_union_view.q,\
tez_vector_dynpart_hashjoin_1.q,\
tez_vector_dynpart_hashjoin_2.q,\
- tez_join_hash.q,\
- tez_join_result_complex.q,\
- tez_join_tests.q,\
- tez_joins_explain.q,\
- tez_schema_evolution.q,\
- tez_self_join.q,\
- tez_union.q,\
- tez_union2.q,\
- tez_union_dynamic_partition.q,\
- tez_union_view.q,\
- tez_union_decimal.q,\
- tez_union_group_by.q,\
- tez_smb_main.q,\
- tez_smb_1.q,\
- vector_complex_all.q,\
- vector_complex_join.q,\
- vector_include_no_sel.q,\
- vector_join_part_col_char.q,\
vectorized_dynamic_partition_pruning.q,\
- tez_multi_union.q,\
- tez_join.q,\
- tez_union_multiinsert.q
-
-
-minillap.query.files=llap_udf.q,\
- orc_llap.q,\
- orc_llap_counters.q,\
- orc_llap_counters1.q,\
- orc_llap_nonvector.q
-
+ windowing_gby.q
encrypted.query.files=encryption_join_unencrypted_tbl.q,\
encryption_insert_partition_static.q,\
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/constprog_semijoin.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/constprog_semijoin.q b/ql/src/test/queries/clientpositive/constprog_semijoin.q
index 34f90a0..f282c27 100644
--- a/ql/src/test/queries/clientpositive/constprog_semijoin.q
+++ b/ql/src/test/queries/clientpositive/constprog_semijoin.q
@@ -1,4 +1,5 @@
set hive.optimize.constant.propagation=true;
+set hive.explain.user=true;
create table table1 (id int, val string, val1 string, dimid int);
insert into table1 (id, val, val1, dimid) values (1, 't1val01', 'val101', 100), (2, 't1val02', 'val102', 200), (3, 't1val03', 'val103', 103), (3, 't1val01', 'val104', 100), (2, 't1val05', 'val105', 200), (3, 't1val01', 'val106', 103), (1, 't1val07', 'val107', 200), (2, 't1val01', 'val108', 200), (3, 't1val09', 'val109', 103), (4,'t1val01', 'val110', 200);
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_3.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/cte_3.q b/ql/src/test/queries/clientpositive/cte_3.q
index aa8adbc..037c0cb 100644
--- a/ql/src/test/queries/clientpositive/cte_3.q
+++ b/ql/src/test/queries/clientpositive/cte_3.q
@@ -1,5 +1,6 @@
set hive.mapred.mode=nonstrict;
set hive.optimize.cte.materialize.threshold=1;
+set hive.explain.user=true;
explain
with q1 as ( select key from src where key = '5')
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_5.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/cte_5.q b/ql/src/test/queries/clientpositive/cte_5.q
index 8968688..8f6ac15 100644
--- a/ql/src/test/queries/clientpositive/cte_5.q
+++ b/ql/src/test/queries/clientpositive/cte_5.q
@@ -1,5 +1,6 @@
set hive.mapred.mode=nonstrict;
set hive.optimize.cte.materialize.threshold=-1;
+set hive.explain.user=true;
create database mydb;
use mydb;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_1.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/cte_mat_1.q b/ql/src/test/queries/clientpositive/cte_mat_1.q
index 2afb960..1131a4b 100644
--- a/ql/src/test/queries/clientpositive/cte_mat_1.q
+++ b/ql/src/test/queries/clientpositive/cte_mat_1.q
@@ -1,5 +1,6 @@
set hive.mapred.mode=nonstrict;
set hive.optimize.cte.materialize.threshold=-1;
+set hive.explain.user=true;
explain
with q1 as (select * from src where key= '5')
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_2.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/cte_mat_2.q b/ql/src/test/queries/clientpositive/cte_mat_2.q
index adcd087..283aa40 100644
--- a/ql/src/test/queries/clientpositive/cte_mat_2.q
+++ b/ql/src/test/queries/clientpositive/cte_mat_2.q
@@ -1,5 +1,6 @@
set hive.mapred.mode=nonstrict;
set hive.optimize.cte.materialize.threshold=3;
+set hive.explain.user=true;
explain
with q1 as (select * from src where key= '5')
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_3.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/cte_mat_3.q b/ql/src/test/queries/clientpositive/cte_mat_3.q
index 650cc24..a6c3a86 100644
--- a/ql/src/test/queries/clientpositive/cte_mat_3.q
+++ b/ql/src/test/queries/clientpositive/cte_mat_3.q
@@ -1,5 +1,6 @@
set hive.mapred.mode=nonstrict;
set hive.optimize.cte.materialize.threshold=2;
+set hive.explain.user=true;
explain
with q1 as (select * from src where key= '5')
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_4.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/cte_mat_4.q b/ql/src/test/queries/clientpositive/cte_mat_4.q
index 2d75963..c52d9a2 100644
--- a/ql/src/test/queries/clientpositive/cte_mat_4.q
+++ b/ql/src/test/queries/clientpositive/cte_mat_4.q
@@ -1,5 +1,6 @@
set hive.mapred.mode=nonstrict;
set hive.optimize.cte.materialize.threshold=2;
+set hive.explain.user=true;
create temporary table q1 (a int, b string);
insert into q1 values (1, 'A');
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/cte_mat_5.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/cte_mat_5.q b/ql/src/test/queries/clientpositive/cte_mat_5.q
index fd0aeda..83d1eab 100644
--- a/ql/src/test/queries/clientpositive/cte_mat_5.q
+++ b/ql/src/test/queries/clientpositive/cte_mat_5.q
@@ -1,5 +1,6 @@
set hive.mapred.mode=nonstrict;
set hive.optimize.cte.materialize.threshold=1;
+set hive.explain.user=true;
create database mydb;
use mydb;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/deleteAnalyze.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/deleteAnalyze.q b/ql/src/test/queries/clientpositive/deleteAnalyze.q
index 7e5371c..26123a6 100644
--- a/ql/src/test/queries/clientpositive/deleteAnalyze.q
+++ b/ql/src/test/queries/clientpositive/deleteAnalyze.q
@@ -1,4 +1,5 @@
set hive.stats.autogather=true;
+set hive.explain.user=true;
dfs ${system:test.dfs.mkdir} ${system:test.tmp.dir}/testdeci2;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/empty_join.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/empty_join.q b/ql/src/test/queries/clientpositive/empty_join.q
index 0c372a0..088b66a 100644
--- a/ql/src/test/queries/clientpositive/empty_join.q
+++ b/ql/src/test/queries/clientpositive/empty_join.q
@@ -1,5 +1,6 @@
set hive.auto.convert.join=true;
set hive.mapjoin.hybridgrace.hashtable=false;
+set hive.explain.user=true;
DROP TABLE IF EXISTS test_1;
CREATE TABLE test_1 AS SELECT 1 AS id;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q b/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q
index bbf33f9..ddb9d82 100644
--- a/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q
+++ b/ql/src/test/queries/clientpositive/nonmr_fetch_threshold.q
@@ -1,4 +1,5 @@
set hive.fetch.task.conversion=more;
+set hive.explain.user=true;
set hive.mapred.mode=nonstrict;
explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10;
explain select cast(key as int) * 10, upper(value) from src limit 10;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/smb_cache.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/smb_cache.q b/ql/src/test/queries/clientpositive/smb_cache.q
index 5800f3f..e415e51 100644
--- a/ql/src/test/queries/clientpositive/smb_cache.q
+++ b/ql/src/test/queries/clientpositive/smb_cache.q
@@ -1,3 +1,4 @@
+set hive.explain.user=true;
create table bug_201_input_a (
userid int
) clustered by (userid) sorted by (userid) into 64 BUCKETS ;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q b/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q
index 01bd33d..8a63635 100644
--- a/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q
+++ b/ql/src/test/queries/clientpositive/vector_aggregate_without_gby.q
@@ -1,4 +1,5 @@
set hive.mapred.mode=nonstrict;
+set hive.explain.user=true;
create table testvec(id int, dt int, greg_dt string) stored as orc;
insert into table testvec
values
@@ -12,4 +13,4 @@ values
set hive.vectorized.execution.enabled=true;
set hive.map.aggr=true;
explain select max(dt), max(greg_dt) from testvec where id=5;
-select max(dt), max(greg_dt) from testvec where id=5;
\ No newline at end of file
+select max(dt), max(greg_dt) from testvec where id=5;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q b/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q
index d7e1e10..2077f8e 100644
--- a/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q
+++ b/ql/src/test/queries/clientpositive/vector_auto_smb_mapjoin_14.q
@@ -1,4 +1,5 @@
set hive.mapred.mode=nonstrict;
+set hive.explain.user=true;
SET hive.vectorized.execution.enabled=true;
;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_bround.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vector_bround.q b/ql/src/test/queries/clientpositive/vector_bround.q
index 9130540..deea00b 100644
--- a/ql/src/test/queries/clientpositive/vector_bround.q
+++ b/ql/src/test/queries/clientpositive/vector_bround.q
@@ -1,4 +1,5 @@
set hive.mapred.mode=nonstrict;
+set hive.explain.user=true;
create table test_vector_bround(v0 double, v1 double) stored as orc;
insert into table test_vector_bround
values
@@ -12,4 +13,4 @@ values
(3.51, 1.351);
set hive.vectorized.execution.enabled=true;
explain select bround(v0), bround(v1, 1) from test_vector_bround;
-select bround(v0), bround(v1, 1) from test_vector_bround;
\ No newline at end of file
+select bround(v0), bround(v1, 1) from test_vector_bround;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q b/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q
index a3cec04..53df2aa 100644
--- a/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q
+++ b/ql/src/test/queries/clientpositive/vector_groupby_mapjoin.q
@@ -19,4 +19,4 @@ select *
from src
where not key in
(select key from src)
-order by key;
\ No newline at end of file
+order by key;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_join_part_col_char.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vector_join_part_col_char.q b/ql/src/test/queries/clientpositive/vector_join_part_col_char.q
index 45a9165..5cfce37 100644
--- a/ql/src/test/queries/clientpositive/vector_join_part_col_char.q
+++ b/ql/src/test/queries/clientpositive/vector_join_part_col_char.q
@@ -2,6 +2,8 @@ set hive.mapred.mode=nonstrict;
set hive.support.concurrency=true;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.vectorized.execution.enabled=true;
+set hive.explain.user=true;
+set hive.metastore.fastpath=false;
drop table if exists char_part_tbl1 ;
drop table if exists char_part_tbl2;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vector_struct_in.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vector_struct_in.q b/ql/src/test/queries/clientpositive/vector_struct_in.q
index 42ee938..50487db 100644
--- a/ql/src/test/queries/clientpositive/vector_struct_in.q
+++ b/ql/src/test/queries/clientpositive/vector_struct_in.q
@@ -1,4 +1,5 @@
set hive.cbo.enable=false;
+set hive.explain.user=true;
set hive.tez.dynamic.partition.pruning=false;
set hive.vectorized.execution.enabled=true;
SET hive.auto.convert.join=true;
@@ -244,4 +245,4 @@ struct(1L, "b", 0.5D),
struct(5L, "b", 1.5D),
struct(1L, "a", 0.5D),
struct(3L, "b", 1.5D)
-) as b from test_4;
\ No newline at end of file
+) as b from test_4;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vectorized_parquet_types.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vectorized_parquet_types.q b/ql/src/test/queries/clientpositive/vectorized_parquet_types.q
index bf41958..297c5af 100644
--- a/ql/src/test/queries/clientpositive/vectorized_parquet_types.q
+++ b/ql/src/test/queries/clientpositive/vectorized_parquet_types.q
@@ -1,5 +1,6 @@
set hive.mapred.mode=nonstrict;
SET hive.vectorized.execution.enabled=true;
+set hive.explain.user=true;
DROP TABLE parquet_types_staging;
DROP TABLE parquet_types;
@@ -80,4 +81,4 @@ SELECT ctinyint,
MAX(cdecimal)
FROM parquet_types
GROUP BY ctinyint
-ORDER BY ctinyint;
\ No newline at end of file
+ORDER BY ctinyint;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/vectorized_timestamp.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/vectorized_timestamp.q b/ql/src/test/queries/clientpositive/vectorized_timestamp.q
index baf0cfa..2784b7a 100644
--- a/ql/src/test/queries/clientpositive/vectorized_timestamp.q
+++ b/ql/src/test/queries/clientpositive/vectorized_timestamp.q
@@ -1,4 +1,5 @@
set hive.fetch.task.conversion=none;
+set hive.explain.user=true;
DROP TABLE IF EXISTS test;
CREATE TABLE test(ts TIMESTAMP) STORED AS ORC;
@@ -24,4 +25,4 @@ SELECT ts FROM test;
EXPLAIN
SELECT MIN(ts), MAX(ts), MAX(ts) - MIN(ts) FROM test;
-SELECT MIN(ts), MAX(ts), MAX(ts) - MIN(ts) FROM test;
\ No newline at end of file
+SELECT MIN(ts), MAX(ts), MAX(ts) - MIN(ts) FROM test;
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/queries/clientpositive/windowing_gby.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/windowing_gby.q b/ql/src/test/queries/clientpositive/windowing_gby.q
index d844f11..e1a536b 100644
--- a/ql/src/test/queries/clientpositive/windowing_gby.q
+++ b/ql/src/test/queries/clientpositive/windowing_gby.q
@@ -1,4 +1,5 @@
set hive.mapred.mode=nonstrict;
+set hive.explain.user=true;
explain
select rank() over (order by return_ratio) as return_rank from
(select sum(wr.cint)/sum(ws.c_int) as return_ratio
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out b/ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out
new file mode 100644
index 0000000..4fc4114
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out
@@ -0,0 +1,153 @@
+PREHOOK: query: -- Bucket pruning only works for ACID when split-update (U=D+I) has been enabled for the table.
+-- For e.g., this can be done by setting 'transactional_properties' = 'default'.
+-- This also means that bucket pruning will not work for ACID tables with legacy behaviour.
+
+CREATE TABLE acidTblDefault(a INT) CLUSTERED BY(a) INTO 16 BUCKETS STORED AS ORC TBLPROPERTIES ('transactional'='true', 'transactional_properties'='default')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@acidTblDefault
+POSTHOOK: query: -- Bucket pruning only works for ACID when split-update (U=D+I) has been enabled for the table.
+-- For e.g., this can be done by setting 'transactional_properties' = 'default'.
+-- This also means that bucket pruning will not work for ACID tables with legacy behaviour.
+
+CREATE TABLE acidTblDefault(a INT) CLUSTERED BY(a) INTO 16 BUCKETS STORED AS ORC TBLPROPERTIES ('transactional'='true', 'transactional_properties'='default')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@acidTblDefault
+PREHOOK: query: INSERT INTO TABLE acidTblDefault SELECT cint FROM alltypesorc WHERE cint IS NOT NULL ORDER BY cint
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+PREHOOK: Output: default@acidtbldefault
+POSTHOOK: query: INSERT INTO TABLE acidTblDefault SELECT cint FROM alltypesorc WHERE cint IS NOT NULL ORDER BY cint
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+POSTHOOK: Output: default@acidtbldefault
+POSTHOOK: Lineage: acidtbldefault.a SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cint, type:int, comment:null), ]
+PREHOOK: query: INSERT INTO TABLE acidTblDefault VALUES (1)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__1
+PREHOOK: Output: default@acidtbldefault
+POSTHOOK: query: INSERT INTO TABLE acidTblDefault VALUES (1)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__1
+POSTHOOK: Output: default@acidtbldefault
+POSTHOOK: Lineage: acidtbldefault.a EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+PREHOOK: query: -- Exactly one of the buckets should be selected out of the 16 buckets
+-- by the following selection query.
+EXPLAIN EXTENDED
+SELECT * FROM acidTblDefault WHERE a = 1
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Exactly one of the buckets should be selected out of the 16 buckets
+-- by the following selection query.
+EXPLAIN EXTENDED
+SELECT * FROM acidTblDefault WHERE a = 1
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: acidtbldefault
+ filterExpr: (a = 1) (type: boolean)
+ buckets included: [1,] of 16
+ Statistics: Num rows: 8983 Data size: 35932 Basic stats: COMPLETE Column stats: NONE
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (a = 1) (type: boolean)
+ Statistics: Num rows: 4491 Data size: 17964 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: 1 (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 4491 Data size: 17964 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 4491 Data size: 17964 Basic stats: COMPLETE Column stats: NONE
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0
+ columns.types int
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: may be used (ACID table)
+ Path -> Alias:
+#### A masked pattern was here ####
+ Path -> Partition:
+#### A masked pattern was here ####
+ Partition
+ base file name: acidtbldefault
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ properties:
+ bucket_count 16
+ bucket_field_name a
+ columns a
+ columns.comments
+ columns.types int
+#### A masked pattern was here ####
+ name default.acidtbldefault
+ numFiles 17
+ numRows 0
+ rawDataSize 0
+ serialization.ddl struct acidtbldefault { i32 a}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ totalSize 35932
+ transactional true
+ transactional_properties default
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+
+ input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+ output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+ properties:
+ bucket_count 16
+ bucket_field_name a
+ columns a
+ columns.comments
+ columns.types int
+#### A masked pattern was here ####
+ name default.acidtbldefault
+ numFiles 17
+ numRows 0
+ rawDataSize 0
+ serialization.ddl struct acidtbldefault { i32 a}
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ totalSize 35932
+ transactional true
+ transactional_properties default
+#### A masked pattern was here ####
+ serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+ name: default.acidtbldefault
+ name: default.acidtbldefault
+ Truncated Path -> Alias:
+ /acidtbldefault [acidtbldefault]
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/acid_globallimit.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/acid_globallimit.q.out b/ql/src/test/results/clientpositive/llap/acid_globallimit.q.out
new file mode 100644
index 0000000..6f0e46e
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/acid_globallimit.q.out
@@ -0,0 +1,52 @@
+PREHOOK: query: -- Global Limit optimization does not work with ACID table. Make sure to skip it for ACID table.
+CREATE TABLE acidtest1(c1 INT, c2 STRING)
+CLUSTERED BY (c1) INTO 3 BUCKETS
+STORED AS ORC
+TBLPROPERTIES ("transactional"="true")
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@acidtest1
+POSTHOOK: query: -- Global Limit optimization does not work with ACID table. Make sure to skip it for ACID table.
+CREATE TABLE acidtest1(c1 INT, c2 STRING)
+CLUSTERED BY (c1) INTO 3 BUCKETS
+STORED AS ORC
+TBLPROPERTIES ("transactional"="true")
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@acidtest1
+PREHOOK: query: insert into table acidtest1 select cint, cstring1 from alltypesorc where cint is not null order by cint
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+PREHOOK: Output: default@acidtest1
+POSTHOOK: query: insert into table acidtest1 select cint, cstring1 from alltypesorc where cint is not null order by cint
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+POSTHOOK: Output: default@acidtest1
+POSTHOOK: Lineage: acidtest1.c1 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cint, type:int, comment:null), ]
+POSTHOOK: Lineage: acidtest1.c2 SIMPLE [(alltypesorc)alltypesorc.FieldSchema(name:cstring1, type:string, comment:null), ]
+PREHOOK: query: select cast (c1 as string) from acidtest1 limit 10
+PREHOOK: type: QUERY
+PREHOOK: Input: default@acidtest1
+#### A masked pattern was here ####
+POSTHOOK: query: select cast (c1 as string) from acidtest1 limit 10
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@acidtest1
+#### A masked pattern was here ####
+-340178543
+-339581189
+-339244391
+356535438
+-338184935
+-338131778
+356416560
+844686816
+355274340
+-821957276
+PREHOOK: query: drop table acidtest1
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@acidtest1
+PREHOOK: Output: default@acidtest1
+POSTHOOK: query: drop table acidtest1
+POSTHOOK: type: DROPTABLE
+POSTHOOK: Input: default@acidtest1
+POSTHOOK: Output: default@acidtest1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/acid_vectorization_missing_cols.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/acid_vectorization_missing_cols.q.out b/ql/src/test/results/clientpositive/llap/acid_vectorization_missing_cols.q.out
new file mode 100644
index 0000000..6acc6e3
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/acid_vectorization_missing_cols.q.out
@@ -0,0 +1,56 @@
+PREHOOK: query: drop table if exists newtable
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: drop table if exists newtable
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: create table newtable(
+ a string,
+ b int,
+ c double)
+row format delimited
+fields terminated by '\t'
+stored as textfile
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@newtable
+POSTHOOK: query: create table newtable(
+ a string,
+ b int,
+ c double)
+row format delimited
+fields terminated by '\t'
+stored as textfile
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@newtable
+PREHOOK: query: drop table if exists newtable_acid
+PREHOOK: type: DROPTABLE
+POSTHOOK: query: drop table if exists newtable_acid
+POSTHOOK: type: DROPTABLE
+PREHOOK: query: create table newtable_acid (b int, a varchar(50),c decimal(3,2), d int)
+clustered by (b) into 2 buckets
+stored as orc
+tblproperties ('transactional'='true')
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@newtable_acid
+POSTHOOK: query: create table newtable_acid (b int, a varchar(50),c decimal(3,2), d int)
+clustered by (b) into 2 buckets
+stored as orc
+tblproperties ('transactional'='true')
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@newtable_acid
+PREHOOK: query: insert into newtable_acid(a,b,c)
+select * from newtable
+PREHOOK: type: QUERY
+PREHOOK: Input: default@newtable
+PREHOOK: Output: default@newtable_acid
+POSTHOOK: query: insert into newtable_acid(a,b,c)
+select * from newtable
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@newtable
+POSTHOOK: Output: default@newtable_acid
+POSTHOOK: Lineage: newtable_acid.a EXPRESSION [(newtable)newtable.FieldSchema(name:a, type:string, comment:null), ]
+POSTHOOK: Lineage: newtable_acid.b SIMPLE [(newtable)newtable.FieldSchema(name:b, type:int, comment:null), ]
+POSTHOOK: Lineage: newtable_acid.c EXPRESSION [(newtable)newtable.FieldSchema(name:c, type:double, comment:null), ]
+POSTHOOK: Lineage: newtable_acid.d SIMPLE []
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out b/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out
index 7e30942..caa41b2 100644
--- a/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out
+++ b/ql/src/test/results/clientpositive/llap/alter_merge_2_orc.q.out
@@ -64,12 +64,10 @@ POSTHOOK: Lineage: src_orc_merge_test_part PARTITION(ds=2012-01-03,ts=2012-01-03
PREHOOK: query: select count(1) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31'
PREHOOK: type: QUERY
PREHOOK: Input: default@src_orc_merge_test_part
-PREHOOK: Input: default@src_orc_merge_test_part@ds=2012-01-03/ts=2012-01-03+14%3A46%3A31
#### A masked pattern was here ####
POSTHOOK: query: select count(1) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31'
POSTHOOK: type: QUERY
POSTHOOK: Input: default@src_orc_merge_test_part
-POSTHOOK: Input: default@src_orc_merge_test_part@ds=2012-01-03/ts=2012-01-03+14%3A46%3A31
#### A masked pattern was here ####
610
PREHOOK: query: select sum(hash(key)), sum(hash(value)) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31'
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out b/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out
index cefe069..0d5ba01 100644
--- a/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out
+++ b/ql/src/test/results/clientpositive/llap/alter_merge_stats_orc.q.out
@@ -89,7 +89,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 3
numRows 1500
rawDataSize 141000
@@ -140,7 +140,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 1500
rawDataSize 141000
@@ -241,7 +241,7 @@ Database: default
Table: src_orc_merge_test_part_stat
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 3
numRows 1500
rawDataSize 141000
@@ -290,7 +290,7 @@ Database: default
Table: src_orc_merge_test_part_stat
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 3
numRows 1500
rawDataSize 141000
@@ -347,7 +347,7 @@ Database: default
Table: src_orc_merge_test_part_stat
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 1500
rawDataSize 141000
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join0.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_join0.q.out b/ql/src/test/results/clientpositive/llap/auto_join0.q.out
index f5be383..3dda24a 100644
--- a/ql/src/test/results/clientpositive/llap/auto_join0.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_join0.q.out
@@ -32,6 +32,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 6 (BROADCAST_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -43,43 +44,45 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -89,42 +92,41 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3
input vertices:
1 Reducer 6
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string)
sort order: ++++
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col0,_col1,_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: llap
@@ -132,10 +134,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_join1.q.out b/ql/src/test/results/clientpositive/llap/auto_join1.q.out
index 2a771fc..3624062 100644
--- a/ql/src/test/results/clientpositive/llap/auto_join1.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_join1.q.out
@@ -27,64 +27,66 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 1 <- Map 2 (BROADCAST_EDGE)
+ Map 2 <- Map 1 (BROADCAST_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col1, _col2
+ outputColumnNames: _col0, _col2
input vertices:
- 1 Map 2
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ 0 Map 1
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: UDFToInteger(_col2) (type: int), _col1 (type: string)
+ expressions: UDFToInteger(_col0) (type: int), _col2 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 115805 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 115805 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dest_j1
Execution mode: llap
- Map 2
- Map Operator Tree:
- TableScan
- alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -113,7 +115,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest_j1
POSTHOOK: Lineage: dest_j1.key EXPRESSION [(src)src1.FieldSchema(name:key, type:string, comment:default), ]
-POSTHOOK: Lineage: dest_j1.value SIMPLE [(src)src1.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: dest_j1.value SIMPLE [(src)src2.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: SELECT sum(hash(dest_j1.key,dest_j1.value)) FROM dest_j1
PREHOOK: type: QUERY
PREHOOK: Input: default@dest_j1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join21.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_join21.q.out b/ql/src/test/results/clientpositive/llap/auto_join21.q.out
index 5ec385c..91ea004 100644
--- a/ql/src/test/results/clientpositive/llap/auto_join21.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_join21.q.out
@@ -15,6 +15,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -24,41 +25,44 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key > 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -75,28 +79,28 @@ STAGE PLANS:
1 key (type: string)
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
[28/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
index a9b439e..be09d28 100644
--- a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
@@ -3,12 +3,8 @@ PREHOOK: type: CREATETABLE
POSTHOOK: query: explain create table src_orc_merge_test_part(key int, value string) partitioned by (ds string, ts string) stored as orc
POSTHOOK: type: CREATETABLE
Stage-0
- Create Table Operator:
- columns:["key int","value string"]
- input format:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
- name:default.src_orc_merge_test_part
- output format:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
- partition columns:["ds string","ts string"]
+ Create Table Operator:
+ name:default.src_orc_merge_test_part
PREHOOK: query: create table src_orc_merge_test_part(key int, value string) partitioned by (ds string, ts string) stored as orc
PREHOOK: type: CREATETABLE
@@ -50,25 +46,20 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-3
- Stats-Aggr Operator
- Stage-0
- Move Operator
- partition:{"ds":"2012-01-03","ts":"2012-01-03+14:46:31"}
- table:{"input format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part"}
- Stage-2
- Dependency Collection{}
- Stage-1
- Map 1
- File Output Operator [FS_3]
- compressed:false
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part"}
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Stats-Aggr Operator
+ Stage-0
+ Move Operator
+ table:{"name:":"default.src_orc_merge_test_part"}
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.src_orc_merge_test_part"}
+ Select Operator [SEL_1] (rows=500 width=95)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
PREHOOK: query: insert overwrite table src_orc_merge_test_part partition(ds='2012-01-03', ts='2012-01-03+14:46:31') select * from src
PREHOOK: type: QUERY
@@ -90,42 +81,30 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-3
- Stats-Aggr Operator
- Stage-0
- Move Operator
- partition:{"ds":"2012-01-03","ts":"2012-01-03+14:46:31"}
- table:{"input format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part"}
- Stage-2
- Dependency Collection{}
- Stage-1
- Reducer 2
- File Output Operator [FS_7]
- compressed:false
- Statistics:Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.src_orc_merge_test_part"}
- Select Operator [SEL_6]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- Limit [LIM_5]
- Number of rows:100
- Statistics:Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_4]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_3]
- sort order:
- Statistics:Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: string), _col1 (type: string)
- Limit [LIM_2]
- Number of rows:100
- Statistics:Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Stats-Aggr Operator
+ Stage-0
+ Move Operator
+ table:{"name:":"default.src_orc_merge_test_part"}
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_7]
+ table:{"name:":"default.src_orc_merge_test_part"}
+ Select Operator [SEL_6] (rows=100 width=95)
+ Output:["_col0","_col1"]
+ Limit [LIM_5] (rows=100 width=178)
+ Number of rows:100
+ Select Operator [SEL_4] (rows=100 width=178)
+ Output:["_col0","_col1"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_3]
+ Limit [LIM_2] (rows=100 width=178)
+ Number of rows:100
+ Select Operator [SEL_1] (rows=500 width=178)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
PREHOOK: query: explain select count(1) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31'
PREHOOK: type: QUERY
@@ -133,36 +112,9 @@ POSTHOOK: query: explain select count(1) from src_orc_merge_test_part where ds='
POSTHOOK: type: QUERY
Plan optimized by CBO.
-Vertex dependency in root stage
-Reducer 2 <- Map 1 (SIMPLE_EDGE)
-
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 2
- File Output Operator [FS_8]
- compressed:false
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [GBY_6]
- | aggregations:["count(VALUE._col0)"]
- | outputColumnNames:["_col0"]
- | Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_5]
- sort order:
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: bigint)
- Group By Operator [GBY_4]
- aggregations:["count(1)"]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_2]
- Statistics:Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src_orc_merge_test_part
- Statistics:Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:1
PREHOOK: query: explain select sum(hash(key)), sum(hash(value)) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31'
PREHOOK: type: QUERY
@@ -174,33 +126,21 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 2
- File Output Operator [FS_8]
- compressed:false
- Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [GBY_6]
- | aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_5]
- sort order:
- Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: bigint), _col1 (type: bigint)
- Group By Operator [GBY_4]
- aggregations:["sum(_col0)","sum(_col1)"]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src_orc_merge_test_part
- Statistics:Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_8]
+ Group By Operator [GBY_6] (rows=1 width=16)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_5]
+ Group By Operator [GBY_4] (rows=1 width=16)
+ Output:["_col0","_col1"],aggregations:["sum(_col0)","sum(_col1)"]
+ Select Operator [SEL_2] (rows=500 width=102)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=500 width=102)
+ default@src_orc_merge_test_part,src_orc_merge_test_part,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
PREHOOK: query: alter table src_orc_merge_test_part partition (ds='2012-01-03', ts='2012-01-03+14:46:31') concatenate
PREHOOK: type: ALTER_PARTITION_MERGE
@@ -220,32 +160,20 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 2
- File Output Operator [FS_8]
- compressed:false
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [GBY_6]
- | aggregations:["count(VALUE._col0)"]
- | outputColumnNames:["_col0"]
- | Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_5]
- sort order:
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: bigint)
- Group By Operator [GBY_4]
- aggregations:["count(1)"]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_2]
- Statistics:Num rows: 1 Data size: 2515 Basic stats: PARTIAL Column stats: NONE
- TableScan [TS_0]
- alias:src_orc_merge_test_part
- Statistics:Num rows: 1 Data size: 2515 Basic stats: PARTIAL Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_8]
+ Group By Operator [GBY_6] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(VALUE._col0)"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_5]
+ Group By Operator [GBY_4] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(1)"]
+ Select Operator [SEL_2] (rows=500 width=102)
+ TableScan [TS_0] (rows=500 width=102)
+ default@src_orc_merge_test_part,src_orc_merge_test_part,Tbl:COMPLETE,Col:COMPLETE
PREHOOK: query: explain select sum(hash(key)), sum(hash(value)) from src_orc_merge_test_part where ds='2012-01-03' and ts='2012-01-03+14:46:31'
PREHOOK: type: QUERY
@@ -257,33 +185,21 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 2
- File Output Operator [FS_8]
- compressed:false
- Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [GBY_6]
- | aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_5]
- sort order:
- Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: bigint), _col1 (type: bigint)
- Group By Operator [GBY_4]
- aggregations:["sum(_col0)","sum(_col1)"]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 24 Data size: 2515 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src_orc_merge_test_part
- Statistics:Num rows: 24 Data size: 2515 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_8]
+ Group By Operator [GBY_6] (rows=1 width=16)
+ Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)","sum(VALUE._col1)"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_5]
+ Group By Operator [GBY_4] (rows=1 width=16)
+ Output:["_col0","_col1"],aggregations:["sum(_col0)","sum(_col1)"]
+ Select Operator [SEL_2] (rows=500 width=102)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=500 width=102)
+ default@src_orc_merge_test_part,src_orc_merge_test_part,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
PREHOOK: query: drop table src_orc_merge_test_part
PREHOOK: type: DROPTABLE
@@ -322,63 +238,37 @@ Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 3
- File Output Operator [FS_17]
- compressed:false
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [GBY_15]
- | aggregations:["sum(VALUE._col0)"]
- | outputColumnNames:["_col0"]
- | Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- |<-Reducer 2 [SIMPLE_EDGE]
- Reduce Output Operator [RS_14]
- sort order:
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: bigint)
- Group By Operator [GBY_13]
- aggregations:["sum(hash(_col0,_col1,_col2,_col3))"]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_11]
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_10]
- key expressions:_col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string)
- sort order:++++
- Statistics:Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator [MAPJOIN_20]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{}
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
- |<-Map 4 [BROADCAST_EDGE]
- | Reduce Output Operator [RS_7]
- | sort order:
- | Statistics:Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col0 (type: string), _col1 (type: string)
- | Select Operator [SEL_5]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_19]
- | predicate:(key < 10) (type: boolean)
- | Statistics:Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_3]
- | alias:src
- | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- |<-Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_18]
- predicate:(key < 10) (type: boolean)
- Statistics:Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_17]
+ Group By Operator [GBY_15] (rows=1 width=8)
+ Output:["_col0"],aggregations:["sum(VALUE._col0)"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_14]
+ Group By Operator [GBY_13] (rows=1 width=8)
+ Output:["_col0"],aggregations:["sum(hash(_col0,_col1,_col2,_col3))"]
+ Select Operator [SEL_11] (rows=27556 width=356)
+ Output:["_col0","_col1","_col2","_col3"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_10]
+ Map Join Operator [MAPJOIN_20] (rows=27556 width=356)
+ Conds:(Inner),Output:["_col0","_col1","_col2","_col3"]
+ <-Map 4 [BROADCAST_EDGE] llap
+ BROADCAST [RS_7]
+ Select Operator [SEL_5] (rows=166 width=178)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_19] (rows=166 width=178)
+ predicate:(key < 10)
+ TableScan [TS_3] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ <-Select Operator [SEL_2] (rows=166 width=178)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_18] (rows=166 width=178)
+ predicate:(key < 10)
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
PREHOOK: query: explain select key, (c_int+1)+2 as x, sum(c_int) from cbo_t1 group by c_float, cbo_t1.c_int, key
PREHOOK: type: QUERY
@@ -390,40 +280,24 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 2
- File Output Operator [FS_7]
- compressed:false
- Statistics:Num rows: 10 Data size: 885 Basic stats: COMPLETE Column stats: COMPLETE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_5]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 10 Data size: 885 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_4]
- | aggregations:["sum(VALUE._col0)"]
- | keys:KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: float)
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 10 Data size: 917 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_3]
- key expressions:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- Map-reduce partition columns:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- sort order:+++
- Statistics:Num rows: 10 Data size: 917 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col3 (type: bigint)
- Group By Operator [GBY_2]
- aggregations:["sum(c_int)"]
- keys:key (type: string), c_int (type: int), c_float (type: float)
- outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 10 Data size: 917 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator [SEL_1]
- outputColumnNames:["key","c_int","c_float"]
- Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
- TableScan [TS_0]
- alias:cbo_t1
- Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_7]
+ Select Operator [SEL_5] (rows=10 width=97)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_4] (rows=10 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_3]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_2] (rows=10 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(c_int)"],keys:key, c_int, c_float
+ Select Operator [SEL_1] (rows=20 width=88)
+ Output:["key","c_int","c_float"]
+ TableScan [TS_0] (rows=20 width=88)
+ default@cbo_t1,cbo_t1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int","c_float"]
PREHOOK: query: explain select x, y, count(*) from (select key, (c_int+c_float+1+2) as x, sum(c_int) as y from cbo_t1 group by c_float, cbo_t1.c_int, key) R group by y, x
PREHOOK: type: QUERY
@@ -436,60 +310,33 @@ Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 3
- File Output Operator [FS_12]
- compressed:false
- Statistics:Num rows: 5 Data size: 100 Basic stats: COMPLETE Column stats: COMPLETE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_11]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 5 Data size: 100 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_10]
- | aggregations:["count(VALUE._col0)"]
- | keys:KEY._col0 (type: bigint), KEY._col1 (type: float)
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 5 Data size: 100 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 2 [SIMPLE_EDGE]
- Reduce Output Operator [RS_9]
- key expressions:_col0 (type: bigint), _col1 (type: float)
- Map-reduce partition columns:_col0 (type: bigint), _col1 (type: float)
- sort order:++
- Statistics:Num rows: 5 Data size: 100 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col2 (type: bigint)
- Group By Operator [GBY_8]
- aggregations:["count()"]
- keys:_col0 (type: bigint), _col1 (type: float)
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 5 Data size: 100 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator [SEL_5]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 10 Data size: 917 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_4]
- | aggregations:["sum(VALUE._col0)"]
- | keys:KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: float)
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 10 Data size: 917 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_3]
- key expressions:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- Map-reduce partition columns:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- sort order:+++
- Statistics:Num rows: 10 Data size: 917 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col3 (type: bigint)
- Group By Operator [GBY_2]
- aggregations:["sum(c_int)"]
- keys:key (type: string), c_int (type: int), c_float (type: float)
- outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 10 Data size: 917 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator [SEL_1]
- outputColumnNames:["key","c_int","c_float"]
- Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
- TableScan [TS_0]
- alias:cbo_t1
- Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_12]
+ Select Operator [SEL_11] (rows=5 width=20)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_10] (rows=5 width=20)
+ Output:["_col0","_col1","_col2"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_9]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_8] (rows=5 width=20)
+ Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col0, _col1
+ Select Operator [SEL_5] (rows=10 width=101)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_4] (rows=10 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_3]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_2] (rows=10 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(c_int)"],keys:key, c_int, c_float
+ Select Operator [SEL_1] (rows=20 width=88)
+ Output:["key","c_int","c_float"]
+ TableScan [TS_0] (rows=20 width=88)
+ default@cbo_t1,cbo_t1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int","c_float"]
PREHOOK: query: explain select cbo_t3.c_int, c, count(*) from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1 where (cbo_t1.c_int + 1 >= 0) and (cbo_t1.c_int > 0 or cbo_t1.c_float >= 0) group by c_float, cbo_t1.c_int, key order by a) cbo_t1 join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2 where (cbo_t2.c_int + 1 >= 0) and (cbo_t2.c_int > 0 or cbo_t2.c_float >= 0) group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on cbo_t1.a=p join cbo_t3 on cbo_t1.a=key where (b + cbo_t2.q >= 0) and (b > 0 or c_int >= 0) group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c
PREHOOK: type: QUERY
@@ -508,172 +355,88 @@ Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
Reducer 9 <- Map 8 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 7
- File Output Operator [FS_45]
- compressed:false
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_44]
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 6 [SIMPLE_EDGE]
- Reduce Output Operator [RS_43]
- key expressions:(UDFToLong(_col0) + _col1) (type: bigint), _col1 (type: bigint)
- sort order:-+
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col0 (type: int), _col2 (type: bigint)
- Select Operator [SEL_41]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_40]
- | aggregations:["count(VALUE._col0)"]
- | keys:KEY._col0 (type: bigint), KEY._col1 (type: int)
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 5 [SIMPLE_EDGE]
- Reduce Output Operator [RS_39]
- key expressions:_col0 (type: bigint), _col1 (type: int)
- Map-reduce partition columns:_col0 (type: bigint), _col1 (type: int)
- sort order:++
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col2 (type: bigint)
- Group By Operator [GBY_38]
- aggregations:["count()"]
- keys:_col2 (type: bigint), _col6 (type: int)
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator [SEL_37]
- outputColumnNames:["_col2","_col6"]
- Statistics:Num rows: 2 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator [FIL_51]
- predicate:((_col1 > 0) or (_col6 >= 0)) (type: boolean)
- Statistics:Num rows: 2 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
- Merge Join Operator [MERGEJOIN_57]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{"0":"_col0 (type: string)","1":"_col0 (type: string)"}
- | outputColumnNames:["_col1","_col2","_col6"]
- | Statistics:Num rows: 3 Data size: 48 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Map 11 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_34]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 18 Data size: 1424 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col1 (type: int)
- | Select Operator [SEL_30]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 18 Data size: 1424 Basic stats: COMPLETE Column stats: COMPLETE
- | Filter Operator [FIL_55]
- | predicate:key is not null (type: boolean)
- | Statistics:Num rows: 18 Data size: 1424 Basic stats: COMPLETE Column stats: COMPLETE
- | TableScan [TS_29]
- | alias:cbo_t3
- | Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 4 [SIMPLE_EDGE]
- Reduce Output Operator [RS_32]
- key expressions:_col0 (type: string)
- Map-reduce partition columns:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col1 (type: int), _col2 (type: bigint)
- Select Operator [SEL_28]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator [FIL_52]
- predicate:((_col1 + _col4) >= 0) (type: boolean)
- Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- Merge Join Operator [MERGEJOIN_56]
- | condition map:[{"":"Inner Join 0 to 1"}]
- | keys:{"0":"_col0 (type: string)","1":"_col0 (type: string)"}
- | outputColumnNames:["_col0","_col1","_col2","_col4"]
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 10 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_25]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col1 (type: int)
- | Select Operator [SEL_20]
- | | outputColumnNames:["_col0","_col1"]
- | | Statistics:Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
- | |<-Reducer 9 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_19]
- | key expressions:_col3 (type: double), _col2 (type: bigint)
- | sort order:-+
- | Statistics:Num rows: 1 Data size: 105 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col0 (type: string), _col1 (type: int)
- | Select Operator [SEL_17]
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 105 Basic stats: COMPLETE Column stats: COMPLETE
- | Group By Operator [GBY_16]
- | | aggregations:["sum(VALUE._col0)"]
- | | keys:KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: float)
- | | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | |<-Map 8 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_15]
- | key expressions:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- | sort order:+++
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col3 (type: bigint)
- | Group By Operator [GBY_14]
- | aggregations:["sum(c_int)"]
- | keys:key (type: string), c_int (type: int), c_float (type: float)
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | Filter Operator [FIL_54]
- | predicate:((((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0.0))) and key is not null) (type: boolean)
- | Statistics:Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: COMPLETE
- | TableScan [TS_11]
- | alias:cbo_t2
- | Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 3 [SIMPLE_EDGE]
- Reduce Output Operator [RS_23]
- key expressions:_col0 (type: string)
- Map-reduce partition columns:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col1 (type: int), _col2 (type: bigint)
- Select Operator [SEL_9]
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 2 [SIMPLE_EDGE]
- Reduce Output Operator [RS_8]
- key expressions:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col1 (type: int), _col2 (type: bigint)
- Select Operator [SEL_6]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_5]
- | aggregations:["sum(VALUE._col0)"]
- | keys:KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: float)
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_4]
- key expressions:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- Map-reduce partition columns:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- sort order:+++
- Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col3 (type: bigint)
- Group By Operator [GBY_3]
- aggregations:["sum(c_int)"]
- keys:key (type: string), c_int (type: int), c_float (type: float)
- outputColumnNames:["_col0","_col1","_col2","_col3"]
- Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator [FIL_53]
- predicate:((((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0.0))) and key is not null) (type: boolean)
- Statistics:Num rows: 3 Data size: 279 Basic stats: COMPLETE Column stats: COMPLETE
- TableScan [TS_0]
- alias:cbo_t1
- Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 7 llap
+ File Output Operator [FS_42]
+ Select Operator [SEL_41] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"]
+ <-Reducer 6 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_40]
+ Select Operator [SEL_38] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_37] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1
+ <-Reducer 5 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_36]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_35] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col2, _col6
+ Select Operator [SEL_34] (rows=3 width=16)
+ Output:["_col2","_col6"]
+ Filter Operator [FIL_33] (rows=3 width=16)
+ predicate:((_col1 > 0) or (_col6 >= 0))
+ Merge Join Operator [MERGEJOIN_52] (rows=3 width=16)
+ Conds:RS_30._col0=RS_31._col0(Inner),Output:["_col1","_col2","_col6"]
+ <-Map 11 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_31]
+ PartitionCols:_col0
+ Select Operator [SEL_29] (rows=18 width=84)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_50] (rows=18 width=84)
+ predicate:key is not null
+ TableScan [TS_27] (rows=20 width=84)
+ default@cbo_t3,cbo_t3,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int"]
+ <-Reducer 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_30]
+ PartitionCols:_col0
+ Select Operator [SEL_26] (rows=1 width=101)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_25] (rows=1 width=101)
+ predicate:((_col1 + _col4) >= 0)
+ Merge Join Operator [MERGEJOIN_51] (rows=1 width=101)
+ Conds:RS_22._col0=RS_23._col0(Inner),Output:["_col0","_col1","_col2","_col4"]
+ <-Reducer 10 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_23]
+ PartitionCols:_col0
+ Select Operator [SEL_20] (rows=1 width=89)
+ Output:["_col0","_col1"]
+ <-Reducer 9 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_19]
+ Select Operator [SEL_17] (rows=1 width=105)
+ Output:["_col0","_col1","_col2","_col3"]
+ Group By Operator [GBY_16] (rows=1 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Map 8 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_15]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_14] (rows=2 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(c_int)"],keys:key, c_int, c_float
+ Filter Operator [FIL_49] (rows=5 width=93)
+ predicate:(((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0)) and key is not null)
+ TableScan [TS_11] (rows=20 width=88)
+ default@cbo_t2,cbo_t2,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int","c_float"]
+ <-Reducer 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_22]
+ PartitionCols:_col0
+ Select Operator [SEL_9] (rows=1 width=97)
+ Output:["_col0","_col1","_col2"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_8]
+ Select Operator [SEL_6] (rows=1 width=97)
+ Output:["_col0","_col1","_col2"]
+ Group By Operator [GBY_5] (rows=1 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_4]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_3] (rows=2 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(c_int)"],keys:key, c_int, c_float
+ Filter Operator [FIL_48] (rows=5 width=93)
+ predicate:(((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0)) and key is not null)
+ TableScan [TS_0] (rows=20 width=88)
+ default@cbo_t1,cbo_t1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int","c_float"]
PREHOOK: query: explain select cbo_t3.c_int, c, count(*) from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1 where (cbo_t1.c_int + 1 >= 0) and (cbo_t1.c_int > 0 or cbo_t1.c_float >= 0) group by c_float, cbo_t1.c_int, key having cbo_t1.c_float > 0 and (c_int >=1 or c_float >= 1) and (c_int + c_float) >= 0 order by b % c asc, b desc) cbo_t1 left outer join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2 where (cbo_t2.c_int + 1 >= 0) and (cbo_t2.c_int > 0 or cbo_t2.c_float >= 0) group by c_float, cbo_t2.c_int, key having cbo_t2.c_float > 0 and (c_int >=1 or c_float >= 1) and (c_int + c_float) >= 0) cbo_t2 on cbo_t1.a=p left outer join cbo_t3 on cbo_t1.a=key where (b + cbo_t2.q >= 0) and (b > 0 or c_int >= 0) group by cbo_t3.c_int, c having cbo_t3.c_int > 0 and (c_int >=1 or c >= 1) and (c_int + c) >= 0 order by cbo_t3.c_int % c asc, cbo_t3.c_int desc
PREHOOK: type: QUERY
@@ -691,163 +454,82 @@ Reducer 7 <- Reducer 6 (SIMPLE_EDGE)
Reducer 9 <- Map 8 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 7
- File Output Operator [FS_44]
- compressed:false
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_43]
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 6 [SIMPLE_EDGE]
- Reduce Output Operator [RS_42]
- key expressions:(UDFToLong(_col0) % _col1) (type: bigint), _col0 (type: int)
- sort order:+-
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col1 (type: bigint), _col2 (type: bigint)
- Select Operator [SEL_40]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_39]
- | aggregations:["count(VALUE._col0)"]
- | keys:KEY._col0 (type: bigint), KEY._col1 (type: int)
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 5 [SIMPLE_EDGE]
- Reduce Output Operator [RS_38]
- key expressions:_col0 (type: bigint), _col1 (type: int)
- Map-reduce partition columns:_col0 (type: bigint), _col1 (type: int)
- sort order:++
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col2 (type: bigint)
- Group By Operator [GBY_37]
- aggregations:["count()"]
- keys:_col2 (type: bigint), _col6 (type: int)
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator [SEL_34]
- outputColumnNames:["_col2","_col6"]
- Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator [FIL_49]
- predicate:((((_col6 > 0) and ((_col6 >= 1) or (_col2 >= 1))) and ((UDFToLong(_col6) + _col2) >= 0)) and ((_col1 > 0) or (_col6 >= 0))) (type: boolean)
- Statistics:Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
- Merge Join Operator [MERGEJOIN_55]
- | condition map:[{"":"Left Outer Join0 to 1"}]
- | keys:{"0":"_col0 (type: string)","1":"_col0 (type: string)"}
- | outputColumnNames:["_col1","_col2","_col6"]
- | Statistics:Num rows: 4 Data size: 64 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Map 10 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_31]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 20 Data size: 1602 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col1 (type: int)
- | Select Operator [SEL_29]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 20 Data size: 1602 Basic stats: COMPLETE Column stats: COMPLETE
- | TableScan [TS_28]
- | alias:cbo_t3
- | Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 4 [SIMPLE_EDGE]
- Reduce Output Operator [RS_30]
- key expressions:_col0 (type: string)
- Map-reduce partition columns:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col1 (type: int), _col2 (type: bigint)
- Select Operator [SEL_27]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator [FIL_50]
- predicate:((_col1 + _col4) >= 0) (type: boolean)
- Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- Merge Join Operator [MERGEJOIN_54]
- | condition map:[{"":"Left Outer Join0 to 1"}]
- | keys:{"0":"_col0 (type: string)","1":"_col0 (type: string)"}
- | outputColumnNames:["_col0","_col1","_col2","_col4"]
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 3 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_23]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col1 (type: int), _col2 (type: bigint)
- | Select Operator [SEL_11]
- | | outputColumnNames:["_col0","_col1","_col2"]
- | | Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
- | |<-Reducer 2 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_10]
- | key expressions:_col3 (type: bigint), _col1 (type: int)
- | sort order:+-
- | Statistics:Num rows: 1 Data size: 105 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col0 (type: string), _col2 (type: bigint)
- | Select Operator [SEL_8]
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 105 Basic stats: COMPLETE Column stats: COMPLETE
- | Group By Operator [GBY_7]
- | | aggregations:["sum(VALUE._col0)"]
- | | keys:KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: float)
- | | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | |<-Map 1 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_6]
- | key expressions:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- | sort order:+++
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col3 (type: bigint)
- | Group By Operator [GBY_5]
- | aggregations:["sum(_col2)"]
- | keys:_col0 (type: string), _col2 (type: int), _col3 (type: float)
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | Select Operator [SEL_2]
- | outputColumnNames:["_col0","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- | Filter Operator [FIL_51]
- | predicate:((((((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0.0))) and (c_float > 0.0)) and ((c_int >= 1) or (c_float >= 1.0))) and ((UDFToFloat(c_int) + c_float) >= 0.0)) (type: boolean)
- | Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- | TableScan [TS_0]
- | alias:cbo_t1
- | Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 9 [SIMPLE_EDGE]
- Reduce Output Operator [RS_24]
- key expressions:_col0 (type: string)
- Map-reduce partition columns:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col1 (type: int)
- Select Operator [SEL_21]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_20]
- | keys:KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: float)
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Map 8 [SIMPLE_EDGE]
- Reduce Output Operator [RS_19]
- key expressions:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- Map-reduce partition columns:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- sort order:+++
- Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_18]
- keys:_col0 (type: string), _col2 (type: int), _col3 (type: float)
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator [SEL_15]
- outputColumnNames:["_col0","_col2","_col3"]
- Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator [FIL_52]
- predicate:((((((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0.0))) and (c_float > 0.0)) and ((c_int >= 1) or (c_float >= 1.0))) and ((UDFToFloat(c_int) + c_float) >= 0.0)) (type: boolean)
- Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- TableScan [TS_13]
- alias:cbo_t2
- Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 7 llap
+ File Output Operator [FS_39]
+ Select Operator [SEL_38] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"]
+ <-Reducer 6 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_37]
+ Group By Operator [GBY_34] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"],aggregations:["count(VALUE._col0)"],keys:KEY._col0, KEY._col1
+ <-Reducer 5 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_33]
+ PartitionCols:_col0, _col1
+ Group By Operator [GBY_32] (rows=1 width=20)
+ Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col6, _col2
+ Select Operator [SEL_31] (rows=1 width=16)
+ Output:["_col6","_col2"]
+ Filter Operator [FIL_30] (rows=1 width=16)
+ predicate:(((_col1 > 0) or (_col6 >= 0)) and ((_col6 >= 1) or (_col2 >= 1)) and ((UDFToLong(_col6) + _col2) >= 0))
+ Merge Join Operator [MERGEJOIN_48] (rows=3 width=16)
+ Conds:RS_27._col0=RS_28._col0(Inner),Output:["_col1","_col2","_col6"]
+ <-Map 10 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_28]
+ PartitionCols:_col0
+ Select Operator [SEL_26] (rows=18 width=84)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_46] (rows=18 width=84)
+ predicate:((c_int > 0) and key is not null)
+ TableScan [TS_24] (rows=20 width=84)
+ default@cbo_t3,cbo_t3,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int"]
+ <-Reducer 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_27]
+ PartitionCols:_col0
+ Select Operator [SEL_23] (rows=1 width=101)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_22] (rows=1 width=101)
+ predicate:((_col1 + _col4) >= 0)
+ Merge Join Operator [MERGEJOIN_47] (rows=1 width=101)
+ Conds:RS_19._col0=RS_20._col0(Left Outer),Output:["_col0","_col1","_col2","_col4"]
+ <-Reducer 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_19]
+ PartitionCols:_col0
+ Select Operator [SEL_9] (rows=1 width=97)
+ Output:["_col0","_col1","_col2"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_8]
+ Select Operator [SEL_6] (rows=1 width=105)
+ Output:["_col0","_col1","_col2","_col3"]
+ Group By Operator [GBY_5] (rows=1 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_4]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_3] (rows=1 width=101)
+ Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(c_int)"],keys:key, c_int, c_float
+ Filter Operator [FIL_44] (rows=1 width=93)
+ predicate:(((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0)) and (c_float > 0) and ((c_int >= 1) or (c_float >= 1)) and ((UDFToFloat(c_int) + c_float) >= 0.0) and key is not null)
+ TableScan [TS_0] (rows=20 width=88)
+ default@cbo_t1,cbo_t1,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int","c_float"]
+ <-Reducer 9 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_20]
+ PartitionCols:_col0
+ Select Operator [SEL_17] (rows=1 width=89)
+ Output:["_col0","_col1"]
+ Group By Operator [GBY_16] (rows=1 width=93)
+ Output:["_col0","_col1","_col2"],keys:KEY._col0, KEY._col1, KEY._col2
+ <-Map 8 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_15]
+ PartitionCols:_col0, _col1, _col2
+ Group By Operator [GBY_14] (rows=1 width=93)
+ Output:["_col0","_col1","_col2"],keys:key, c_int, c_float
+ Filter Operator [FIL_45] (rows=1 width=93)
+ predicate:(((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0)) and (c_float > 0) and ((c_int >= 1) or (c_float >= 1)) and ((UDFToFloat(c_int) + c_float) >= 0.0) and key is not null)
+ TableScan [TS_11] (rows=20 width=88)
+ default@cbo_t2,cbo_t2,Tbl:COMPLETE,Col:COMPLETE,Output:["key","c_int","c_float"]
PREHOOK: query: explain select cbo_t3.c_int, c, count(*) from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1 where (cbo_t1.c_int + 1 >= 0) and (cbo_t1.c_int > 0 or cbo_t1.c_float >= 0) group by c_float, cbo_t1.c_int, key having cbo_t1.c_float > 0 and (c_int >=1 or c_float >= 1) and (c_int + c_float) >= 0 order by b+c, a desc) cbo_t1 right outer join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2 where (cbo_t2.c_int + 1 >= 0) and (cbo_t2.c_int > 0 or cbo_t2.c_float >= 0) group by c_float, cbo_t2.c_int, key having cbo_t2.c_float > 0 and (c_int >=1 or c_float >= 1) and (c_int + c_float) >= 0) cbo_t2 on cbo_t1.a=p right outer join cbo_t3 on cbo_t1.a=key where (b + cbo_t2.q >= 2) and (b > 0 or c_int >= 0) group by cbo_t3.c_int, c
PREHOOK: type: QUERY
@@ -863,136 +545,69 @@ Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
Reducer 7 <- Map 6 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 5
- File Output Operator [FS_35]
- compressed:false
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_34]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_33]
- | aggregations:["count(VALUE._col0)"]
- | keys:KEY._col0 (type: bigint), KEY._col1 (type: int)
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 4 [SIMPLE_EDGE]
- Reduce Output Operator [RS_32]
- key expressions:_col0 (type: bigint), _col1 (type: int)
- Map-reduce partition columns:_col0 (type: bigint), _col1 (type: int)
- sort order:++
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col2 (type: bigint)
- Group By Operator [GBY_31]
- aggregations:["count()"]
- keys:_col2 (type: bigint), _col6 (type: int)
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator [SEL_30]
- outputColumnNames:["_col2","_col6"]
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator [FIL_38]
- predicate:(((_col1 + _col4) >= 2) and ((_col1 > 0) or (_col6 >= 0))) (type: boolean)
- Statistics:Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Merge Join Operator [MERGEJOIN_41]
- | condition map:[{"":"Right Outer Join0 to 1"},{"":"Right Outer Join0 to 2"}]
- | keys:{"0":"_col0 (type: string)","1":"_col0 (type: string)","2":"_col0 (type: string)"}
- | outputColumnNames:["_col1","_col2","_col4","_col6"]
- | Statistics:Num rows: 4 Data size: 80 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Map 8 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_27]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 20 Data size: 1602 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col1 (type: int)
- | Select Operator [SEL_24]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 20 Data size: 1602 Basic stats: COMPLETE Column stats: COMPLETE
- | TableScan [TS_23]
- | alias:cbo_t3
- | Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 3 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_25]
- | key expressions:_col0 (type: string)
- | Map-reduce partition columns:_col0 (type: string)
- | sort order:+
- | Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col1 (type: int), _col2 (type: bigint)
- | Select Operator [SEL_11]
- | | outputColumnNames:["_col0","_col1","_col2"]
- | | Statistics:Num rows: 1 Data size: 97 Basic stats: COMPLETE Column stats: COMPLETE
- | |<-Reducer 2 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_10]
- | key expressions:_col3 (type: bigint), _col0 (type: string)
- | sort order:+-
- | Statistics:Num rows: 1 Data size: 105 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col1 (type: int), _col2 (type: bigint)
- | Select Operator [SEL_8]
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 105 Basic stats: COMPLETE Column stats: COMPLETE
- | Group By Operator [GBY_7]
- | | aggregations:["sum(VALUE._col0)"]
- | | keys:KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: float)
- | | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | |<-Map 1 [SIMPLE_EDGE]
- | Reduce Output Operator [RS_6]
- | key expressions:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- | Map-reduce partition columns:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- | sort order:+++
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | value expressions:_col3 (type: bigint)
- | Group By Operator [GBY_5]
- | aggregations:["sum(_col2)"]
- | keys:_col0 (type: string), _col2 (type: int), _col3 (type: float)
- | outputColumnNames:["_col0","_col1","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 101 Basic stats: COMPLETE Column stats: COMPLETE
- | Select Operator [SEL_2]
- | outputColumnNames:["_col0","_col2","_col3"]
- | Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- | Filter Operator [FIL_39]
- | predicate:((((((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0.0))) and (c_float > 0.0)) and ((c_int >= 1) or (c_float >= 1.0))) and ((UDFToFloat(c_int) + c_float) >= 0.0)) (type: boolean)
- | Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- | TableScan [TS_0]
- | alias:cbo_t1
- | Statistics:Num rows: 20 Data size: 262 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Reducer 7 [SIMPLE_EDGE]
- Reduce Output Operator [RS_26]
- key expressions:_col0 (type: string)
- Map-reduce partition columns:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions:_col1 (type: int)
- Select Operator [SEL_21]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_20]
- | keys:KEY._col0 (type: string), KEY._col1 (type: int), KEY._col2 (type: float)
- | outputColumnNames:["_col0","_col1","_col2"]
- | Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- |<-Map 6 [SIMPLE_EDGE]
- Reduce Output Operator [RS_19]
- key expressions:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- Map-reduce partition columns:_col0 (type: string), _col1 (type: int), _col2 (type: float)
- sort order:+++
- Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator [GBY_18]
- keys:_col0 (type: string), _col2 (type: int), _col3 (type: float)
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator [SEL_15]
- outputColumnNames:["_col0","_col2","_col3"]
- Statistics:Num rows: 1 Data size: 93 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator [FIL_40]
- predicate:((((((c_int + 1) >= 0) and ((c_int > 0) or (c_float >= 0.0))) and (c_float > 0.0)) and ((c_int >= 1) or (c_float >= 1.0))) and ((UDFToFloat(c_int) + c_float) >= 0.0)) (type: boolean)
- Statistics:Num rows: 1 Data size: 93 Basic stat
<TRUNCATED>
[39/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/bucketpruning1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/bucketpruning1.q.out b/ql/src/test/results/clientpositive/llap/bucketpruning1.q.out
new file mode 100644
index 0000000..f6ddfe9
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/bucketpruning1.q.out
@@ -0,0 +1,1697 @@
+PREHOOK: query: CREATE TABLE srcbucket_pruned(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 16 BUCKETS STORED AS TEXTFILE
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@srcbucket_pruned
+POSTHOOK: query: CREATE TABLE srcbucket_pruned(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 16 BUCKETS STORED AS TEXTFILE
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@srcbucket_pruned
+PREHOOK: query: -- cannot prune 2-key scenarios without a smarter optimizer
+CREATE TABLE srcbucket_unpruned(key int, value string) partitioned by (ds string) CLUSTERED BY (key,value) INTO 16 BUCKETS STORED AS TEXTFILE
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@srcbucket_unpruned
+POSTHOOK: query: -- cannot prune 2-key scenarios without a smarter optimizer
+CREATE TABLE srcbucket_unpruned(key int, value string) partitioned by (ds string) CLUSTERED BY (key,value) INTO 16 BUCKETS STORED AS TEXTFILE
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@srcbucket_unpruned
+PREHOOK: query: -- good cases
+
+explain extended
+select * from srcbucket_pruned where key = 1
+PREHOOK: type: QUERY
+POSTHOOK: query: -- good cases
+
+explain extended
+select * from srcbucket_pruned where key = 1
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (key = 1) (type: boolean)
+ buckets included: [1,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key = 1) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 1 (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 16
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 16
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (key = 16) (type: boolean)
+ buckets included: [0,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key = 16) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 16 (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 17
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 17
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (key = 17) (type: boolean)
+ buckets included: [1,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key = 17) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 17 (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 16+1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 16+1
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (key = 17) (type: boolean)
+ buckets included: [1,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key = 17) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 17 (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = '11'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = '11'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (key = 11) (type: boolean)
+ buckets included: [11,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key = 11) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 11 (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 and ds='2008-04-08'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 and ds='2008-04-08'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key = 1) and (ds = '2008-04-08')) (type: boolean)
+ buckets included: [1,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key = 1) and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 1 (type: int), value (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 98 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 98 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 and ds='2008-04-08' and value='One'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 and ds='2008-04-08' and value='One'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key = 1) and (ds = '2008-04-08') and (value = 'One')) (type: boolean)
+ buckets included: [1,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key = 1) and (ds = '2008-04-08') and (value = 'One')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 1 (type: int), 'One' (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 185 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 185 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where value='One' and key = 1 and ds='2008-04-08'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where value='One' and key = 1 and ds='2008-04-08'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((value = 'One') and (key = 1) and (ds = '2008-04-08')) (type: boolean)
+ buckets included: [1,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((value = 'One') and (key = 1) and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 1 (type: int), 'One' (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 185 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 185 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key in (2,3)
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key in (2,3)
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (key) IN (2, 3) (type: boolean)
+ buckets included: [2,3,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key) IN (2, 3) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key in (2,3) and ds='2008-04-08'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key in (2,3) and ds='2008-04-08'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key) IN (2, 3) and (ds = '2008-04-08')) (type: boolean)
+ buckets included: [2,3,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key) IN (2, 3) and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key in (2,3) and ds='2008-04-08' and value='One'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key in (2,3) and ds='2008-04-08' and value='One'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key) IN (2, 3) and (ds = '2008-04-08') and (value = 'One')) (type: boolean)
+ buckets included: [2,3,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key) IN (2, 3) and (ds = '2008-04-08') and (value = 'One')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), 'One' (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where value='One' and key in (2,3) and ds='2008-04-08'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where value='One' and key in (2,3) and ds='2008-04-08'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key) IN (2, 3) and (value = 'One') and (ds = '2008-04-08')) (type: boolean)
+ buckets included: [2,3,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key) IN (2, 3) and (value = 'One') and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), 'One' (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where (key=1 or key=2) and ds='2008-04-08'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where (key=1 or key=2) and ds='2008-04-08'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (((key = 1) or (key = 2)) and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (((key = 1) or (key = 2)) and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where (key=1 or key=2) and value = 'One' and ds='2008-04-08'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where (key=1 or key=2) and value = 'One' and ds='2008-04-08'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (((key = 1) or (key = 2)) and (value = 'One') and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (((key = 1) or (key = 2)) and (value = 'One') and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), 'One' (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: -- compat case (-15 = 1 & 15)
+
+explain extended
+select * from srcbucket_pruned where key = -15
+PREHOOK: type: QUERY
+POSTHOOK: query: -- compat case (-15 = 1 & 15)
+
+explain extended
+select * from srcbucket_pruned where key = -15
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (key = -15) (type: boolean)
+ buckets included: [1,15,] of 16
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key = -15) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: -15 (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: -- valid but irrelevant case (all buckets selected)
+
+explain extended
+select * from srcbucket_pruned where key in (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17)
+PREHOOK: type: QUERY
+POSTHOOK: query: -- valid but irrelevant case (all buckets selected)
+
+explain extended
+select * from srcbucket_pruned where key in (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17)
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (key) IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key) IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key in (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17) and ds='2008-04-08'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key in (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17) and ds='2008-04-08'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key) IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key) IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 94 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key in (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17) and ds='2008-04-08' and value='One'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key in (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17) and ds='2008-04-08' and value='One'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key) IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) and (ds = '2008-04-08') and (value = 'One')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key) IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) and (ds = '2008-04-08') and (value = 'One')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), 'One' (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where value='One' and key in (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17) and ds='2008-04-08'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where value='One' and key in (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17) and ds='2008-04-08'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key) IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) and (value = 'One') and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key) IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17) and (value = 'One') and (ds = '2008-04-08')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), 'One' (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 181 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: -- valid, but unimplemented cases
+
+explain extended
+select * from srcbucket_pruned where key = 1 and ds='2008-04-08' or key = 2
+PREHOOK: type: QUERY
+POSTHOOK: query: -- valid, but unimplemented cases
+
+explain extended
+select * from srcbucket_pruned where key = 1 and ds='2008-04-08' or key = 2
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: (((key = 1) and (ds = '2008-04-08')) or (key = 2)) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (((key = 1) and (ds = '2008-04-08')) or (key = 2)) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 and ds='2008-04-08' and (value='One' or value = 'Two')
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 and ds='2008-04-08' and (value='One' or value = 'Two')
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key = 1) and (ds = '2008-04-08') and ((value = 'One') or (value = 'Two'))) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key = 1) and (ds = '2008-04-08') and ((value = 'One') or (value = 'Two'))) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 1 (type: int), value (type: string), '2008-04-08' (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 98 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 98 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 or value = "One" or key = 2 and value = "Two"
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 or value = "One" or key = 2 and value = "Two"
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key = 1) or (value = 'One') or ((key = 2) and (value = 'Two'))) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key = 1) or (value = 'One') or ((key = 2) and (value = 'Two'))) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: -- Invalid cases
+
+explain extended
+select * from srcbucket_pruned where key = 'x11'
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Invalid cases
+
+explain extended
+select * from srcbucket_pruned where key = 'x11'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: false (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 or value = "One"
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 or value = "One"
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key = 1) or (value = 'One')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key = 1) or (value = 'One')) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 or value = "One" or key = 2
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_pruned where key = 1 or value = "One" or key = 2
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_pruned
+ filterExpr: ((key = 1) or (value = 'One') or (key = 2)) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: ((key = 1) or (value = 'One') or (key = 2)) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_unpruned where key in (3, 5)
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_unpruned where key in (3, 5)
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_unpruned
+ filterExpr: (key) IN (3, 5) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key) IN (3, 5) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain extended
+select * from srcbucket_unpruned where key = 1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain extended
+select * from srcbucket_unpruned where key = 1
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: srcbucket_unpruned
+ filterExpr: (key = 1) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ GatherStats: false
+ Filter Operator
+ isSamplingPred: false
+ predicate: (key = 1) (type: boolean)
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: 1 (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: PARTIAL
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1,_col2
+ columns.types int:string:string
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
+ Execution mode: llap
+ LLAP IO: unknown
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/column_names_with_leading_and_trailing_spaces.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/column_names_with_leading_and_trailing_spaces.q.out b/ql/src/test/results/clientpositive/llap/column_names_with_leading_and_trailing_spaces.q.out
new file mode 100644
index 0000000..18314b5
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/column_names_with_leading_and_trailing_spaces.q.out
@@ -0,0 +1,85 @@
+PREHOOK: query: create table space (` left` string, ` middle ` string, `right ` string)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@space
+POSTHOOK: query: create table space (` left` string, ` middle ` string, `right ` string)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@space
+PREHOOK: query: desc formatted space
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@space
+POSTHOOK: query: desc formatted space
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@space
+# col_name data_type comment
+
+ left string
+ middle string
+right string
+
+# Detailed Table Information
+Database: default
+#### A masked pattern was here ####
+Retention: 0
+#### A masked pattern was here ####
+Table Type: MANAGED_TABLE
+Table Parameters:
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 0
+ numRows 0
+ rawDataSize 0
+ totalSize 0
+#### A masked pattern was here ####
+
+# Storage Information
+SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+InputFormat: org.apache.hadoop.mapred.TextInputFormat
+OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+Compressed: No
+Num Buckets: -1
+Bucket Columns: []
+Sort Columns: []
+Storage Desc Params:
+ serialization.format 1
+PREHOOK: query: insert into space values ("1", "2", "3")
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__1
+PREHOOK: Output: default@space
+POSTHOOK: query: insert into space values ("1", "2", "3")
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__1
+POSTHOOK: Output: default@space
+POSTHOOK: Lineage: space. left SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: space. middle SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+POSTHOOK: Lineage: space.right SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col3, type:string, comment:), ]
+PREHOOK: query: select * from space
+PREHOOK: type: QUERY
+PREHOOK: Input: default@space
+#### A masked pattern was here ####
+POSTHOOK: query: select * from space
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@space
+#### A masked pattern was here ####
+1 2 3
+PREHOOK: query: insert into space (` middle `) values("2")
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__2
+PREHOOK: Output: default@space
+POSTHOOK: query: insert into space (` middle `) values("2")
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__2
+POSTHOOK: Output: default@space
+POSTHOOK: Lineage: space. left SIMPLE []
+POSTHOOK: Lineage: space. middle SIMPLE [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: space.right SIMPLE []
+PREHOOK: query: select * from space order by ` left`
+PREHOOK: type: QUERY
+PREHOOK: Input: default@space
+#### A masked pattern was here ####
+POSTHOOK: query: select * from space order by ` left`
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@space
+#### A masked pattern was here ####
+NULL 2 NULL
+1 2 3
[08/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/skewjoin.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/skewjoin.q.out b/ql/src/test/results/clientpositive/llap/skewjoin.q.out
index e5d851d..617045c 100644
--- a/ql/src/test/results/clientpositive/llap/skewjoin.q.out
+++ b/ql/src/test/results/clientpositive/llap/skewjoin.q.out
@@ -91,6 +91,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -99,39 +100,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: src2
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -141,15 +144,15 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col1, _col2
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col2
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: UDFToInteger(_col2) (type: int), _col1 (type: string)
+ expressions: UDFToInteger(_col0) (type: int), _col2 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 115805 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 115805 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -183,7 +186,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest_j1
POSTHOOK: Lineage: dest_j1.key EXPRESSION [(src)src1.FieldSchema(name:key, type:string, comment:default), ]
-POSTHOOK: Lineage: dest_j1.value SIMPLE [(src)src1.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: dest_j1.value SIMPLE [(src)src2.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: SELECT sum(hash(key)), sum(hash(value)) FROM dest_j1
PREHOOK: type: QUERY
PREHOOK: Input: default@dest_j1
@@ -212,6 +215,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -231,6 +235,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -246,6 +251,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -261,6 +267,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
@@ -276,6 +283,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -299,8 +307,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 3 Data size: 99 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -349,6 +357,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -368,6 +377,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -383,6 +393,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -398,6 +409,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 20 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
@@ -413,6 +425,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -436,8 +449,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 3 Data size: 99 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -478,6 +491,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -498,21 +512,23 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: key (type: string), val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key + 1) is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (key + 1) (type: double)
sort order: +
Map-reduce partition columns: (key + 1) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: key (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -523,7 +539,7 @@ STAGE PLANS:
0 UDFToDouble(key) (type: double)
1 (key + 1) (type: double)
outputColumnNames: _col0, _col1, _col5
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 47850 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(hash(_col0)), sum(hash(_col1)), sum(hash(_col5))
mode: hash
@@ -534,7 +550,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 24 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1), sum(VALUE._col2)
@@ -545,8 +561,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 24 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -587,6 +603,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -596,39 +613,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -638,35 +657,35 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col1, _col2
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: hash(_col0) (type: int), hash(_col1) (type: int)
+ expressions: hash(_col1) (type: int), hash(_col2) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -715,6 +734,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -724,39 +744,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: (key is not null and UDFToDouble(substring(value, 5)) is not null) (type: boolean)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ predicate: (key is not null and value is not null) (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), UDFToDouble(substring(_col1, 5)) (type: double)
sort order: ++
Map-reduce partition columns: _col0 (type: string), UDFToDouble(substring(_col1, 5)) (type: double)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: (key is not null and (UDFToDouble(substring(value, 5)) + 1.0) is not null) (type: boolean)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ predicate: (key is not null and value is not null) (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), (UDFToDouble(substring(_col1, 5)) + 1.0) (type: double)
sort order: ++
Map-reduce partition columns: _col0 (type: string), (UDFToDouble(substring(_col1, 5)) + 1.0) (type: double)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -767,34 +789,34 @@ STAGE PLANS:
0 _col0 (type: string), UDFToDouble(substring(_col1, 5)) (type: double)
1 _col0 (type: string), (UDFToDouble(substring(_col1, 5)) + 1.0) (type: double)
outputColumnNames: _col2, _col3
- Statistics: Num rows: 137 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col2) (type: int), hash(_col3) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1460 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 890 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -851,123 +873,111 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
- Reducer 5 <- Map 4 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: (((UDFToDouble(key) < 80.0) and (UDFToDouble(key) < 100.0)) and key is not null) (type: boolean)
- Statistics: Num rows: 28 Data size: 297 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((UDFToDouble(key) < 100.0) and (UDFToDouble(key) < 80.0)) (type: boolean)
+ Statistics: Num rows: 55 Data size: 4785 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 28 Data size: 297 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 4785 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 28 Data size: 297 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 4785 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: ((UDFToDouble(key) < 100.0) and (UDFToDouble(key) < 80.0)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((UDFToDouble(key) < 80.0) and (UDFToDouble(key) < 100.0)) (type: boolean)
+ Statistics: Num rows: 55 Data size: 4785 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 4785 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 4785 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
- Map 6
+ LLAP IO: no inputs
+ Map 5
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((UDFToDouble(key) < 100.0) and (UDFToDouble(key) < 80.0)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
+ Inner Join 0 to 2
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col1, _col3
- Statistics: Num rows: 66 Data size: 706 Basic stats: COMPLETE Column stats: NONE
+ 2 _col0 (type: string)
+ outputColumnNames: _col0, _col3
+ Statistics: Num rows: 314 Data size: 55892 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: hash(_col1) (type: int), hash(_col3) (type: int)
+ expressions: hash(_col0) (type: int), hash(_col3) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 66 Data size: 706 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 55892 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 5
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: string)
- 1 _col0 (type: string)
- outputColumnNames: _col0, _col2
- Statistics: Num rows: 60 Data size: 642 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 60 Data size: 642 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col2 (type: string)
Stage: Stage-0
Fetch Operator
@@ -1013,6 +1023,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1030,6 +1041,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: key (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -1042,6 +1054,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 30 Basic stats: COMPLETE Column stats: NONE
value expressions: val (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1063,7 +1076,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
@@ -1074,8 +1087,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/smb_cache.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/smb_cache.q.out b/ql/src/test/results/clientpositive/llap/smb_cache.q.out
new file mode 100644
index 0000000..932b211
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/smb_cache.q.out
@@ -0,0 +1,347 @@
+PREHOOK: query: create table bug_201_input_a (
+ userid int
+) clustered by (userid) sorted by (userid) into 64 BUCKETS
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@bug_201_input_a
+POSTHOOK: query: create table bug_201_input_a (
+ userid int
+) clustered by (userid) sorted by (userid) into 64 BUCKETS
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@bug_201_input_a
+PREHOOK: query: create table bug_201_input_b (
+ userid int
+) clustered by (userid) sorted by (userid) into 64 BUCKETS
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@bug_201_input_b
+POSTHOOK: query: create table bug_201_input_b (
+ userid int
+) clustered by (userid) sorted by (userid) into 64 BUCKETS
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@bug_201_input_b
+PREHOOK: query: insert overwrite table bug_201_input_a
+select distinct(userid) as userid from (
+ select arr as userid from (
+ select explode(array (
+ 12804352 ,251326720,50029057 ,251155969,60217858 ,251995906,78744835 ,250561795,13637380 ,207184132,58189573 ,251988997,
+ 62314246 ,251565574,63912199 ,250889479,57424648 ,208269832,39819529 ,251811337,39396106 ,250124554,25833739 ,251722507,
+ 48908812 ,252057100,39894541 ,251633677,20268046 ,251462926,46375183 ,251292175,64902160 ,251879440,40980241 ,204206353,
+ 50411026 ,251790610,54030355 ,251367187,29097748 ,205968148,12755989 ,250773013,45685270 ,208406038,47788567 ,208235287,
+ 57724696 ,207559192,13083673 ,206630425,41717530 ,250929946,50642971 ,250759195,43144732 ,250841116,51059485 ,250670365,
+ 39771166 ,250752286,53244703 ,250834207,43725088 ,207961888,46586401 ,207285793,77241634 ,251837986,19967011 ,251919907,
+ 37230628 ,251496484,47419429 ,251578405,50786086 ,250144294,48593959 ,251742247,44885800 ,251318824,52295209 ,251400745,
+ 66274090 ,251988010,28960555 ,208357675,51024940 ,206923564,32156461 ,251981101,47398702 ,252063022,48238639 ,251386927,
+ 77377840 ,251468848,64068145 ,207333169,16142386 ,251380018,15971635 ,251461939,60018484 ,250027828,43171381 ,250867765,
+ 47548726 ,250191670,37776439 ,208330039,59588152 ,251871544,75335737 ,251953465,57477946 ,251782714,63623995 ,250348603,
+ 57641788 ,250935868,42058045 ,250007101,59574334 ,251352382,42474559 ,250928959,52663360 ,252021568,60578113 ,251598145,
+ 60407362 ,251174722,55941187 ,208302403,65119300 ,251843908,61916485 ,251673157,65535814 ,207790150,62838343 ,208124743,
+ 26030152 ,251666248,45315145 ,204498505,59799370 ,251577418,72514891 ,250648651,64258636 ,208281676,53475661 ,251823181,
+ 30564430 ,207940174,32162383 ,250976335,62059600 ,251563600,64920913 ,251392849,50347858 ,250716754,40828243 ,251304019,
+ 44447572 ,204388948,14966869 ,251973205,59013718 ,250791766,12856663 ,251631703,40227160 ,250450264,20600665 ,252048217,
+ 48223834 ,251119450,76099675 ,207741787,49145692 ,250019932,50490973 ,207905629,58405726 ,250689118,47370079 ,250013023,
+ 44419936 ,250347616,36416353 ,250934881,62776162 ,251269474,48455779 ,251098723,43736932 ,251938660,12740197 ,251767909,
+ 54007654 ,252102502,49794151 ,251931751,63267688 ,252013672,28480873 ,252095593,63684202 ,251419498,76652395 ,252006763,
+ 64606060 ,251077996,53317741 ,251665261,21310318 ,250989166,41353327 ,208116847,52805488 ,208451440,43033201 ,251992945,
+ 48673906 ,251064178,15655795 ,207686515,44794996 ,251228020,14303605 ,251815285,48243574 ,251897206,65507191 ,207508855,
+ 63820408 ,250797688,45457273 ,251637625,62973562 ,251466874,42083707 ,252054139,63642748 ,251630716,40731517 ,251712637,
+ 29948542 ,252047230,40137343 ,250360447,43756672 ,251200384,13517953 ,251534977,77273218 ,251616898,54867331 ,250435459,
+ 46105732 ,250517380,48714373 ,208150405,79369606 ,250933894,57216391 ,250257799,20408200 ,250845064,61675657 ,206962057,
+ 15013258 ,250756234,54512011 ,250585483,70512268 ,251172748,78932365 ,251254669,12814222 , 251841934,49533583 , 251923855,
+ 44309392 , 208040848,73953937 , 250824337,56601490 , 251664274,35458963 , 208033939,50953876 , 251828116,74281621 , 204155029,
+ 64509334 , 250223254,44630167 , 252073879,52292248 , 207685528,63997081 , 251732377,46897306 , 250803610,60370843 , 251138203,
+ 65506204 , 251725468,48406429 , 205821085,15893662 , 250878622,38716063 , 251213215,35007904 , 251295136,66926497 , 251629729,
+ 56143522 , 251964322,73407139 , 250277539,71720356 , 251117476,80645797 , 205465765,43079590 , 250270630,75756199 , 208156327,
+ 58656424 , 251445160,57474985 , 251779753,45428650 , 251356330,62186923 , 250680235,54183340 , 250004140,55781293 , 207637165,
+ 15941038 , 251684014,14254255 , 251513263,42130096 , 251342512,34884529 , 208217521,70087858 , 251253682,50208691 , 250830259,
+ 54333364 , 250406836,17019829 , 251752117,75468982 , 251834038,17689015 , 208203703,22571704 , 250481848,52974265 , 250816441,
+ 52803514 , 251656378,61981627 , 251485627,13045180 , 250556860,71494333 , 250133437,81935806 , 251478718,50433727 , 252065983,
+ 26764480 , 250631872,64747201 , 251977153,64829122 , 206830786,47982019 , 251888323,64992964 , 207499972,52188613 , 251799493,
+ 39384262 , 251376070,41992903 , 250699975,41822152 , 250781896,39882697 , 250358473,56135626 , 251198410,35751115 , 251785675,
+ 75249868 , 251867596,55118029 , 207479245,70107598 , 251526094,50481103 , 207895759,55869136 , 251689936,45086161 , 251519185,
+ 46431442 , 251348434,53335507 , 251683027,39520468 , 251512276,53246677 , 250836181,42211030 , 251928790,56442583 , 251252695,
+ 75727576 , 251334616,15673561 , 250911193,42286042 , 250993114,63845083 , 251833051,37396444 , 251156956,77653213 , 251744221,
+ 16335838 , 251068126,61393375 , 250897375,53642464 , 250221280,58525153 , 206843617,56585698 , 207683554,30642403 , 250972387,
+ 63319012 , 250801636,76287205 , 207929317,44785126 , 251723494,50425831 , 251300071,45706984 , 251634664,13952233 , 251463913,
+ 77960170 , 251293162,27760363 , 251627755,69280492 , 206734060,13521901 , 208332013,35333614 , 250862830,56892655 , 251702767,
+ 49394416 , 251532016,79291633 , 250097905,50316274 , 206467570,45344755 , 251525107,17127412 , 250596340,53846773 , 206966005,
+ 53170678 , 251012854,46430455 , 252105463,59651320 , 251682040,54427129 , 251763961,54509050 , 252098554,65203195 , 251422459,
+ 40270588 , 250241020,13316605 , 208379389,45235198 , 251668222,60477439 , 251497471,
+ 101510977 ,114200836 ,156174985 ,60512971 ,181554703 ,
+ 148365841 ,52703827 ,182530846 ,149341984 ,116153122 ,
+ 162031843 ,141532840 ,154222699 ,109320121 ,155198842
+ )) as arr )a )b
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+PREHOOK: Output: default@bug_201_input_a
+POSTHOOK: query: insert overwrite table bug_201_input_a
+select distinct(userid) as userid from (
+ select arr as userid from (
+ select explode(array (
+ 12804352 ,251326720,50029057 ,251155969,60217858 ,251995906,78744835 ,250561795,13637380 ,207184132,58189573 ,251988997,
+ 62314246 ,251565574,63912199 ,250889479,57424648 ,208269832,39819529 ,251811337,39396106 ,250124554,25833739 ,251722507,
+ 48908812 ,252057100,39894541 ,251633677,20268046 ,251462926,46375183 ,251292175,64902160 ,251879440,40980241 ,204206353,
+ 50411026 ,251790610,54030355 ,251367187,29097748 ,205968148,12755989 ,250773013,45685270 ,208406038,47788567 ,208235287,
+ 57724696 ,207559192,13083673 ,206630425,41717530 ,250929946,50642971 ,250759195,43144732 ,250841116,51059485 ,250670365,
+ 39771166 ,250752286,53244703 ,250834207,43725088 ,207961888,46586401 ,207285793,77241634 ,251837986,19967011 ,251919907,
+ 37230628 ,251496484,47419429 ,251578405,50786086 ,250144294,48593959 ,251742247,44885800 ,251318824,52295209 ,251400745,
+ 66274090 ,251988010,28960555 ,208357675,51024940 ,206923564,32156461 ,251981101,47398702 ,252063022,48238639 ,251386927,
+ 77377840 ,251468848,64068145 ,207333169,16142386 ,251380018,15971635 ,251461939,60018484 ,250027828,43171381 ,250867765,
+ 47548726 ,250191670,37776439 ,208330039,59588152 ,251871544,75335737 ,251953465,57477946 ,251782714,63623995 ,250348603,
+ 57641788 ,250935868,42058045 ,250007101,59574334 ,251352382,42474559 ,250928959,52663360 ,252021568,60578113 ,251598145,
+ 60407362 ,251174722,55941187 ,208302403,65119300 ,251843908,61916485 ,251673157,65535814 ,207790150,62838343 ,208124743,
+ 26030152 ,251666248,45315145 ,204498505,59799370 ,251577418,72514891 ,250648651,64258636 ,208281676,53475661 ,251823181,
+ 30564430 ,207940174,32162383 ,250976335,62059600 ,251563600,64920913 ,251392849,50347858 ,250716754,40828243 ,251304019,
+ 44447572 ,204388948,14966869 ,251973205,59013718 ,250791766,12856663 ,251631703,40227160 ,250450264,20600665 ,252048217,
+ 48223834 ,251119450,76099675 ,207741787,49145692 ,250019932,50490973 ,207905629,58405726 ,250689118,47370079 ,250013023,
+ 44419936 ,250347616,36416353 ,250934881,62776162 ,251269474,48455779 ,251098723,43736932 ,251938660,12740197 ,251767909,
+ 54007654 ,252102502,49794151 ,251931751,63267688 ,252013672,28480873 ,252095593,63684202 ,251419498,76652395 ,252006763,
+ 64606060 ,251077996,53317741 ,251665261,21310318 ,250989166,41353327 ,208116847,52805488 ,208451440,43033201 ,251992945,
+ 48673906 ,251064178,15655795 ,207686515,44794996 ,251228020,14303605 ,251815285,48243574 ,251897206,65507191 ,207508855,
+ 63820408 ,250797688,45457273 ,251637625,62973562 ,251466874,42083707 ,252054139,63642748 ,251630716,40731517 ,251712637,
+ 29948542 ,252047230,40137343 ,250360447,43756672 ,251200384,13517953 ,251534977,77273218 ,251616898,54867331 ,250435459,
+ 46105732 ,250517380,48714373 ,208150405,79369606 ,250933894,57216391 ,250257799,20408200 ,250845064,61675657 ,206962057,
+ 15013258 ,250756234,54512011 ,250585483,70512268 ,251172748,78932365 ,251254669,12814222 , 251841934,49533583 , 251923855,
+ 44309392 , 208040848,73953937 , 250824337,56601490 , 251664274,35458963 , 208033939,50953876 , 251828116,74281621 , 204155029,
+ 64509334 , 250223254,44630167 , 252073879,52292248 , 207685528,63997081 , 251732377,46897306 , 250803610,60370843 , 251138203,
+ 65506204 , 251725468,48406429 , 205821085,15893662 , 250878622,38716063 , 251213215,35007904 , 251295136,66926497 , 251629729,
+ 56143522 , 251964322,73407139 , 250277539,71720356 , 251117476,80645797 , 205465765,43079590 , 250270630,75756199 , 208156327,
+ 58656424 , 251445160,57474985 , 251779753,45428650 , 251356330,62186923 , 250680235,54183340 , 250004140,55781293 , 207637165,
+ 15941038 , 251684014,14254255 , 251513263,42130096 , 251342512,34884529 , 208217521,70087858 , 251253682,50208691 , 250830259,
+ 54333364 , 250406836,17019829 , 251752117,75468982 , 251834038,17689015 , 208203703,22571704 , 250481848,52974265 , 250816441,
+ 52803514 , 251656378,61981627 , 251485627,13045180 , 250556860,71494333 , 250133437,81935806 , 251478718,50433727 , 252065983,
+ 26764480 , 250631872,64747201 , 251977153,64829122 , 206830786,47982019 , 251888323,64992964 , 207499972,52188613 , 251799493,
+ 39384262 , 251376070,41992903 , 250699975,41822152 , 250781896,39882697 , 250358473,56135626 , 251198410,35751115 , 251785675,
+ 75249868 , 251867596,55118029 , 207479245,70107598 , 251526094,50481103 , 207895759,55869136 , 251689936,45086161 , 251519185,
+ 46431442 , 251348434,53335507 , 251683027,39520468 , 251512276,53246677 , 250836181,42211030 , 251928790,56442583 , 251252695,
+ 75727576 , 251334616,15673561 , 250911193,42286042 , 250993114,63845083 , 251833051,37396444 , 251156956,77653213 , 251744221,
+ 16335838 , 251068126,61393375 , 250897375,53642464 , 250221280,58525153 , 206843617,56585698 , 207683554,30642403 , 250972387,
+ 63319012 , 250801636,76287205 , 207929317,44785126 , 251723494,50425831 , 251300071,45706984 , 251634664,13952233 , 251463913,
+ 77960170 , 251293162,27760363 , 251627755,69280492 , 206734060,13521901 , 208332013,35333614 , 250862830,56892655 , 251702767,
+ 49394416 , 251532016,79291633 , 250097905,50316274 , 206467570,45344755 , 251525107,17127412 , 250596340,53846773 , 206966005,
+ 53170678 , 251012854,46430455 , 252105463,59651320 , 251682040,54427129 , 251763961,54509050 , 252098554,65203195 , 251422459,
+ 40270588 , 250241020,13316605 , 208379389,45235198 , 251668222,60477439 , 251497471,
+ 101510977 ,114200836 ,156174985 ,60512971 ,181554703 ,
+ 148365841 ,52703827 ,182530846 ,149341984 ,116153122 ,
+ 162031843 ,141532840 ,154222699 ,109320121 ,155198842
+ )) as arr )a )b
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+POSTHOOK: Output: default@bug_201_input_a
+POSTHOOK: Lineage: bug_201_input_a.userid SCRIPT []
+PREHOOK: query: insert overwrite table bug_201_input_b
+select distinct(userid) as userid from (
+ select arr as userid from (
+ select explode(array (
+ 55632256 ,243051712 ,39037825 ,163984129 ,22443394 ,147389698 ,68322115 ,193268419,
+ 51727684 ,176673988 ,35133253 ,160079557 ,81011974 ,205958278 ,64417543 ,251836999,
+ 47823112 ,172769416 ,31228681 ,156174985 ,14634250 ,202053706 ,60512971 ,247932427,
+ 43918540 ,168864844 ,89797261 ,152270413 ,73202830 ,73202830 ,56608399 ,244027855,
+ 40013968 ,164960272 ,85892689 ,148365841 ,69298258 ,69298258 ,52703827 ,177650131,
+ 36109396 ,161055700 ,19514965 ,206934421 ,65393686 ,190339990 ,48799255 ,111272407,
+ 94677976 ,157151128 ,15610393 ,203029849 ,61489114 ,248908570 ,44894683 ,169840987,
+ 90773404 ,153246556 ,11705821 ,74178973 ,57584542 ,245003998 ,40990111 ,165936415,
+ 24395680 ,149341984 ,70274401 ,70274401 ,53679970 ,178626274 ,37085539 ,162031843,
+ 145437412 ,207910564 ,66369829 ,191316133 ,49775398 ,112248550 ,33180967 ,158127271,
+ 79059688 ,204005992 ,62465257 ,187411561 ,45870826 ,170817130 ,29276395 ,154222699,
+ 12681964 ,75155116 ,58560685 ,245980141 ,41966254 ,166912558 ,150318127 ,150318127,
+ 8777392 ,71250544 ,54656113 ,179602417 ,38061682 ,163007986 ,146413555 ,146413555,
+ 67345972 ,192292276 ,50751541 ,175697845 ,34157110 ,159103414 ,17562679 ,204982135,
+ 63441400 ,250860856 ,46846969 ,171793273 ,30252538 ,155198842 ,76131259 ,201077563,
+ 59536828 ,246956284 ,42942397 ,167888701 ,26347966 ,151294270 ,9753535 ,72226687,
+ 101510977 ,114200836 ,156174985 ,60512971 ,181554703 ,
+ 148365841 ,52703827 ,182530846 ,149341984 ,116153122 ,
+ 162031843 ,141532840 ,154222699 ,109320121 ,155198842
+ )) as arr )a ) b
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+PREHOOK: Output: default@bug_201_input_b
+POSTHOOK: query: insert overwrite table bug_201_input_b
+select distinct(userid) as userid from (
+ select arr as userid from (
+ select explode(array (
+ 55632256 ,243051712 ,39037825 ,163984129 ,22443394 ,147389698 ,68322115 ,193268419,
+ 51727684 ,176673988 ,35133253 ,160079557 ,81011974 ,205958278 ,64417543 ,251836999,
+ 47823112 ,172769416 ,31228681 ,156174985 ,14634250 ,202053706 ,60512971 ,247932427,
+ 43918540 ,168864844 ,89797261 ,152270413 ,73202830 ,73202830 ,56608399 ,244027855,
+ 40013968 ,164960272 ,85892689 ,148365841 ,69298258 ,69298258 ,52703827 ,177650131,
+ 36109396 ,161055700 ,19514965 ,206934421 ,65393686 ,190339990 ,48799255 ,111272407,
+ 94677976 ,157151128 ,15610393 ,203029849 ,61489114 ,248908570 ,44894683 ,169840987,
+ 90773404 ,153246556 ,11705821 ,74178973 ,57584542 ,245003998 ,40990111 ,165936415,
+ 24395680 ,149341984 ,70274401 ,70274401 ,53679970 ,178626274 ,37085539 ,162031843,
+ 145437412 ,207910564 ,66369829 ,191316133 ,49775398 ,112248550 ,33180967 ,158127271,
+ 79059688 ,204005992 ,62465257 ,187411561 ,45870826 ,170817130 ,29276395 ,154222699,
+ 12681964 ,75155116 ,58560685 ,245980141 ,41966254 ,166912558 ,150318127 ,150318127,
+ 8777392 ,71250544 ,54656113 ,179602417 ,38061682 ,163007986 ,146413555 ,146413555,
+ 67345972 ,192292276 ,50751541 ,175697845 ,34157110 ,159103414 ,17562679 ,204982135,
+ 63441400 ,250860856 ,46846969 ,171793273 ,30252538 ,155198842 ,76131259 ,201077563,
+ 59536828 ,246956284 ,42942397 ,167888701 ,26347966 ,151294270 ,9753535 ,72226687,
+ 101510977 ,114200836 ,156174985 ,60512971 ,181554703 ,
+ 148365841 ,52703827 ,182530846 ,149341984 ,116153122 ,
+ 162031843 ,141532840 ,154222699 ,109320121 ,155198842
+ )) as arr )a ) b
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+POSTHOOK: Output: default@bug_201_input_b
+POSTHOOK: Lineage: bug_201_input_b.userid SCRIPT []
+PREHOOK: query: explain
+select
+t1.userid,
+ fa.userid as fa_userid
+ from bug_201_input_b as t1
+ join bug_201_input_a as fa on (t1.userid = fa.userid)
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+t1.userid,
+ fa.userid as fa_userid
+ from bug_201_input_b as t1
+ join bug_201_input_a as fa on (t1.userid = fa.userid)
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Merge Join Operator [MERGEJOIN_15] (rows=579 width=8)
+ Conds:RS_6._col0=RS_7._col0(Inner),Output:["_col0","_col1"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=130 width=8)
+ Output:["_col0"]
+ Filter Operator [FIL_13] (rows=130 width=8)
+ predicate:userid is not null
+ TableScan [TS_0] (rows=130 width=8)
+ default@bug_201_input_b,t1,Tbl:COMPLETE,Col:NONE,Output:["userid"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=527 width=8)
+ Output:["_col0"]
+ Filter Operator [FIL_14] (rows=527 width=8)
+ predicate:userid is not null
+ TableScan [TS_3] (rows=527 width=8)
+ default@bug_201_input_a,fa,Tbl:COMPLETE,Col:NONE,Output:["userid"]
+
+PREHOOK: query: select
+t1.userid,
+ fa.userid as fa_userid
+ from bug_201_input_b as t1
+ join bug_201_input_a as fa on (t1.userid = fa.userid)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@bug_201_input_a
+PREHOOK: Input: default@bug_201_input_b
+#### A masked pattern was here ####
+POSTHOOK: query: select
+t1.userid,
+ fa.userid as fa_userid
+ from bug_201_input_b as t1
+ join bug_201_input_a as fa on (t1.userid = fa.userid)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@bug_201_input_a
+POSTHOOK: Input: default@bug_201_input_b
+#### A masked pattern was here ####
+52703827 52703827
+60512971 60512971
+101510977 101510977
+109320121 109320121
+114200836 114200836
+116153122 116153122
+141532840 141532840
+148365841 148365841
+149341984 149341984
+154222699 154222699
+155198842 155198842
+156174985 156174985
+162031843 162031843
+181554703 181554703
+182530846 182530846
+Warning: Value had a \n character in it.
+PREHOOK: query: explain
+select
+t1.userid,
+ fa.userid as fa_userid
+ from bug_201_input_b as t1
+ join bug_201_input_a as fa on (t1.userid = fa.userid)
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+t1.userid,
+ fa.userid as fa_userid
+ from bug_201_input_b as t1
+ join bug_201_input_a as fa on (t1.userid = fa.userid)
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Merge Join Operator [MERGEJOIN_15] (rows=579 width=8)
+ Conds:RS_6._col0=RS_7._col0(Inner),Output:["_col0","_col1"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=130 width=8)
+ Output:["_col0"]
+ Filter Operator [FIL_13] (rows=130 width=8)
+ predicate:userid is not null
+ TableScan [TS_0] (rows=130 width=8)
+ default@bug_201_input_b,t1,Tbl:COMPLETE,Col:NONE,Output:["userid"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=527 width=8)
+ Output:["_col0"]
+ Filter Operator [FIL_14] (rows=527 width=8)
+ predicate:userid is not null
+ TableScan [TS_3] (rows=527 width=8)
+ default@bug_201_input_a,fa,Tbl:COMPLETE,Col:NONE,Output:["userid"]
+
+PREHOOK: query: select
+t1.userid,
+ fa.userid as fa_userid
+ from bug_201_input_b as t1
+ join bug_201_input_a as fa on (t1.userid = fa.userid)
+PREHOOK: type: QUERY
+PREHOOK: Input: default@bug_201_input_a
+PREHOOK: Input: default@bug_201_input_b
+#### A masked pattern was here ####
+POSTHOOK: query: select
+t1.userid,
+ fa.userid as fa_userid
+ from bug_201_input_b as t1
+ join bug_201_input_a as fa on (t1.userid = fa.userid)
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@bug_201_input_a
+POSTHOOK: Input: default@bug_201_input_b
+#### A masked pattern was here ####
+52703827 52703827
+60512971 60512971
+101510977 101510977
+109320121 109320121
+114200836 114200836
+116153122 116153122
+141532840 141532840
+148365841 148365841
+149341984 149341984
+154222699 154222699
+155198842 155198842
+156174985 156174985
+162031843 162031843
+181554703 181554703
+182530846 182530846
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/stats_noscan_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/stats_noscan_1.q.out b/ql/src/test/results/clientpositive/llap/stats_noscan_1.q.out
index 96e0e43..561b263 100644
--- a/ql/src/test/results/clientpositive/llap/stats_noscan_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/stats_noscan_1.q.out
@@ -101,10 +101,8 @@ Database: default
Table: analyze_srcpart
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
- numRows -1
- rawDataSize -1
totalSize 5812
#### A masked pattern was here ####
@@ -141,10 +139,8 @@ Database: default
Table: analyze_srcpart
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
- numRows -1
- rawDataSize -1
totalSize 5812
#### A masked pattern was here ####
@@ -181,10 +177,7 @@ Database: default
Table: analyze_srcpart
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE false
numFiles 1
- numRows -1
- rawDataSize -1
totalSize 5812
#### A masked pattern was here ####
@@ -221,10 +214,7 @@ Database: default
Table: analyze_srcpart
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE false
numFiles 1
- numRows -1
- rawDataSize -1
totalSize 5812
#### A masked pattern was here ####
@@ -373,10 +363,8 @@ Database: default
Table: analyze_srcpart_partial
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
- numRows -1
- rawDataSize -1
totalSize 5812
#### A masked pattern was here ####
@@ -413,10 +401,8 @@ Database: default
Table: analyze_srcpart_partial
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
- numRows -1
- rawDataSize -1
totalSize 5812
#### A masked pattern was here ####
@@ -453,10 +439,7 @@ Database: default
Table: analyze_srcpart_partial
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE false
numFiles 1
- numRows -1
- rawDataSize -1
totalSize 5812
#### A masked pattern was here ####
@@ -493,10 +476,7 @@ Database: default
Table: analyze_srcpart_partial
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE false
numFiles 1
- numRows -1
- rawDataSize -1
totalSize 5812
#### A masked pattern was here ####
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/stats_only_null.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/stats_only_null.q.out b/ql/src/test/results/clientpositive/llap/stats_only_null.q.out
index fe26cf5..d59d4ac 100644
--- a/ql/src/test/results/clientpositive/llap/stats_only_null.q.out
+++ b/ql/src/test/results/clientpositive/llap/stats_only_null.q.out
@@ -79,6 +79,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -102,8 +103,9 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 40 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: bigint), _col3 (type: bigint), _col4 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1), count(VALUE._col2), count(VALUE._col3), count(VALUE._col4)
@@ -114,8 +116,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 40 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -137,6 +139,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -145,11 +148,11 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: stats_null_part
- Statistics: Num rows: 10 Data size: 120 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 200 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: a (type: double), b (type: int), c (type: string), d (type: smallint)
outputColumnNames: a, b, c, d
- Statistics: Num rows: 10 Data size: 120 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 200 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), count(a), count(b), count(c), count(d)
mode: hash
@@ -160,8 +163,9 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 40 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint), _col1 (type: bigint), _col2 (type: bigint), _col3 (type: bigint), _col4 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1), count(VALUE._col2), count(VALUE._col3), count(VALUE._col4)
@@ -172,8 +176,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 40 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -234,7 +238,7 @@ Database: default
Table: stats_null_part
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\",\"c\":\"true\",\"d\":\"true\"}}
numFiles 1
numRows 6
rawDataSize 71
@@ -275,7 +279,7 @@ Database: default
Table: stats_null_part
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\",\"c\":\"true\",\"d\":\"true\"}}
numFiles 1
numRows 4
rawDataSize 49
@@ -387,10 +391,10 @@ POSTHOOK: Input: default@stats_null_part
POSTHOOK: Input: default@stats_null_part@dt=1
POSTHOOK: Input: default@stats_null_part@dt=__HIVE_DEFAULT_PARTITION__
#### A masked pattern was here ####
-PREHOOK: query: describe formatted stats_null_part a partition(dt = 1)
+PREHOOK: query: describe formatted stats_null_part partition(dt = 1) a
PREHOOK: type: DESCTABLE
PREHOOK: Input: default@stats_null_part
-POSTHOOK: query: describe formatted stats_null_part a partition(dt = 1)
+POSTHOOK: query: describe formatted stats_null_part partition(dt = 1) a
POSTHOOK: type: DESCTABLE
POSTHOOK: Input: default@stats_null_part
# col_name data_type min max num_nulls distinct_count avg_col_len max_col_len num_trues num_falses comment
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/subquery_exists.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/subquery_exists.q.out b/ql/src/test/results/clientpositive/llap/subquery_exists.q.out
index 6b1374a..1a006d8 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_exists.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_exists.q.out
@@ -31,6 +31,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -39,43 +40,45 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((value > 'val_9') and key is not null) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
- key expressions: _col1 (type: string), _col0 (type: string)
+ key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Map-reduce partition columns: _col1 (type: string), _col0 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: a
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((value > 'val_9') and key is not null) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: value (type: string), key (type: string)
+ expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -83,16 +86,16 @@ STAGE PLANS:
condition map:
Left Semi Join 0 to 1
keys:
- 0 _col1 (type: string), _col0 (type: string)
+ 0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 91 Data size: 969 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 91 Data size: 969 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -212,3 +215,104 @@ POSTHOOK: Input: default@src
97 val_97
98 val_98
98 val_98
+PREHOOK: query: -- upper case in subq
+explain
+select *
+from src b
+where exists
+ (select a.key
+ from src a
+ where b.VALUE = a.VALUE
+ )
+PREHOOK: type: QUERY
+POSTHOOK: query: -- upper case in subq
+explain
+select *
+from src b
+where exists
+ (select a.key
+ from src a
+ where b.VALUE = a.VALUE
+ )
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: b
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: value is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: string)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 3
+ Map Operator Tree:
+ TableScan
+ alias: a
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: value is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 214 Data size: 19474 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 214 Data size: 19474 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Merge Join Operator
+ condition map:
+ Left Semi Join 0 to 1
+ keys:
+ 0 _col1 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
[19/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/metadataonly1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/metadataonly1.q.out b/ql/src/test/results/clientpositive/llap/metadataonly1.q.out
index e1dd7f2..98261bc 100644
--- a/ql/src/test/results/clientpositive/llap/metadataonly1.q.out
+++ b/ql/src/test/results/clientpositive/llap/metadataonly1.q.out
@@ -10,25 +10,6 @@ PREHOOK: query: explain extended select max(ds) from TEST1
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select max(ds) from TEST1
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST1
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- max
- TOK_TABLE_OR_COL
- ds
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -36,6 +17,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -44,48 +26,51 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string)
auto parallelism: false
Execution mode: llap
+ LLAP IO: unknown
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -119,25 +104,6 @@ PREHOOK: query: explain extended select max(ds) from TEST1
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select max(ds) from TEST1
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST1
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- max
- TOK_TABLE_OR_COL
- ds
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -145,6 +111,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -153,44 +120,52 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string)
auto parallelism: false
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10004default.test1{ds=1} [test1]
+ nullscan://null/default.test1/part_ds=1_ [test1]
Path -> Partition:
- -mr-10004default.test1{ds=1}
+ nullscan://null/default.test1/part_ds=1_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
ds 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -213,31 +188,32 @@ STAGE PLANS:
name: default.test1
name: default.test1
Truncated Path -> Alias:
- -mr-10004default.test1{ds=1} [test1]
+ nullscan://null/default.test1/part_ds=1_ [test1]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -266,25 +242,6 @@ PREHOOK: query: explain extended select count(distinct ds) from TEST1
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(distinct ds) from TEST1
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST1
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONDI
- count
- TOK_TABLE_OR_COL
- ds
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -292,6 +249,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -300,45 +258,53 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT ds)
keys: ds (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
+ null sort order: a
sort order: +
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
auto parallelism: false
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10004default.test1{ds=1} [test1]
+ nullscan://null/default.test1/part_ds=1_ [test1]
Path -> Partition:
- -mr-10004default.test1{ds=1}
+ nullscan://null/default.test1/part_ds=1_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
ds 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -361,31 +327,32 @@ STAGE PLANS:
name: default.test1
name: default.test1
Truncated Path -> Alias:
- -mr-10004default.test1{ds=1} [test1]
+ nullscan://null/default.test1/part_ds=1_ [test1]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col0:0._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -414,25 +381,6 @@ PREHOOK: query: explain extended select count(ds) from TEST1
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(ds) from TEST1
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST1
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- count
- TOK_TABLE_OR_COL
- ds
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -440,6 +388,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -448,24 +397,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: bigint)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -477,17 +428,22 @@ STAGE PLANS:
partition values:
ds 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -512,29 +468,30 @@ STAGE PLANS:
Truncated Path -> Alias:
/test1/ds=1 [test1]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -572,52 +529,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain extended
select count(*) from TEST1 a2 join (select max(ds) m from TEST1) b on a2.ds=b.m
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- TEST1
- a2
- TOK_SUBQUERY
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST1
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- max
- TOK_TABLE_OR_COL
- ds
- m
- b
- =
- .
- TOK_TABLE_OR_COL
- a2
- ds
- .
- TOK_TABLE_OR_COL
- b
- m
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -625,6 +536,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -635,20 +547,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
tag: 0
auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -660,17 +574,22 @@ STAGE PLANS:
partition values:
ds 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -700,17 +619,22 @@ STAGE PLANS:
partition values:
ds 2
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -738,25 +662,27 @@ STAGE PLANS:
Map 4
Map Operator Tree:
TableScan
- alias: a2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ alias: test1
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -768,17 +694,22 @@ STAGE PLANS:
partition values:
ds 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -808,17 +739,22 @@ STAGE PLANS:
partition values:
ds 2
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -841,8 +777,8 @@ STAGE PLANS:
name: default.test1
name: default.test1
Truncated Path -> Alias:
- /test1/ds=1 [a2]
- /test1/ds=2 [a2]
+ /test1/ds=1 [test1]
+ /test1/ds=2 [test1]
Reducer 2
Execution mode: llap
Needs Tagging: false
@@ -853,43 +789,45 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Position of Big Table: 0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Position of Big Table: 1
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: bigint)
auto parallelism: false
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -904,32 +842,33 @@ STAGE PLANS:
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
isSamplingPred: false
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
tag: 1
auto parallelism: true
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Dynamic Partitioning Event Operator
Target column: ds (string)
Target Input: a2
Partition key expr: ds
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Target Vertex: Map 1
Stage: Stage-0
@@ -984,31 +923,6 @@ PREHOOK: query: explain extended select ds, count(distinct hr) from TEST2 group
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select ds, count(distinct hr) from TEST2 group by ds
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST2
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- ds
- TOK_SELEXPR
- TOK_FUNCTIONDI
- count
- TOK_TABLE_OR_COL
- hr
- TOK_GROUPBY
- TOK_TABLE_OR_COL
- ds
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -1016,6 +930,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1024,31 +939,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: ds, hr
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT hr)
keys: ds (type: string), hr (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 376 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
+ null sort order: aa
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 376 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
auto parallelism: true
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10004default.test2{ds=1, hr=1} [test2]
- -mr-10005default.test2{ds=1, hr=2} [test2]
- -mr-10006default.test2{ds=1, hr=3} [test2]
+ nullscan://null/default.test2/part_ds=1_hr=1_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=2_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=3_ [test2]
Path -> Partition:
- -mr-10004default.test2{ds=1, hr=1}
+ nullscan://null/default.test2/part_ds=1_hr=1_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1056,17 +974,22 @@ STAGE PLANS:
ds 1
hr 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1088,7 +1011,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test2
name: default.test2
- -mr-10005default.test2{ds=1, hr=2}
+ nullscan://null/default.test2/part_ds=1_hr=2_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1096,17 +1019,22 @@ STAGE PLANS:
ds 1
hr 2
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1128,7 +1056,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test2
name: default.test2
- -mr-10006default.test2{ds=1, hr=3}
+ nullscan://null/default.test2/part_ds=1_hr=3_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1136,17 +1064,22 @@ STAGE PLANS:
ds 1
hr 3
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1169,11 +1102,11 @@ STAGE PLANS:
name: default.test2
name: default.test2
Truncated Path -> Alias:
- -mr-10004default.test2{ds=1, hr=1} [test2]
- -mr-10005default.test2{ds=1, hr=2} [test2]
- -mr-10006default.test2{ds=1, hr=3} [test2]
+ nullscan://null/default.test2/part_ds=1_hr=1_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=2_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=3_ [test2]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -1181,22 +1114,23 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1
columns.types string:bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1229,31 +1163,6 @@ PREHOOK: query: explain extended select ds, count(hr) from TEST2 group by ds
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select ds, count(hr) from TEST2 group by ds
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST2
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- ds
- TOK_SELEXPR
- TOK_FUNCTION
- count
- TOK_TABLE_OR_COL
- hr
- TOK_GROUPBY
- TOK_TABLE_OR_COL
- ds
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -1261,6 +1170,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1269,27 +1179,29 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: ds, hr
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(hr)
keys: ds (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col1 (type: bigint)
auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -1302,17 +1214,22 @@ STAGE PLANS:
ds 1
hr 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1343,17 +1260,22 @@ STAGE PLANS:
ds 1
hr 2
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1384,17 +1306,22 @@ STAGE PLANS:
ds 1
hr 3
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1421,7 +1348,7 @@ STAGE PLANS:
/test2/ds=1/hr=2 [test2]
/test2/ds=1/hr=3 [test2]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -1429,22 +1356,23 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1
columns.types string:bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1476,25 +1404,6 @@ PREHOOK: query: explain extended select max(ds) from TEST1
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select max(ds) from TEST1
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST1
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTION
- max
- TOK_TABLE_OR_COL
- ds
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -1502,6 +1411,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1510,45 +1420,53 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string)
outputColumnNames: ds
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(ds)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
+ null sort order:
sort order:
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
value expressions: _col0 (type: string)
auto parallelism: false
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10004default.test1{ds=1} [test1]
- -mr-10005default.test1{ds=2} [test1]
+ nullscan://null/default.test1/part_ds=1_ [test1]
+ nullscan://null/default.test1/part_ds=2_ [test1]
Path -> Partition:
- -mr-10004default.test1{ds=1}
+ nullscan://null/default.test1/part_ds=1_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
ds 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1570,24 +1488,29 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test1
name: default.test1
- -mr-10005default.test1{ds=2}
+ nullscan://null/default.test1/part_ds=2_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
ds 2
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test1
+ numFiles 0
+ numRows 0
partition_columns ds
partition_columns.types string
+ rawDataSize 0
serialization.ddl struct test1 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1610,32 +1533,33 @@ STAGE PLANS:
name: default.test1
name: default.test1
Truncated Path -> Alias:
- -mr-10004default.test1{ds=1} [test1]
- -mr-10005default.test1{ds=2} [test1]
+ nullscan://null/default.test1/part_ds=1_ [test1]
+ nullscan://null/default.test1/part_ds=2_ [test1]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1717,31 +1641,6 @@ PREHOOK: query: explain extended select ds, count(distinct hr) from TEST2 group
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select ds, count(distinct hr) from TEST2 group by ds
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- TEST2
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- ds
- TOK_SELEXPR
- TOK_FUNCTIONDI
- count
- TOK_TABLE_OR_COL
- hr
- TOK_GROUPBY
- TOK_TABLE_OR_COL
- ds
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -1749,6 +1648,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1757,33 +1657,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: ds (type: string), hr (type: string)
outputColumnNames: ds, hr
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 368 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT hr)
keys: ds (type: string), hr (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 376 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
+ null sort order: aa
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 376 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
auto parallelism: true
+ Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
- -mr-10004default.test2{ds=01_10_10, hr=01} [test2]
- -mr-10005default.test2{ds=01_10_20, hr=02} [test2]
- -mr-10006default.test2{ds=1, hr=1} [test2]
- -mr-10007default.test2{ds=1, hr=2} [test2]
- -mr-10008default.test2{ds=1, hr=3} [test2]
+ nullscan://null/default.test2/part_ds=01_10_10_hr=01_ [test2]
+ nullscan://null/default.test2/part_ds=01_10_20_hr=02_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=1_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=2_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=3_ [test2]
Path -> Partition:
- -mr-10004default.test2{ds=01_10_10, hr=01}
+ nullscan://null/default.test2/part_ds=01_10_10_hr=01_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1791,17 +1694,22 @@ STAGE PLANS:
ds 01:10:10
hr 01
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1823,7 +1731,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test2
name: default.test2
- -mr-10005default.test2{ds=01_10_20, hr=02}
+ nullscan://null/default.test2/part_ds=01_10_20_hr=02_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1831,17 +1739,22 @@ STAGE PLANS:
ds 01:10:20
hr 02
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1863,7 +1776,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test2
name: default.test2
- -mr-10006default.test2{ds=1, hr=1}
+ nullscan://null/default.test2/part_ds=1_hr=1_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1871,17 +1784,22 @@ STAGE PLANS:
ds 1
hr 1
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1903,7 +1821,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test2
name: default.test2
- -mr-10007default.test2{ds=1, hr=2}
+ nullscan://null/default.test2/part_ds=1_hr=2_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1911,17 +1829,22 @@ STAGE PLANS:
ds 1
hr 2
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1943,7 +1866,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test2
name: default.test2
- -mr-10008default.test2{ds=1, hr=3}
+ nullscan://null/default.test2/part_ds=1_hr=3_
Partition
input format: org.apache.hadoop.hive.ql.io.OneNullRowInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1951,17 +1874,22 @@ STAGE PLANS:
ds 1
hr 3
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns a,b
columns.comments
columns.types int:double
#### A masked pattern was here ####
name default.test2
+ numFiles 0
+ numRows 0
partition_columns ds/hr
partition_columns.types string:string
+ rawDataSize 0
serialization.ddl struct test2 { i32 a, double b}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.NullStructSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.NullStructSerDe
@@ -1984,13 +1912,13 @@ STAGE PLANS:
name: default.test2
name: default.test2
Truncated Path -> Alias:
- -mr-10004default.test2{ds=01_10_10, hr=01} [test2]
- -mr-10005default.test2{ds=01_10_20, hr=02} [test2]
- -mr-10006default.test2{ds=1, hr=1} [test2]
- -mr-10007default.test2{ds=1, hr=2} [test2]
- -mr-10008default.test2{ds=1, hr=3} [test2]
+ nullscan://null/default.test2/part_ds=01_10_10_hr=01_ [test2]
+ nullscan://null/default.test2/part_ds=01_10_20_hr=02_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=1_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=2_ [test2]
+ nullscan://null/default.test2/part_ds=1_hr=3_ [test2]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -1998,22 +1926,23 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1
columns.types string:bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
[10/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part.q.out
index 81d0fcc..b9bb18c 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part.q.out
@@ -186,10 +186,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=11)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=11)
- default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=15)
+ default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_permute_select
@@ -445,10 +445,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=15)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=15)
- default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=19)
+ default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_string_permute_select
@@ -730,10 +730,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=55)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=55)
- default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=59)
+ default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_double
PREHOOK: type: QUERY
@@ -879,10 +879,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=9 width=147)
+ Select Operator [SEL_1] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=147)
- default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=151)
+ default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_timestamp
PREHOOK: type: QUERY
@@ -1003,10 +1003,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=9 width=91)
+ Select Operator [SEL_1] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=91)
- default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=95)
+ default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_date
PREHOOK: type: QUERY
@@ -1157,10 +1157,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=33)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_string
PREHOOK: type: QUERY
@@ -1304,10 +1304,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=140)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=140)
- default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=144)
+ default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char
PREHOOK: type: QUERY
@@ -1451,10 +1451,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=32)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=32)
- default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=36)
+ default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char_trunc
PREHOOK: type: QUERY
@@ -1598,10 +1598,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=33)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar
PREHOOK: type: QUERY
@@ -1745,10 +1745,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=30)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=30)
- default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=34)
+ default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar_trunc
PREHOOK: type: QUERY
@@ -1889,10 +1889,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=38)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=38)
- default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=42)
+ default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_string
PREHOOK: type: QUERY
@@ -2033,10 +2033,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=116)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=116)
- default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=120)
+ default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char
PREHOOK: type: QUERY
@@ -2177,10 +2177,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=39)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=39)
- default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=43)
+ default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char_trunc
PREHOOK: type: QUERY
@@ -2321,10 +2321,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=36)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=36)
- default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=40)
+ default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar
PREHOOK: type: QUERY
@@ -2465,10 +2465,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=36)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=36)
- default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=40)
+ default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar_trunc
PREHOOK: type: QUERY
@@ -2620,10 +2620,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=79)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=79)
- default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=83)
+ default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_string_group_string_group_string
PREHOOK: type: QUERY
@@ -2764,10 +2764,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=93)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=93)
- default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=97)
+ default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_char
PREHOOK: type: QUERY
@@ -2908,10 +2908,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=68)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=68)
- default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=72)
+ default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_varchar
PREHOOK: type: QUERY
@@ -3069,10 +3069,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=44)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=10 width=44)
- default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=10 width=48)
+ default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_lower_to_higher_numeric_group_tinyint
PREHOOK: type: QUERY
@@ -3219,10 +3219,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=44)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=10 width=44)
- default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=10 width=48)
+ default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_lower_to_higher_numeric_group_smallint
PREHOOK: type: QUERY
@@ -3366,10 +3366,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=42)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=42)
- default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=46)
+ default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_lower_to_higher_numeric_group_int
PREHOOK: type: QUERY
@@ -3510,10 +3510,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=41)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=41)
- default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=45)
+ default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_lower_to_higher_numeric_group_bigint
PREHOOK: type: QUERY
@@ -3651,10 +3651,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=33)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_decimal
PREHOOK: type: QUERY
@@ -3789,10 +3789,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=14)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=14)
- default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=18)
+ default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_float
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_complex.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_complex.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_complex.q.out
index 69581b5..af86449 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_complex.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_complex.q.out
@@ -164,10 +164,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=151)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=6 width=151)
- default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:NONE,Output:["insert_num","s1","b"]
+ TableScan [TS_0] (rows=6 width=155)
+ default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","s1","b"]
PREHOOK: query: select insert_num,part,s1,b from part_change_various_various_struct1
PREHOOK: type: QUERY
@@ -428,10 +428,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=8 width=113)
+ Select Operator [SEL_1] (rows=8 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=8 width=113)
- default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s2"]
+ TableScan [TS_0] (rows=8 width=117)
+ default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s2"]
PREHOOK: query: select insert_num,part,b,s2 from part_add_various_various_struct2
PREHOOK: type: QUERY
@@ -616,10 +616,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=4 width=84)
+ Select Operator [SEL_1] (rows=4 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=4 width=84)
- default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s3"]
+ TableScan [TS_0] (rows=4 width=88)
+ default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s3"]
PREHOOK: query: select insert_num,part,b,s3 from part_add_to_various_various_struct4
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_primitive.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_primitive.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_primitive.q.out
index 4e8c9bf..fcecca8 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_primitive.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_nonvec_mapwork_part_all_primitive.q.out
@@ -140,10 +140,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=96)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
- TableScan [TS_0] (rows=6 width=96)
- default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
+ TableScan [TS_0] (rows=6 width=100)
+ default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,b from part_change_various_various_boolean
PREHOOK: type: QUERY
@@ -309,10 +309,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=119)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=119)
- default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=123)
+ default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_tinyint
PREHOOK: type: QUERY
@@ -479,10 +479,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=126)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=126)
- default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=130)
+ default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_smallint
PREHOOK: type: QUERY
@@ -649,10 +649,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=130)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=130)
- default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=134)
+ default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_int
PREHOOK: type: QUERY
@@ -819,10 +819,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=135)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=135)
- default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=139)
+ default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_bigint
PREHOOK: type: QUERY
@@ -987,10 +987,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=134)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=134)
- default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=138)
+ default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_float
PREHOOK: type: QUERY
@@ -1155,10 +1155,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=141)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=141)
- default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=145)
+ default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_double
PREHOOK: type: QUERY
@@ -1323,10 +1323,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=174)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=174)
- default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=178)
+ default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_decimal
PREHOOK: type: QUERY
@@ -1497,10 +1497,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=141)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=141)
- default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=145)
+ default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_string
PREHOOK: type: QUERY
@@ -1671,10 +1671,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=193)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=193)
- default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=197)
+ default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char
PREHOOK: type: QUERY
@@ -1845,10 +1845,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=129)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=129)
- default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=133)
+ default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char_trunc
PREHOOK: type: QUERY
@@ -2019,10 +2019,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=143)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=143)
- default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=147)
+ default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar
PREHOOK: type: QUERY
@@ -2193,10 +2193,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=138)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=138)
- default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=142)
+ default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar_trunc
PREHOOK: type: QUERY
@@ -2364,10 +2364,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=131)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
- TableScan [TS_0] (rows=7 width=131)
- default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
+ TableScan [TS_0] (rows=7 width=135)
+ default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,b from part_change_various_various_timestamp
PREHOOK: type: QUERY
@@ -2508,10 +2508,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=96)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=6 width=96)
- default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=6 width=100)
+ default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_various_various_date
PREHOOK: type: QUERY
@@ -2746,10 +2746,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=13 width=105)
+ Select Operator [SEL_1] (rows=13 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=13 width=105)
- default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=13 width=109)
+ default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_same_type_different_params
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part.q.out
index 934d57b..8df02a2 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part.q.out
@@ -190,10 +190,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=11)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=11)
- default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=15)
+ default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_permute_select
@@ -449,10 +449,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=15)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=15)
- default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=19)
+ default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_string_permute_select
@@ -734,10 +734,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=38)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=38)
- default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=42)
+ default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_double
PREHOOK: type: QUERY
@@ -883,10 +883,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=9 width=132)
+ Select Operator [SEL_3] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=132)
- default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=136)
+ default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_timestamp
PREHOOK: type: QUERY
@@ -1007,10 +1007,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=9 width=61)
+ Select Operator [SEL_3] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=61)
- default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=65)
+ default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_date
PREHOOK: type: QUERY
@@ -1161,10 +1161,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=33)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_string
PREHOOK: type: QUERY
@@ -1308,10 +1308,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=33)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char
PREHOOK: type: QUERY
@@ -1455,10 +1455,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=30)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=30)
- default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=34)
+ default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char_trunc
PREHOOK: type: QUERY
@@ -1602,10 +1602,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=33)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar
PREHOOK: type: QUERY
@@ -1749,10 +1749,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=30)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=30)
- default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=34)
+ default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar_trunc
PREHOOK: type: QUERY
@@ -1893,10 +1893,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=38)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=38)
- default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=42)
+ default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_string
PREHOOK: type: QUERY
@@ -2037,10 +2037,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=38)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=38)
- default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=42)
+ default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char
PREHOOK: type: QUERY
@@ -2181,10 +2181,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=35)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=35)
- default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=39)
+ default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char_trunc
PREHOOK: type: QUERY
@@ -2325,10 +2325,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=36)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=36)
- default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=40)
+ default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar
PREHOOK: type: QUERY
@@ -2469,10 +2469,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=36)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=36)
- default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=40)
+ default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar_trunc
PREHOOK: type: QUERY
@@ -2624,10 +2624,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=54)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=54)
- default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=58)
+ default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_string_group_string_group_string
PREHOOK: type: QUERY
@@ -2768,10 +2768,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=46)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=46)
- default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=50)
+ default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_char
PREHOOK: type: QUERY
@@ -2912,10 +2912,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=45)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=45)
- default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=49)
+ default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_varchar
PREHOOK: type: QUERY
@@ -3073,10 +3073,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=44)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=10 width=44)
- default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=10 width=48)
+ default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_lower_to_higher_numeric_group_tinyint
PREHOOK: type: QUERY
@@ -3223,10 +3223,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=44)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=10 width=44)
- default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=10 width=48)
+ default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_lower_to_higher_numeric_group_smallint
PREHOOK: type: QUERY
@@ -3370,10 +3370,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=42)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=42)
- default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=46)
+ default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_lower_to_higher_numeric_group_int
PREHOOK: type: QUERY
@@ -3514,10 +3514,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=41)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=41)
- default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=45)
+ default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_lower_to_higher_numeric_group_bigint
PREHOOK: type: QUERY
@@ -3655,10 +3655,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=33)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_decimal
PREHOOK: type: QUERY
@@ -3793,10 +3793,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=14)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=14)
- default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=18)
+ default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_float
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_complex.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_complex.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_complex.q.out
index 16f6f2d..39d85aa 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_complex.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_complex.q.out
@@ -168,10 +168,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=151)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=6 width=151)
- default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:NONE,Output:["insert_num","s1","b"]
+ TableScan [TS_0] (rows=6 width=155)
+ default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","s1","b"]
PREHOOK: query: select insert_num,part,s1,b from part_change_various_various_struct1
PREHOOK: type: QUERY
@@ -432,10 +432,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=8 width=113)
+ Select Operator [SEL_1] (rows=8 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=8 width=113)
- default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s2"]
+ TableScan [TS_0] (rows=8 width=117)
+ default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s2"]
PREHOOK: query: select insert_num,part,b,s2 from part_add_various_various_struct2
PREHOOK: type: QUERY
@@ -620,10 +620,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=4 width=84)
+ Select Operator [SEL_1] (rows=4 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=4 width=84)
- default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s3"]
+ TableScan [TS_0] (rows=4 width=88)
+ default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s3"]
PREHOOK: query: select insert_num,part,b,s3 from part_add_to_various_various_struct4
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_primitive.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_primitive.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_primitive.q.out
index 5ea61b0..e78e01f 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_primitive.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_vec_mapwork_part_all_primitive.q.out
@@ -144,10 +144,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=6 width=96)
+ Select Operator [SEL_3] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
- TableScan [TS_0] (rows=6 width=96)
- default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
+ TableScan [TS_0] (rows=6 width=100)
+ default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,b from part_change_various_various_boolean
PREHOOK: type: QUERY
@@ -313,10 +313,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=105)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=105)
- default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=109)
+ default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_tinyint
PREHOOK: type: QUERY
@@ -483,10 +483,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=111)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=111)
- default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=115)
+ default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_smallint
PREHOOK: type: QUERY
@@ -653,10 +653,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=114)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=114)
- default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=118)
+ default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_int
PREHOOK: type: QUERY
@@ -823,10 +823,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=119)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=119)
- default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=123)
+ default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_bigint
PREHOOK: type: QUERY
@@ -991,10 +991,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=118)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=118)
- default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=122)
+ default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_float
PREHOOK: type: QUERY
@@ -1159,10 +1159,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=124)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=124)
- default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=128)
+ default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_double
PREHOOK: type: QUERY
@@ -1327,10 +1327,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=159)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=159)
- default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=163)
+ default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_decimal
PREHOOK: type: QUERY
@@ -1501,10 +1501,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=141)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=141)
- default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=145)
+ default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_string
PREHOOK: type: QUERY
@@ -1675,10 +1675,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=127)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=127)
- default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=131)
+ default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char
PREHOOK: type: QUERY
@@ -1849,10 +1849,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=121)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=121)
- default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=125)
+ default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char_trunc
PREHOOK: type: QUERY
@@ -2023,10 +2023,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=143)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=143)
- default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=147)
+ default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar
PREHOOK: type: QUERY
@@ -2197,10 +2197,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=138)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=138)
- default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=142)
+ default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar_trunc
PREHOOK: type: QUERY
@@ -2368,10 +2368,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=124)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
- TableScan [TS_0] (rows=7 width=124)
- default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
+ TableScan [TS_0] (rows=7 width=128)
+ default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,b from part_change_various_various_timestamp
PREHOOK: type: QUERY
@@ -2512,10 +2512,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=6 width=97)
+ Select Operator [SEL_3] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=6 width=97)
- default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=6 width=101)
+ default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_various_various_date
PREHOOK: type: QUERY
@@ -2750,10 +2750,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=13 width=96)
+ Select Operator [SEL_3] (rows=13 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=13 width=96)
- default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=13 width=100)
+ default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_same_type_different_params
PREHOOK: type: QUERY
[04/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_union.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_union.q.out b/ql/src/test/results/clientpositive/llap/tez_union.q.out
index f5fb3d6..9955e12 100644
--- a/ql/src/test/results/clientpositive/llap/tez_union.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_union.q.out
@@ -25,14 +25,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -42,10 +42,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1050 Data size: 11155 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1719 Data size: 305982 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -56,33 +56,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1050 Data size: 11155 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1719 Data size: 305982 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -183,76 +183,76 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 6
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 8
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -264,15 +264,15 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4878 Data size: 39024 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 4
Execution mode: llap
@@ -281,10 +281,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -359,38 +359,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -400,10 +400,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
0 Map 1
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 424386 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 424386 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -414,14 +414,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -431,10 +431,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
0 Map 1
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 424386 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 424386 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -539,14 +539,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -556,7 +556,7 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 4
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -569,15 +569,15 @@ STAGE PLANS:
input vertices:
1 Map 9
2 Map 10
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col0 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string)
sort order: +
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -585,62 +585,62 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s8
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: s3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: s4
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -650,7 +650,7 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 6
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -663,15 +663,15 @@ STAGE PLANS:
input vertices:
1 Map 9
2 Map 10
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col0 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string)
sort order: +
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -679,33 +679,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s5
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: s6
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -715,7 +715,7 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 8
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -728,15 +728,15 @@ STAGE PLANS:
input vertices:
1 Map 9
2 Map 10
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col0 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string)
sort order: +
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -744,48 +744,48 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s7
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 9
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -794,10 +794,10 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), VALUE._col1 (type: string), KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 3630 Data size: 38563 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 21754 Data size: 5677794 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -979,14 +979,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -996,10 +996,10 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2438 Data size: 212106 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1010,33 +1010,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: s4
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1046,10 +1046,10 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Map 5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2438 Data size: 212106 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1060,19 +1060,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s5
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Union 2
@@ -1172,11 +1172,11 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Outer Join0 to 1
@@ -1186,10 +1186,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3
input vertices:
1 Map 4
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 868284 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 868284 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1200,11 +1200,11 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Outer Join0 to 1
@@ -1214,10 +1214,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3
input vertices:
1 Map 4
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 868284 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 868284 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1228,22 +1228,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -1284,43 +1284,43 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 87000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: s
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Right Outer Join0 to 1
@@ -1330,10 +1330,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
0 Union 2
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 424386 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2439 Data size: 424386 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_union2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_union2.q.out b/ql/src/test/results/clientpositive/llap/tez_union2.q.out
index 0f3abd0..51d45d2 100644
--- a/ql/src/test/results/clientpositive/llap/tez_union2.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_union2.q.out
@@ -52,63 +52,63 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 750 Data size: 7968 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 352 Data size: 62656 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 750 Data size: 7968 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 352 Data size: 62656 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -118,10 +118,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 375 Data size: 3984 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 352 Data size: 62656 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 375 Data size: 3984 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 352 Data size: 62656 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -133,30 +133,30 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: complete
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 17835 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 35670 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 750 Data size: 7968 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 352 Data size: 62656 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 750 Data size: 7968 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 352 Data size: 62656 Basic stats: COMPLETE Column stats: COMPLETE
Union 2
Vertex: Union 2
Union 5
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_union_group_by.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_union_group_by.q.out b/ql/src/test/results/clientpositive/llap/tez_union_group_by.q.out
index f9f3397..22f24c9 100644
--- a/ql/src/test/results/clientpositive/llap/tez_union_group_by.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_union_group_by.q.out
@@ -160,25 +160,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Filter Operator
predicate: ((date < '2014-09-02') and (u <> 0)) (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: u (type: bigint), date (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: min(_col1)
keys: _col0 (type: bigint)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 3 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: unknown
@@ -201,25 +201,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Filter Operator
predicate: ((date < '2014-09-02') and (u <> 0)) (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: u (type: bigint), date (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: min(_col1)
keys: _col0 (type: bigint)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 3 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: unknown
@@ -227,25 +227,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Filter Operator
predicate: ((date < '2014-09-02') and (u <> 0)) (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: u (type: bigint), date (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: min(_col1)
keys: _col0 (type: bigint)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 3 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: unknown
@@ -253,15 +253,15 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Filter Operator
predicate: (t is not null and (date >= '2014-03-04') and (date < '2014-09-03') and (u <> 0)) (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Reduce Output Operator
key expressions: t (type: string), st (type: string)
sort order: ++
Map-reduce partition columns: t (type: string), st (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
value expressions: u (type: bigint)
Execution mode: llap
LLAP IO: unknown
@@ -273,19 +273,19 @@ STAGE PLANS:
keys: KEY._col0 (type: bigint)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Filter Operator
predicate: (_col1 <= '2014-09-02') (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: _col0 (type: bigint)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
Reducer 4
Execution mode: llap
Reduce Operator Tree:
@@ -296,14 +296,14 @@ STAGE PLANS:
0 _col0 (type: bigint)
1 _col0 (type: bigint)
outputColumnNames: _col0, _col2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 202 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col2 (type: bigint), _col0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 202 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 202 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -319,17 +319,17 @@ STAGE PLANS:
1 t (type: string), st (type: string)
nullSafes: [false, true]
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 202 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col0 (type: bigint)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 202 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 202 Basic stats: COMPLETE Column stats: NONE
Reducer 9
Execution mode: llap
Reduce Operator Tree:
@@ -337,12 +337,12 @@ STAGE PLANS:
keys: KEY._col0 (type: bigint)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 202 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 202 Basic stats: COMPLETE Column stats: NONE
Union 2
Vertex: Union 2
[03/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_union_multiinsert.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_union_multiinsert.q.out b/ql/src/test/results/clientpositive/llap/tez_union_multiinsert.q.out
index 9b77106..9b7612d 100644
--- a/ql/src/test/results/clientpositive/llap/tez_union_multiinsert.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_union_multiinsert.q.out
@@ -86,70 +86,70 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 133000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 501 Data size: 136272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 136272 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: s0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -167,29 +167,29 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 501 Data size: 136272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 136272 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: llap
Reduce Operator Tree:
@@ -198,10 +198,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 205 Data size: 19680 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 205 Data size: 19680 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -215,10 +215,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 280280 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 280280 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -975,33 +975,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
@@ -1026,37 +1026,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 133000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 501 Data size: 136272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 136272 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -1067,10 +1067,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 205 Data size: 19680 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 205 Data size: 19680 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1084,10 +1084,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 280280 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 280280 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1108,29 +1108,29 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 501 Data size: 136272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 136272 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Union 2
Vertex: Union 2
@@ -1878,33 +1878,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
@@ -1929,33 +1929,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -1966,10 +1966,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 205 Data size: 19680 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 205 Data size: 19680 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -1983,10 +1983,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 280280 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 280280 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -2009,23 +2009,23 @@ STAGE PLANS:
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 500 Data size: 140000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 1001 Data size: 464464 Basic stats: COMPLETE Column stats: COMPLETE
Union 2
Vertex: Union 2
@@ -2782,33 +2782,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 70000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 70000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 232464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 232464 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -2828,23 +2828,23 @@ STAGE PLANS:
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 70000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 70000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 232464 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 232464 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: llap
Reduce Operator Tree:
@@ -2853,10 +2853,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 205 Data size: 19680 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 205 Data size: 19680 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -2870,10 +2870,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 140280 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 501 Data size: 140280 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -3629,21 +3629,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 68000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 68000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -3662,12 +3662,12 @@ STAGE PLANS:
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 68000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), substr(_col1, 5) (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 68000 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: llap
Reduce Operator Tree:
@@ -3675,31 +3675,31 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 272 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 68000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), substr(_col1, 5) (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 125 Data size: 35000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 280 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 125 Data size: 35000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT substr(_col1, 5))
keys: _col0 (type: string), _col1 (type: string)
mode: complete
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 464 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 116000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col2 (type: bigint)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 456 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 114000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 456 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 250 Data size: 114000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -3713,10 +3713,10 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 125 Data size: 12000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 96 Basic stats: COMPLETE Column stats: PARTIAL
+ Statistics: Num rows: 125 Data size: 12000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_1.q.out b/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_1.q.out
index def567b..c3aebc7 100644
--- a/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_1.q.out
@@ -33,19 +33,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint BETWEEN 1000000 AND 3000000 (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 1546640 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 1546640 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int)
sort order: +
Map-reduce partition columns: _col2 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 1546640 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: llap
LLAP IO: all inputs
@@ -53,19 +53,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000) (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 862030 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 862030 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int)
sort order: +
Map-reduce partition columns: _col2 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 862030 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: llap
LLAP IO: all inputs
@@ -79,11 +79,11 @@ STAGE PLANS:
0 _col2 (type: int)
1 _col2 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 4517320 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int)
sort order: +
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 4517320 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean), _col12 (type: tinyint), _col13 (type: smallint), _col14 (type: int), _col15 (type: bigint), _col16 (type: float), _col17 (type: double), _col18 (type: string), _col19 (type: string), _col20 (type: timestamp), _col21 (type: timestamp), _col22 (type: boolean), _col23 (type: boolean)
Reducer 3
Execution mode: llap
@@ -91,10 +91,10 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: tinyint), VALUE._col1 (type: smallint), KEY.reducesinkkey0 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: double), VALUE._col5 (type: string), VALUE._col6 (type: string), VALUE._col7 (type: timestamp), VALUE._col8 (type: timestamp), VALUE._col9 (type: boolean), VALUE._col10 (type: boolean), VALUE._col11 (type: tinyint), VALUE._col12 (type: smallint), VALUE._col13 (type: int), VALUE._col14 (type: bigint), VALUE._col15 (type: float), VALUE._col16 (type: double), VALUE._col17 (type: string), VALUE._col18 (type: string), VALUE._col19 (type: timestamp), VALUE._col20 (type: timestamp), VALUE._col21 (type: boolean), VALUE._col22 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 4517320 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 4517320 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -165,38 +165,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint BETWEEN 1000000 AND 3000000 (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 18348 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 18348 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 18348 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 110088 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000) (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -208,15 +208,15 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 58288 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 3
Execution mode: llap
@@ -225,10 +225,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -293,19 +293,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 73396 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint BETWEEN 1000000 AND 3000000 (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 36700 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: csmallint (type: smallint), cint (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 36700 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
sort order: +
Map-reduce partition columns: _col1 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 36700 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: smallint)
Execution mode: llap
LLAP IO: all inputs
@@ -313,19 +313,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 110088 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000) (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -338,18 +338,18 @@ STAGE PLANS:
0 _col1 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 29144 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: smallint)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: smallint)
sort order: +
Map-reduce partition columns: _col0 (type: smallint)
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
@@ -359,11 +359,11 @@ STAGE PLANS:
keys: KEY._col0 (type: smallint)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3379 Data size: 726540 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
sort order: +
- Statistics: Num rows: 3379 Data size: 726540 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: smallint)
Reducer 4
Execution mode: llap
@@ -371,10 +371,10 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: smallint), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3379 Data size: 726540 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 3379 Data size: 726540 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out b/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out
index d4d1250..189841a 100644
--- a/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out
@@ -43,19 +43,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (csmallint < 100) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col1) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col1) (type: int)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: llap
LLAP IO: all inputs
@@ -63,38 +63,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col0) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (UDFToInteger(_col0) + 0) (type: int)
sort order: +
Map-reduce partition columns: (UDFToInteger(_col0) + 0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -109,11 +109,11 @@ STAGE PLANS:
1 UDFToInteger(_col0) (type: int)
2 (UDFToInteger(_col0) + 0) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: smallint), _col0 (type: tinyint), _col2 (type: int)
sort order: +++
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Reducer 3
Execution mode: llap
@@ -121,10 +121,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: tinyint), KEY.reducesinkkey0 (type: smallint), KEY.reducesinkkey2 (type: int), VALUE._col0 (type: bigint), VALUE._col1 (type: float), VALUE._col2 (type: double), VALUE._col3 (type: string), VALUE._col4 (type: string), VALUE._col5 (type: timestamp), VALUE._col6 (type: timestamp), VALUE._col7 (type: boolean), VALUE._col8 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -238,19 +238,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (csmallint < 100) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col1) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col1) (type: int)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: vectorized, llap
LLAP IO: all inputs
@@ -258,38 +258,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col0) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (UDFToInteger(_col0) + 0) (type: int)
sort order: +
Map-reduce partition columns: (UDFToInteger(_col0) + 0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -304,11 +304,11 @@ STAGE PLANS:
1 UDFToInteger(_col0) (type: int)
2 (UDFToInteger(_col0) + 0) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: smallint), _col0 (type: tinyint), _col2 (type: int)
sort order: +++
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Reducer 3
Execution mode: vectorized, llap
@@ -316,10 +316,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: tinyint), KEY.reducesinkkey0 (type: smallint), KEY.reducesinkkey2 (type: int), VALUE._col0 (type: bigint), VALUE._col1 (type: float), VALUE._col2 (type: double), VALUE._col3 (type: string), VALUE._col4 (type: string), VALUE._col5 (type: timestamp), VALUE._col6 (type: timestamp), VALUE._col7 (type: boolean), VALUE._col8 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -431,19 +431,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (csmallint < 100) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col1) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col1) (type: int)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4096 Data size: 1031250 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: vectorized, llap
LLAP IO: all inputs
@@ -451,38 +451,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: UDFToInteger(_col0) (type: int)
sort order: +
Map-reduce partition columns: UDFToInteger(_col0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: (UDFToInteger(_col0) + 0) (type: int)
sort order: +
Map-reduce partition columns: (UDFToInteger(_col0) + 0) (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -497,11 +497,11 @@ STAGE PLANS:
1 UDFToInteger(_col0) (type: int)
2 (UDFToInteger(_col0) + 0) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: smallint), _col0 (type: tinyint), _col2 (type: int)
sort order: +++
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Reducer 3
Execution mode: vectorized, llap
@@ -509,10 +509,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: tinyint), KEY.reducesinkkey0 (type: smallint), KEY.reducesinkkey2 (type: int), VALUE._col0 (type: bigint), VALUE._col1 (type: float), VALUE._col2 (type: double), VALUE._col3 (type: string), VALUE._col4 (type: string), VALUE._col5 (type: timestamp), VALUE._col6 (type: timestamp), VALUE._col7 (type: boolean), VALUE._col8 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 9011 Data size: 1937438 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2710 Data size: 840100 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/transform1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/transform1.q.out b/ql/src/test/results/clientpositive/llap/transform1.q.out
index 49d2a5c..87e4482 100644
--- a/ql/src/test/results/clientpositive/llap/transform1.q.out
+++ b/ql/src/test/results/clientpositive/llap/transform1.q.out
@@ -41,8 +41,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -115,8 +115,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
[24/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out b/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out
index b8f24c4..de81828 100644
--- a/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out
@@ -49,25 +49,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -79,15 +79,15 @@ STAGE PLANS:
input vertices:
0 Map 1
2 Map 4
- Statistics: Num rows: 4400 Data size: 46745 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 594 Data size: 4752 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -95,15 +95,15 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -113,10 +113,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -180,25 +180,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -210,15 +210,15 @@ STAGE PLANS:
input vertices:
0 Map 1
2 Map 4
- Statistics: Num rows: 4400 Data size: 46745 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 594 Data size: 4752 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -226,15 +226,15 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -244,10 +244,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -324,25 +324,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -357,15 +357,15 @@ STAGE PLANS:
0 Map 1
2 Map 4
3 Map 5
- Statistics: Num rows: 6600 Data size: 70118 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5803 Data size: 46424 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -373,30 +373,30 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: w
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -406,10 +406,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -477,25 +477,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -510,15 +510,15 @@ STAGE PLANS:
0 Map 1
2 Map 4
3 Map 5
- Statistics: Num rows: 6600 Data size: 70118 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5803 Data size: 46424 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -526,30 +526,30 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: w
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -559,10 +559,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -650,40 +650,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 10
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -695,15 +695,15 @@ STAGE PLANS:
input vertices:
0 Map 1
2 Map 6
- Statistics: Num rows: 4400 Data size: 46745 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 594 Data size: 4752 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -711,40 +711,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 8
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 182000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 182000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -756,15 +756,15 @@ STAGE PLANS:
input vertices:
0 Map 7
2 Map 10
- Statistics: Num rows: 4400 Data size: 46745 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 545 Data size: 4360 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -775,17 +775,17 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: bigint)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 5
Execution mode: llap
Reduce Operator Tree:
@@ -793,10 +793,10 @@ STAGE PLANS:
keys: KEY._col0 (type: bigint)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -808,17 +808,17 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: bigint)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Union 4
Vertex: Union 4
@@ -900,40 +900,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 10
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -945,15 +945,15 @@ STAGE PLANS:
input vertices:
0 Map 1
2 Map 6
- Statistics: Num rows: 4400 Data size: 46745 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 594 Data size: 4752 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -961,40 +961,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 8
Map Operator Tree:
TableScan
alias: z
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 182000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 182000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1006,15 +1006,15 @@ STAGE PLANS:
input vertices:
0 Map 7
2 Map 10
- Statistics: Num rows: 4400 Data size: 46745 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 545 Data size: 4360 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -1025,17 +1025,17 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: bigint)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 5
Execution mode: llap
Reduce Operator Tree:
@@ -1043,10 +1043,10 @@ STAGE PLANS:
keys: KEY._col0 (type: bigint)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1058,17 +1058,17 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: bigint)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: bigint)
sort order: +
Map-reduce partition columns: _col0 (type: bigint)
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Union 4
Vertex: Union 4
@@ -1160,15 +1160,15 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((value < 'zzzzzzzzzz') and (key < 'zzzzzzzz')) (type: boolean)
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 350 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 350 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -1176,10 +1176,10 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: z1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 'zzzzzzzz') (type: boolean)
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 666 Data size: 57942 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1192,7 +1192,7 @@ STAGE PLANS:
input vertices:
0 Map 1
2 Map 4
- Statistics: Num rows: 1465 Data size: 15565 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 46 Data size: 4094 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1204,15 +1204,15 @@ STAGE PLANS:
input vertices:
1 Map 5
2 Map 6
- Statistics: Num rows: 3223 Data size: 34243 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 981 Data size: 7848 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -1220,45 +1220,45 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((value < 'zzzzzzzz') and (key < 'zzzzzzzz')) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: z2
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key < 'zzzzzzzzzz') and (value < 'zzzzzzzzzz')) (type: boolean)
- Statistics: Num rows: 222 Data size: 2358 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 222 Data size: 39516 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 222 Data size: 2358 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 222 Data size: 39516 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 6
Map Operator Tree:
TableScan
alias: y2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (value < 'zzzzzzzzzz') (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 15106 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 15106 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -1268,10 +1268,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1355,15 +1355,15 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((value < 'zzzzzzzzzz') and (key < 'zzzzzzzz')) (type: boolean)
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 350 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 350 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -1371,10 +1371,10 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: z1
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 174000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 'zzzzzzzz') (type: boolean)
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 666 Data size: 57942 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1387,7 +1387,7 @@ STAGE PLANS:
input vertices:
0 Map 1
2 Map 4
- Statistics: Num rows: 1465 Data size: 15565 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 46 Data size: 4094 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1399,15 +1399,15 @@ STAGE PLANS:
input vertices:
1 Map 5
2 Map 6
- Statistics: Num rows: 3223 Data size: 34243 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 981 Data size: 7848 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -1415,45 +1415,45 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: y1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((value < 'zzzzzzzz') and (key < 'zzzzzzzz')) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: z2
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 356000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key < 'zzzzzzzzzz') and (value < 'zzzzzzzzzz')) (type: boolean)
- Statistics: Num rows: 222 Data size: 2358 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 222 Data size: 39516 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 222 Data size: 2358 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 222 Data size: 39516 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 6
Map Operator Tree:
TableScan
alias: y2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (value < 'zzzzzzzzzz') (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 15106 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 15106 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Reducer 3
@@ -1463,10 +1463,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/identity_project_remove_skip.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/identity_project_remove_skip.q.out b/ql/src/test/results/clientpositive/llap/identity_project_remove_skip.q.out
index 2033310..3375fa8 100644
--- a/ql/src/test/results/clientpositive/llap/identity_project_remove_skip.q.out
+++ b/ql/src/test/results/clientpositive/llap/identity_project_remove_skip.q.out
@@ -23,56 +23,49 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE)
- Reducer 4 <- Map 3 (SIMPLE_EDGE), Reducer 2 (BROADCAST_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (BROADCAST_EDGE)
+ Reducer 4 <- Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: ((key is not null and (value = 'val_105')) and (key = '105')) (type: boolean)
- Statistics: Num rows: 62 Data size: 658 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((value = 'val_105') and (key = '105')) (type: boolean)
+ Statistics: Num rows: 2 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 62 Data size: 658 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: '105' (type: string)
sort order: +
- Statistics: Num rows: 62 Data size: 658 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key = '105') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: '105' (type: string)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
- Statistics: Num rows: 62 Data size: 658 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: '105' (type: string)
- sort order: +
- Map-reduce partition columns: '105' (type: string)
- Statistics: Num rows: 62 Data size: 658 Basic stats: COMPLETE Column stats: NONE
- Reducer 4
- Execution mode: uber
- Reduce Operator Tree:
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -80,20 +73,29 @@ STAGE PLANS:
0 '105' (type: string)
1 '105' (type: string)
input vertices:
- 0 Reducer 2
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ 1 Reducer 4
+ Statistics: Num rows: 4 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: '105' (type: string), 'val_105' (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Reducer 4
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: '105' (type: string)
+ sort order: +
+ Map-reduce partition columns: '105' (type: string)
+ Statistics: Num rows: 2 Data size: 174 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/insert1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/insert1.q.out b/ql/src/test/results/clientpositive/llap/insert1.q.out
index 623ed2d..4b30156 100644
--- a/ql/src/test/results/clientpositive/llap/insert1.q.out
+++ b/ql/src/test/results/clientpositive/llap/insert1.q.out
@@ -26,7 +26,7 @@ POSTHOOK: query: insert overwrite table insert1 select a.key, a.value from inser
POSTHOOK: type: QUERY
POSTHOOK: Input: default@insert2
POSTHOOK: Output: default@insert1
-POSTHOOK: Lineage: insert1.key SIMPLE [(insert2)a.FieldSchema(name:key, type:int, comment:null), ]
+POSTHOOK: Lineage: insert1.key SIMPLE []
POSTHOOK: Lineage: insert1.value SIMPLE [(insert2)a.FieldSchema(name:value, type:string, comment:null), ]
PREHOOK: query: explain insert into table insert1 select a.key, a.value from insert2 a WHERE (a.key=-1)
PREHOOK: type: QUERY
@@ -64,6 +64,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -117,6 +118,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -186,6 +188,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: x.insert1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -239,6 +242,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.insert1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -315,6 +319,7 @@ STAGE PLANS:
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: x.insert1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-3
Dependency Collection
[37/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out b/ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out
index d474b7e..5702c6e 100644
--- a/ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out
+++ b/ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out
@@ -27,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -36,39 +37,41 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -78,24 +81,20 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ outputColumnNames: _col0
+ Statistics: Num rows: 60 Data size: 5160 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count(1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -104,34 +103,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -176,6 +175,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -185,39 +185,41 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -227,24 +229,20 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ outputColumnNames: _col0
+ Statistics: Num rows: 60 Data size: 5160 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count(1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -253,34 +251,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -333,71 +331,69 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 1 <- Map 4 (BROADCAST_EDGE)
- Reducer 2 <- Map 1 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+ Map 2 <- Map 1 (BROADCAST_EDGE)
+ Reducer 3 <- Map 2 (SIMPLE_EDGE)
+ Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: string)
- 1 _col0 (type: string)
- outputColumnNames: _col1
- input vertices:
- 1 Map 4
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Select Operator
- expressions: _col1 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
- Map 4
+ LLAP IO: no inputs
+ Map 2
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- Reducer 2
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0
+ input vertices:
+ 0 Map 1
+ Statistics: Num rows: 60 Data size: 5160 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count(1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 3
Execution mode: llap
Reduce Operator Tree:
Group By Operator
@@ -405,34 +401,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
- Reducer 3
- Execution mode: uber
+ Reducer 4
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -483,6 +479,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -493,43 +490,45 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 17835 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 17835 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -540,18 +539,18 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 1128 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 1128 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
@@ -561,34 +560,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 1128 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 1128 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -633,6 +632,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -643,43 +643,45 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 13 Data size: 99 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 17835 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 17835 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -690,18 +692,18 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 1128 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 1128 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
@@ -711,34 +713,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 1128 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 1128 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -789,6 +791,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -798,60 +801,58 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Right Outer Join0 to 1
+ Left Outer Join0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ outputColumnNames: _col0
+ Statistics: Num rows: 60 Data size: 5160 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count(1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -860,34 +861,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -932,6 +933,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -941,60 +943,58 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Right Outer Join0 to 1
+ Left Outer Join0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ outputColumnNames: _col0
+ Statistics: Num rows: 60 Data size: 5160 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count(1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -1003,34 +1003,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1316 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1081,6 +1081,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1090,56 +1091,62 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Right Outer Join0 to 1
+ Left Outer Join0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ outputColumnNames: _col1
+ Statistics: Num rows: 60 Data size: 5220 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col1 (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 60 Data size: 5220 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count(1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -1148,34 +1155,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1220,6 +1227,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1229,56 +1237,62 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Right Outer Join0 to 1
+ Left Outer Join0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ outputColumnNames: _col1
+ Statistics: Num rows: 60 Data size: 5220 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col1 (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 60 Data size: 5220 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count(1)
+ keys: _col0 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -1287,34 +1301,34 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: hash(_col0) (type: int), hash(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col0), sum(_col1)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint), _col1 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), sum(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1363,6 +1377,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1371,75 +1386,77 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Right Outer Join0 to 1
+ Left Outer Join0 to 1
keys:
0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col3
+ Statistics: Num rows: 25 Data size: 4425 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col2 (type: string), _col1 (type: string)
+ expressions: _col0 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4425 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2220 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2220 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2220 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2220 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1495,6 +1512,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1503,75 +1521,77 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Right Outer Join0 to 1
+ Left Outer Join0 to 1
keys:
0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col1 (type: string)
- outputColumnNames: _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col3
+ Statistics: Num rows: 25 Data size: 4425 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col2 (type: string), _col1 (type: string)
+ expressions: _col0 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4425 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2220 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2220 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2220 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2220 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1635,6 +1655,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1644,56 +1665,62 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: y
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: x
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2150 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
- alias: x
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ alias: y
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
- Left Outer Join0 to 1
+ Right Outer Join0 to 1
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(1)
- keys: _col0 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ outputColumnNames: _col1
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: _col1 (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats:
<TRUNCATED>
[23/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/insert_into1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/insert_into1.q.out b/ql/src/test/results/clientpositive/llap/insert_into1.q.out
index cfbfadc..d4c514a 100644
--- a/ql/src/test/results/clientpositive/llap/insert_into1.q.out
+++ b/ql/src/test/results/clientpositive/llap/insert_into1.q.out
@@ -27,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -35,34 +36,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 100
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 17800 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -146,6 +149,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -154,34 +158,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 100
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 17800 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -265,6 +271,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -273,34 +280,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 950 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 950 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -371,6 +380,266 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@insert_into1
#### A masked pattern was here ####
10
+PREHOOK: query: explain insert overwrite table insert_into1 select 1, 'a'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert overwrite table insert_into1 select 1, 'a'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: _dummy_table
+ Row Limit Per Split: 1
+ Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: 1 (type: int), 'a' (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.insert_into1
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.insert_into1
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: insert overwrite table insert_into1 select 1, 'a'
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+PREHOOK: Output: default@insert_into1
+POSTHOOK: query: insert overwrite table insert_into1 select 1, 'a'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+POSTHOOK: Output: default@insert_into1
+POSTHOOK: Lineage: insert_into1.key SIMPLE []
+POSTHOOK: Lineage: insert_into1.value SIMPLE []
+PREHOOK: query: explain insert into insert_into1 select 2, 'b'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert into insert_into1 select 2, 'b'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: _dummy_table
+ Row Limit Per Split: 1
+ Statistics: Num rows: 1 Data size: 1 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: 2 (type: int), 'b' (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 89 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.insert_into1
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ replace: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.insert_into1
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: insert into insert_into1 select 2, 'b'
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+PREHOOK: Output: default@insert_into1
+POSTHOOK: query: insert into insert_into1 select 2, 'b'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+POSTHOOK: Output: default@insert_into1
+POSTHOOK: Lineage: insert_into1.key SIMPLE []
+POSTHOOK: Lineage: insert_into1.value SIMPLE []
+PREHOOK: query: select * from insert_into1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@insert_into1
+#### A masked pattern was here ####
+POSTHOOK: query: select * from insert_into1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@insert_into1
+#### A masked pattern was here ####
+1 a
+2 b
+PREHOOK: query: explain
+insert into table insert_into1 values(1, 'abc')
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+insert into table insert_into1 values(1, 'abc')
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: values__tmp__table__1
+ Statistics: Num rows: 1 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: UDFToInteger(tmp_values_col1) (type: int), tmp_values_col2 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.insert_into1
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ replace: false
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.insert_into1
+
+PREHOOK: query: insert into table insert_into1 values(1, 'abc')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__2
+PREHOOK: Output: default@insert_into1
+POSTHOOK: query: insert into table insert_into1 values(1, 'abc')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__2
+POSTHOOK: Output: default@insert_into1
+POSTHOOK: Lineage: insert_into1.key EXPRESSION [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: insert_into1.value SIMPLE [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: explain
+SELECT COUNT(*) FROM insert_into1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+SELECT COUNT(*) FROM insert_into1
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: insert_into1
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from insert_into1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@insert_into1
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from insert_into1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@insert_into1
+#### A masked pattern was here ####
+3
PREHOOK: query: DROP TABLE insert_into1
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@insert_into1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/insert_into2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/insert_into2.q.out b/ql/src/test/results/clientpositive/llap/insert_into2.q.out
index 6eb8129..c0d2178 100644
--- a/ql/src/test/results/clientpositive/llap/insert_into2.q.out
+++ b/ql/src/test/results/clientpositive/llap/insert_into2.q.out
@@ -27,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -35,34 +36,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 100
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 17800 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -187,6 +190,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -195,34 +199,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 100
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 17800 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -316,6 +322,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -324,34 +331,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 50
- Statistics: Num rows: 50 Data size: 500 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 8900 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 50 Data size: 500 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 4750 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 50 Data size: 500 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 4750 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -430,6 +439,85 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@insert_into2
#### A masked pattern was here ####
50
+PREHOOK: query: insert into table insert_into2 partition (ds='2') values(1, 'abc')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__1
+PREHOOK: Output: default@insert_into2@ds=2
+POSTHOOK: query: insert into table insert_into2 partition (ds='2') values(1, 'abc')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__1
+POSTHOOK: Output: default@insert_into2@ds=2
+POSTHOOK: Lineage: insert_into2 PARTITION(ds=2).key EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: insert_into2 PARTITION(ds=2).value SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: explain
+SELECT COUNT(*) FROM insert_into2 where ds='2'
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+SELECT COUNT(*) FROM insert_into2 where ds='2'
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: insert_into2
+ Statistics: Num rows: 50 Data size: 930 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ Statistics: Num rows: 50 Data size: 930 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from insert_into2 where ds='2'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@insert_into2
+PREHOOK: Input: default@insert_into2@ds=2
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from insert_into2 where ds='2'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@insert_into2
+POSTHOOK: Input: default@insert_into2@ds=2
+#### A masked pattern was here ####
+51
PREHOOK: query: DROP TABLE insert_into2
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@insert_into2
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/join0.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/join0.q.out b/ql/src/test/results/clientpositive/llap/join0.q.out
index f177afc..82440e7 100644
--- a/ql/src/test/results/clientpositive/llap/join0.q.out
+++ b/ql/src/test/results/clientpositive/llap/join0.q.out
@@ -36,17 +36,17 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -54,17 +54,17 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -78,21 +78,21 @@ STAGE PLANS:
0
1
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string)
sort order: ++++
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27556 Data size: 9809936 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/join1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/join1.q.out b/ql/src/test/results/clientpositive/llap/join1.q.out
index f719181..cb0a735 100644
--- a/ql/src/test/results/clientpositive/llap/join1.q.out
+++ b/ql/src/test/results/clientpositive/llap/join1.q.out
@@ -27,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -35,39 +36,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: src2
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -77,15 +80,15 @@ STAGE PLANS:
keys:
0 _col0 (type: string)
1 _col0 (type: string)
- outputColumnNames: _col1, _col2
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0, _col2
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: UDFToInteger(_col2) (type: int), _col1 (type: string)
+ expressions: UDFToInteger(_col0) (type: int), _col2 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 115805 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 115805 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -119,7 +122,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@src
POSTHOOK: Output: default@dest_j1
POSTHOOK: Lineage: dest_j1.key EXPRESSION [(src)src1.FieldSchema(name:key, type:string, comment:default), ]
-POSTHOOK: Lineage: dest_j1.value SIMPLE [(src)src1.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: dest_j1.value SIMPLE [(src)src2.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: SELECT dest_j1.* FROM dest_j1
PREHOOK: type: QUERY
PREHOOK: Input: default@dest_j1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/join_nullsafe.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/join_nullsafe.q.out b/ql/src/test/results/clientpositive/llap/join_nullsafe.q.out
index 5ec2db7..7e3b67d 100644
--- a/ql/src/test/results/clientpositive/llap/join_nullsafe.q.out
+++ b/ql/src/test/results/clientpositive/llap/join_nullsafe.q.out
@@ -31,6 +31,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -47,6 +48,7 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -59,6 +61,7 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
value expressions: key (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -79,8 +82,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 3 Data size: 28 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -121,6 +124,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -132,14 +136,15 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: int)
sort order: +
Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -147,14 +152,15 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: value (type: int)
sort order: +
Map-reduce partition columns: value (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
value expressions: key (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -162,14 +168,15 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: int)
sort order: +
Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -182,17 +189,17 @@ STAGE PLANS:
1 value (type: int)
2 key (type: int)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -222,6 +229,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -238,6 +246,7 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -250,6 +259,7 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
value expressions: key (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -262,6 +272,7 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -284,8 +295,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -342,6 +353,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -353,13 +365,14 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: int), value (type: int)
sort order: ++
Map-reduce partition columns: key (type: int), value (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -367,13 +380,14 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: value (type: int), key (type: int)
sort order: ++
Map-reduce partition columns: value (type: int), key (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -381,13 +395,14 @@ STAGE PLANS:
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: int), value (type: int)
sort order: ++
Map-reduce partition columns: key (type: int), value (type: int)
- Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -401,17 +416,17 @@ STAGE PLANS:
2 key (type: int), value (type: int)
nullSafes: [true, false]
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: int), _col5 (type: int), _col6 (type: int), _col10 (type: int), _col11 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 4 Data size: 37 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -441,6 +456,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -456,6 +472,7 @@ STAGE PLANS:
Map-reduce partition columns: key (type: int), value (type: int)
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -467,6 +484,7 @@ STAGE PLANS:
Map-reduce partition columns: value (type: int), key (type: int)
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -478,6 +496,7 @@ STAGE PLANS:
Map-reduce partition columns: key (type: int), value (type: int)
Statistics: Num rows: 3 Data size: 26 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -500,8 +519,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 6 Data size: 57 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1584,6 +1603,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1603,6 +1623,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -1618,6 +1639,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: key (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1638,8 +1660,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
[47/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out
index 377e4b5..1bef238 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_12.q.out
@@ -138,71 +138,11 @@ POSTHOOK: query: load data local inpath '../../data/files/smallsrcsortbucket3out
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@bucket_medium@ds=2008-04-08
-Warning: Map Join MAPJOIN[33][bigTable=?] in task 'Map 3' is a cross product
+Warning: Map Join MAPJOIN[34][bigTable=?] in task 'Map 3' is a cross product
PREHOOK: query: explain extended select count(*) FROM bucket_small a JOIN bucket_medium b ON a.key = b.key JOIN bucket_big c ON c.key = b.key JOIN bucket_medium d ON c.key = b.key
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_small a JOIN bucket_medium b ON a.key = b.key JOIN bucket_big c ON c.key = b.key JOIN bucket_medium d ON c.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_JOIN
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_medium
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- c
- =
- .
- TOK_TABLE_OR_COL
- c
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_TABREF
- TOK_TABNAME
- bucket_medium
- d
- =
- .
- TOK_TABLE_OR_COL
- c
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -210,9 +150,9 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 2 <- Map 1 (BROADCAST_EDGE)
- Map 3 <- Map 2 (BROADCAST_EDGE), Map 5 (BROADCAST_EDGE)
+ Map 3 <- Map 1 (BROADCAST_EDGE), Map 2 (BROADCAST_EDGE), Map 5 (BROADCAST_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
@@ -220,20 +160,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -245,7 +191,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -291,33 +236,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 1 Data size: 170 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 170 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- outputColumnNames: _col6
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 1 Data size: 125 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: _col6 (type: string), _col6 (type: string)
- sort order: ++
- Map-reduce partition columns: _col6 (type: string), _col6 (type: string)
- Statistics: Num rows: 1 Data size: 125 Basic stats: COMPLETE Column stats: NONE
- tag: 0
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -329,7 +267,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 3
bucket_field_name key
columns key,value
@@ -375,48 +312,55 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 2 => 1
- keys:
- 0 _col6 (type: string), _col6 (type: string)
- 1 key (type: string), key (type: string)
- input vertices:
- 0 Map 2
- Position of Big Table: 1
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
- Estimated key counts: Map 5 => 1
+ Inner Join 1 to 2
+ Estimated key counts: Map 1 => 1, Map 2 => 1
keys:
- 0
- 1
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ 2 _col0 (type: string)
input vertices:
- 1 Map 5
- Position of Big Table: 0
- Statistics: Num rows: 69 Data size: 7032 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ 0 Map 1
+ 1 Map 2
+ Position of Big Table: 2
+ Statistics: Num rows: 255 Data size: 27614 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 5 => 1
+ keys:
+ 0
+ 1
+ input vertices:
+ 1 Map 5
+ Position of Big Table: 0
+ Statistics: Num rows: 255 Data size: 29909 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -428,7 +372,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -476,7 +419,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -523,14 +465,18 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: d
- Statistics: Num rows: 1 Data size: 170 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 170 Basic stats: PARTIAL Column stats: NONE
- tag: 1
- auto parallelism: false
+ Select Operator
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ tag: 1
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -542,7 +488,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 3
bucket_field_name key
columns key,value
@@ -585,7 +530,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_medium/ds=2008-04-08 [d]
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -601,13 +546,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -621,7 +567,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Map Join MAPJOIN[33][bigTable=?] in task 'Map 3' is a cross product
+Warning: Map Join MAPJOIN[34][bigTable=?] in task 'Map 3' is a cross product
PREHOOK: query: select count(*) FROM bucket_small a JOIN bucket_medium b ON a.key = b.key JOIN bucket_big c ON c.key = b.key JOIN bucket_medium d ON c.key = b.key
PREHOOK: type: QUERY
PREHOOK: Input: default@bucket_big
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out
index 639ea41..a539e03 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_13.q.out
@@ -83,6 +83,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-2
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 2 (BROADCAST_EDGE)
#### A masked pattern was here ####
@@ -94,7 +95,7 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -104,19 +105,18 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col5, _col6
input vertices:
1 Map 2
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col5 (type: int), _col6 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col2 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -125,16 +125,17 @@ STAGE PLANS:
Select Operator
expressions: _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dest2
Execution mode: llap
+ LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
@@ -142,14 +143,15 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: int)
sort order: +
Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-3
Dependency Collection
@@ -295,6 +297,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-2
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 2 (BROADCAST_EDGE)
#### A masked pattern was here ####
@@ -306,7 +309,7 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -316,19 +319,18 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col5, _col6
input vertices:
1 Map 2
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col5 (type: int), _col6 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col2 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -337,16 +339,17 @@ STAGE PLANS:
Select Operator
expressions: _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dest2
Execution mode: llap
+ LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
@@ -354,14 +357,15 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: int)
sort order: +
Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-3
Dependency Collection
@@ -507,6 +511,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-2
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 2 (BROADCAST_EDGE)
#### A masked pattern was here ####
@@ -518,7 +523,7 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -528,19 +533,18 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col5, _col6
input vertices:
1 Map 2
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col5 (type: int), _col6 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col2 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -549,16 +553,17 @@ STAGE PLANS:
Select Operator
expressions: _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dest2
Execution mode: llap
+ LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
@@ -566,14 +571,15 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: int)
sort order: +
Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-3
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out
index ad5a814..9bab958 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_14.q.out
@@ -49,6 +49,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -59,39 +60,48 @@ STAGE PLANS:
TableScan
alias: a
Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Left Outer Join0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -102,8 +112,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -148,6 +158,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -158,39 +169,48 @@ STAGE PLANS:
TableScan
alias: a
Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 189 Data size: 1891 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Right Outer Join0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- input vertices:
- 0 Map 1
- Statistics: Num rows: 207 Data size: 2080 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 189 Data size: 1891 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Right Outer Join0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 0 Map 1
+ Statistics: Num rows: 207 Data size: 2080 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -201,8 +221,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out
index 8c2e080..8059b1c 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_15.q.out
@@ -47,6 +47,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -57,39 +58,48 @@ STAGE PLANS:
TableScan
alias: a
Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Left Outer Join0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -100,8 +110,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -123,6 +133,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -133,39 +144,48 @@ STAGE PLANS:
TableScan
alias: a
Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 20 Data size: 160 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Right Outer Join0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- input vertices:
- 0 Map 1
- Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Right Outer Join0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 0 Map 1
+ Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -176,8 +196,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out
index 66869af..0b1d648 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out
@@ -89,38 +89,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- Since the leftmost table is assumed as the big table, arrange the tables in the join accordingly
explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -128,6 +96,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -137,36 +106,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 54 Data size: 5500 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 29 Data size: 3025 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 59 Data size: 6525 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -178,7 +152,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -226,7 +199,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -273,20 +245,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 113 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 113 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -298,7 +276,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -341,7 +318,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_small/ds=2008-04-08 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -357,13 +334,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -400,38 +378,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- The mapjoin should fail resulting in the sort-merge join
explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -439,6 +385,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -448,36 +395,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 54 Data size: 5500 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 29 Data size: 3025 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 54 Data size: 5932 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 59 Data size: 6525 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -489,7 +441,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -537,7 +488,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -584,20 +534,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 113 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 113 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -609,7 +565,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -652,7 +607,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_small/ds=2008-04-08 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -668,13 +623,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
[25/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out b/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out
index 2facae5..8ec11eb 100644
--- a/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out
@@ -56,14 +56,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint < 2000000000) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -72,15 +72,15 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 4505 Data size: 968719 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26150 Data size: 209200 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: all inputs
@@ -88,19 +88,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: cd
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint < 2000000000) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -110,10 +110,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -181,14 +181,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint < 2000000000) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -197,15 +197,15 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 4505 Data size: 968719 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26150 Data size: 209200 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: all inputs
@@ -213,19 +213,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: cd
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint < 2000000000) (type: boolean)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 4096 Data size: 880654 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -235,10 +235,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -302,14 +302,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint is not null (type: boolean)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -318,15 +318,15 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 19518 Data size: 156144 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: all inputs
@@ -334,19 +334,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: cd
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint is not null (type: boolean)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -356,10 +356,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -423,14 +423,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint is not null (type: boolean)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -439,15 +439,15 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 19518 Data size: 156144 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: all inputs
@@ -455,19 +455,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: cd
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint is not null (type: boolean)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 9173 Data size: 27396 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -477,10 +477,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -542,11 +542,11 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Outer Join0 to 1
@@ -555,15 +555,15 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26150 Data size: 209200 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: all inputs
@@ -571,16 +571,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: cd
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -590,10 +590,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -655,11 +655,11 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Outer Join0 to 1
@@ -668,15 +668,15 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26150 Data size: 209200 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Execution mode: llap
LLAP IO: all inputs
@@ -684,16 +684,16 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: cd
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -703,10 +703,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -802,14 +802,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -818,7 +818,7 @@ STAGE PLANS:
1 _col0 (type: string)
input vertices:
1 Map 3
- Statistics: Num rows: 577 Data size: 13721 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 577 Data size: 18341 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -834,19 +834,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p2
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -925,14 +925,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -941,7 +941,7 @@ STAGE PLANS:
1 _col0 (type: string)
input vertices:
1 Map 3
- Statistics: Num rows: 577 Data size: 13721 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 577 Data size: 18341 Basic stats: COMPLETE Column stats: NONE
HybridGraceHashJoin: true
Group By Operator
aggregations: count()
@@ -958,19 +958,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p2
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -1049,14 +1049,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1065,7 +1065,7 @@ STAGE PLANS:
1 _col0 (type: string)
input vertices:
1 Map 3
- Statistics: Num rows: 577 Data size: 13721 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 577 Data size: 18341 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1081,19 +1081,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p2
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -1172,14 +1172,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1188,7 +1188,7 @@ STAGE PLANS:
1 _col0 (type: string)
input vertices:
1 Map 3
- Statistics: Num rows: 577 Data size: 13721 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 577 Data size: 18341 Basic stats: COMPLETE Column stats: NONE
HybridGraceHashJoin: true
Group By Operator
aggregations: count()
@@ -1205,19 +1205,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p2
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 525 Data size: 12474 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 525 Data size: 16674 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
[11/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/sample1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/sample1.q.out b/ql/src/test/results/clientpositive/llap/sample1.q.out
index 68eb0d9..979c5e5 100644
--- a/ql/src/test/results/clientpositive/llap/sample1.q.out
+++ b/ql/src/test/results/clientpositive/llap/sample1.q.out
@@ -18,45 +18,6 @@ INSERT OVERWRITE TABLE dest1 SELECT s.*
FROM srcpart TABLESAMPLE (BUCKET 1 OUT OF 1 ON rand()) s
WHERE s.ds='2008-04-08' and s.hr='11'
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- srcpart
- TOK_TABLEBUCKETSAMPLE
- 1
- 1
- TOK_FUNCTION
- rand
- s
- TOK_INSERT
- TOK_DESTINATION
- TOK_TAB
- TOK_TABNAME
- dest1
- TOK_SELECT
- TOK_SELEXPR
- TOK_ALLCOLREF
- TOK_TABNAME
- s
- TOK_WHERE
- and
- =
- .
- TOK_TABLE_OR_COL
- s
- ds
- '2008-04-08'
- =
- .
- TOK_TABLE_OR_COL
- s
- hr
- '11'
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-2 depends on stages: Stage-1
@@ -72,36 +33,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Filter Operator
isSamplingPred: true
predicate: (((hash(rand()) & 2147483647) % 1) = 0) (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(key) (type: int), value (type: string), '2008-04-08' (type: string), '11' (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 68750 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 68750 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns key,value,dt,hr
columns.comments
columns.types int:string:string:string
#### A masked pattern was here ####
name default.dest1
+ numFiles 0
+ numRows 0
+ rawDataSize 0
serialization.ddl struct dest1 { i32 key, string value, string dt, string hr}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dest1
@@ -109,6 +75,7 @@ STAGE PLANS:
GatherStats: true
MultiFileSpray: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -121,7 +88,7 @@ STAGE PLANS:
ds 2008-04-08
hr 11
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -173,15 +140,20 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns key,value,dt,hr
columns.comments
columns.types int:string:string:string
#### A masked pattern was here ####
name default.dest1
+ numFiles 0
+ numRows 0
+ rawDataSize 0
serialization.ddl struct dest1 { i32 key, string value, string dt, string hr}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ totalSize 0
#### A masked pattern was here ####
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dest1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part.q.out
index c6a3a0a..9162533 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part.q.out
@@ -186,10 +186,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=99)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=99)
- default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=103)
+ default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_permute_select
@@ -445,10 +445,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=153)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=153)
- default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=157)
+ default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_string_permute_select
@@ -730,10 +730,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=234)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=234)
- default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=238)
+ default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_double
PREHOOK: type: QUERY
@@ -879,10 +879,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=9 width=530)
+ Select Operator [SEL_1] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=530)
- default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=534)
+ default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_timestamp
PREHOOK: type: QUERY
@@ -1003,10 +1003,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=9 width=529)
+ Select Operator [SEL_1] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=529)
- default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=533)
+ default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_date
PREHOOK: type: QUERY
@@ -1157,10 +1157,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=313)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=313)
- default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=317)
+ default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_string
PREHOOK: type: QUERY
@@ -1304,10 +1304,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=422)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=422)
- default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=426)
+ default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char
PREHOOK: type: QUERY
@@ -1451,10 +1451,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=314)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=314)
- default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=318)
+ default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char_trunc
PREHOOK: type: QUERY
@@ -1598,10 +1598,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=313)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=313)
- default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=317)
+ default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar
PREHOOK: type: QUERY
@@ -1745,10 +1745,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=311)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=311)
- default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=315)
+ default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar_trunc
PREHOOK: type: QUERY
@@ -1889,10 +1889,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=304)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=304)
- default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=308)
+ default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_string
PREHOOK: type: QUERY
@@ -2033,10 +2033,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=383)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=383)
- default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=387)
+ default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char
PREHOOK: type: QUERY
@@ -2177,10 +2177,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=306)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=306)
- default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=310)
+ default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char_trunc
PREHOOK: type: QUERY
@@ -2321,10 +2321,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=304)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=304)
- default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=308)
+ default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar
PREHOOK: type: QUERY
@@ -2465,10 +2465,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=302)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=302)
- default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=306)
+ default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar_trunc
PREHOOK: type: QUERY
@@ -2620,10 +2620,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=496)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=496)
- default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=500)
+ default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_string_group_string_group_string
PREHOOK: type: QUERY
@@ -2764,10 +2764,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=427)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=427)
- default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=431)
+ default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_char
PREHOOK: type: QUERY
@@ -2908,10 +2908,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=402)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=402)
- default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=406)
+ default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_varchar
PREHOOK: type: QUERY
@@ -3069,10 +3069,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=183)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=10 width=183)
- default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=10 width=187)
+ default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_lower_to_higher_numeric_group_tinyint
PREHOOK: type: QUERY
@@ -3219,10 +3219,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=182)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=10 width=182)
- default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=10 width=186)
+ default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_lower_to_higher_numeric_group_smallint
PREHOOK: type: QUERY
@@ -3366,10 +3366,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=178)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=178)
- default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=182)
+ default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_lower_to_higher_numeric_group_int
PREHOOK: type: QUERY
@@ -3510,10 +3510,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=177)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=177)
- default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=181)
+ default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_lower_to_higher_numeric_group_bigint
PREHOOK: type: QUERY
@@ -3651,10 +3651,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=189)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=189)
- default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=193)
+ default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_decimal
PREHOOK: type: QUERY
@@ -3789,10 +3789,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=10 width=99)
+ Select Operator [SEL_1] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=99)
- default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=103)
+ default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_float
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_complex.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_complex.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_complex.q.out
index 7b31742..990375a 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_complex.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_complex.q.out
@@ -164,10 +164,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=785)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=6 width=785)
- default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:NONE,Output:["insert_num","s1","b"]
+ TableScan [TS_0] (rows=6 width=789)
+ default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","s1","b"]
PREHOOK: query: select insert_num,part,s1,b from part_change_various_various_struct1
PREHOOK: type: QUERY
@@ -428,10 +428,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=8 width=610)
+ Select Operator [SEL_1] (rows=8 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=8 width=610)
- default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s2"]
+ TableScan [TS_0] (rows=8 width=614)
+ default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s2"]
PREHOOK: query: select insert_num,part,b,s2 from part_add_various_various_struct2
PREHOOK: type: QUERY
@@ -616,10 +616,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=4 width=289)
+ Select Operator [SEL_1] (rows=4 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=4 width=289)
- default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s3"]
+ TableScan [TS_0] (rows=4 width=293)
+ default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s3"]
PREHOOK: query: select insert_num,part,b,s3 from part_add_to_various_various_struct4
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_primitive.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_primitive.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_primitive.q.out
index 1ec8895..41dfdfb 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_primitive.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_orc_nonvec_mapwork_part_all_primitive.q.out
@@ -140,10 +140,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=285)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
- TableScan [TS_0] (rows=6 width=285)
- default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
+ TableScan [TS_0] (rows=6 width=289)
+ default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,b from part_change_various_various_boolean
PREHOOK: type: QUERY
@@ -309,10 +309,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=438)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=438)
- default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=442)
+ default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_tinyint
PREHOOK: type: QUERY
@@ -479,10 +479,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=439)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=439)
- default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=443)
+ default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_smallint
PREHOOK: type: QUERY
@@ -649,10 +649,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=435)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=435)
- default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=439)
+ default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_int
PREHOOK: type: QUERY
@@ -819,10 +819,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=447)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=447)
- default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=451)
+ default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_bigint
PREHOOK: type: QUERY
@@ -987,10 +987,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=437)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=437)
- default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=441)
+ default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_float
PREHOOK: type: QUERY
@@ -1155,10 +1155,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=448)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=448)
- default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=452)
+ default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_double
PREHOOK: type: QUERY
@@ -1323,10 +1323,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=701)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=701)
- default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=705)
+ default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_decimal
PREHOOK: type: QUERY
@@ -1497,10 +1497,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=673)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=673)
- default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=677)
+ default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_string
PREHOOK: type: QUERY
@@ -1671,10 +1671,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=725)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=725)
- default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=729)
+ default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char
PREHOOK: type: QUERY
@@ -1845,10 +1845,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=661)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=661)
- default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=665)
+ default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char_trunc
PREHOOK: type: QUERY
@@ -2019,10 +2019,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=672)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=672)
- default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=676)
+ default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar
PREHOOK: type: QUERY
@@ -2193,10 +2193,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=667)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=667)
- default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=671)
+ default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar_trunc
PREHOOK: type: QUERY
@@ -2364,10 +2364,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=7 width=406)
+ Select Operator [SEL_1] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
- TableScan [TS_0] (rows=7 width=406)
- default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
+ TableScan [TS_0] (rows=7 width=410)
+ default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,b from part_change_various_various_timestamp
PREHOOK: type: QUERY
@@ -2508,10 +2508,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=407)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=6 width=407)
- default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=6 width=411)
+ default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_various_various_date
PREHOOK: type: QUERY
@@ -2746,10 +2746,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=13 width=668)
+ Select Operator [SEL_1] (rows=13 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=13 width=668)
- default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=13 width=672)
+ default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_same_type_different_params
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part.q.out
index 8c78b66..892d5c0 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part.q.out
@@ -186,10 +186,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=99)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=99)
- default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=103)
+ default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_permute_select
@@ -445,10 +445,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=153)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=153)
- default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=157)
+ default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_string_permute_select
@@ -730,10 +730,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=234)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=234)
- default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=238)
+ default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_double
PREHOOK: type: QUERY
@@ -879,10 +879,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=9 width=530)
+ Select Operator [SEL_3] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=530)
- default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=534)
+ default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_timestamp
PREHOOK: type: QUERY
@@ -1003,10 +1003,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=9 width=529)
+ Select Operator [SEL_3] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=529)
- default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=533)
+ default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_date
PREHOOK: type: QUERY
@@ -1157,10 +1157,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=313)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=313)
- default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=317)
+ default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_string
PREHOOK: type: QUERY
@@ -1304,10 +1304,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=422)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=422)
- default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=426)
+ default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char
PREHOOK: type: QUERY
@@ -1451,10 +1451,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=314)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=314)
- default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=318)
+ default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char_trunc
PREHOOK: type: QUERY
@@ -1598,10 +1598,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=313)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=313)
- default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=317)
+ default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar
PREHOOK: type: QUERY
@@ -1745,10 +1745,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=311)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=311)
- default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=315)
+ default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar_trunc
PREHOOK: type: QUERY
@@ -1889,10 +1889,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=304)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=304)
- default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=308)
+ default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_string
PREHOOK: type: QUERY
@@ -2033,10 +2033,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=383)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=383)
- default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=387)
+ default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char
PREHOOK: type: QUERY
@@ -2177,10 +2177,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=306)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=306)
- default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=310)
+ default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char_trunc
PREHOOK: type: QUERY
@@ -2321,10 +2321,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=304)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=304)
- default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=308)
+ default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar
PREHOOK: type: QUERY
@@ -2465,10 +2465,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=302)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=302)
- default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=306)
+ default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar_trunc
PREHOOK: type: QUERY
@@ -2620,10 +2620,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=496)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=496)
- default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=500)
+ default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_string_group_string_group_string
PREHOOK: type: QUERY
@@ -2764,10 +2764,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=427)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=427)
- default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=431)
+ default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_char
PREHOOK: type: QUERY
@@ -2908,10 +2908,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=402)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=402)
- default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=406)
+ default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_varchar
PREHOOK: type: QUERY
@@ -3069,10 +3069,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=183)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=10 width=183)
- default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=10 width=187)
+ default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_lower_to_higher_numeric_group_tinyint
PREHOOK: type: QUERY
@@ -3219,10 +3219,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=182)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=10 width=182)
- default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=10 width=186)
+ default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_lower_to_higher_numeric_group_smallint
PREHOOK: type: QUERY
@@ -3366,10 +3366,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=178)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=178)
- default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=182)
+ default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_lower_to_higher_numeric_group_int
PREHOOK: type: QUERY
@@ -3510,10 +3510,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=177)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=177)
- default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=181)
+ default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_lower_to_higher_numeric_group_bigint
PREHOOK: type: QUERY
@@ -3651,10 +3651,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=189)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=189)
- default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=193)
+ default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_decimal
PREHOOK: type: QUERY
@@ -3789,10 +3789,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=99)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=99)
- default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=103)
+ default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_float
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_complex.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_complex.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_complex.q.out
index 4f84617..26f39c4 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_complex.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_complex.q.out
@@ -164,10 +164,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=785)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=6 width=785)
- default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:NONE,Output:["insert_num","s1","b"]
+ TableScan [TS_0] (rows=6 width=789)
+ default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","s1","b"]
PREHOOK: query: select insert_num,part,s1,b from part_change_various_various_struct1
PREHOOK: type: QUERY
@@ -428,10 +428,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=8 width=610)
+ Select Operator [SEL_1] (rows=8 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=8 width=610)
- default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s2"]
+ TableScan [TS_0] (rows=8 width=614)
+ default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s2"]
PREHOOK: query: select insert_num,part,b,s2 from part_add_various_various_struct2
PREHOOK: type: QUERY
@@ -616,10 +616,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=4 width=289)
+ Select Operator [SEL_1] (rows=4 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=4 width=289)
- default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s3"]
+ TableScan [TS_0] (rows=4 width=293)
+ default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s3"]
PREHOOK: query: select insert_num,part,b,s3 from part_add_to_various_various_struct4
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_primitive.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_primitive.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_primitive.q.out
index 8b929a4..963ef0d 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_primitive.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_orc_vec_mapwork_part_all_primitive.q.out
@@ -140,10 +140,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=6 width=285)
+ Select Operator [SEL_3] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
- TableScan [TS_0] (rows=6 width=285)
- default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
+ TableScan [TS_0] (rows=6 width=289)
+ default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,b from part_change_various_various_boolean
PREHOOK: type: QUERY
@@ -309,10 +309,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=438)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=438)
- default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=442)
+ default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_tinyint
PREHOOK: type: QUERY
@@ -479,10 +479,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=439)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=439)
- default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=443)
+ default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_smallint
PREHOOK: type: QUERY
@@ -649,10 +649,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=435)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=435)
- default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=439)
+ default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_int
PREHOOK: type: QUERY
@@ -819,10 +819,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=447)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=447)
- default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=451)
+ default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_bigint
PREHOOK: type: QUERY
@@ -987,10 +987,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=437)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=437)
- default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=441)
+ default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_float
PREHOOK: type: QUERY
@@ -1155,10 +1155,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=448)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=448)
- default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=452)
+ default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_double
PREHOOK: type: QUERY
@@ -1323,10 +1323,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=701)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=701)
- default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=705)
+ default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_decimal
PREHOOK: type: QUERY
@@ -1497,10 +1497,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=673)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=673)
- default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=677)
+ default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_string
PREHOOK: type: QUERY
@@ -1671,10 +1671,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=725)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=725)
- default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=729)
+ default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char
PREHOOK: type: QUERY
@@ -1845,10 +1845,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=661)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=661)
- default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=665)
+ default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char_trunc
PREHOOK: type: QUERY
@@ -2019,10 +2019,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=672)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=672)
- default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=676)
+ default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar
PREHOOK: type: QUERY
@@ -2193,10 +2193,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=667)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=667)
- default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=671)
+ default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar_trunc
PREHOOK: type: QUERY
@@ -2364,10 +2364,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=406)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
- TableScan [TS_0] (rows=7 width=406)
- default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
+ TableScan [TS_0] (rows=7 width=410)
+ default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,b from part_change_various_various_timestamp
PREHOOK: type: QUERY
@@ -2508,10 +2508,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=6 width=407)
+ Select Operator [SEL_3] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=6 width=407)
- default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=6 width=411)
+ default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_various_various_date
PREHOOK: type: QUERY
@@ -2746,10 +2746,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=13 width=668)
+ Select Operator [SEL_3] (rows=13 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=13 width=668)
- default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=13 width=672)
+ default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_same_type_different_params
PREHOOK: type: QUERY
[43/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out
index 5459914..9549778 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out
@@ -61,6 +61,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -73,27 +74,31 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -101,15 +106,20 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -120,8 +130,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -168,6 +178,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -180,31 +191,35 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
outputColumnNames: _col0
- input vertices:
- 1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -212,28 +227,33 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -295,6 +315,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 4 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -308,31 +329,33 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
outputColumnNames: _col0
- input vertices:
- 1 Map 4
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 4
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -340,47 +363,51 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
keys: KEY._col0 (type: int)
mode: mergepartial
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Select Operator
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -459,6 +486,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Map 4 <- Map 6 (BROADCAST_EDGE)
@@ -473,31 +501,35 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
outputColumnNames: _col0
- input vertices:
- 1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -505,13 +537,18 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -519,31 +556,35 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
outputColumnNames: _col0
- input vertices:
- 1 Map 6
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 6
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 6
Map Operator Tree:
TableScan
@@ -551,22 +592,27 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -576,18 +622,17 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
1 Reducer 5
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: bigint), _col3 (type: bigint)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 2 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: llap
@@ -597,12 +642,12 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint)
Stage: Stage-0
@@ -680,6 +725,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -706,7 +752,6 @@ STAGE PLANS:
input vertices:
1 Map 3
Statistics: Num rows: 3 Data size: 23 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
Group By Operator
aggregations: count()
mode: hash
@@ -717,6 +762,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -735,8 +781,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -747,8 +794,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -811,9 +858,10 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 2 <- Map 1 (BROADCAST_EDGE)
- Reducer 3 <- Map 2 (SIMPLE_EDGE)
+ Map 1 <- Map 3 (BROADCAST_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -822,48 +870,53 @@ STAGE PLANS:
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((key < 8) and (key < 6)) and key is not null) (type: boolean)
+ predicate: ((key < 8) and (key < 6)) (type: boolean)
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
- Map 2
+ LLAP IO: no inputs
+ Map 3
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 key (type: int)
- input vertices:
- 0 Map 1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ predicate: ((key < 8) and (key < 6)) (type: boolean)
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
Execution mode: llap
- Reducer 3
- Execution mode: uber
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -874,8 +927,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -962,6 +1015,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -973,7 +1027,7 @@ STAGE PLANS:
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((key < 8) and (key < 6)) and key is not null) (type: boolean)
+ predicate: ((key < 8) and (key < 6)) (type: boolean)
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
@@ -988,7 +1042,6 @@ STAGE PLANS:
input vertices:
1 Map 3
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
Group By Operator
aggregations: count()
mode: hash
@@ -999,13 +1052,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((key < 8) and (key < 6)) and key is not null) (type: boolean)
+ predicate: ((key < 8) and (key < 6)) (type: boolean)
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
@@ -1017,8 +1071,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1029,8 +1084,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1107,6 +1162,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -1133,7 +1189,6 @@ STAGE PLANS:
input vertices:
1 Map 3
Statistics: Num rows: 3 Data size: 23 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
Group By Operator
aggregations: count()
mode: hash
@@ -1144,6 +1199,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -1162,8 +1218,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1174,8 +1231,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1228,6 +1285,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -1238,13 +1296,13 @@ STAGE PLANS:
TableScan
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: (key + 1) (type: int)
- outputColumnNames: _col0
+ Filter Operator
+ predicate: (key + 1) is not null (type: boolean)
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: (key + 1) (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -1253,8 +1311,7 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1265,26 +1322,28 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: (key + 1) (type: int)
- outputColumnNames: _col0
+ Filter Operator
+ predicate: (key + 1) is not null (type: boolean)
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: (key + 1) (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1295,8 +1354,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1345,9 +1404,10 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 2 <- Map 1 (BROADCAST_EDGE)
- Reducer 3 <- Map 2 (SIMPLE_EDGE)
+ Map 1 <- Map 3 (BROADCAST_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1362,42 +1422,47 @@ STAGE PLANS:
expressions: key (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 3 Data size: 23 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
- Map 2
+ LLAP IO: no inputs
+ Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 key (type: int)
- input vertices:
- 0 Map 1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ predicate: (key < 6) (type: boolean)
+ Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
Execution mode: llap
- Reducer 3
- Execution mode: uber
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1408,8 +1473,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1456,9 +1521,10 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 2 <- Map 1 (BROADCAST_EDGE)
- Reducer 3 <- Map 2 (SIMPLE_EDGE)
+ Map 1 <- Map 3 (BROADCAST_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1473,42 +1539,47 @@ STAGE PLANS:
expressions: key (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 3 Data size: 23 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
- Map 2
+ LLAP IO: no inputs
+ Map 3
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 _col0 (type: int)
- input vertices:
- 1 Map 1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ predicate: (key < 6) (type: boolean)
+ Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
Execution mode: llap
- Reducer 3
- Execution mode: uber
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1519,8 +1590,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1577,6 +1648,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE), Map 4 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -1606,7 +1678,6 @@ STAGE PLANS:
1 Map 3
2 Map 4
Statistics: Num rows: 6 Data size: 46 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
Group By Operator
aggregations: count()
mode: hash
@@ -1617,6 +1688,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -1635,6 +1707,7 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -1653,8 +1726,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1665,8 +1739,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1737,9 +1811,10 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 2 <- Map 1 (BROADCAST_EDGE)
- Reducer 3 <- Map 2 (SIMPLE_EDGE)
+ Map 1 <- Map 3 (BROADCAST_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1748,48 +1823,53 @@ STAGE PLANS:
alias: a
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((key < 8) and (key < 6)) and key is not null) (type: boolean)
+ predicate: ((key < 8) and (key < 6)) (type: boolean)
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
- Map 2
+ LLAP IO: no inputs
+ Map 3
Map Operator Tree:
TableScan
alias: b
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 key (type: int)
- input vertices:
- 0 Map 1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ predicate: ((key < 8) and (key < 6)) (type: boolean)
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
Execution mode: llap
- Reducer 3
- Execution mode: uber
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1800,8 +1880,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1860,6 +1940,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -1872,27 +1953,31 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- input vertices:
- 1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -1900,15 +1985,20 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1919,8 +2009,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1967,6 +2057,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -1979,31 +2070,35 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
outputColumnNames: _col0
- input vertices:
- 1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -2011,28 +2106,33 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2094,6 +2194,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 4 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -2107,31 +2208,33 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
outputColumnNames: _col0
- input vertices:
- 1 Map 4
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 4
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -2139,47 +2242,51 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Group By Operator
- aggregations: count(VALUE._col0)
keys: KEY._col0 (type: int)
mode: mergepartial
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col0
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Select Operator
- Statistics: Num rows: 2 Data size: 15 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2258,6 +2365,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Map 4 <- Map 6 (BROADCAST_EDGE)
@@ -2272,31 +2380,35 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 35 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
outputColumnNames: _col0
- input vertices:
- 1 Map 3
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- keys: _col0 (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 38 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 3
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ keys: _col0 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 11 Data size: 77 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
@@ -2304,13 +2416,18 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 70 Basic stats: COMPLETE Column stats: NONE
<TRUNCATED>
[21/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/mergejoin.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/mergejoin.q.out b/ql/src/test/results/clientpositive/llap/mergejoin.q.out
index 77bb875..b73b427 100644
--- a/ql/src/test/results/clientpositive/llap/mergejoin.q.out
+++ b/ql/src/test/results/clientpositive/llap/mergejoin.q.out
@@ -15,6 +15,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -24,41 +25,43 @@ STAGE PLANS:
TableScan
alias: a
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -69,13 +72,13 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 21180 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 21180 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -256,6 +259,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -266,39 +270,41 @@ STAGE PLANS:
TableScan
alias: a
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -308,7 +314,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -319,7 +325,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -330,8 +336,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1343,6 +1349,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1352,33 +1359,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
- filterExpr: (key) IN (RS[4]) (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1388,7 +1396,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1399,7 +1407,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1410,8 +1418,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1452,6 +1460,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1461,33 +1470,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- filterExpr: (key) IN (RS[5]) (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1497,7 +1507,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1508,7 +1518,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1519,8 +1529,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1561,6 +1571,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1570,32 +1581,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1605,7 +1618,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1616,7 +1629,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1627,8 +1640,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1665,6 +1678,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Map 6 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
@@ -1676,59 +1690,62 @@ STAGE PLANS:
TableScan
alias: a
filterExpr: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 5
Map Operator Tree:
TableScan
alias: b
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
alias: c
filterExpr: value is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: value (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1739,12 +1756,12 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col1
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -1754,7 +1771,7 @@ STAGE PLANS:
keys:
0 _col1 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 605 Data size: 56870 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 61710 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1765,7 +1782,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 4
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1776,8 +1793,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1814,6 +1831,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1824,39 +1842,41 @@ STAGE PLANS:
TableScan
alias: a
filterExpr: value is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: value (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
filterExpr: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: value (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1866,7 +1886,7 @@ STAGE PLANS:
keys:
0 _col1 (type: string)
1 _col1 (type: string)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1877,7 +1897,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1888,8 +1908,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1932,6 +1952,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 7 <- Union 3 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -1944,77 +1965,81 @@ STAGE PLANS:
TableScan
alias: s1
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
- alias: s1
+ alias: s3
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 7
Map Operator Tree:
TableScan
- alias: s1
+ alias: s2
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 508 Data size: 47770 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 508 Data size: 51836 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 8
Map Operator Tree:
TableScan
alias: b
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -2025,12 +2050,12 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 266 Data size: 25022 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 27152 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 508 Data size: 47770 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 508 Data size: 51836 Basic stats: COMPLETE Column stats: NONE
Reducer 4
Execution mode: llap
Reduce Operator Tree:
@@ -2040,7 +2065,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 558 Data size: 52547 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 558 Data size: 57019 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2051,7 +2076,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 5
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -2062,8 +2087,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 3
Vertex: Union 3
@@ -2085,6 +2110,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -2095,39 +2121,41 @@ STAGE PLANS:
TableScan
alias: a
filterExpr: value is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: value (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
filterExpr: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: value (type: string)
outputColumnNames: _col1
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -2137,7 +2165,7 @@ STAGE PLANS:
keys:
0 _col1 (type: string)
1 _col1 (type: string)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2148,7 +2176,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -2159,8 +2187,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2195,6 +2223,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Map 6 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
@@ -2206,59 +2235,62 @@ STAGE PLANS:
TableScan
alias: a
filterExpr: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 5
Map Operator Tree:
TableScan
alias: b
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
alias: c
filterExpr: value is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: value (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 2225 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -2269,12 +2301,12 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col1
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -2284,7 +2316,7 @@ STAGE PLANS:
keys:
0 _col1 (type: string)
1 _col0 (type: string)
- Statistics: Num rows: 605 Data size: 56870 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 61710 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2295,7 +2327,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 4
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -2306,8 +2338,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2352,6 +2384,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 7 <- Union 3 (CONTAINS)
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE), Union 3 (CONTAINS)
@@ -2364,77 +2397,81 @@ STAGE PLANS:
TableScan
alias: s1
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 6
Map Operator Tree:
TableScan
- alias: s1
+ alias: s3
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 7
Map Operator Tree:
TableScan
- alias: s1
+ alias: s2
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 508 Data size: 47770 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 508 Data size: 51836 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 8
Map Operator Tree:
TableScan
alias: b
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -2445,12 +2482,12 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 266 Data size: 25022 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 27152 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 508 Data size: 47770 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 508 Data size: 51836 Basic stats: COMPLETE Column stats: NONE
Reducer 4
Execution mode: llap
Reduce Operator Tree:
@@ -2460,7 +2497,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 558 Data size: 52547 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 558 Data size: 57019 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2471,7 +2508,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 5
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -2482,8 +2519,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 3
Vertex: Union 3
@@ -2519,6 +2556,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
@@ -2531,49 +2569,51 @@ STAGE PLANS:
TableScan
alias: t1
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Map 5
Map Operator Tree:
TableScan
alias: t2
filterExpr: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Execution mode: vectorized, llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: vectorized, llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 22748 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 24684 Basic stats: COMPLETE Column stats: NONE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -2583,7 +2623,7 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 550 Data size: 51700 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 56100 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -2594,7 +2634,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 4
- Execution mode: vectorized, uber
+ Execution mode: vectorized, llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -2605,8 +2645,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: vectorized, llap
@@ -2614,12 +2654,12 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 47000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 51000 Basic stats: COMPLETE Column stats: NONE
Stage: Stage-0
Fetch Operator
@@ -2654,6 +2694,7 @@ POSTHOOK: Input: default@tab_part
POSTHOOK: Input: default@tab_part@ds=2008-04-08
#### A masked pattern was here ####
480
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select * from (select * from tab where tab.key = 0)a full outer join (select * from tab_part where tab_part.key = 98)b on a.key = b.key
PREHOOK: type: QUERY
PREHOOK: Input: default@tab
@@ -2673,22 +2714,22 @@ POSTHOOK: Input: default@tab_part@ds=2008-04-08
0 val_0 2008-04-08 NULL NULL NULL
NULL NULL NULL 98 val_98 2008-04-08
NULL NULL NULL 98 val_98 2008-04-08
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select * from (select * from tab where tab.key = 0)a right outer join (select * from tab_part where tab_part.key = 98)b on a.key = b.key
PREHOOK: type: QUERY
PREHOOK: Input: default@tab
-PREHOOK: Input: default@tab@ds=2008-04-08
PREHOOK: Input: default@tab_part
PREHOOK: Input: default@tab_part@ds=2008-04-08
#### A masked pattern was here ####
POSTHOOK: query: select * from (select * from tab where tab.key = 0)a right outer join (select * from tab_part where tab_part.key = 98)b on a.key = b.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@tab
-POSTHOOK: Input: default@tab@ds=2008-04-08
POSTHOOK: Input: default@tab_part
POSTHOOK: Input: default@tab_part@ds=2008-04-08
#### A masked pattern was here ####
NULL NULL NULL 98 val_98 2008-04-08
NULL NULL NULL 98 val_98 2008-04-08
+Warning: Shuffle Join MERGEJOIN[19][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select * from
(select * from tab where tab.key = 0)a
full outer join
@@ -2709,13 +2750,14 @@ POSTHOOK: Input: default@tab@ds=2008-04-08
POSTHOOK: Input: default@tab_part
POSTHOOK: Input: default@tab_part@ds=2008-04-08
#### A masked pattern was here ####
+Warning: Shuffle Join MERGEJOIN[20][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[21][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from
(select * from tab where tab.key = 0)a
full outer join
(select * from tab_part where tab_part.key = 98)b on a.key = b.key join tab_part c on b.key = c.key
PREHOOK: type: QUERY
PREHOOK: Input: default@tab
-PREHOOK: Input: default@tab@ds=2008-04-08
PREHOOK: Input: default@tab_part
PREHOOK: Input: default@tab_part@ds=2008-04-08
#### A masked pattern was here ####
@@ -2725,7 +2767,6 @@ full outer join
(select * from tab_part where tab_part.key = 98)b on a.key = b.key join tab_part c on b.key = c.key
POSTHOOK: type: QUERY
POSTHOOK: Input: default@tab
-POSTHOOK: Input: default@tab@ds=2008-04-08
POSTHOOK: Input: default@tab_part
POSTHOOK: Input: default@tab_part@ds=2008-04-08
#### A masked pattern was here ####
@@ -2733,7 +2774,8 @@ NULL NULL NULL 98 val_98 2008-04-08 98 val_98 2008-04-08
NULL NULL NULL 98 val_98 2008-04-08 98 val_98 2008-04-08
NULL NULL NULL 98 val_98 2008-04-08 98 val_98 2008-04-08
NULL NULL NULL 98 val_98 2008-04-08 98 val_98 2008-04-08
-Warning: Shuffle Join MERGEJOIN[20][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[18][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[19][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from
(select * from tab where tab.key = 0)a
join
@@ -3260,6 +3302,8 @@ NULL NULL NULL NULL NULL NULL 97 val_97 2008-04-08
NULL NULL NULL NULL NULL NULL 97 val_97 2008-04-08
NULL NULL NULL NULL NULL NULL 98 val_98 2008-04-08
NULL NULL NULL NULL NULL NULL 98 val_98 2008-04-08
+Warning: Shuffle Join MERGEJOIN[19][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[20][tables = [$hdt$_0, $hdt$_1, $hdt$_2]] in Stage 'Reducer 3' is a cross product
PREHOOK: query: select * from
(select * from tab where tab.key = 0)a
join
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/mergejoin_3way.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/mergejoin_3way.q.out b/ql/src/test/results/clientpositive/llap/mergejoin_3way.q.out
new file mode 100644
index 0000000..e644051
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/mergejoin_3way.q.out
@@ -0,0 +1,30 @@
+PREHOOK: query: select
+ a.key, b.value, c.value
+from
+ src a,
+ src1 b,
+ src1 c
+where
+ a.key = b.key and a.key = c.key
+ and b.key != '' and b.value != ''
+ and a.value > 'wal_6789'
+ and c.value > 'wal_6789'
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+PREHOOK: Input: default@src1
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ a.key, b.value, c.value
+from
+ src a,
+ src1 b,
+ src1 c
+where
+ a.key = b.key and a.key = c.key
+ and b.key != '' and b.value != ''
+ and a.value > 'wal_6789'
+ and c.value > 'wal_6789'
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+POSTHOOK: Input: default@src1
+#### A masked pattern was here ####
[09/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part.q.out
index 9fcf610..b35d3b1 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part.q.out
@@ -190,10 +190,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=11)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=11)
- default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=15)
+ default@part_add_int_permute_select,part_add_int_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_permute_select
@@ -449,10 +449,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=15)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=15)
- default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:NONE,Output:["insert_num","a","b"]
+ TableScan [TS_0] (rows=10 width=19)
+ default@part_add_int_string_permute_select,part_add_int_string_permute_select,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","a","b"]
PREHOOK: query: -- SELECT permutation columns to make sure NULL defaulting works right
select insert_num,part,a,b from part_add_int_string_permute_select
@@ -734,10 +734,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=38)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=38)
- default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=42)
+ default@part_change_string_group_double,part_change_string_group_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_double
PREHOOK: type: QUERY
@@ -883,10 +883,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=9 width=132)
+ Select Operator [SEL_3] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=132)
- default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=136)
+ default@part_change_date_group_string_group_timestamp,part_change_date_group_string_group_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_timestamp
PREHOOK: type: QUERY
@@ -1007,10 +1007,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=9 width=61)
+ Select Operator [SEL_3] (rows=9 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=9 width=61)
- default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=9 width=65)
+ default@part_change_date_group_string_group_date,part_change_date_group_string_group_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_date_group_string_group_date
PREHOOK: type: QUERY
@@ -1161,10 +1161,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=33)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_numeric_group_string_group_multi_ints_string,part_change_numeric_group_string_group_multi_ints_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_string
PREHOOK: type: QUERY
@@ -1308,10 +1308,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=33)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_numeric_group_string_group_multi_ints_char,part_change_numeric_group_string_group_multi_ints_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char
PREHOOK: type: QUERY
@@ -1455,10 +1455,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=30)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=30)
- default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=34)
+ default@part_change_numeric_group_string_group_multi_ints_char_trunc,part_change_numeric_group_string_group_multi_ints_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_char_trunc
PREHOOK: type: QUERY
@@ -1602,10 +1602,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=33)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_numeric_group_string_group_multi_ints_varchar,part_change_numeric_group_string_group_multi_ints_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar
PREHOOK: type: QUERY
@@ -1749,10 +1749,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=30)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=30)
- default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=34)
+ default@part_change_numeric_group_string_group_multi_ints_varchar_trunc,part_change_numeric_group_string_group_multi_ints_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_numeric_group_string_group_multi_ints_varchar_trunc
PREHOOK: type: QUERY
@@ -1893,10 +1893,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=38)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=38)
- default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=42)
+ default@part_change_numeric_group_string_group_floating_string,part_change_numeric_group_string_group_floating_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_string
PREHOOK: type: QUERY
@@ -2037,10 +2037,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=38)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=38)
- default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=42)
+ default@part_change_numeric_group_string_group_floating_char,part_change_numeric_group_string_group_floating_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char
PREHOOK: type: QUERY
@@ -2181,10 +2181,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=35)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=35)
- default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=39)
+ default@part_change_numeric_group_string_group_floating_char_trunc,part_change_numeric_group_string_group_floating_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_char_trunc
PREHOOK: type: QUERY
@@ -2325,10 +2325,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=36)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=36)
- default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=40)
+ default@part_change_numeric_group_string_group_floating_varchar,part_change_numeric_group_string_group_floating_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar
PREHOOK: type: QUERY
@@ -2469,10 +2469,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=36)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=36)
- default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=40)
+ default@part_change_numeric_group_string_group_floating_varchar_trunc,part_change_numeric_group_string_group_floating_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_numeric_group_string_group_floating_varchar_trunc
PREHOOK: type: QUERY
@@ -2624,10 +2624,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=54)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=54)
- default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=58)
+ default@part_change_string_group_string_group_string,part_change_string_group_string_group_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_string_group_string_group_string
PREHOOK: type: QUERY
@@ -2768,10 +2768,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=46)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=46)
- default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=50)
+ default@part_change_string_group_string_group_char,part_change_string_group_string_group_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_char
PREHOOK: type: QUERY
@@ -2912,10 +2912,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=45)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=45)
- default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=49)
+ default@part_change_string_group_string_group_varchar,part_change_string_group_string_group_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_string_group_string_group_varchar
PREHOOK: type: QUERY
@@ -3073,10 +3073,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=44)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=10 width=44)
- default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=10 width=48)
+ default@part_change_lower_to_higher_numeric_group_tinyint,part_change_lower_to_higher_numeric_group_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_lower_to_higher_numeric_group_tinyint
PREHOOK: type: QUERY
@@ -3223,10 +3223,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=44)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7"]
- TableScan [TS_0] (rows=10 width=44)
- default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","b"]
+ TableScan [TS_0] (rows=10 width=48)
+ default@part_change_lower_to_higher_numeric_group_smallint,part_change_lower_to_higher_numeric_group_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,b from part_change_lower_to_higher_numeric_group_smallint
PREHOOK: type: QUERY
@@ -3370,10 +3370,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=42)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=10 width=42)
- default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=10 width=46)
+ default@part_change_lower_to_higher_numeric_group_int,part_change_lower_to_higher_numeric_group_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_lower_to_higher_numeric_group_int
PREHOOK: type: QUERY
@@ -3514,10 +3514,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=41)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- TableScan [TS_0] (rows=10 width=41)
- default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","b"]
+ TableScan [TS_0] (rows=10 width=45)
+ default@part_change_lower_to_higher_numeric_group_bigint,part_change_lower_to_higher_numeric_group_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,b from part_change_lower_to_higher_numeric_group_bigint
PREHOOK: type: QUERY
@@ -3655,10 +3655,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=33)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=33)
- default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=37)
+ default@part_change_lower_to_higher_numeric_group_decimal,part_change_lower_to_higher_numeric_group_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_decimal
PREHOOK: type: QUERY
@@ -3793,10 +3793,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=10 width=14)
+ Select Operator [SEL_3] (rows=10 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=10 width=14)
- default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","b"]
+ TableScan [TS_0] (rows=10 width=18)
+ default@part_change_lower_to_higher_numeric_group_float,part_change_lower_to_higher_numeric_group_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","b"]
PREHOOK: query: select insert_num,part,c1,b from part_change_lower_to_higher_numeric_group_float
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_complex.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_complex.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_complex.q.out
index 446ea52..e0bdf5d 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_complex.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_complex.q.out
@@ -170,10 +170,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=6 width=151)
+ Select Operator [SEL_1] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=6 width=151)
- default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:NONE,Output:["insert_num","s1","b"]
+ TableScan [TS_0] (rows=6 width=155)
+ default@part_change_various_various_struct1,part_change_various_various_struct1,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","s1","b"]
PREHOOK: query: select insert_num,part,s1,b from part_change_various_various_struct1
PREHOOK: type: QUERY
@@ -434,10 +434,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=8 width=113)
+ Select Operator [SEL_1] (rows=8 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=8 width=113)
- default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s2"]
+ TableScan [TS_0] (rows=8 width=117)
+ default@part_add_various_various_struct2,part_add_various_various_struct2,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s2"]
PREHOOK: query: select insert_num,part,b,s2 from part_add_various_various_struct2
PREHOOK: type: QUERY
@@ -622,10 +622,10 @@ Stage-0
Stage-1
Map 1 llap
File Output Operator [FS_2]
- Select Operator [SEL_1] (rows=4 width=84)
+ Select Operator [SEL_1] (rows=4 width=4)
Output:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0] (rows=4 width=84)
- default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:NONE,Output:["insert_num","b","s3"]
+ TableScan [TS_0] (rows=4 width=88)
+ default@part_add_to_various_various_struct4,part_add_to_various_various_struct4,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","b","s3"]
PREHOOK: query: select insert_num,part,b,s3 from part_add_to_various_various_struct4
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_primitive.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_primitive.q.out b/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_primitive.q.out
index 5bb2d64..8525c72 100644
--- a/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_primitive.q.out
+++ b/ql/src/test/results/clientpositive/llap/schema_evol_text_vecrow_mapwork_part_all_primitive.q.out
@@ -144,10 +144,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=6 width=96)
+ Select Operator [SEL_3] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
- TableScan [TS_0] (rows=6 width=96)
- default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
+ TableScan [TS_0] (rows=6 width=100)
+ default@part_change_various_various_boolean,part_change_various_various_boolean,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,b from part_change_various_various_boolean
PREHOOK: type: QUERY
@@ -313,10 +313,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=105)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=105)
- default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=109)
+ default@part_change_various_various_tinyint,part_change_various_various_tinyint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_tinyint
PREHOOK: type: QUERY
@@ -483,10 +483,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=111)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=111)
- default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=115)
+ default@part_change_various_various_smallint,part_change_various_various_smallint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_smallint
PREHOOK: type: QUERY
@@ -653,10 +653,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=114)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=114)
- default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=118)
+ default@part_change_various_various_int,part_change_various_various_int,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_int
PREHOOK: type: QUERY
@@ -823,10 +823,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=119)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=119)
- default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=123)
+ default@part_change_various_various_bigint,part_change_various_various_bigint,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_bigint
PREHOOK: type: QUERY
@@ -991,10 +991,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=118)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=118)
- default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=122)
+ default@part_change_various_various_float,part_change_various_various_float,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_float
PREHOOK: type: QUERY
@@ -1159,10 +1159,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=124)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=124)
- default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=128)
+ default@part_change_various_various_double,part_change_various_various_double,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_double
PREHOOK: type: QUERY
@@ -1327,10 +1327,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=159)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13"]
- TableScan [TS_0] (rows=7 width=159)
- default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
+ TableScan [TS_0] (rows=7 width=163)
+ default@part_change_various_various_decimal,part_change_various_various_decimal,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,b from part_change_various_various_decimal
PREHOOK: type: QUERY
@@ -1501,10 +1501,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=141)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=141)
- default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=145)
+ default@part_change_various_various_string,part_change_various_various_string,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_string
PREHOOK: type: QUERY
@@ -1675,10 +1675,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=127)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=127)
- default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=131)
+ default@part_change_various_various_char,part_change_various_various_char,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char
PREHOOK: type: QUERY
@@ -1849,10 +1849,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=121)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=121)
- default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=125)
+ default@part_change_various_various_char_trunc,part_change_various_various_char_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_char_trunc
PREHOOK: type: QUERY
@@ -2023,10 +2023,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=143)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=143)
- default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=147)
+ default@part_change_various_various_varchar,part_change_various_various_varchar,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar
PREHOOK: type: QUERY
@@ -2197,10 +2197,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=138)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15"]
- TableScan [TS_0] (rows=7 width=138)
- default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
+ TableScan [TS_0] (rows=7 width=142)
+ default@part_change_various_various_varchar_trunc,part_change_various_various_varchar_trunc,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","c13","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,c13,b from part_change_various_various_varchar_trunc
PREHOOK: type: QUERY
@@ -2368,10 +2368,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=7 width=124)
+ Select Operator [SEL_3] (rows=7 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14"]
- TableScan [TS_0] (rows=7 width=124)
- default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
+ TableScan [TS_0] (rows=7 width=128)
+ default@part_change_various_various_timestamp,part_change_various_various_timestamp,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","c7","c8","c9","c10","c11","c12","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,c11,c12,b from part_change_various_various_timestamp
PREHOOK: type: QUERY
@@ -2512,10 +2512,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=6 width=97)
+ Select Operator [SEL_3] (rows=6 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6"]
- TableScan [TS_0] (rows=6 width=97)
- default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","b"]
+ TableScan [TS_0] (rows=6 width=101)
+ default@part_change_various_various_date,part_change_various_various_date,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,b from part_change_various_various_date
PREHOOK: type: QUERY
@@ -2750,10 +2750,10 @@ Stage-0
Stage-1
Map 1 vectorized, llap
File Output Operator [FS_4]
- Select Operator [SEL_3] (rows=13 width=96)
+ Select Operator [SEL_3] (rows=13 width=4)
Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8"]
- TableScan [TS_0] (rows=13 width=96)
- default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:NONE,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
+ TableScan [TS_0] (rows=13 width=100)
+ default@part_change_same_type_different_params,part_change_same_type_different_params,Tbl:COMPLETE,Col:PARTIAL,Output:["insert_num","c1","c2","c3","c4","c5","c6","b"]
PREHOOK: query: select insert_num,part,c1,c2,c3,c4,c5,c6,b from part_change_same_type_different_params
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/script_pipe.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/script_pipe.q.out b/ql/src/test/results/clientpositive/llap/script_pipe.q.out
index 2e68189..f1caec4 100644
--- a/ql/src/test/results/clientpositive/llap/script_pipe.q.out
+++ b/ql/src/test/results/clientpositive/llap/script_pipe.q.out
@@ -11,6 +11,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -19,41 +20,43 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: string), VALUE._col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 1
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
Transform Operator
command: true
output info:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -81,24 +84,24 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string), key (type: string), value (type: string), key (type: string), value (type: string), key (type: string), value (type: string), key (type: string), value (type: string), key (type: string), value (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 534000 Basic stats: COMPLETE Column stats: COMPLETE
Transform Operator
command: head -n 1
output info:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 534000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 534000 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/scriptfile1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/scriptfile1.q.out b/ql/src/test/results/clientpositive/llap/scriptfile1.q.out
index d72ff5c..bf202f9 100644
--- a/ql/src/test/results/clientpositive/llap/scriptfile1.q.out
+++ b/ql/src/test/results/clientpositive/llap/scriptfile1.q.out
@@ -1,6 +1,9 @@
PREHOOK: query: -- SORT_QUERY_RESULTS
-- EXCLUDE_OS_WINDOWS
+
+-- NO_SESSION_REUSE
+
CREATE TABLE dest1(key INT, value STRING)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
@@ -8,6 +11,9 @@ PREHOOK: Output: default@dest1
POSTHOOK: query: -- SORT_QUERY_RESULTS
-- EXCLUDE_OS_WINDOWS
+
+-- NO_SESSION_REUSE
+
CREATE TABLE dest1(key INT, value STRING)
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out b/ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out
index fbc8567..985086d 100644
--- a/ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out
+++ b/ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out
@@ -13,6 +13,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -21,36 +22,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -708,6 +710,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -716,35 +719,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < '3') (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -930,6 +934,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Map 4 <- Union 2 (CONTAINS)
@@ -940,56 +945,58 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 525 Data size: 5503 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 525 Data size: 5503 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 525 Data size: 5503 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 525 Data size: 5503 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 262 Data size: 2746 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 262 Data size: 2746 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 2
Vertex: Union 2
@@ -2466,6 +2473,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -2474,36 +2482,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -3157,6 +3166,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -3165,35 +3175,36 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < '3') (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -3379,6 +3390,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Map 4 <- Union 2 (CONTAINS)
@@ -3389,56 +3401,58 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 525 Data size: 5503 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 525 Data size: 5503 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 525 Data size: 5503 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 525 Data size: 5503 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 262 Data size: 2746 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 262 Data size: 2746 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 262 Data size: 46636 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 2
Vertex: Union 2
[48/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out
index 9a3b7fe..9a586fa 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_1.q.out
@@ -109,38 +109,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- Since size is being used to find the big table, the order of the tables in the join does not matter
explain extended select count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -148,6 +116,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -157,20 +126,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -182,7 +157,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -228,36 +202,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -269,7 +248,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -317,7 +295,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -361,7 +338,7 @@ STAGE PLANS:
/bucket_big/ds=2008-04-08 [b]
/bucket_big/ds=2008-04-09 [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -377,13 +354,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -418,38 +396,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -457,6 +403,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -466,36 +413,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -507,7 +459,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -555,7 +506,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -602,20 +552,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -627,7 +583,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -670,7 +625,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_small/ds=2008-04-08 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -686,13 +641,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -727,38 +683,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -766,6 +690,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -775,36 +700,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -816,7 +746,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -864,7 +793,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -911,20 +839,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -936,7 +870,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -979,7 +912,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_small/ds=2008-04-08 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -995,13 +928,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out
index ae1a8d2..4a049c7 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out
@@ -71,6 +71,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 5 (BROADCAST_EDGE), Union 2 (CONTAINS)
Map 4 <- Map 5 (BROADCAST_EDGE), Union 2 (CONTAINS)
@@ -89,30 +90,26 @@ STAGE PLANS:
expressions: key (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 6 Data size: 42 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- input vertices:
- 1 Map 5
- Statistics: Num rows: 6 Data size: 46 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 5
+ Statistics: Num rows: 6 Data size: 46 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -125,30 +122,26 @@ STAGE PLANS:
expressions: key (type: int)
outputColumnNames: _col0
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 6 Data size: 42 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- input vertices:
- 1 Map 5
- Statistics: Num rows: 6 Data size: 46 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ input vertices:
+ 1 Map 5
+ Statistics: Num rows: 6 Data size: 46 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ value expressions: _col0 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
@@ -172,8 +165,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: int)
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -184,8 +178,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 2
Vertex: Union 2
@@ -248,10 +242,10 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Map 3 <- Reducer 2 (BROADCAST_EDGE)
- Reducer 2 <- Map 1 (SIMPLE_EDGE)
- Reducer 4 <- Map 3 (SIMPLE_EDGE)
+ Map 2 <- Map 1 (BROADCAST_EDGE)
+ Reducer 3 <- Map 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -263,20 +257,18 @@ STAGE PLANS:
predicate: (key < 6) (type: boolean)
Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
Group By Operator
- aggregations: count()
- bucketGroup: true
keys: key (type: int)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
+ mode: final
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 3 Data size: 21 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint)
+ Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
- Map 3
+ LLAP IO: no inputs
+ Map 2
Map Operator Tree:
TableScan
alias: a
@@ -295,9 +287,8 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
input vertices:
- 0 Reducer 2
+ 0 Map 1
Statistics: Num rows: 3 Data size: 23 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
Group By Operator
aggregations: count()
mode: hash
@@ -308,29 +299,12 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Execution mode: llap
- Reducer 2
+ LLAP IO: no inputs
+ Reducer 3
Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
- keys: KEY._col0 (type: int)
- mode: mergepartial
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 7 Basic stats: COMPLETE Column stats: NONE
- Reducer 4
- Execution mode: uber
- Reduce Operator Tree:
- Group By Operator
- aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
@@ -338,8 +312,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out
index 3c50dac..5cb3db5 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_11.q.out
@@ -105,38 +105,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_small a JOIN bucket
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -144,6 +112,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -153,20 +122,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -178,7 +153,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -223,36 +197,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -264,7 +243,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -311,7 +289,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -354,7 +331,7 @@ STAGE PLANS:
/bucket_big/ds=2008-04-08 [b]
/bucket_big/ds=2008-04-09 [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -370,13 +347,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -419,38 +397,6 @@ POSTHOOK: query: -- Since size is being used to find the big table, the order of
-- bucketized mapjoin is not done
explain extended select count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -458,6 +404,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -467,20 +414,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -492,7 +445,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -537,36 +489,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -578,7 +535,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -625,7 +581,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -668,7 +623,7 @@ STAGE PLANS:
/bucket_big/ds=2008-04-08 [b]
/bucket_big/ds=2008-04-09 [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -684,13 +639,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -727,43 +683,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- The join is converted to a bucketed mapjoin with a mapjoin hint
explain extended select /*+ mapjoin(a) */ count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_HINTLIST
- TOK_HINT
- TOK_MAPJOIN
- TOK_HINTARGLIST
- a
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -771,6 +690,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -780,20 +700,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
tag: 0
auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -805,7 +727,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -850,12 +771,12 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -866,20 +787,21 @@ STAGE PLANS:
input vertices:
0 Map 1
Position of Big Table: 1
- Statistics: Num rows: 63 Data size: 6393 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 127 Data size: 13807 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
+ null sort order:
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
tag: -1
value expressions: _col0 (type: bigint)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -891,7 +813,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -938,7 +859,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -981,7 +901,7 @@ STAGE PLANS:
/bucket_big/ds=2008-04-08 [b]
/bucket_big/ds=2008-04-09 [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -997,13 +917,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -1040,58 +961,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- HIVE-7023
explain extended select /* + MAPJOIN(a,b) */ count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key JOIN bucket_big c ON a.key = c.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- c
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- c
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_HINTLIST
- TOK_HINT
- TOK_MAPJOIN
- TOK_HINTARGLIST
- a
- b
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -1099,6 +968,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE), Map 4 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -1108,20 +978,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 122 Basic stats: COMPLETE Column stats: NONE
tag: 0
auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -1133,7 +1005,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -1178,17 +1049,17 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
Inner Join 0 to 2
- Estimated key counts: Map 1 => 1, Map 4 => 58
+ Estimated key counts: Map 1 => 1, Map 4 => 116
keys:
0 key (type: string)
1 key (type: string)
@@ -1197,20 +1068,21 @@ STAGE PLANS:
0 Map 1
2 Map 4
Position of Big Table: 1
- Statistics: Num rows: 127 Data size: 12786 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 255 Data size: 27614 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
+ null sort order:
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
tag: -1
value expressions: _col0 (type: bigint)
auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -1222,7 +1094,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -1269,7 +1140,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -1315,20 +1185,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 116 Data size: 11624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: key (type: string)
+ null sort order: a
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 116 Data size: 12552 Basic stats: COMPLETE Column stats: NONE
tag: 2
auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -1340,7 +1212,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -1387,7 +1258,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -1430,7 +1300,7 @@ STAGE PLANS:
/bucket_big/ds=2008-04-08 [c]
/bucket_big/ds=2008-04-09 [c]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -1446,13 +1316,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
[30/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out b/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out
index fa15157..5569011 100644
--- a/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out
@@ -123,6 +123,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -144,13 +145,14 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -194,6 +196,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -217,8 +220,10 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 10 Data size: 240 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -234,12 +239,12 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 10 Data size: 240 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 10 Data size: 240 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -283,6 +288,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -304,14 +310,15 @@ STAGE PLANS:
sort order: ++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -353,6 +360,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -374,14 +382,15 @@ STAGE PLANS:
sort order: +++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), KEY._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -493,6 +502,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -514,13 +524,14 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -564,6 +575,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -587,8 +599,10 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 10 Data size: 240 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -604,12 +618,12 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 10 Data size: 240 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 10 Data size: 240 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -653,6 +667,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -674,14 +689,15 @@ STAGE PLANS:
sort order: ++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -723,6 +739,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -744,14 +761,15 @@ STAGE PLANS:
sort order: +++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), KEY._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -873,7 +891,7 @@ Database: default
Table: over1k_part
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 2
numRows 32
rawDataSize 830
@@ -915,7 +933,7 @@ Database: default
Table: over1k_part
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 2
numRows 6
rawDataSize 156
@@ -957,7 +975,7 @@ Database: default
Table: over1k_part_limit
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 2
numRows 14
rawDataSize 362
@@ -999,7 +1017,7 @@ Database: default
Table: over1k_part_limit
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 2
numRows 6
rawDataSize 156
@@ -1040,7 +1058,7 @@ Database: default
Table: over1k_part_buck
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 8
numRows 32
rawDataSize 830
@@ -1081,8 +1099,8 @@ Database: default
Table: over1k_part_buck
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 8
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 6
rawDataSize 156
totalSize 162
@@ -1122,7 +1140,7 @@ Database: default
Table: over1k_part_buck_sort
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 8
numRows 32
rawDataSize 830
@@ -1163,8 +1181,8 @@ Database: default
Table: over1k_part_buck_sort
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
- numFiles 8
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ numFiles 4
numRows 6
rawDataSize 156
totalSize 162
@@ -1265,6 +1283,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1287,8 +1306,9 @@ STAGE PLANS:
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: smallint), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: smallint), KEY.reducesinkkey0 (type: int), VALUE._col1 (type: bigint), VALUE._col2 (type: float), VALUE._col3 (type: tinyint)
@@ -1335,9 +1355,9 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -1353,29 +1373,18 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: _col1 (type: int)
- sort order: +
+ key expressions: _col4 (type: tinyint), _col1 (type: int)
+ sort order: ++
+ Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col2 (type: bigint), _col3 (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), KEY.reducesinkkey0 (type: int), VALUE._col1 (type: bigint), VALUE._col2 (type: float), VALUE._col3 (type: tinyint)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col4 (type: tinyint), _col1 (type: int)
- sort order: ++
- Map-reduce partition columns: _col4 (type: tinyint)
- Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
- Reducer 3
- Execution mode: uber
- Reduce Operator Tree:
- Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), KEY._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -1419,6 +1428,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1437,8 +1447,10 @@ STAGE PLANS:
key expressions: _col2 (type: int)
sort order: +
Statistics: Num rows: 4443 Data size: 106636 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1461,12 +1473,12 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 10 Data size: 240 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint)
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), KEY._col4 (type: tinyint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 10 Data size: 240 Basic stats: COMPLETE Column stats: NONE
File Output Operator
@@ -1510,6 +1522,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1533,8 +1546,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: tinyint), _col1 (type: smallint), _col2 (type: int), _col3 (type: bigint), _col4 (type: float)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: tinyint), KEY._col1 (type: smallint), KEY._col2 (type: int), KEY._col3 (type: bigint), KEY._col4 (type: float)
@@ -1588,6 +1602,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1611,8 +1626,9 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: tinyint), KEY._col1 (type: smallint), KEY._col2 (type: int), KEY._col3 (type: bigint), KEY._col4 (type: float)
@@ -1693,7 +1709,7 @@ Database: default
Table: over1k_part2
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 16
rawDataSize 415
@@ -1735,7 +1751,7 @@ Database: default
Table: over1k_part2
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 3
rawDataSize 78
@@ -1840,7 +1856,7 @@ Database: default
Table: over1k_part2
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 16
rawDataSize 415
@@ -1882,7 +1898,7 @@ Database: default
Table: over1k_part2
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 3
rawDataSize 78
@@ -1986,6 +2002,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -2009,8 +2026,9 @@ STAGE PLANS:
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col4 (type: tinyint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), KEY.reducesinkkey0 (type: float), VALUE._col3 (type: tinyint)
@@ -2056,6 +2074,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -2077,14 +2096,15 @@ STAGE PLANS:
sort order: +++
Map-reduce partition columns: _col4 (type: tinyint)
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint), _col3 (type: float), _col4 (type: tinyint), '_bucket_number' (type: string)
+ value expressions: _col0 (type: smallint), _col1 (type: int), _col2 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: tinyint), VALUE._bucket_number (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3, _col4, _bucket_number
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: int), VALUE._col2 (type: bigint), KEY._col3 (type: float), KEY._col4 (type: tinyint), KEY.'_bucket_number' (type: string)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, '_bucket_number'
Statistics: Num rows: 4442 Data size: 106611 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
@@ -2154,7 +2174,7 @@ Database: default
Table: over1k_part_buck_sort2
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 16
rawDataSize 415
@@ -2195,7 +2215,7 @@ Database: default
Table: over1k_part_buck_sort2
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 3
rawDataSize 78
@@ -2297,7 +2317,7 @@ Database: default
Table: over1k_part_buck_sort2
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 16
rawDataSize 415
@@ -2338,7 +2358,7 @@ Database: default
Table: over1k_part_buck_sort2
#### A masked pattern was here ####
Partition Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 3
rawDataSize 78
@@ -2399,3 +2419,855 @@ POSTHOOK: Input: default@over1k_part_buck_sort2@t=27
POSTHOOK: Input: default@over1k_part_buck_sort2@t=__HIVE_DEFAULT_PARTITION__
#### A masked pattern was here ####
19
+PREHOOK: query: create table over1k_part3(
+ si smallint,
+ b bigint,
+ f float)
+ partitioned by (s string, t tinyint, i int)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: create table over1k_part3(
+ si smallint,
+ b bigint,
+ f float)
+ partitioned by (s string, t tinyint, i int)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@over1k_part3
+PREHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where s="foo"
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where s="foo"
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: over1k
+ Statistics: Num rows: 859 Data size: 106636 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (s = 'foo') (type: boolean)
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: si (type: smallint), b (type: bigint), f (type: float), 'foo' (type: string), t (type: tinyint), i (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ sort order: +++
+ Map-reduce partition columns: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: smallint), _col1 (type: bigint), _col2 (type: float)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: bigint), VALUE._col2 (type: float), KEY._col3 (type: string), KEY._col4 (type: tinyint), KEY._col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ i
+ s
+ t
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: over1k
+ Statistics: Num rows: 859 Data size: 106636 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (t = 27) (type: boolean)
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: si (type: smallint), b (type: bigint), f (type: float), s (type: string), 27 (type: tinyint), i (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ sort order: +++
+ Map-reduce partition columns: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: smallint), _col1 (type: bigint), _col2 (type: float)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: bigint), VALUE._col2 (type: float), KEY._col3 (type: string), KEY._col4 (type: tinyint), KEY._col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ i
+ s
+ t
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: over1k
+ Statistics: Num rows: 859 Data size: 106636 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: (i = 100) (type: boolean)
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: si (type: smallint), b (type: bigint), f (type: float), s (type: string), t (type: tinyint), 100 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ sort order: +++
+ Map-reduce partition columns: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: smallint), _col1 (type: bigint), _col2 (type: float)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: bigint), VALUE._col2 (type: float), KEY._col3 (type: string), KEY._col4 (type: tinyint), KEY._col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 429 Data size: 53255 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ i
+ s
+ t
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: over1k
+ Statistics: Num rows: 859 Data size: 106636 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: ((i = 100) and (t = 27)) (type: boolean)
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: si (type: smallint), b (type: bigint), f (type: float), s (type: string), 27 (type: tinyint), 100 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ sort order: +++
+ Map-reduce partition columns: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: smallint), _col1 (type: bigint), _col2 (type: float)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: bigint), VALUE._col2 (type: float), KEY._col3 (type: string), KEY._col4 (type: tinyint), KEY._col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ i
+ s
+ t
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and s="foo"
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and s="foo"
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: over1k
+ Statistics: Num rows: 859 Data size: 106636 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: ((i = 100) and (s = 'foo')) (type: boolean)
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: si (type: smallint), b (type: bigint), f (type: float), 'foo' (type: string), t (type: tinyint), 100 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ sort order: +++
+ Map-reduce partition columns: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: smallint), _col1 (type: bigint), _col2 (type: float)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: bigint), VALUE._col2 (type: float), KEY._col3 (type: string), KEY._col4 (type: tinyint), KEY._col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ i
+ s
+ t
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27 and s="foo"
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27 and s="foo"
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: over1k
+ Statistics: Num rows: 859 Data size: 106636 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: ((t = 27) and (s = 'foo')) (type: boolean)
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: si (type: smallint), b (type: bigint), f (type: float), 'foo' (type: string), 27 (type: tinyint), i (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ sort order: +++
+ Map-reduce partition columns: _col3 (type: string), _col4 (type: tinyint), _col5 (type: int)
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: smallint), _col1 (type: bigint), _col2 (type: float)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: smallint), VALUE._col1 (type: bigint), VALUE._col2 (type: float), KEY._col3 (type: string), KEY._col4 (type: tinyint), KEY._col5 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 214 Data size: 26565 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ i
+ s
+ t
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27 and s="foo"
+PREHOOK: type: QUERY
+POSTHOOK: query: explain insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27 and s="foo"
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-2 depends on stages: Stage-1
+ Stage-0 depends on stages: Stage-2
+ Stage-3 depends on stages: Stage-0
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: over1k
+ Statistics: Num rows: 859 Data size: 106636 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: ((i = 100) and (t = 27) and (s = 'foo')) (type: boolean)
+ Statistics: Num rows: 107 Data size: 13282 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: si (type: smallint), b (type: bigint), f (type: float), 'foo' (type: string), 27 (type: tinyint), 100 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 107 Data size: 13282 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 107 Data size: 13282 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-2
+ Dependency Collection
+
+ Stage: Stage-0
+ Move Operator
+ tables:
+ partition:
+ i
+ s
+ t
+ replace: true
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ name: default.over1k_part3
+
+ Stage: Stage-3
+ Stats-Aggr Operator
+
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where s="foo"
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where s="foo"
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+POSTHOOK: Output: default@over1k_part3@s=alice quirinius/t=27/i=65636
+POSTHOOK: Output: default@over1k_part3@s=bob ovid/t=27/i=65619
+POSTHOOK: Output: default@over1k_part3@s=david allen/t=27/i=65617
+POSTHOOK: Output: default@over1k_part3@s=ethan laertes/t=27/i=65628
+POSTHOOK: Output: default@over1k_part3@s=irene underhill/t=27/i=65787
+POSTHOOK: Output: default@over1k_part3@s=jessica zipper/t=27/i=65778
+POSTHOOK: Output: default@over1k_part3@s=mike zipper/t=27/i=65779
+POSTHOOK: Output: default@over1k_part3@s=oscar carson/t=27/i=65624
+POSTHOOK: Output: default@over1k_part3@s=oscar ovid/t=27/i=65536
+POSTHOOK: Output: default@over1k_part3@s=priscilla zipper/t=27/i=65622
+POSTHOOK: Output: default@over1k_part3@s=quinn allen/t=27/i=65708
+POSTHOOK: Output: default@over1k_part3@s=rachel carson/t=27/i=65677
+POSTHOOK: Output: default@over1k_part3@s=tom brown/t=27/i=65675
+POSTHOOK: Output: default@over1k_part3@s=ulysses underhill/t=27/i=65570
+POSTHOOK: Output: default@over1k_part3@s=wendy van buren/t=27/i=65680
+POSTHOOK: Output: default@over1k_part3@s=xavier quirinius/t=27/i=65599
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=alice quirinius,t=27,i=65636).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=alice quirinius,t=27,i=65636).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=alice quirinius,t=27,i=65636).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=bob ovid,t=27,i=65619).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=bob ovid,t=27,i=65619).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=bob ovid,t=27,i=65619).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=david allen,t=27,i=65617).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=david allen,t=27,i=65617).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=david allen,t=27,i=65617).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ethan laertes,t=27,i=65628).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ethan laertes,t=27,i=65628).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ethan laertes,t=27,i=65628).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=irene underhill,t=27,i=65787).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=irene underhill,t=27,i=65787).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=irene underhill,t=27,i=65787).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=jessica zipper,t=27,i=65778).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=jessica zipper,t=27,i=65778).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=jessica zipper,t=27,i=65778).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=mike zipper,t=27,i=65779).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=mike zipper,t=27,i=65779).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=mike zipper,t=27,i=65779).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar carson,t=27,i=65624).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar carson,t=27,i=65624).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar carson,t=27,i=65624).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar ovid,t=27,i=65536).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar ovid,t=27,i=65536).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar ovid,t=27,i=65536).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=priscilla zipper,t=27,i=65622).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=priscilla zipper,t=27,i=65622).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=priscilla zipper,t=27,i=65622).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=quinn allen,t=27,i=65708).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=quinn allen,t=27,i=65708).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=quinn allen,t=27,i=65708).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=rachel carson,t=27,i=65677).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=rachel carson,t=27,i=65677).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=rachel carson,t=27,i=65677).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=tom brown,t=27,i=65675).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=tom brown,t=27,i=65675).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=tom brown,t=27,i=65675).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ulysses underhill,t=27,i=65570).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ulysses underhill,t=27,i=65570).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ulysses underhill,t=27,i=65570).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=wendy van buren,t=27,i=65680).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=wendy van buren,t=27,i=65680).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=wendy van buren,t=27,i=65680).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=xavier quirinius,t=27,i=65599).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=xavier quirinius,t=27,i=65599).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=xavier quirinius,t=27,i=65599).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and s="foo"
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and s="foo"
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27 and s="foo"
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27 and s="foo"
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27 and s="foo"
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27 and s="foo"
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: select sum(hash(*)) from over1k_part3
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k_part3
+PREHOOK: Input: default@over1k_part3@s=alice quirinius/t=27/i=65636
+PREHOOK: Input: default@over1k_part3@s=bob ovid/t=27/i=65619
+PREHOOK: Input: default@over1k_part3@s=david allen/t=27/i=65617
+PREHOOK: Input: default@over1k_part3@s=ethan laertes/t=27/i=65628
+PREHOOK: Input: default@over1k_part3@s=irene underhill/t=27/i=65787
+PREHOOK: Input: default@over1k_part3@s=jessica zipper/t=27/i=65778
+PREHOOK: Input: default@over1k_part3@s=mike zipper/t=27/i=65779
+PREHOOK: Input: default@over1k_part3@s=oscar carson/t=27/i=65624
+PREHOOK: Input: default@over1k_part3@s=oscar ovid/t=27/i=65536
+PREHOOK: Input: default@over1k_part3@s=priscilla zipper/t=27/i=65622
+PREHOOK: Input: default@over1k_part3@s=quinn allen/t=27/i=65708
+PREHOOK: Input: default@over1k_part3@s=rachel carson/t=27/i=65677
+PREHOOK: Input: default@over1k_part3@s=tom brown/t=27/i=65675
+PREHOOK: Input: default@over1k_part3@s=ulysses underhill/t=27/i=65570
+PREHOOK: Input: default@over1k_part3@s=wendy van buren/t=27/i=65680
+PREHOOK: Input: default@over1k_part3@s=xavier quirinius/t=27/i=65599
+#### A masked pattern was here ####
+POSTHOOK: query: select sum(hash(*)) from over1k_part3
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k_part3
+POSTHOOK: Input: default@over1k_part3@s=alice quirinius/t=27/i=65636
+POSTHOOK: Input: default@over1k_part3@s=bob ovid/t=27/i=65619
+POSTHOOK: Input: default@over1k_part3@s=david allen/t=27/i=65617
+POSTHOOK: Input: default@over1k_part3@s=ethan laertes/t=27/i=65628
+POSTHOOK: Input: default@over1k_part3@s=irene underhill/t=27/i=65787
+POSTHOOK: Input: default@over1k_part3@s=jessica zipper/t=27/i=65778
+POSTHOOK: Input: default@over1k_part3@s=mike zipper/t=27/i=65779
+POSTHOOK: Input: default@over1k_part3@s=oscar carson/t=27/i=65624
+POSTHOOK: Input: default@over1k_part3@s=oscar ovid/t=27/i=65536
+POSTHOOK: Input: default@over1k_part3@s=priscilla zipper/t=27/i=65622
+POSTHOOK: Input: default@over1k_part3@s=quinn allen/t=27/i=65708
+POSTHOOK: Input: default@over1k_part3@s=rachel carson/t=27/i=65677
+POSTHOOK: Input: default@over1k_part3@s=tom brown/t=27/i=65675
+POSTHOOK: Input: default@over1k_part3@s=ulysses underhill/t=27/i=65570
+POSTHOOK: Input: default@over1k_part3@s=wendy van buren/t=27/i=65680
+POSTHOOK: Input: default@over1k_part3@s=xavier quirinius/t=27/i=65599
+#### A masked pattern was here ####
+17814641134
+PREHOOK: query: -- cross verify results with SDPO disabled
+drop table over1k_part3
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@over1k_part3
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: -- cross verify results with SDPO disabled
+drop table over1k_part3
+POSTHOOK: type: DROPTABLE
+POSTHOOK: Input: default@over1k_part3
+POSTHOOK: Output: default@over1k_part3
+PREHOOK: query: create table over1k_part3(
+ si smallint,
+ b bigint,
+ f float)
+ partitioned by (s string, t tinyint, i int)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: create table over1k_part3(
+ si smallint,
+ b bigint,
+ f float)
+ partitioned by (s string, t tinyint, i int)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@over1k_part3
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where s="foo"
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where s="foo"
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+POSTHOOK: Output: default@over1k_part3@s=alice quirinius/t=27/i=65636
+POSTHOOK: Output: default@over1k_part3@s=bob ovid/t=27/i=65619
+POSTHOOK: Output: default@over1k_part3@s=david allen/t=27/i=65617
+POSTHOOK: Output: default@over1k_part3@s=ethan laertes/t=27/i=65628
+POSTHOOK: Output: default@over1k_part3@s=irene underhill/t=27/i=65787
+POSTHOOK: Output: default@over1k_part3@s=jessica zipper/t=27/i=65778
+POSTHOOK: Output: default@over1k_part3@s=mike zipper/t=27/i=65779
+POSTHOOK: Output: default@over1k_part3@s=oscar carson/t=27/i=65624
+POSTHOOK: Output: default@over1k_part3@s=oscar ovid/t=27/i=65536
+POSTHOOK: Output: default@over1k_part3@s=priscilla zipper/t=27/i=65622
+POSTHOOK: Output: default@over1k_part3@s=quinn allen/t=27/i=65708
+POSTHOOK: Output: default@over1k_part3@s=rachel carson/t=27/i=65677
+POSTHOOK: Output: default@over1k_part3@s=tom brown/t=27/i=65675
+POSTHOOK: Output: default@over1k_part3@s=ulysses underhill/t=27/i=65570
+POSTHOOK: Output: default@over1k_part3@s=wendy van buren/t=27/i=65680
+POSTHOOK: Output: default@over1k_part3@s=xavier quirinius/t=27/i=65599
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=alice quirinius,t=27,i=65636).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=alice quirinius,t=27,i=65636).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=alice quirinius,t=27,i=65636).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=bob ovid,t=27,i=65619).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=bob ovid,t=27,i=65619).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=bob ovid,t=27,i=65619).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=david allen,t=27,i=65617).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=david allen,t=27,i=65617).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=david allen,t=27,i=65617).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ethan laertes,t=27,i=65628).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ethan laertes,t=27,i=65628).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ethan laertes,t=27,i=65628).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=irene underhill,t=27,i=65787).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=irene underhill,t=27,i=65787).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=irene underhill,t=27,i=65787).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=jessica zipper,t=27,i=65778).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=jessica zipper,t=27,i=65778).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=jessica zipper,t=27,i=65778).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=mike zipper,t=27,i=65779).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=mike zipper,t=27,i=65779).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=mike zipper,t=27,i=65779).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar carson,t=27,i=65624).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar carson,t=27,i=65624).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar carson,t=27,i=65624).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar ovid,t=27,i=65536).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar ovid,t=27,i=65536).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=oscar ovid,t=27,i=65536).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=priscilla zipper,t=27,i=65622).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=priscilla zipper,t=27,i=65622).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=priscilla zipper,t=27,i=65622).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=quinn allen,t=27,i=65708).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=quinn allen,t=27,i=65708).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=quinn allen,t=27,i=65708).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=rachel carson,t=27,i=65677).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=rachel carson,t=27,i=65677).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=rachel carson,t=27,i=65677).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=tom brown,t=27,i=65675).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=tom brown,t=27,i=65675).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=tom brown,t=27,i=65675).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ulysses underhill,t=27,i=65570).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ulysses underhill,t=27,i=65570).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=ulysses underhill,t=27,i=65570).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=wendy van buren,t=27,i=65680).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=wendy van buren,t=27,i=65680).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=wendy van buren,t=27,i=65680).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=xavier quirinius,t=27,i=65599).b SIMPLE [(over1k)over1k.FieldSchema(name:b, type:bigint, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=xavier quirinius,t=27,i=65599).f SIMPLE [(over1k)over1k.FieldSchema(name:f, type:float, comment:null), ]
+POSTHOOK: Lineage: over1k_part3 PARTITION(s=xavier quirinius,t=27,i=65599).si SIMPLE [(over1k)over1k.FieldSchema(name:si, type:smallint, comment:null), ]
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and s="foo"
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and s="foo"
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27 and s="foo"
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where t=27 and s="foo"
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27 and s="foo"
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k
+PREHOOK: Output: default@over1k_part3
+POSTHOOK: query: insert overwrite table over1k_part3 partition(s,t,i) select si,b,f,s,t,i from over1k where i=100 and t=27 and s="foo"
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k
+PREHOOK: query: select sum(hash(*)) from over1k_part3
+PREHOOK: type: QUERY
+PREHOOK: Input: default@over1k_part3
+PREHOOK: Input: default@over1k_part3@s=alice quirinius/t=27/i=65636
+PREHOOK: Input: default@over1k_part3@s=bob ovid/t=27/i=65619
+PREHOOK: Input: default@over1k_part3@s=david allen/t=27/i=65617
+PREHOOK: Input: default@over1k_part3@s=ethan laertes/t=27/i=65628
+PREHOOK: Input: default@over1k_part3@s=irene underhill/t=27/i=65787
+PREHOOK: Input: default@over1k_part3@s=jessica zipper/t=27/i=65778
+PREHOOK: Input: default@over1k_part3@s=mike zipper/t=27/i=65779
+PREHOOK: Input: default@over1k_part3@s=oscar carson/t=27/i=65624
+PREHOOK: Input: default@over1k_part3@s=oscar ovid/t=27/i=65536
+PREHOOK: Input: default@over1k_part3@s=priscilla zipper/t=27/i=65622
+PREHOOK: Input: default@over1k_part3@s=quinn allen/t=27/i=65708
+PREHOOK: Input: default@over1k_part3@s=rachel carson/t=27/i=65677
+PREHOOK: Input: default@over1k_part3@s=tom brown/t=27/i=65675
+PREHOOK: Input: default@over1k_part3@s=ulysses underhill/t=27/i=65570
+PREHOOK: Input: default@over1k_part3@s=wendy van buren/t=27/i=65680
+PREHOOK: Input: default@over1k_part3@s=xavier quirinius/t=27/i=65599
+#### A masked pattern was here ####
+POSTHOOK: query: select sum(hash(*)) from over1k_part3
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@over1k_part3
+POSTHOOK: Input: default@over1k_part3@s=alice quirinius/t=27/i=65636
+POSTHOOK: Input: default@over1k_part3@s=bob ovid/t=27/i=65619
+POSTHOOK: Input: default@over1k_part3@s=david allen/t=27/i=65617
+POSTHOOK: Input: default@over1k_part3@s=ethan laertes/t=27/i=65628
+POSTHOOK: Input: default@over1k_part3@s=irene underhill/t=27/i=65787
+POSTHOOK: Input: default@over1k_part3@s=jessica zipper/t=27/i=65778
+POSTHOOK: Input: default@over1k_part3@s=mike zipper/t=27/i=65779
+POSTHOOK: Input: default@over1k_part3@s=oscar carson/t=27/i=65624
+POSTHOOK: Input: default@over1k_part3@s=oscar ovid/t=27/i=65536
+POSTHOOK: Input: default@over1k_part3@s=priscilla zipper/t=27/i=65622
+POSTHOOK: Input: default@over1k_part3@s=quinn allen/t=27/i=65708
+POSTHOOK: Input: default@over1k_part3@s=rachel carson/t=27/i=65677
+POSTHOOK: Input: default@over1k_part3@s=tom brown/t=27/i=65675
+POSTHOOK: Input: default@over1k_part3@s=ulysses underhill/t=27/i=65570
+POSTHOOK: Input: default@over1k_part3@s=wendy van buren/t=27/i=65680
+POSTHOOK: Input: default@over1k_part3@s=xavier quirinius/t=27/i=65599
+#### A masked pattern was here ####
+17814641134
[12/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/ptf_streaming.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/ptf_streaming.q.out b/ql/src/test/results/clientpositive/llap/ptf_streaming.q.out
index a4fed1e..440b95d 100644
--- a/ql/src/test/results/clientpositive/llap/ptf_streaming.q.out
+++ b/ql/src/test/results/clientpositive/llap/ptf_streaming.q.out
@@ -31,6 +31,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -40,21 +41,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -64,24 +66,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopstreaming
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -91,7 +93,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -115,17 +117,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -205,6 +207,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -215,31 +218,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_name (type: string), p_mfgr (type: string), p_size (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: p2
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -250,12 +255,12 @@ STAGE PLANS:
0 p_partkey (type: int)
1 p_partkey (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 3
Execution mode: llap
@@ -263,7 +268,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -273,24 +278,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopstreaming
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -300,7 +305,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -311,17 +316,17 @@ STAGE PLANS:
window function: GenericUDAFLagEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6467 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), (_col5 - lag_window_0) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6583 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6583 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -397,6 +402,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Map 4 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
@@ -406,35 +412,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 16094 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_partkey (type: int), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: int), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col1 (type: string), VALUE._col2 (type: string), VALUE._col3 (type: int), VALUE._col4 (type: string), VALUE._col5 (type: double), VALUE._col6 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -444,19 +452,19 @@ STAGE PLANS:
Partition table definition
input alias: abc
name: noopstreaming
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col0: int, _col1: string, _col2: string, _col3: string, _col4: string, _col5: int, _col6: string, _col7: double, _col8: string
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 23062 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string)
Reducer 3
Execution mode: llap
@@ -468,13 +476,13 @@ STAGE PLANS:
0 _col0 (type: int)
1 p_partkey (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 17951 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -550,6 +558,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -559,7 +568,7 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -569,26 +578,27 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmapstreaming
- order by: p_name, p_size(DESC)
+ order by: p_name ASC NULLS FIRST, p_size DESC NULLS LAST
output shape: p_name: string, p_mfgr: string, p_size: int
partition by: p_mfgr
raw input shape:
transforms raw input: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string), p_size (type: int)
sort order: ++-
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), KEY.reducesinkkey2 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -598,24 +608,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmapstreaming
- order by: _col1, _col5(DESC)
+ order by: _col1 ASC NULLS FIRST, _col5 DESC NULLS LAST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
transforms raw input: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int)
sort order: ++-
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), KEY.reducesinkkey2 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -625,7 +635,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1, _col5(DESC)
+ order by: _col1 ASC NULLS FIRST, _col5 DESC NULLS LAST
partition by: _col2
raw input shape:
window functions:
@@ -636,17 +646,17 @@ STAGE PLANS:
window function: GenericUDAFRankEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5902 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -724,6 +734,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -733,7 +744,7 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -743,27 +754,28 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmapstreaming
- order by: p_name
+ order by: p_name ASC NULLS FIRST
output shape: p_name: string, p_mfgr: string, p_size: int, p_retailprice: double
partition by: p_mfgr
raw input shape:
transforms raw input: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -773,25 +785,25 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmapstreaming
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
transforms raw input: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -801,7 +813,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -825,17 +837,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -919,6 +931,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -929,21 +942,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -953,11 +967,11 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -967,7 +981,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmapstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
@@ -975,17 +989,17 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
Execution mode: llap
@@ -993,7 +1007,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1003,7 +1017,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmapstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
@@ -1011,24 +1025,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1038,7 +1052,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -1062,17 +1076,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1158,6 +1172,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1168,21 +1183,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1192,11 +1208,11 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1206,7 +1222,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
@@ -1214,17 +1230,17 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
Execution mode: llap
@@ -1232,7 +1248,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1242,7 +1258,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
@@ -1250,24 +1266,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1277,7 +1293,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -1301,17 +1317,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1397,6 +1413,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1407,21 +1424,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6006 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1431,11 +1449,11 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1445,7 +1463,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmapstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
@@ -1453,17 +1471,17 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noop
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 3
Execution mode: llap
@@ -1471,7 +1489,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1481,7 +1499,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmapstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
@@ -1489,24 +1507,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noop
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1516,7 +1534,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -1540,17 +1558,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumDouble
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12974 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), _col5 (type: int), rank_window_0 (type: int), dense_rank_window_1 (type: int), sum_window_2 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1640,6 +1658,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
@@ -1650,35 +1669,37 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6110 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_name (type: string)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6110 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_partkey (type: int), p_size (type: int), p_retailprice (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: p1
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: p_partkey is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_partkey (type: int)
sort order: +
Map-reduce partition columns: p_partkey (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: int), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col0, _col1, _col2, _col5, _col7
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 13078 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1688,19 +1709,19 @@ STAGE PLANS:
Partition table definition
input alias: abc
name: noopstreaming
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
output shape: _col0: int, _col1: string, _col2: string, _col5: int, _col7: double
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 13078 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 13078 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 13078 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col2 (type: string), _col5 (type: int), _col7 (type: double)
Reducer 3
Execution mode: llap
@@ -1712,20 +1733,20 @@ STAGE PLANS:
0 _col0 (type: int)
1 p_partkey (type: int)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6699 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 6699 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int), _col7 (type: double)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int), VALUE._col5 (type: double)
outputColumnNames: _col1, _col2, _col5, _col7
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 22243 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1735,7 +1756,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -1772,17 +1793,17 @@ STAGE PLANS:
window function: GenericUDAFLagEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 22243 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), rank_window_0 (type: int), dense_rank_window_1 (type: int), count_window_2 (type: bigint), _col7 (type: double), sum_window_3 (type: double), _col5 (type: int), (_col5 - lag_window_4) (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 7511 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 7511 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1888,6 +1909,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1898,21 +1920,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string)
sort order: +
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_name (type: string), p_size (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col4 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1922,18 +1945,18 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopstreaming
- order by: _col2
+ order by: _col2 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
Partition table definition
input alias: ptf_2
name: noop
- order by: _col2
+ order by: _col2 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1943,7 +1966,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2, _col1
raw input shape:
@@ -1951,17 +1974,17 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2, _col1
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Map-side function: true
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string), _col1 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 3
Execution mode: llap
@@ -1969,7 +1992,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -1979,7 +2002,7 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopwithmap
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2, _col1
raw input shape:
@@ -1987,24 +2010,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_2
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2, _col1
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string), _col1 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2014,7 +2037,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
partition by: _col2, _col1
raw input shape:
window functions:
@@ -2038,17 +2061,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumLong
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), rank_window_0 (type: int), dense_rank_window_1 (type: int), _col5 (type: int), sum_window_2 (type: bigint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 6214 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2162,6 +2185,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -2173,21 +2197,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string)
sort order: +
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: p_name (type: string), p_size (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col4 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2197,23 +2222,23 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopstreaming
- order by: _col2
+ order by: _col2 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
Partition table definition
input alias: ptf_2
name: noop
- order by: _col2
+ order by: _col2 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string), _col1 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 3
Execution mode: llap
@@ -2221,7 +2246,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2231,16 +2256,16 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noopstreaming
- order by: _col2, _col1
+ order by: _col2 ASC NULLS FIRST, _col1 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2, _col1
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string)
sort order: +
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string), _col5 (type: int)
Reducer 4
Execution mode: llap
@@ -2248,7 +2273,7 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col4 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2258,24 +2283,24 @@ STAGE PLANS:
Partition table definition
input alias: ptf_1
name: noop
- order by: _col2
+ order by: _col2 ASC NULLS FIRST
output shape: _col1: string, _col2: string, _col5: int
partition by: _col2
raw input shape:
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col2 (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col5 (type: int)
Reducer 5
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: string), VALUE._col3 (type: int)
outputColumnNames: _col1, _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -2285,7 +2310,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col1
+ order by: _col1 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -2309,17 +2334,17 @@ STAGE PLANS:
name: sum
window function: GenericUDAFSumLong
window frame: PRECEDING(MAX)~
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 12766 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col1 (type: string), rank_window_0 (type: int), dense_rank_window_1 (type: int), _col5 (type: int), sum_window_2 (type: bigint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4
<TRUNCATED>
[36/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/count.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/count.q.out b/ql/src/test/results/clientpositive/llap/count.q.out
index e8d5a8a..5c38049 100644
--- a/ql/src/test/results/clientpositive/llap/count.q.out
+++ b/ql/src/test/results/clientpositive/llap/count.q.out
@@ -42,6 +42,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -68,8 +69,9 @@ STAGE PLANS:
Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
value expressions: _col5 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0), sum(VALUE._col2)
@@ -81,8 +83,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 2 Data size: 39 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -114,6 +116,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -139,8 +142,9 @@ STAGE PLANS:
Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
value expressions: _col4 (type: bigint), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: bigint), _col9 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), count(VALUE._col1), count(VALUE._col2), count(VALUE._col3), count(VALUE._col4), count(VALUE._col5), count(DISTINCT KEY._col0:0._col0), count(DISTINCT KEY._col0:1._col0), count(DISTINCT KEY._col0:2._col0), count(DISTINCT KEY._col0:3._col0), count(DISTINCT KEY._col0:4._col0, KEY._col0:4._col1), count(DISTINCT KEY._col0:5._col0, KEY._col0:5._col1), count(DISTINCT KEY._col0:6._col0, KEY._col0:6._col1), count(DISTINCT KEY._col0:7._col0, KEY._col0:7._col1), count(DISTINCT KEY._col0:8._col0, KEY._col0:8._col1), count(DISTINCT KEY._col0:9._col0, KEY._col0:9._col1), count(DISTINCT KEY._col0:10._col0, KEY._col0:10._col1, KEY._col0:10._col2), count(DISTINCT KEY._col0:11._col0, KEY._col0:11._col1, KEY._col0:11._col2), count(DISTINCT KEY._col0:12._col0, KEY._col0:12._col1, KEY._col0:12._col2), count(DISTINCT KEY._col0:13._col0, KEY._col0:13._col1, KEY._col0:13._col2), count(DISTINCT KEY._col0:14._col0, KEY._col0:14._col1, KEY._col0:14._col2, KEY.
_col0:14._col3)
@@ -151,8 +155,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 200 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -181,6 +185,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -201,8 +206,9 @@ STAGE PLANS:
Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
value expressions: d (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0), sum(VALUE._col0)
@@ -214,8 +220,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 2 Data size: 39 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -247,6 +253,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -265,8 +272,9 @@ STAGE PLANS:
sort order: ++++
Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(1), count(), count(KEY._col0:0._col0), count(KEY._col0:1._col0), count(KEY._col0:2._col0), count(KEY._col0:3._col0), count(DISTINCT KEY._col0:0._col0), count(DISTINCT KEY._col0:1._col0), count(DISTINCT KEY._col0:2._col0), count(DISTINCT KEY._col0:3._col0), count(DISTINCT KEY._col0:4._col0, KEY._col0:4._col1), count(DISTINCT KEY._col0:5._col0, KEY._col0:5._col1), count(DISTINCT KEY._col0:6._col0, KEY._col0:6._col1), count(DISTINCT KEY._col0:7._col0, KEY._col0:7._col1), count(DISTINCT KEY._col0:8._col0, KEY._col0:8._col1), count(DISTINCT KEY._col0:9._col0, KEY._col0:9._col1), count(DISTINCT KEY._col0:10._col0, KEY._col0:10._col1, KEY._col0:10._col2), count(DISTINCT KEY._col0:11._col0, KEY._col0:11._col1, KEY._col0:11._col2), count(DISTINCT KEY._col0:12._col0, KEY._col0:12._col1, KEY._col0:12._col2), count(DISTINCT KEY._col0:13._col0, KEY._col0:13._col1, KEY._col0:13._col2), count(DISTINCT KEY._col0:14._col0, KEY._col0:14._col1, KEY._col0:14._col2, K
EY._col0:14._col3)
@@ -277,8 +285,618 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 168 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(1), count(*), count(a), count(b), count(c), count(d), count(distinct a), count(distinct b), count(distinct c), count(distinct d), count(distinct a,b), count(distinct b,c), count(distinct c,d), count(distinct a,d), count(distinct a,c), count(distinct b,d), count(distinct a,b,c), count(distinct b,c,d), count(distinct a,c,d), count(distinct a,b,d), count(distinct a,b,c,d) from abcd
+PREHOOK: type: QUERY
+PREHOOK: Input: default@abcd
+#### A masked pattern was here ####
+POSTHOOK: query: select count(1), count(*), count(a), count(b), count(c), count(d), count(distinct a), count(distinct b), count(distinct c), count(distinct d), count(distinct a,b), count(distinct b,c), count(distinct c,d), count(distinct a,d), count(distinct a,c), count(distinct b,d), count(distinct a,b,c), count(distinct b,c,d), count(distinct a,c,d), count(distinct a,b,d), count(distinct a,b,c,d) from abcd
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@abcd
+#### A masked pattern was here ####
+7 7 6 6 6 7 3 3 6 7 4 5 6 6 5 6 4 5 5 5 4
+PREHOOK: query: --first aggregation with literal. gbinfo was generating wrong expression
+explain select count(1), count(*), count(a), count(b), count(c), count(d), count(distinct a), count(distinct b), count(distinct c), count(distinct d), count(distinct a,b), count(distinct b,c), count(distinct c,d), count(distinct a,d), count(distinct a,c), count(distinct b,d), count(distinct a,b,c), count(distinct b,c,d), count(distinct a,c,d), count(distinct a,b,d), count(distinct a,b,c,d) from abcd
+PREHOOK: type: QUERY
+POSTHOOK: query: --first aggregation with literal. gbinfo was generating wrong expression
+explain select count(1), count(*), count(a), count(b), count(c), count(d), count(distinct a), count(distinct b), count(distinct c), count(distinct d), count(distinct a,b), count(distinct b,c), count(distinct c,d), count(distinct a,d), count(distinct a,c), count(distinct b,d), count(distinct a,b,c), count(distinct b,c,d), count(distinct a,c,d), count(distinct a,b,d), count(distinct a,b,c,d) from abcd
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: abcd
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: a (type: int), b (type: int), c (type: int), d (type: int)
+ outputColumnNames: $f1, $f2, $f3, $f4
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count(1), count(), count($f1), count($f2), count($f3), count($f4), count(DISTINCT $f1), count(DISTINCT $f2), count(DISTINCT $f3), count(DISTINCT $f4), count(DISTINCT $f1, $f2), count(DISTINCT $f2, $f3), count(DISTINCT $f3, $f4), count(DISTINCT $f1, $f4), count(DISTINCT $f1, $f3), count(DISTINCT $f2, $f4), count(DISTINCT $f1, $f2, $f3), count(DISTINCT $f2, $f3, $f4), count(DISTINCT $f1, $f3, $f4), count(DISTINCT $f1, $f2, $f4), count(DISTINCT $f1, $f2, $f3, $f4)
+ keys: $f1 (type: int), $f2 (type: int), $f3 (type: int), $f4 (type: int)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23, _col24
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col3 (type: int)
+ sort order: ++++
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col4 (type: bigint), _col5 (type: bigint), _col6 (type: bigint), _col7 (type: bigint), _col8 (type: bigint), _col9 (type: bigint)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0), count(VALUE._col1), count(VALUE._col2), count(VALUE._col3), count(VALUE._col4), count(VALUE._col5), count(DISTINCT KEY._col0:0._col0), count(DISTINCT KEY._col0:1._col0), count(DISTINCT KEY._col0:2._col0), count(DISTINCT KEY._col0:3._col0), count(DISTINCT KEY._col0:4._col0, KEY._col0:4._col1), count(DISTINCT KEY._col0:5._col0, KEY._col0:5._col1), count(DISTINCT KEY._col0:6._col0, KEY._col0:6._col1), count(DISTINCT KEY._col0:7._col0, KEY._col0:7._col1), count(DISTINCT KEY._col0:8._col0, KEY._col0:8._col1), count(DISTINCT KEY._col0:9._col0, KEY._col0:9._col1), count(DISTINCT KEY._col0:10._col0, KEY._col0:10._col1, KEY._col0:10._col2), count(DISTINCT KEY._col0:11._col0, KEY._col0:11._col1, KEY._col0:11._col2), count(DISTINCT KEY._col0:12._col0, KEY._col0:12._col1, KEY._col0:12._col2), count(DISTINCT KEY._col0:13._col0, KEY._col0:13._col1, KEY._col0:13._col2), count(DISTINCT KEY._col0:14._col0, KEY._col0:14._col1, KEY._col0:14._col2, KEY.
_col0:14._col3)
+ mode: mergepartial
+ outputColumnNames: $f0, $f1, $f2, $f3, $f4, $f5, $f6, $f7, $f8, $f9, $f10, $f11, $f12, $f13, $f14, $f15, $f16, $f17, $f18, $f19, $f20
+ Statistics: Num rows: 1 Data size: 336 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 336 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(1), count(*), count(a), count(b), count(c), count(d), count(distinct a), count(distinct b), count(distinct c), count(distinct d), count(distinct a,b), count(distinct b,c), count(distinct c,d), count(distinct a,d), count(distinct a,c), count(distinct b,d), count(distinct a,b,c), count(distinct b,c,d), count(distinct a,c,d), count(distinct a,b,d), count(distinct a,b,c,d) from abcd
+PREHOOK: type: QUERY
+PREHOOK: Input: default@abcd
+#### A masked pattern was here ####
+POSTHOOK: query: select count(1), count(*), count(a), count(b), count(c), count(d), count(distinct a), count(distinct b), count(distinct c), count(distinct d), count(distinct a,b), count(distinct b,c), count(distinct c,d), count(distinct a,d), count(distinct a,c), count(distinct b,d), count(distinct a,b,c), count(distinct b,c,d), count(distinct a,c,d), count(distinct a,b,d), count(distinct a,b,c,d) from abcd
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@abcd
+#### A masked pattern was here ####
+7 7 6 6 6 7 3 3 6 7 4 5 6 6 5 6 4 5 5 5 4
+PREHOOK: query: explain select count(distinct b) from abcd group by a
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select count(distinct b) from abcd group by a
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: abcd
+ Statistics: Num rows: 9 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: a (type: int), b (type: int)
+ outputColumnNames: a, b
+ Statistics: Num rows: 9 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: a (type: int), b (type: int)
+ sort order: ++
+ Map-reduce partition columns: a (type: int), b (type: int)
+ Statistics: Num rows: 9 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: int), KEY._col1 (type: int)
+ mode: complete
+ outputColumnNames: a, b
+ Statistics: Num rows: 4 Data size: 34 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: a (type: int)
+ sort order: +
+ Map-reduce partition columns: a (type: int)
+ Statistics: Num rows: 4 Data size: 34 Basic stats: COMPLETE Column stats: NONE
+ value expressions: b (type: int)
+ Reducer 3
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ keys: KEY._col0 (type: int)
+ mode: complete
+ outputColumnNames: a, $f1
+ Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: $f1 (type: bigint)
+ outputColumnNames: _o__c0
+ Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(distinct b) from abcd group by a
+PREHOOK: type: QUERY
+PREHOOK: Input: default@abcd
+#### A masked pattern was here ####
+POSTHOOK: query: select count(distinct b) from abcd group by a
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@abcd
+#### A masked pattern was here ####
+1
+1
+1
+2
+PREHOOK: query: explain select count(distinct b) from abcd group by b
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select count(distinct b) from abcd group by b
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: abcd
+ Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: b (type: int)
+ outputColumnNames: b
+ Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: b (type: int)
+ sort order: +
+ Map-reduce partition columns: b (type: int)
+ Statistics: Num rows: 19 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: int)
+ mode: complete
+ outputColumnNames: b
+ Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: b (type: int)
+ sort order: +
+ Map-reduce partition columns: b (type: int)
+ Statistics: Num rows: 9 Data size: 36 Basic stats: COMPLETE Column stats: NONE
+ Reducer 3
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(KEY._col0)
+ keys: KEY._col0 (type: int)
+ mode: complete
+ outputColumnNames: b, $f1
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: $f1 (type: bigint)
+ outputColumnNames: _o__c0
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(distinct b) from abcd group by b
+PREHOOK: type: QUERY
+PREHOOK: Input: default@abcd
+#### A masked pattern was here ####
+POSTHOOK: query: select count(distinct b) from abcd group by b
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@abcd
+#### A masked pattern was here ####
+0
+1
+1
+1
+PREHOOK: query: explain select count(distinct b) from abcd group by c
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select count(distinct b) from abcd group by c
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: abcd
+ Statistics: Num rows: 9 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: b (type: int), c (type: int)
+ outputColumnNames: b, c
+ Statistics: Num rows: 9 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: b (type: int), c (type: int)
+ sort order: ++
+ Map-reduce partition columns: b (type: int), c (type: int)
+ Statistics: Num rows: 9 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: int), KEY._col1 (type: int)
+ mode: complete
+ outputColumnNames: b, c
+ Statistics: Num rows: 4 Data size: 34 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: c (type: int)
+ sort order: +
+ Map-reduce partition columns: c (type: int)
+ Statistics: Num rows: 4 Data size: 34 Basic stats: COMPLETE Column stats: NONE
+ value expressions: b (type: int)
+ Reducer 3
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ keys: KEY._col0 (type: int)
+ mode: complete
+ outputColumnNames: c, $f1
+ Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: $f1 (type: bigint)
+ outputColumnNames: _o__c0
+ Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 2 Data size: 17 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(distinct b) from abcd group by c
+PREHOOK: type: QUERY
+PREHOOK: Input: default@abcd
+#### A masked pattern was here ####
+POSTHOOK: query: select count(distinct b) from abcd group by c
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@abcd
+#### A masked pattern was here ####
+0
+1
+1
+1
+1
+1
+1
+PREHOOK: query: explain select count(b), count(distinct c) from abcd group by d
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select count(b), count(distinct c) from abcd group by d
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: abcd
+ Statistics: Num rows: 6 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: b (type: int), c (type: int), d (type: int)
+ outputColumnNames: b, c, d
+ Statistics: Num rows: 6 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: d (type: int), c (type: int)
+ sort order: ++
+ Map-reduce partition columns: d (type: int)
+ Statistics: Num rows: 6 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ value expressions: b (type: int)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0), count(DISTINCT KEY._col1:0._col0)
+ keys: KEY._col0 (type: int)
+ mode: complete
+ outputColumnNames: d, $f1, $f2
+ Statistics: Num rows: 3 Data size: 39 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: $f1 (type: bigint), $f2 (type: bigint)
+ outputColumnNames: _o__c0, _o__c1
+ Statistics: Num rows: 3 Data size: 39 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 3 Data size: 39 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(b), count(distinct c) from abcd group by d
+PREHOOK: type: QUERY
+PREHOOK: Input: default@abcd
+#### A masked pattern was here ####
+POSTHOOK: query: select count(b), count(distinct c) from abcd group by d
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@abcd
+#### A masked pattern was here ####
+0 1
+1 0
+1 1
+1 1
+1 1
+1 1
+1 1
+PREHOOK: query: --non distinct aggregate with same column as group by key
+explain select a, count(distinct b), count(distinct c), sum(d), sum(d+d), sum(d*3), sum(b), sum(c), sum(a), sum(distinct a), sum(distinct b) from abcd group by a
+PREHOOK: type: QUERY
+POSTHOOK: query: --non distinct aggregate with same column as group by key
+explain select a, count(distinct b), count(distinct c), sum(d), sum(d+d), sum(d*3), sum(b), sum(c), sum(a), sum(distinct a), sum(distinct b) from abcd group by a
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: abcd
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: a (type: int), b (type: int), c (type: int), d (type: int), (d + d) (type: int), (d * 3) (type: int)
+ outputColumnNames: $f0, $f1, $f2, $f3, $f4, $f5
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: $f0 (type: int), $f1 (type: int), $f2 (type: int)
+ sort order: +++
+ Map-reduce partition columns: $f0 (type: int)
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ value expressions: $f3 (type: int), $f4 (type: int), $f5 (type: int)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0), sum(VALUE._col0), sum(VALUE._col1), sum(VALUE._col2), sum(KEY._col1:0._col0), sum(KEY._col1:1._col0), sum(KEY._col0), sum(DISTINCT KEY._col1:2._col0), sum(DISTINCT KEY._col1:3._col0)
+ keys: KEY._col0 (type: int)
+ mode: complete
+ outputColumnNames: $f0, $f1, $f2, $f3, $f4, $f5, $f6, $f7, $f8, $f9, $f10
+ Statistics: Num rows: 2 Data size: 39 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 2 Data size: 39 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select a, count(distinct b), count(distinct c), sum(d), sum(d+d), sum(d*3), sum(b), sum(c), sum(a), sum(distinct a), sum(distinct b) from abcd group by a
+PREHOOK: type: QUERY
+PREHOOK: Input: default@abcd
+#### A masked pattern was here ####
+POSTHOOK: query: select a, count(distinct b), count(distinct c), sum(d), sum(d+d), sum(d*3), sum(b), sum(c), sum(a), sum(distinct a), sum(distinct b) from abcd group by a
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@abcd
+#### A masked pattern was here ####
+10 2 2 10 20 30 1200 95 30 10 1100
+100 1 1 3 6 9 100 10 100 100 100
+12 1 2 9 18 27 100 155 24 12 100
+NULL 1 1 6 12 18 35 23 NULL NULL 35
+PREHOOK: query: --non distinct aggregate with same column as distinct aggregate
+explain select a, count(distinct b), count(distinct c), sum(d), sum(c) from abcd group by a
+PREHOOK: type: QUERY
+POSTHOOK: query: --non distinct aggregate with same column as distinct aggregate
+explain select a, count(distinct b), count(distinct c), sum(d), sum(c) from abcd group by a
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: abcd
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: a (type: int), b (type: int), c (type: int), d (type: int)
+ outputColumnNames: a, b, c, d
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: a (type: int), b (type: int), c (type: int)
+ sort order: +++
+ Map-reduce partition columns: a (type: int)
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ value expressions: d (type: int)
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0), sum(VALUE._col0), sum(KEY._col1:1._col0)
+ keys: KEY._col0 (type: int)
+ mode: complete
+ outputColumnNames: a, $f1, $f2, $f3, $f4
+ Statistics: Num rows: 2 Data size: 39 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 2 Data size: 39 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select a, count(distinct b), count(distinct c), sum(d), sum(c) from abcd group by a
+PREHOOK: type: QUERY
+PREHOOK: Input: default@abcd
+#### A masked pattern was here ####
+POSTHOOK: query: select a, count(distinct b), count(distinct c), sum(d), sum(c) from abcd group by a
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@abcd
+#### A masked pattern was here ####
+10 2 2 10 95
+100 1 1 3 10
+12 1 2 9 155
+NULL 1 1 6 23
+PREHOOK: query: --aggregation with literal
+explain select count(1), count(*), count(a), count(b), count(c), count(d), count(distinct a), count(distinct b), count(distinct c), count(distinct d), count(distinct a,b), count(distinct b,c), count(distinct c,d), count(distinct a,d), count(distinct a,c), count(distinct b,d), count(distinct a,b,c), count(distinct b,c,d), count(distinct a,c,d), count(distinct a,b,d), count(distinct a,b,c,d) from abcd
+PREHOOK: type: QUERY
+POSTHOOK: query: --aggregation with literal
+explain select count(1), count(*), count(a), count(b), count(c), count(d), count(distinct a), count(distinct b), count(distinct c), count(distinct d), count(distinct a,b), count(distinct b,c), count(distinct c,d), count(distinct a,d), count(distinct a,c), count(distinct b,d), count(distinct a,b,c), count(distinct b,c,d), count(distinct a,c,d), count(distinct a,b,d), count(distinct a,b,c,d) from abcd
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: abcd
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: a (type: int), b (type: int), c (type: int), d (type: int)
+ outputColumnNames: $f1, $f2, $f3, $f4
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: $f1 (type: int), $f2 (type: int), $f3 (type: int), $f4 (type: int)
+ sort order: ++++
+ Statistics: Num rows: 4 Data size: 78 Basic stats: COMPLETE Column stats: NONE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(1), count(), count(KEY._col0:0._col0), count(KEY._col0:1._col0), count(KEY._col0:2._col0), count(KEY._col0:3._col0), count(DISTINCT KEY._col0:0._col0), count(DISTINCT KEY._col0:1._col0), count(DISTINCT KEY._col0:2._col0), count(DISTINCT KEY._col0:3._col0), count(DISTINCT KEY._col0:4._col0, KEY._col0:4._col1), count(DISTINCT KEY._col0:5._col0, KEY._col0:5._col1), count(DISTINCT KEY._col0:6._col0, KEY._col0:6._col1), count(DISTINCT KEY._col0:7._col0, KEY._col0:7._col1), count(DISTINCT KEY._col0:8._col0, KEY._col0:8._col1), count(DISTINCT KEY._col0:9._col0, KEY._col0:9._col1), count(DISTINCT KEY._col0:10._col0, KEY._col0:10._col1, KEY._col0:10._col2), count(DISTINCT KEY._col0:11._col0, KEY._col0:11._col1, KEY._col0:11._col2), count(DISTINCT KEY._col0:12._col0, KEY._col0:12._col1, KEY._col0:12._col2), count(DISTINCT KEY._col0:13._col0, KEY._col0:13._col1, KEY._col0:13._col2), count(DISTINCT KEY._col0:14._col0, KEY._col0:14._col1, KEY._col0:14._col2, K
EY._col0:14._col3)
+ mode: complete
+ outputColumnNames: $f0, $f1, $f2, $f3, $f4, $f5, $f6, $f7, $f8, $f9, $f10, $f11, $f12, $f13, $f14, $f15, $f16, $f17, $f18, $f19, $f20
+ Statistics: Num rows: 1 Data size: 168 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 168 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cross_join.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cross_join.q.out b/ql/src/test/results/clientpositive/llap/cross_join.q.out
index 6ff8c2d..8578dbf 100644
--- a/ql/src/test/results/clientpositive/llap/cross_join.q.out
+++ b/ql/src/test/results/clientpositive/llap/cross_join.q.out
@@ -1,4 +1,4 @@
-Warning: Shuffle Join MERGEJOIN[11][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[9][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: -- current
explain select src.key from src join src src2
PREHOOK: type: QUERY
@@ -12,6 +12,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -20,20 +21,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: src
+ alias: src2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE
@@ -41,6 +43,7 @@ STAGE PLANS:
sort order:
Statistics: Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -51,13 +54,13 @@ STAGE PLANS:
0
1
outputColumnNames: _col0
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250000 Data size: 21750000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250000 Data size: 21750000 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -66,7 +69,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[11][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[9][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: -- ansi cross join
explain select src.key from src cross join src src2
PREHOOK: type: QUERY
@@ -80,6 +83,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -88,20 +92,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: src
+ alias: src2
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE
@@ -109,6 +114,7 @@ STAGE PLANS:
sort order:
Statistics: Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -119,13 +125,13 @@ STAGE PLANS:
0
1
outputColumnNames: _col0
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250000 Data size: 21750000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250000 Data size: 21750000 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -147,6 +153,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -155,38 +162,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: src2
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -197,13 +206,13 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -212,3 +221,204 @@ STAGE PLANS:
Processor Tree:
ListSink
+Warning: Map Join MAPJOIN[9][bigTable=?] in task 'Map 1' is a cross product
+PREHOOK: query: explain select src.key from src join src src2
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select src.key from src join src src2
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Map 1 <- Map 2 (BROADCAST_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 2
+ Statistics: Num rows: 250000 Data size: 21750000 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 250000 Data size: 21750000 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ Statistics: Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+Warning: Map Join MAPJOIN[9][bigTable=?] in task 'Map 1' is a cross product
+PREHOOK: query: explain select src.key from src cross join src src2
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select src.key from src cross join src src2
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Map 1 <- Map 2 (BROADCAST_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0
+ 1
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 2
+ Statistics: Num rows: 250000 Data size: 21750000 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 250000 Data size: 21750000 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ Statistics: Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 500 Data size: 2000 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: explain select src.key from src cross join src src2 on src.key=src2.key
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select src.key from src cross join src src2 on src.key=src2.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Map 1 <- Map 2 (BROADCAST_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: src
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0
+ input vertices:
+ 1 Map 2
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1219 Data size: 106053 Basic stats: COMPLETE Column stats: COMPLETE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: src2
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
[40/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out b/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out
index 8b1c126..f6500eb 100644
--- a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out
+++ b/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out
@@ -128,14 +128,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -145,7 +145,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -155,14 +155,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col3
input vertices:
1 Map 3
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col3 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 605 Data size: 11267 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -173,38 +173,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: llap
LLAP IO: no inputs
@@ -281,14 +281,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -298,14 +298,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col3
input vertices:
0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 97900 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 97900 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 97900 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -341,14 +341,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab_part
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: ((key > 1) and (key > 2)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -358,10 +358,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 2
- Statistics: Num rows: 60 Data size: 642 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 1126 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 60 Data size: 642 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 1126 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -372,19 +372,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab_part
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: ((key > 2) and (key > 1)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
@@ -416,14 +416,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab_part
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key > 1) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 3091 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 3091 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Left Outer Join0 to 1
@@ -433,10 +433,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 2
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 182 Data size: 3400 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 182 Data size: 3400 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -447,19 +447,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab_part
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: ((key > 2) and (key > 1)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
@@ -491,33 +491,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab_part
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: ((key > 1) and (key > 2)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 1024 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: tab_part
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key > 2) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 3091 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 3091 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Right Outer Join0 to 1
@@ -527,10 +527,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
0 Map 1
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 182 Data size: 3400 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 182 Data size: 3400 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -564,39 +564,39 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: key (type: int)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Reducer 2
@@ -606,7 +606,7 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 121 Data size: 1283 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 121 Data size: 2251 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -616,10 +616,10 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 3
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -651,39 +651,39 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tab
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: value (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: UDFToDouble(_col0) (type: double)
sort order: +
Map-reduce partition columns: UDFToDouble(_col0) (type: double)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -694,7 +694,7 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 121 Data size: 1283 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 121 Data size: 2251 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -704,14 +704,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col2
input vertices:
1 Map 3
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: string), _col2 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 266 Data size: 2822 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
[13/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/ptf_matchpath.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/ptf_matchpath.q.out b/ql/src/test/results/clientpositive/llap/ptf_matchpath.q.out
index 78e84a0..b45e5ed 100644
--- a/ql/src/test/results/clientpositive/llap/ptf_matchpath.q.out
+++ b/ql/src/test/results/clientpositive/llap/ptf_matchpath.q.out
@@ -69,6 +69,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -85,8 +86,9 @@ STAGE PLANS:
Statistics: Num rows: 24 Data size: 5379 Basic stats: COMPLETE Column stats: NONE
value expressions: origin_city_name (type: string), arr_delay (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey1 (type: int), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type: int), VALUE._col2 (type: float), KEY.reducesinkkey0 (type: string)
@@ -102,7 +104,7 @@ STAGE PLANS:
input alias: ptf_1
arguments: 'LATE.LATE+', 'LATE', (_col5 > 15.0), 'origin_city_name, fl_num, year, month, day_of_month, size(tpath) as sz, tpath[0].day_of_month as tpath'
name: matchpath
- order by: _col2, _col3, _col4
+ order by: _col2 ASC NULLS FIRST, _col3 ASC NULLS FIRST, _col4 ASC NULLS FIRST
output shape: tpath: int
partition by: _col6
raw input shape:
@@ -116,8 +118,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 24 Data size: 5379 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -197,6 +199,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -213,8 +216,9 @@ STAGE PLANS:
Statistics: Num rows: 24 Data size: 5379 Basic stats: COMPLETE Column stats: NONE
value expressions: origin_city_name (type: string), arr_delay (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type: int), KEY.reducesinkkey4 (type: int), VALUE._col2 (type: float), KEY.reducesinkkey1 (type: string)
@@ -230,7 +234,7 @@ STAGE PLANS:
input alias: ptf_1
arguments: 'LATE.LATE+', 'LATE', (_col5 > 15.0), 'origin_city_name, fl_num, year, month, day_of_month, size(tpath) as sz, tpath[0].day_of_month as tpath'
name: matchpath
- order by: _col6, _col2, _col3, _col4
+ order by: _col6 ASC NULLS FIRST, _col2 ASC NULLS FIRST, _col3 ASC NULLS FIRST, _col4 ASC NULLS FIRST
output shape: tpath: int
partition by: 0
raw input shape:
@@ -240,15 +244,15 @@ STAGE PLANS:
predicate: (fl_num = 1142) (type: boolean)
Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: origin_city_name (type: string), '1142' (type: string), year (type: int), month (type: int), day_of_month (type: int), sz (type: int), tpath (type: int)
+ expressions: origin_city_name (type: string), fl_num (type: string), year (type: int), month (type: int), day_of_month (type: int), sz (type: int), tpath (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -315,6 +319,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -323,28 +328,29 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: flights_tiny
- Statistics: Num rows: 44 Data size: 5379 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 24 Data size: 5379 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (fl_num = -1142) (type: boolean)
- Statistics: Num rows: 22 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: origin_city_name (type: string), year (type: int), month (type: int), day_of_month (type: int), arr_delay (type: float)
- outputColumnNames: _col0, _col2, _col3, _col4, _col5
- Statistics: Num rows: 22 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
+ expressions: origin_city_name (type: string), year (type: int), month (type: int), day_of_month (type: int), arr_delay (type: float), fl_num (type: string)
+ outputColumnNames: _col0, _col2, _col3, _col4, _col5, _col6
+ Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: 0 (type: int), '-1142' (type: string), _col2 (type: int), _col3 (type: int), _col4 (type: int)
+ key expressions: 0 (type: int), _col6 (type: string), _col2 (type: int), _col3 (type: int), _col4 (type: int)
sort order: +++++
Map-reduce partition columns: 0 (type: int)
- Statistics: Num rows: 22 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: string), _col5 (type: float)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
- expressions: VALUE._col0 (type: string), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type: int), KEY.reducesinkkey4 (type: int), VALUE._col2 (type: float), '-1142' (type: string)
+ expressions: VALUE._col0 (type: string), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type: int), KEY.reducesinkkey4 (type: int), VALUE._col2 (type: float), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col2, _col3, _col4, _col5, _col6
- Statistics: Num rows: 22 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
PTF Operator
Function definitions:
Input definition
@@ -355,22 +361,22 @@ STAGE PLANS:
input alias: ptf_1
arguments: 'LATE.LATE+', 'LATE', (_col5 > 15.0), 'origin_city_name, fl_num, year, month, day_of_month, size(tpath) as sz, tpath[0].day_of_month as tpath'
name: matchpath
- order by: _col6, _col2, _col3, _col4
+ order by: _col6 ASC NULLS FIRST, _col2 ASC NULLS FIRST, _col3 ASC NULLS FIRST, _col4 ASC NULLS FIRST
output shape: tpath: int
partition by: 0
raw input shape:
referenced columns: _col0, _col6, _col2, _col3, _col4, tpath, _col5
- Statistics: Num rows: 22 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: origin_city_name (type: string), '-1142' (type: string), year (type: int), month (type: int), day_of_month (type: int), sz (type: int), tpath (type: int)
+ expressions: origin_city_name (type: string), fl_num (type: string), year (type: int), month (type: int), day_of_month (type: int), sz (type: int), tpath (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
- Statistics: Num rows: 22 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 22 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12 Data size: 2689 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
[07/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/subquery_in.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/subquery_in.q.out b/ql/src/test/results/clientpositive/llap/subquery_in.q.out
index 034557f..321f1cc 100644
--- a/ql/src/test/results/clientpositive/llap/subquery_in.q.out
+++ b/ql/src/test/results/clientpositive/llap/subquery_in.q.out
@@ -21,6 +21,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -29,44 +30,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key > '9') (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: s1
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key > '9') (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 14442 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 14442 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 69 Data size: 6003 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 69 Data size: 6003 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -77,13 +80,13 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 182 Data size: 1939 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -142,6 +145,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -150,43 +154,45 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > '9') and value is not null) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: a
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > '9') and value is not null) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -197,13 +203,13 @@ STAGE PLANS:
0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 91 Data size: 969 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 91 Data size: 969 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 356 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -270,6 +276,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
@@ -280,32 +287,35 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: p_name (type: string), p_size (type: int), UDFToDouble(p_size) (type: double)
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: _col2 is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: UDFToDouble(p_size) is not null (type: boolean)
+ Statistics: Num rows: 26 Data size: 3250 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: p_name (type: string), p_size (type: int), UDFToDouble(p_size) (type: double)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 26 Data size: 3458 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: double)
sort order: +
Map-reduce partition columns: _col2 (type: double)
- Statistics: Num rows: 13 Data size: 1573 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 3458 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: part
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: p_mfgr (type: string), p_size (type: int)
sort order: ++
Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -316,13 +326,13 @@ STAGE PLANS:
0 _col2 (type: double)
1 _col0 (type: double)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 125 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 125 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: llap
@@ -330,7 +340,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: int)
outputColumnNames: _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 9620 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -340,7 +350,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col5
+ order by: _col5 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -351,22 +361,22 @@ STAGE PLANS:
window function: GenericUDAFRankEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 9620 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (rank_window_0 <= 2) (type: boolean)
- Statistics: Num rows: 8 Data size: 968 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 8 Data size: 2960 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col5 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 8 Data size: 968 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 8 Data size: 2960 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: avg(_col0)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Statistics: Num rows: 1 Data size: 76 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: struct<count:bigint,sum:double,input:int>)
Reducer 5
Execution mode: llap
@@ -375,20 +385,20 @@ STAGE PLANS:
aggregations: avg(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: double)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double)
sort order: +
Map-reduce partition columns: _col0 (type: double)
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
@@ -443,6 +453,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
@@ -453,32 +464,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: (p_size is not null and p_mfgr is not null) (type: boolean)
- Statistics: Num rows: 7 Data size: 847 Basic stats: COMPLETE Column stats: NONE
+ predicate: (p_mfgr is not null and p_size is not null) (type: boolean)
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: p_name (type: string), p_mfgr (type: string), p_size (type: int)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 7 Data size: 847 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
- key expressions: _col2 (type: int), _col1 (type: string)
+ key expressions: _col1 (type: string), _col2 (type: int)
sort order: ++
- Map-reduce partition columns: _col2 (type: int), _col1 (type: string)
- Statistics: Num rows: 7 Data size: 847 Basic stats: COMPLETE Column stats: NONE
+ Map-reduce partition columns: _col1 (type: string), _col2 (type: int)
+ Statistics: Num rows: 26 Data size: 5798 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: b
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: p_mfgr (type: string), p_size (type: int)
- sort order: ++
- Map-reduce partition columns: p_mfgr (type: string)
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ alias: part
+ Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: p_mfgr is not null (type: boolean)
+ Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: p_mfgr (type: string), p_size (type: int)
+ sort order: ++
+ Map-reduce partition columns: p_mfgr (type: string)
+ Statistics: Num rows: 26 Data size: 2652 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -486,20 +503,20 @@ STAGE PLANS:
condition map:
Left Semi Join 0 to 1
keys:
- 0 _col2 (type: int), _col1 (type: string)
- 1 _col0 (type: int), _col1 (type: string)
+ 0 _col1 (type: string), _col2 (type: int)
+ 1 _col0 (type: string), _col1 (type: int)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 7 Data size: 931 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 223 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: string), _col0 (type: string), _col2 (type: int)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 7 Data size: 931 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 223 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 7 Data size: 931 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 223 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: llap
@@ -507,7 +524,7 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: int)
outputColumnNames: _col2, _col5
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 9620 Basic stats: COMPLETE Column stats: COMPLETE
PTF Operator
Function definitions:
Input definition
@@ -517,7 +534,7 @@ STAGE PLANS:
Windowing table definition
input alias: ptf_1
name: windowingtablefunction
- order by: _col5
+ order by: _col5 ASC NULLS FIRST
partition by: _col2
raw input shape:
window functions:
@@ -528,25 +545,25 @@ STAGE PLANS:
window function: GenericUDAFRankEvaluator
window frame: PRECEDING(MAX)~FOLLOWING(MAX)
isPivotResult: true
- Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 26 Data size: 9620 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: ((rank_window_0 <= 2) and _col2 is not null) (type: boolean)
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ predicate: (rank_window_0 <= 2) (type: boolean)
+ Statistics: Num rows: 8 Data size: 2960 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col2 (type: string), _col5 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 8 Data size: 2960 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: min(_col1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int)
Reducer 5
Execution mode: llap
@@ -556,24 +573,20 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col1 is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: int), _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 204 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: _col0 (type: string), _col1 (type: int)
+ mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- keys: _col0 (type: int), _col1 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: string)
- sort order: ++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: string)
- Statistics: Num rows: 2 Data size: 242 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 102 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string), _col1 (type: int)
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: string), _col1 (type: int)
+ Statistics: Num rows: 1 Data size: 102 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
@@ -632,6 +645,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE)
@@ -641,39 +655,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > '9') and value is not null) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
- alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ alias: a
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > '9') and value is not null) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: key (type: string), value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -684,13 +700,13 @@ STAGE PLANS:
0 _col0 (type: string), _col1 (type: string)
1 _col0 (type: string), _col1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 91 Data size: 969 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 91 Data size: 969 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 178 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 4
Execution mode: llap
@@ -699,17 +715,17 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 83 Data size: 14774 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: string), _col1 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 7298 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string), _col1 (type: string)
- Statistics: Num rows: 41 Data size: 435 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 41 Data size: 7298 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
@@ -791,6 +807,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
@@ -800,64 +817,67 @@ STAGE PLANS:
Map 1
Map Operator Tree:
TableScan
- alias: lineitem
- Statistics: Num rows: 100 Data size: 11999 Basic stats: COMPLETE Column stats: NONE
+ alias: li
+ Statistics: Num rows: 100 Data size: 1600 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: (((l_linenumber = 1) and l_orderkey is not null) and l_partkey is not null) (type: boolean)
- Statistics: Num rows: 13 Data size: 1559 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((l_linenumber = 1) and l_partkey is not null and l_orderkey is not null) (type: boolean)
+ Statistics: Num rows: 17 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: l_orderkey (type: int), l_partkey (type: int), l_suppkey (type: int)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 13 Data size: 1559 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 17 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 13 Data size: 1559 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 17 Data size: 272 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: int), _col2 (type: int)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: lineitem
- Statistics: Num rows: 100 Data size: 11999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 9200 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((l_shipmode = 'AIR') and l_orderkey is not null) (type: boolean)
- Statistics: Num rows: 25 Data size: 2999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1288 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: l_orderkey (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 2999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 56 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col0 (type: int)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 2999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 25 Data size: 2999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 16 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: lineitem
- Statistics: Num rows: 100 Data size: 11999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 400 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: l_partkey is not null (type: boolean)
- Statistics: Num rows: 50 Data size: 5999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 400 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: l_partkey (type: int)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 50 Data size: 5999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 200 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 50 Data size: 5999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 200 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -868,12 +888,12 @@ STAGE PLANS:
0 _col0 (type: int)
1 _col0 (type: int)
outputColumnNames: _col1, _col2
- Statistics: Num rows: 27 Data size: 3298 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
sort order: +
Map-reduce partition columns: _col1 (type: int)
- Statistics: Num rows: 27 Data size: 3298 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 13 Data size: 104 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col2 (type: int)
Reducer 3
Execution mode: llap
@@ -884,18 +904,18 @@ STAGE PLANS:
keys:
0 _col1 (type: int)
1 _col0 (type: int)
- outputColumnNames: _col1, _col2
- Statistics: Num rows: 29 Data size: 3627 Basic stats: COMPLETE Column stats: NONE
+ outputColumnNames: _col2, _col4
+ Statistics: Num rows: 4 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int)
+ expressions: _col4 (type: int), _col2 (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 29 Data size: 3627 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 29 Data size: 3627 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 32 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: llap
@@ -904,12 +924,12 @@ STAGE PLANS:
keys: KEY._col0 (type: int)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 25 Data size: 2999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 200 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 25 Data size: 2999 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 50 Data size: 200 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/temp_table.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/temp_table.q.out b/ql/src/test/results/clientpositive/llap/temp_table.q.out
index 8ec5941..e97b3dd 100644
--- a/ql/src/test/results/clientpositive/llap/temp_table.q.out
+++ b/ql/src/test/results/clientpositive/llap/temp_table.q.out
@@ -18,23 +18,24 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((UDFToDouble(key) % 2.0) = 0.0) (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.foo
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -89,23 +90,24 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((UDFToDouble(key) % 2.0) = 1.0) (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.bar
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -167,6 +169,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -184,10 +187,12 @@ STAGE PLANS:
key expressions: _col0 (type: string)
sort order: +
Statistics: Num rows: 247 Data size: 2609 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
@@ -200,8 +205,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -239,6 +244,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Union 2 (CONTAINS)
Map 4 <- Union 2 (CONTAINS)
@@ -258,8 +264,10 @@ STAGE PLANS:
key expressions: _col0 (type: string)
sort order: +
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
@@ -273,10 +281,12 @@ STAGE PLANS:
key expressions: _col0 (type: string)
sort order: +
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
@@ -289,8 +299,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Union 2
Vertex: Union 2
@@ -467,3 +477,112 @@ POSTHOOK: query: DROP TABLE bay
POSTHOOK: type: DROPTABLE
POSTHOOK: Input: default@bay
POSTHOOK: Output: default@bay
+PREHOOK: query: create table s as select * from src limit 10
+PREHOOK: type: CREATETABLE_AS_SELECT
+PREHOOK: Input: default@src
+PREHOOK: Output: database:default
+PREHOOK: Output: default@s
+POSTHOOK: query: create table s as select * from src limit 10
+POSTHOOK: type: CREATETABLE_AS_SELECT
+POSTHOOK: Input: default@src
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@s
+POSTHOOK: Lineage: s.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: s.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+PREHOOK: query: select count(*) from s
+PREHOOK: type: QUERY
+PREHOOK: Input: default@s
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from s
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@s
+#### A masked pattern was here ####
+10
+PREHOOK: query: create temporary table s as select * from s limit 2
+PREHOOK: type: CREATETABLE_AS_SELECT
+PREHOOK: Input: default@s
+PREHOOK: Output: database:default
+PREHOOK: Output: default@s
+POSTHOOK: query: create temporary table s as select * from s limit 2
+POSTHOOK: type: CREATETABLE_AS_SELECT
+POSTHOOK: Input: default@s
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@s
+PREHOOK: query: select count(*) from s
+PREHOOK: type: QUERY
+PREHOOK: Input: default@s
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from s
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@s
+#### A masked pattern was here ####
+2
+PREHOOK: query: with s as ( select * from src limit 1)
+select count(*) from s
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: with s as ( select * from src limit 1)
+select count(*) from s
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+1
+PREHOOK: query: with src as ( select * from s)
+select count(*) from src
+PREHOOK: type: QUERY
+PREHOOK: Input: default@s
+#### A masked pattern was here ####
+POSTHOOK: query: with src as ( select * from s)
+select count(*) from src
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@s
+#### A masked pattern was here ####
+2
+PREHOOK: query: drop table s
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@s
+PREHOOK: Output: default@s
+POSTHOOK: query: drop table s
+POSTHOOK: type: DROPTABLE
+POSTHOOK: Input: default@s
+POSTHOOK: Output: default@s
+PREHOOK: query: select count(*) from s
+PREHOOK: type: QUERY
+PREHOOK: Input: default@s
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from s
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@s
+#### A masked pattern was here ####
+10
+PREHOOK: query: with s as ( select * from src limit 1)
+select count(*) from s
+PREHOOK: type: QUERY
+PREHOOK: Input: default@src
+#### A masked pattern was here ####
+POSTHOOK: query: with s as ( select * from src limit 1)
+select count(*) from s
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@src
+#### A masked pattern was here ####
+1
+PREHOOK: query: with src as ( select * from s)
+select count(*) from src
+PREHOOK: type: QUERY
+PREHOOK: Input: default@s
+#### A masked pattern was here ####
+POSTHOOK: query: with src as ( select * from s)
+select count(*) from src
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@s
+#### A masked pattern was here ####
+10
+PREHOOK: query: drop table s
+PREHOOK: type: DROPTABLE
+PREHOOK: Input: default@s
+PREHOOK: Output: default@s
+POSTHOOK: query: drop table s
+POSTHOOK: type: DROPTABLE
+POSTHOOK: Input: default@s
+POSTHOOK: Output: default@s
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out b/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out
index 18103d7..c83da8b 100644
--- a/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_bmj_schema_evolution.q.out
@@ -87,14 +87,14 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test
- Statistics: Num rows: 1000 Data size: 52312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 60312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1000 Data size: 52312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 60312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1000 Data size: 52312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 60312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -104,11 +104,11 @@ STAGE PLANS:
outputColumnNames: _col0, _col1
input vertices:
1 Map 3
- Statistics: Num rows: 1100 Data size: 57543 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 66343 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
- Statistics: Num rows: 1100 Data size: 57543 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 66343 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: some inputs
@@ -137,10 +137,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: int), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1100 Data size: 57543 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 66343 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 57543 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1100 Data size: 66343 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_dml.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_dml.q.out b/ql/src/test/results/clientpositive/llap/tez_dml.q.out
index f620010..d09c1f9 100644
--- a/ql/src/test/results/clientpositive/llap/tez_dml.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_dml.q.out
@@ -28,22 +28,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: value (type: string)
outputColumnNames: value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(value)
keys: value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -55,11 +55,11 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 3
Execution mode: llap
@@ -67,10 +67,10 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -879,17 +879,17 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key % 2) = 0) (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(key) (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -897,14 +897,14 @@ STAGE PLANS:
name: default.even
Filter Operator
predicate: ((key % 2) = 1) (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 44500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(key) (type: int), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_1.q.out b/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_1.q.out
index e4fa28c..3c6ef9a 100644
--- a/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_1.q.out
@@ -33,19 +33,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint BETWEEN 1000000 AND 3000000 (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 1546640 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 1546640 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int)
sort order: +
Map-reduce partition columns: _col2 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 1546640 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: llap
LLAP IO: all inputs
@@ -53,19 +53,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 3093170 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000) (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 862030 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), csmallint (type: smallint), cint (type: int), cbigint (type: bigint), cfloat (type: float), cdouble (type: double), cstring1 (type: string), cstring2 (type: string), ctimestamp1 (type: timestamp), ctimestamp2 (type: timestamp), cboolean1 (type: boolean), cboolean2 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 862030 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int)
sort order: +
Map-reduce partition columns: _col2 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 862030 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean)
Execution mode: llap
LLAP IO: all inputs
@@ -79,11 +79,11 @@ STAGE PLANS:
0 _col2 (type: int)
1 _col2 (type: int)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 4517320 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col2 (type: int)
sort order: +
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 4517320 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: tinyint), _col1 (type: smallint), _col3 (type: bigint), _col4 (type: float), _col5 (type: double), _col6 (type: string), _col7 (type: string), _col8 (type: timestamp), _col9 (type: timestamp), _col10 (type: boolean), _col11 (type: boolean), _col12 (type: tinyint), _col13 (type: smallint), _col14 (type: int), _col15 (type: bigint), _col16 (type: float), _col17 (type: double), _col18 (type: string), _col19 (type: string), _col20 (type: timestamp), _col21 (type: timestamp), _col22 (type: boolean), _col23 (type: boolean)
Reducer 3
Execution mode: llap
@@ -91,10 +91,10 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: tinyint), VALUE._col1 (type: smallint), KEY.reducesinkkey0 (type: int), VALUE._col2 (type: bigint), VALUE._col3 (type: float), VALUE._col4 (type: double), VALUE._col5 (type: string), VALUE._col6 (type: string), VALUE._col7 (type: timestamp), VALUE._col8 (type: timestamp), VALUE._col9 (type: boolean), VALUE._col10 (type: boolean), VALUE._col11 (type: tinyint), VALUE._col12 (type: smallint), VALUE._col13 (type: int), VALUE._col14 (type: bigint), VALUE._col15 (type: float), VALUE._col16 (type: double), VALUE._col17 (type: string), VALUE._col18 (type: string), VALUE._col19 (type: timestamp), VALUE._col20 (type: timestamp), VALUE._col21 (type: boolean), VALUE._col22 (type: boolean)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 4517320 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 4517320 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -165,38 +165,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 36696 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint BETWEEN 1000000 AND 3000000 (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 18348 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 18348 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 18348 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Map 4
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 110088 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000) (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -208,15 +208,15 @@ STAGE PLANS:
keys:
0 _col0 (type: int)
1 _col0 (type: int)
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 58288 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 3
Execution mode: llap
@@ -225,10 +225,10 @@ STAGE PLANS:
aggregations: count(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -293,19 +293,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 73396 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: cint BETWEEN 1000000 AND 3000000 (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 36700 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: csmallint (type: smallint), cint (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 36700 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: int)
sort order: +
Map-reduce partition columns: _col1 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 36700 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: smallint)
Execution mode: llap
LLAP IO: all inputs
@@ -313,19 +313,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 110088 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000) (type: boolean)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cint (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3424 Data size: 30684 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: all inputs
Reducer 2
@@ -338,18 +338,18 @@ STAGE PLANS:
0 _col1 (type: int)
1 _col0 (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 7286 Data size: 29144 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
keys: _col0 (type: smallint)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: smallint)
sort order: +
Map-reduce partition columns: _col0 (type: smallint)
- Statistics: Num rows: 6758 Data size: 1453080 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
@@ -359,11 +359,11 @@ STAGE PLANS:
keys: KEY._col0 (type: smallint)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3379 Data size: 726540 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
sort order: +
- Statistics: Num rows: 3379 Data size: 726540 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: smallint)
Reducer 4
Execution mode: llap
@@ -371,10 +371,10 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: smallint), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3379 Data size: 726540 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 3379 Data size: 726540 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2765 Data size: 33180 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
[46/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out
index 096e350..d99d425 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_3.q.out
@@ -89,38 +89,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- Since size is being used to find the big table, the order of the tables in the join does not matter
explain extended select count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -128,6 +96,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -137,20 +106,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 2 Data size: 228 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -162,7 +137,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -210,7 +184,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -257,36 +230,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 29 Data size: 2906 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 31 Data size: 3196 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
+ Statistics: Num rows: 63 Data size: 6903 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -298,7 +276,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -341,7 +318,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_big/ds=2008-04-08 [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -357,13 +334,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -398,38 +376,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -437,6 +383,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -446,36 +393,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 29 Data size: 2906 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 31 Data size: 3196 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 63 Data size: 6903 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -487,7 +439,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -533,20 +484,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 2 Data size: 228 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -558,7 +515,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -606,7 +562,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -650,7 +605,7 @@ STAGE PLANS:
/bucket_small/ds=2008-04-08 [b]
/bucket_small/ds=2008-04-09 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -666,13 +621,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -707,38 +663,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -746,6 +670,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -755,36 +680,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 29 Data size: 2906 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 31 Data size: 3196 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 58 Data size: 6276 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 63 Data size: 6903 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -796,7 +726,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -842,20 +771,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 2 Data size: 228 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 114 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 244 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -867,7 +802,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -915,7 +849,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -959,7 +892,7 @@ STAGE PLANS:
/bucket_small/ds=2008-04-08 [b]
/bucket_small/ds=2008-04-09 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -975,13 +908,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out
index 4d6ed75..069f08d 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out
@@ -105,38 +105,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- Since size is being used to find the big table, the order of the tables in the join does not matter
explain extended select count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -144,6 +112,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -153,20 +122,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 4 Data size: 452 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -178,7 +153,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -226,7 +200,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -273,36 +246,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 14 Data size: 1425 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 1 => 2
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 0 Map 1
- Position of Big Table: 1
- Statistics: Num rows: 15 Data size: 1567 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 1 => 4
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 0 Map 1
+ Position of Big Table: 1
+ Statistics: Num rows: 29 Data size: 3262 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -314,7 +292,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -357,7 +334,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_big/ds=2008-04-08 [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -373,13 +350,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -414,38 +392,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -453,6 +399,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -462,36 +409,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 14 Data size: 1425 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 2
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 15 Data size: 1567 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 4
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 29 Data size: 3262 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -503,7 +455,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -549,20 +500,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 4 Data size: 452 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -574,7 +531,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -622,7 +578,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -666,7 +621,7 @@ STAGE PLANS:
/bucket_small/ds=2008-04-08 [b]
/bucket_small/ds=2008-04-09 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -682,13 +637,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -723,38 +679,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -762,6 +686,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -771,36 +696,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 14 Data size: 1425 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 2
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 15 Data size: 1567 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 27 Data size: 2966 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 4
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 29 Data size: 3262 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -812,7 +742,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 2
bucket_field_name key
columns key,value
@@ -858,20 +787,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 4 Data size: 452 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 4 Data size: 484 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -883,7 +818,6 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -931,7 +865,6 @@ STAGE PLANS:
partition values:
ds 2008-04-09
properties:
- COLUMN_STATS_ACCURATE true
bucket_count 4
bucket_field_name key
columns key,value
@@ -975,7 +908,7 @@ STAGE PLANS:
/bucket_small/ds=2008-04-08 [b]
/bucket_small/ds=2008-04-09 [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -991,13 +924,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out
index b463b43..5e45f9c 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out
@@ -76,38 +76,6 @@ PREHOOK: type: QUERY
POSTHOOK: query: -- Since size is being used to find the big table, the order of the tables in the join does not matter
explain extended select count(*) FROM bucket_small a JOIN bucket_big b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -115,6 +83,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 3 <- Map 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -128,7 +97,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 113 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -137,7 +110,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 4
bucket_field_name key
@@ -147,6 +119,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_small
numFiles 4
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_small { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -157,7 +131,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 4
bucket_field_name key
@@ -167,6 +140,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_small
numFiles 4
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_small { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -185,26 +160,31 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 14 Data size: 1425 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- Position of Big Table: 1
- Statistics: Num rows: 15 Data size: 1567 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ Position of Big Table: 1
+ Statistics: Num rows: 29 Data size: 3025 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
Path -> Alias:
#### A masked pattern was here ####
@@ -214,7 +194,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 2
bucket_field_name key
@@ -224,6 +203,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_big
numFiles 2
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_big { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -234,7 +215,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 2
bucket_field_name key
@@ -244,6 +224,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_big
numFiles 2
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_big { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -255,7 +237,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_big [b]
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -271,13 +253,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -306,38 +289,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -345,6 +296,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -358,7 +310,11 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 113 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -367,7 +323,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 4
bucket_field_name key
@@ -377,6 +332,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_small
numFiles 4
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_small { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -387,7 +344,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 4
bucket_field_name key
@@ -397,6 +353,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_small
numFiles 4
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_small { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -415,26 +373,31 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 14 Data size: 1425 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- Position of Big Table: 0
- Statistics: Num rows: 15 Data size: 1567 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ Position of Big Table: 0
+ Statistics: Num rows: 29 Data size: 3025 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
Path -> Alias:
#### A masked pattern was here ####
@@ -444,7 +407,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 2
bucket_field_name key
@@ -454,6 +416,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_big
numFiles 2
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_big { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -464,7 +428,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 2
bucket_field_name key
@@ -474,6 +437,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_big
numFiles 2
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_big { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -485,7 +450,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_big [a]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -501,13 +466,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -536,38 +502,6 @@ PREHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_s
PREHOOK: type: QUERY
POSTHOOK: query: explain extended select count(*) FROM bucket_big a JOIN bucket_small b ON a.key = b.key
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- bucket_big
- a
- TOK_TABREF
- TOK_TABNAME
- bucket_small
- b
- =
- .
- TOK_TABLE_OR_COL
- a
- key
- .
- TOK_TABLE_OR_COL
- b
- key
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_FUNCTIONSTAR
- count
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -575,6 +509,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -589,31 +524,36 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 14 Data size: 1425 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator
- condition map:
- Inner Join 0 to 1
- Estimated key counts: Map 3 => 1
- keys:
- 0 key (type: string)
- 1 key (type: string)
- input vertices:
- 1 Map 3
- Position of Big Table: 0
- Statistics: Num rows: 15 Data size: 1567 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 27 Data size: 2750 Basic stats: COMPLETE Column stats: NONE
+ Map Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Estimated key counts: Map 3 => 2
+ keys:
+ 0 _col0 (type: string)
+ 1 _col0 (type: string)
+ input vertices:
+ 1 Map 3
+ Position of Big Table: 0
+ Statistics: Num rows: 29 Data size: 3025 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- tag: -1
- value expressions: _col0 (type: bigint)
- auto parallelism: false
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: bigint)
+ auto parallelism: false
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -623,7 +563,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 2
bucket_field_name key
@@ -633,6 +572,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_big
numFiles 2
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_big { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -643,7 +584,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 2
bucket_field_name key
@@ -653,6 +593,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_big
numFiles 2
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_big { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -672,15 +614,21 @@ STAGE PLANS:
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 113 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: string)
- sort order: +
- Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 1 Data size: 113 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 2 Data size: 226 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -690,7 +638,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 4
bucket_field_name key
@@ -700,6 +647,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_small
numFiles 4
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_small { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -710,7 +659,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
SORTBUCKETCOLSPREFIX TRUE
bucket_count 4
bucket_field_name key
@@ -720,6 +668,8 @@ STAGE PLANS:
#### A masked pattern was here ####
name default.bucket_small
numFiles 4
+ numRows 0
+ rawDataSize 0
serialization.ddl struct bucket_small { string key, string value}
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
@@ -731,7 +681,7 @@ STAGE PLANS:
Truncated Path -> Alias:
/bucket_small [b]
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Group By Operator
@@ -747,13 +697,14 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0
columns.types bigint
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
[22/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out b/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out
index e39ac18..3fe4837 100644
--- a/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out
+++ b/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out
@@ -19,6 +19,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -27,34 +28,35 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 3560 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 3560 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -104,6 +106,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -112,34 +115,35 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: -
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 3560 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 3560 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -189,6 +193,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -197,43 +202,44 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: value (type: string), (UDFToDouble(key) + 1.0) (type: double)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(_col1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: _col1 (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 1980 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 1980 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -285,6 +291,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -293,43 +300,44 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: value (type: string), (UDFToDouble(key) + 1.0) (type: double)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: avg(_col1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 36594 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 36594 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: _col1 (type: struct<count:bigint,sum:double,input:double>)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: avg(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 1980 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 1980 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -381,6 +389,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -389,40 +398,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: alltypesorc
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 73400 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: cdouble (type: double)
outputColumnNames: cdouble
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 73400 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: cdouble (type: double)
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4265 Data size: 25480 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double)
sort order: +
Map-reduce partition columns: _col0 (type: double)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4265 Data size: 25480 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
Execution mode: llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: double)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4265 Data size: 25480 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 4300 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 128 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 4300 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 128 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -472,6 +482,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -480,45 +491,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: alltypesorc
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 110096 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), cdouble (type: double)
outputColumnNames: ctinyint, cdouble
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 110096 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: ctinyint (type: tinyint), cdouble (type: double)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 55052 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: tinyint), _col1 (type: double)
sort order: ++
Map-reduce partition columns: _col0 (type: tinyint)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 55052 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: tinyint), KEY._col1 (type: double)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 55052 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(_col1)
keys: _col0 (type: tinyint)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3072 Data size: 660491 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 95 Data size: 1048 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 4300 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 224 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 4300 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 224 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -568,6 +580,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -576,45 +589,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: alltypesorc
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 110096 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), cdouble (type: double)
outputColumnNames: ctinyint, cdouble
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 110096 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: ctinyint (type: tinyint), cdouble (type: double)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 55052 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: tinyint), _col1 (type: double)
sort order: ++
Map-reduce partition columns: _col0 (type: tinyint)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 55052 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: tinyint), KEY._col1 (type: double)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 6144 Data size: 55052 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(_col1)
keys: _col0 (type: tinyint)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3072 Data size: 660491 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 95 Data size: 1048 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 4300 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 224 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 4300 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 224 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -666,6 +680,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -674,42 +689,43 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: alltypesorc
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 1779850 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: ctinyint (type: tinyint), cstring1 (type: string), cstring2 (type: string)
outputColumnNames: ctinyint, cstring1, cstring2
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 1779850 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(DISTINCT cstring1), count(DISTINCT cstring2)
keys: ctinyint (type: tinyint), cstring1 (type: string), cstring2 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 1976458 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: tinyint), _col1 (type: string), _col2 (type: string)
sort order: +++
Map-reduce partition columns: _col0 (type: tinyint)
- Statistics: Num rows: 12288 Data size: 2641964 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 12288 Data size: 1976458 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
Execution mode: llap
+ LLAP IO: all inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col1:0._col0), count(DISTINCT KEY._col1:1._col0)
keys: KEY._col0 (type: tinyint)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 95 Data size: 1808 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 4300 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 384 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 4300 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 384 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -787,6 +803,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -796,24 +813,25 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: value (type: string), key (type: string)
outputColumnNames: value, key
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(key)
keys: value (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: double)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -822,29 +840,29 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: double)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: _col0 (type: string)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: double)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 1980 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 1980 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -904,6 +922,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE)
@@ -915,48 +934,50 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -965,18 +986,18 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 2
- Statistics: Num rows: 2 Data size: 20 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 3
Execution mode: llap
@@ -988,16 +1009,16 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 2 Data size: 22 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 380 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 4
- Statistics: Num rows: 2 Data size: 22 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 380 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 2 Data size: 22 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 380 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: llap
@@ -1007,13 +1028,13 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 3
- Statistics: Num rows: 3 Data size: 30 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 285 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 3 Data size: 30 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 285 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: _col0 (type: string), _col1 (type: bigint)
Reducer 6
@@ -1022,18 +1043,18 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), VALUE._col1 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 3 Data size: 30 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 285 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 3
- Statistics: Num rows: 3 Data size: 30 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 285 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 2 Data size: 20 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 285 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 20 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 285 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Stage: Stage-0
@@ -1057,6 +1078,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1065,37 +1087,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: value (type: string)
sort order: +
Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 0.3
value expressions: key (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
keys: KEY._col0 (type: string)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 214 Data size: 21186 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 20
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 1980 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 20 Data size: 200 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 20 Data size: 1980 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1147,6 +1170,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1155,34 +1179,35 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 407500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 407500 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 2.0E-5
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: string), VALUE._col0 (type: string), VALUE._col0 (type: string), VALUE._col0 (type: string), VALUE._col0 (type: string), VALUE._col0 (type: string), VALUE._col0 (type: string), VALUE._col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 407500 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 100
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 81500 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 81500 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1314,6 +1339,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1323,18 +1349,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: concat(key, value, value, value, value, value, value, value, value, value) (type: string), key (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1343,32 +1370,32 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 96000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: double)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
TopN Hash Memory Usage: 2.0E-5
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: double)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 4000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 100
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 800 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 100 Data size: 800 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/load_dyn_part1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/load_dyn_part1.q.out b/ql/src/test/results/clientpositive/llap/load_dyn_part1.q.out
index 8488240..efd79a6 100644
--- a/ql/src/test/results/clientpositive/llap/load_dyn_part1.q.out
+++ b/ql/src/test/results/clientpositive/llap/load_dyn_part1.q.out
@@ -73,17 +73,17 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 1092000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (ds <= '2008-04-08') (type: boolean)
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 666 Data size: 363636 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string), ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 666 Data size: 363636 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 666 Data size: 363636 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -91,20 +91,21 @@ STAGE PLANS:
name: default.nzhang_part1
Filter Operator
predicate: (ds > '2008-04-08') (type: boolean)
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 666 Data size: 363636 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string), hr (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 666 Data size: 241092 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 666 Data size: 241092 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.nzhang_part2
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-3
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/load_dyn_part2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/load_dyn_part2.q.out b/ql/src/test/results/clientpositive/llap/load_dyn_part2.q.out
index c0ed71c..52d754a 100644
--- a/ql/src/test/results/clientpositive/llap/load_dyn_part2.q.out
+++ b/ql/src/test/results/clientpositive/llap/load_dyn_part2.q.out
@@ -47,6 +47,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -55,27 +56,28 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 724000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string), hr (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 724000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 724000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: string), VALUE._col1 (type: string), VALUE._col2 (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 724000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 724000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/load_dyn_part3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/load_dyn_part3.q.out b/ql/src/test/results/clientpositive/llap/load_dyn_part3.q.out
index 65a3037..7a5b0fc 100644
--- a/ql/src/test/results/clientpositive/llap/load_dyn_part3.q.out
+++ b/ql/src/test/results/clientpositive/llap/load_dyn_part3.q.out
@@ -59,20 +59,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: srcpart
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 1092000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string), ds (type: string), hr (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 1092000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2000 Data size: 1092000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.nzhang_part3
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/mapjoin2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/mapjoin2.q.out b/ql/src/test/results/clientpositive/llap/mapjoin2.q.out
new file mode 100644
index 0000000..a7f7d15
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/mapjoin2.q.out
@@ -0,0 +1,95 @@
+PREHOOK: query: create table tbl (n bigint, t string)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@tbl
+POSTHOOK: query: create table tbl (n bigint, t string)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@tbl
+PREHOOK: query: insert into tbl values (1, 'one')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__1
+PREHOOK: Output: default@tbl
+POSTHOOK: query: insert into tbl values (1, 'one')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__1
+POSTHOOK: Output: default@tbl
+POSTHOOK: Lineage: tbl.n EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: tbl.t SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+PREHOOK: query: insert into tbl values(2, 'two')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__2
+PREHOOK: Output: default@tbl
+POSTHOOK: query: insert into tbl values(2, 'two')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__2
+POSTHOOK: Output: default@tbl
+POSTHOOK: Lineage: tbl.n EXPRESSION [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: tbl.t SIMPLE [(values__tmp__table__2)values__tmp__table__2.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+Warning: Map Join MAPJOIN[13][bigTable=?] in task 'Map 1' is a cross product
+PREHOOK: query: select a.n, a.t, isnull(b.n), isnull(b.t) from (select * from tbl where n = 1) a left outer join (select * from tbl where 1 = 2) b on a.n = b.n
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl
+#### A masked pattern was here ####
+POSTHOOK: query: select a.n, a.t, isnull(b.n), isnull(b.t) from (select * from tbl where n = 1) a left outer join (select * from tbl where 1 = 2) b on a.n = b.n
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl
+#### A masked pattern was here ####
+1 one true true
+Warning: Map Join MAPJOIN[13][bigTable=?] in task 'Map 2' is a cross product
+PREHOOK: query: select isnull(a.n), isnull(a.t), b.n, b.t from (select * from tbl where 2 = 1) a right outer join (select * from tbl where n = 2) b on a.n = b.n
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl
+#### A masked pattern was here ####
+POSTHOOK: query: select isnull(a.n), isnull(a.t), b.n, b.t from (select * from tbl where 2 = 1) a right outer join (select * from tbl where n = 2) b on a.n = b.n
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl
+#### A masked pattern was here ####
+true true 2 two
+Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+PREHOOK: query: select isnull(a.n), isnull(a.t), isnull(b.n), isnull(b.t) from (select * from tbl where n = 1) a full outer join (select * from tbl where n = 2) b on a.n = b.n
+PREHOOK: type: QUERY
+PREHOOK: Input: default@tbl
+#### A masked pattern was here ####
+POSTHOOK: query: select isnull(a.n), isnull(a.t), isnull(b.n), isnull(b.t) from (select * from tbl where n = 1) a full outer join (select * from tbl where n = 2) b on a.n = b.n
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@tbl
+#### A masked pattern was here ####
+false false true true
+true true false false
+PREHOOK: query: select a.key, a.a_one, b.b_one, a.a_zero, b.b_zero from ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a join ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b on a.key = b.key
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: select a.key, a.a_one, b.b_one, a.a_zero, b.b_zero from ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a join ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b on a.key = b.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+11 1 1 0 0
+PREHOOK: query: select a.key, a.a_one, b.b_one, a.a_zero, b.b_zero from ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a left outer join ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b on a.key = b.key
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: select a.key, a.a_one, b.b_one, a.a_zero, b.b_zero from ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a left outer join ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b on a.key = b.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+11 1 1 0 0
+PREHOOK: query: select a.key, a.a_one, b.b_one, a.a_zero, b.b_zero from ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a right outer join ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b on a.key = b.key
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: select a.key, a.a_one, b.b_one, a.a_zero, b.b_zero from ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a right outer join ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b on a.key = b.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+11 1 1 0 0
+PREHOOK: query: select a.key, a.a_one, b.b_one, a.a_zero, b.b_zero from ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a full outer join ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b on a.key = b.key
+PREHOOK: type: QUERY
+PREHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+POSTHOOK: query: select a.key, a.a_one, b.b_one, a.a_zero, b.b_zero from ( SELECT 11 key, 0 confuse_you, 1 a_one, 0 a_zero ) a full outer join ( SELECT 11 key, 0 confuse_you, 1 b_one, 0 b_zero ) b on a.key = b.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: _dummy_database@_dummy_table
+#### A masked pattern was here ####
+11 1 1 0 0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/mapreduce1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/mapreduce1.q.out b/ql/src/test/results/clientpositive/llap/mapreduce1.q.out
index 6eb5ecc..37f92d9 100644
--- a/ql/src/test/results/clientpositive/llap/mapreduce1.q.out
+++ b/ql/src/test/results/clientpositive/llap/mapreduce1.q.out
@@ -31,6 +31,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -39,34 +40,34 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), UDFToInteger((key / 10)) (type: int), UDFToInteger((key % 10)) (type: int), value (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 93000 Basic stats: COMPLETE Column stats: COMPLETE
Transform Operator
command: cat
output info:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 93000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string), _col2 (type: string)
sort order: ++
Map-reduce partition columns: _col3 (type: string), _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 93000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string)
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: UDFToInteger(VALUE._col0) (type: int), UDFToInteger(VALUE._col1) (type: int), UDFToInteger(VALUE._col2) (type: int), VALUE._col3 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 6000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 6000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/mapreduce2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/mapreduce2.q.out b/ql/src/test/results/clientpositive/llap/mapreduce2.q.out
index 2b18150..1904768 100644
--- a/ql/src/test/results/clientpositive/llap/mapreduce2.q.out
+++ b/ql/src/test/results/clientpositive/llap/mapreduce2.q.out
@@ -29,6 +29,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -37,33 +38,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), UDFToInteger((key / 10)) (type: int), UDFToInteger((key % 10)) (type: int), value (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 93000 Basic stats: COMPLETE Column stats: COMPLETE
Transform Operator
command: cat
output info:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 93000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
Map-reduce partition columns: _col3 (type: string), _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 93000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string)
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: UDFToInteger(VALUE._col0) (type: int), UDFToInteger(VALUE._col1) (type: int), UDFToInteger(VALUE._col2) (type: int), VALUE._col3 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 6000 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 6000 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/merge1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/merge1.q.out b/ql/src/test/results/clientpositive/llap/merge1.q.out
index ce3cd2c..4ec5d06 100644
--- a/ql/src/test/results/clientpositive/llap/merge1.q.out
+++ b/ql/src/test/results/clientpositive/llap/merge1.q.out
@@ -27,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -35,40 +36,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), UDFToInteger(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 1640 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 1640 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -485,20 +487,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test_src
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dest1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -550,20 +553,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test_src
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.dest1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/merge2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/merge2.q.out b/ql/src/test/results/clientpositive/llap/merge2.q.out
index e515db1..383c598 100644
--- a/ql/src/test/results/clientpositive/llap/merge2.q.out
+++ b/ql/src/test/results/clientpositive/llap/merge2.q.out
@@ -27,6 +27,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -35,40 +36,41 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(1)
keys: _col0 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 250 Data size: 23750 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), UDFToInteger(_col1) (type: int)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 1640 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 1640 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -485,20 +487,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test_src
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
@@ -550,20 +553,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: test_src
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1000 Data size: 18624 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.test1
Execution mode: llap
+ LLAP IO: no inputs
Stage: Stage-2
Dependency Collection
[49/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join30.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_join30.q.out b/ql/src/test/results/clientpositive/llap/auto_join30.q.out
index 3af7e85..d2cfa62 100644
--- a/ql/src/test/results/clientpositive/llap/auto_join30.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_join30.q.out
@@ -21,54 +21,69 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (BROADCAST_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
- Reducer 5 <- Map 4 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (SIMPLE_EDGE), Reducer 2 (BROADCAST_EDGE)
+ Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
- Map 4
+ LLAP IO: no inputs
+ Map 3
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reducer 4
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -77,46 +92,32 @@ STAGE PLANS:
1 _col0 (type: string)
outputColumnNames: _col2, _col3
input vertices:
- 1 Reducer 5
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ 0 Reducer 2
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
- Reducer 3
- Execution mode: uber
+ Reducer 5
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 5
- Execution mode: llap
- Reduce Operator Tree:
- Select Operator
- expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: string)
Stage: Stage-0
Fetch Operator
@@ -166,6 +167,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (BROADCAST_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -176,38 +178,40 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Outer Join0 to 1
@@ -217,31 +221,30 @@ STAGE PLANS:
outputColumnNames: _col2, _col3
input vertices:
1 Reducer 5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 5
Execution mode: llap
@@ -249,12 +252,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Stage: Stage-0
@@ -305,6 +308,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 4 <- Map 3 (SIMPLE_EDGE), Reducer 2 (BROADCAST_EDGE)
@@ -315,50 +319,52 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Right Outer Join0 to 1
@@ -368,31 +374,30 @@ STAGE PLANS:
outputColumnNames: _col2, _col3
input vertices:
0 Reducer 2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 5
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -449,10 +454,11 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (BROADCAST_EDGE), Reducer 7 (BROADCAST_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
- Reducer 5 <- Map 4 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+ Reducer 4 <- Map 3 (SIMPLE_EDGE), Reducer 2 (BROADCAST_EDGE), Reducer 7 (BROADCAST_EDGE)
+ Reducer 5 <- Reducer 4 (SIMPLE_EDGE)
Reducer 7 <- Map 6 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
@@ -460,62 +466,77 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
- Map 4
+ LLAP IO: no inputs
+ Map 3
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 6
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reducer 4
+ Execution mode: llap
+ Reduce Operator Tree:
+ Select Operator
+ expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -526,59 +547,45 @@ STAGE PLANS:
2 _col0 (type: string)
outputColumnNames: _col2, _col3
input vertices:
- 1 Reducer 5
+ 0 Reducer 2
2 Reducer 7
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 2974 Data size: 529372 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
- Reducer 3
- Execution mode: uber
+ Reducer 5
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Reducer 5
- Execution mode: llap
- Reduce Operator Tree:
- Select Operator
- expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: string)
Reducer 7
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: VALUE._col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
@@ -640,6 +647,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
@@ -652,58 +660,61 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -716,30 +727,30 @@ STAGE PLANS:
1 _col0 (type: string)
2 _col0 (type: string)
outputColumnNames: _col2, _col3
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2974 Data size: 529372 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: llap
@@ -747,12 +758,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Reducer 8
Execution mode: llap
@@ -760,12 +771,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
@@ -827,6 +838,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
@@ -839,58 +851,61 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -903,30 +918,30 @@ STAGE PLANS:
1 _col0 (type: string)
2 _col0 (type: string)
outputColumnNames: _col2, _col3
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2974 Data size: 529372 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: llap
@@ -934,12 +949,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Reducer 8
Execution mode: llap
@@ -947,12 +962,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
@@ -1014,6 +1029,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
@@ -1026,58 +1042,61 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -1090,30 +1109,30 @@ STAGE PLANS:
1 _col0 (type: string)
2 _col0 (type: string)
outputColumnNames: _col2, _col3
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2974 Data size: 529372 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: llap
@@ -1121,12 +1140,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Reducer 8
Execution mode: llap
@@ -1134,12 +1153,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
@@ -1201,6 +1220,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Reducer 6 (SIMPLE_EDGE), Reducer 8 (SIMPLE_EDGE)
@@ -1213,58 +1233,61 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 7
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: string)
sort order: +
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -1277,30 +1300,30 @@ STAGE PLANS:
1 _col0 (type: string)
2 _col0 (type: string)
outputColumnNames: _col2, _col3
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2974 Data size: 529372 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: sum(hash(_col2,_col3))
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: bigint)
Reducer 4
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Reducer 6
Execution mode: llap
@@ -1308,12 +1331,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Reducer 8
Execution mode: llap
@@ -1321,12 +1344,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Stage: Stage-0
Fetch Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join_filters.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_join_filters.q.out b/ql/src/test/results/clientpositive/llap/auto_join_filters.q.out
index 8fde41d..d1d9408 100644
--- a/ql/src/test/results/clientpositive/llap/auto_join_filters.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_join_filters.q.out
@@ -14,7 +14,7 @@ POSTHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/in3.txt' INTO TABLE my
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@myinput1
-Warning: Map Join MAPJOIN[19][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[18][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a JOIN myinput1 b on a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -24,7 +24,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
3078400
-Warning: Map Join MAPJOIN[15][bigTable=a] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[16][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a LEFT OUTER JOIN myinput1 b on a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -34,7 +34,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
4937935
-Warning: Map Join MAPJOIN[15][bigTable=b] in task 'Map 2' is a cross product
+Warning: Map Join MAPJOIN[16][bigTable=?] in task 'Map 2' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a RIGHT OUTER JOIN myinput1 b on a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -300,7 +300,7 @@ POSTHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/in2.txt' into table sm
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@smb_input2
-Warning: Map Join MAPJOIN[19][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[18][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a JOIN myinput1 b on a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -310,7 +310,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
3078400
-Warning: Map Join MAPJOIN[19][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[16][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a LEFT OUTER JOIN myinput1 b on a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -319,8 +319,8 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a L
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
-Warning: Map Join MAPJOIN[19][bigTable=?] in task 'Map 2' is a cross product
+4937935
+Warning: Map Join MAPJOIN[16][bigTable=?] in task 'Map 2' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a RIGHT OUTER JOIN myinput1 b on a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -329,8 +329,8 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a R
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
-Warning: Shuffle Join MERGEJOIN[22][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
+3080335
+Warning: Shuffle Join MERGEJOIN[14][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a FULL OUTER JOIN myinput1 b on a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -339,7 +339,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a F
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4939870
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a JOIN myinput1 b ON a.key = b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -384,7 +384,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a L
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4937935
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a LEFT OUTER JOIN myinput1 b ON a.value = b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -393,7 +393,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a L
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4937935
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a LEFT OUTER JOIN myinput1 b ON a.key = b.key AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -402,7 +402,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a L
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4937935
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a LEFT OUTER JOIN myinput1 b ON a.key = b.key and a.value=b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -411,7 +411,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a L
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4937935
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a RIGHT OUTER JOIN myinput1 b ON a.key = b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -420,7 +420,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a R
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+3080335
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a RIGHT OUTER JOIN myinput1 b ON a.key = b.key AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -429,7 +429,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a R
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+3080335
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a RIGHT OUTER JOIN myinput1 b ON a.value = b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -438,7 +438,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a R
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+3080335
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a RIGHT OUTER JOIN myinput1 b ON a.key=b.key and a.value = b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -447,7 +447,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a R
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+3080335
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a FULL OUTER JOIN myinput1 b ON a.key = b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -456,7 +456,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a F
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4939870
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a FULL OUTER JOIN myinput1 b ON a.key = b.key AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -465,7 +465,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a F
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4939870
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a FULL OUTER JOIN myinput1 b ON a.value = b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -474,7 +474,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a F
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4939870
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a FULL OUTER JOIN myinput1 b ON a.value = b.value and a.key=b.key AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -483,7 +483,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a F
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+4939870
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) from myinput1 a LEFT OUTER JOIN myinput1 b ON (a.value=b.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value) RIGHT OUTER JOIN myinput1 c ON (b.value=c.value AND c.key > 40 AND c.value > 50 AND c.key = c.value AND b.key > 40 AND b.value > 50 AND b.key = b.value)
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -501,7 +501,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) from myinput1 a R
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+3080335
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a LEFT OUTER JOIN myinput1 b RIGHT OUTER JOIN myinput1 c ON a.value = b.value and b.value = c.value AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value AND c.key > 40 AND c.value > 50 AND c.key = c.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -528,7 +528,7 @@ POSTHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) from myinput1 a R
POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
-3078400
+3080335
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a LEFT OUTER JOIN myinput1 b RIGHT OUTER JOIN myinput1 c ON a.value = b.value and b.key = c.key AND a.key > 40 AND a.value > 50 AND a.key = a.value AND b.key > 40 AND b.value > 50 AND b.key = b.value AND c.key > 40 AND c.value > 50 AND c.key = c.value
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join_nulls.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_join_nulls.q.out b/ql/src/test/results/clientpositive/llap/auto_join_nulls.q.out
index b9a759f..5984e8f 100644
--- a/ql/src/test/results/clientpositive/llap/auto_join_nulls.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_join_nulls.q.out
@@ -14,7 +14,7 @@ POSTHOOK: query: LOAD DATA LOCAL INPATH '../../data/files/in1.txt' INTO TABLE my
POSTHOOK: type: LOAD
#### A masked pattern was here ####
POSTHOOK: Output: default@myinput1
-Warning: Map Join MAPJOIN[16][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[14][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a JOIN myinput1 b
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -24,7 +24,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
13630578
-Warning: Map Join MAPJOIN[15][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[14][bigTable=?] in task 'Map 1' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a LEFT OUTER JOIN myinput1 b
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
@@ -34,7 +34,7 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@myinput1
#### A masked pattern was here ####
13630578
-Warning: Map Join MAPJOIN[15][bigTable=?] in task 'Map 1' is a cross product
+Warning: Map Join MAPJOIN[14][bigTable=?] in task 'Map 2' is a cross product
PREHOOK: query: SELECT sum(hash(a.key,a.value,b.key,b.value)) FROM myinput1 a RIGHT OUTER JOIN myinput1 b
PREHOOK: type: QUERY
PREHOOK: Input: default@myinput1
[45/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out
index 16a6f43..9e83db1 100644
--- a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out
@@ -95,6 +95,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -107,42 +108,55 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- outputColumnNames: _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col1 (type: string)
- sort order: +
- Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: string)
- sort order: +
- Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -151,8 +165,8 @@ STAGE PLANS:
Inner Join 0 to 1
keys:
0 _col1 (type: string)
- 1 value (type: string)
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ 1 _col0 (type: string)
+ Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -163,7 +177,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -174,8 +188,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -214,6 +228,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -226,42 +241,55 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- outputColumnNames: _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col1 (type: string)
- sort order: +
- Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
TableScan
alias: d
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: string)
- sort order: +
- Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -270,8 +298,8 @@ STAGE PLANS:
Inner Join 0 to 1
keys:
0 _col1 (type: string)
- 1 value (type: string)
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ 1 _col0 (type: string)
+ Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -282,7 +310,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -293,8 +321,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -333,6 +361,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -345,42 +374,55 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- outputColumnNames: _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col1 (type: string)
- sort order: +
- Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
TableScan
alias: h
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: string)
- sort order: +
- Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -389,8 +431,8 @@ STAGE PLANS:
Inner Join 0 to 1
keys:
0 _col1 (type: string)
- 1 value (type: string)
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ 1 _col0 (type: string)
+ Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -401,7 +443,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -412,8 +454,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -448,65 +490,78 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
- alias: a
+ alias: b
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: UDFToDouble(key) is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(key) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(key) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- Map 4
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
- alias: b
+ alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: UDFToDouble(key) is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(key) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(key) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: UDFToDouble(_col0) (type: double)
+ sort order: +
+ Map-reduce partition columns: UDFToDouble(_col0) (type: double)
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: UDFToDouble(key) is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(key) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(key) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: UDFToDouble(_col0) (type: double)
+ sort order: +
+ Map-reduce partition columns: UDFToDouble(_col0) (type: double)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
- Inner Join 0 to 2
keys:
- 0 UDFToDouble(key) (type: double)
- 1 UDFToDouble(key) (type: double)
- 2 UDFToDouble(key) (type: double)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ 0 UDFToDouble(_col0) (type: double)
+ 1 UDFToDouble(_col0) (type: double)
+ Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -517,7 +572,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -528,8 +583,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -564,6 +619,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -575,42 +631,54 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: c
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- Inner Join 0 to 2
- keys:
- 0 key (type: int)
- 1 key (type: int)
- 2 key (type: int)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Inner Join 0 to 2
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ 2 _col0 (type: int)
+ Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -621,8 +689,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -657,6 +725,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -669,27 +738,35 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- outputColumnNames: _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col1 (type: string)
- sort order: +
- Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
@@ -698,13 +775,18 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: string)
- sort order: +
- Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -713,8 +795,8 @@ STAGE PLANS:
Inner Join 0 to 1
keys:
0 _col1 (type: string)
- 1 value (type: string)
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ 1 _col0 (type: string)
+ Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -725,7 +807,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -736,8 +818,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -772,6 +854,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -784,42 +867,55 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- outputColumnNames: _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col1 (type: string)
- sort order: +
- Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: string)
- sort order: +
- Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -828,8 +924,8 @@ STAGE PLANS:
Inner Join 0 to 1
keys:
0 _col1 (type: string)
- 1 value (type: string)
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ 1 _col0 (type: string)
+ Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -840,7 +936,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -851,8 +947,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -887,65 +983,78 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
- alias: a
+ alias: b
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: UDFToDouble(key) is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(key) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(key) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- Map 4
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
- alias: b
+ alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: UDFToDouble(key) is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(key) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(key) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: UDFToDouble(_col0) (type: double)
+ sort order: +
+ Map-reduce partition columns: UDFToDouble(_col0) (type: double)
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
TableScan
alias: c
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: UDFToDouble(key) is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(key) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(key) (type: double)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: UDFToDouble(_col0) (type: double)
+ sort order: +
+ Map-reduce partition columns: UDFToDouble(_col0) (type: double)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Merge Join Operator
condition map:
Inner Join 0 to 1
- Inner Join 0 to 2
keys:
- 0 UDFToDouble(key) (type: double)
- 1 UDFToDouble(key) (type: double)
- 2 UDFToDouble(key) (type: double)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ 0 UDFToDouble(_col0) (type: double)
+ 1 UDFToDouble(_col0) (type: double)
+ Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -956,7 +1065,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -967,8 +1076,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1003,6 +1112,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1014,42 +1124,54 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: c
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- Inner Join 0 to 2
- keys:
- 0 key (type: int)
- 1 key (type: int)
- 2 key (type: int)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count()
- mode: hash
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ Inner Join 0 to 2
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ 2 _col0 (type: int)
+ Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: bigint)
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
Execution mode: llap
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1060,8 +1182,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1096,6 +1218,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1108,27 +1231,35 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Map Operator Tree:
TableScan
alias: a
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (key is not null and value is not null) (type: boolean)
- Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 key (type: int)
- 1 key (type: int)
- outputColumnNames: _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col1 (type: string)
- sort order: +
- Map-reduce partition columns: _col1 (type: string)
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col1
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col1 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col1 (type: string)
+ Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
Map 5
Map Operator Tree:
@@ -1137,13 +1268,18 @@ STAGE PLANS:
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: value is not null (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: string)
- sort order: +
- Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1152,8 +1288,8 @@ STAGE PLANS:
Inner Join 0 to 1
keys:
0 _col1 (type: string)
- 1 value (type: string)
- Statistics: Num rows: 302 Data size: 3213 Basic stats: COMPLETE Column stats: NONE
+ 1 _col0 (type: string)
+ Statistics: Num rows: 605 Data size: 6427 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -1164,7 +1300,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: bigint)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
@@ -1175,8 +1311,8 @@ STAGE PLANS:
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
[18/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/mrr.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/mrr.q.out b/ql/src/test/results/clientpositive/llap/mrr.q.out
index 7e2a554..6272d04 100644
--- a/ql/src/test/results/clientpositive/llap/mrr.q.out
+++ b/ql/src/test/results/clientpositive/llap/mrr.q.out
@@ -25,22 +25,22 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -52,11 +52,11 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 3
Execution mode: llap
@@ -64,10 +64,10 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -420,38 +420,38 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -465,17 +465,17 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col1, _col2
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col1 (type: string), _col2 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 609 Data size: 108402 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 609 Data size: 108402 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
Execution mode: llap
Reduce Operator Tree:
@@ -483,27 +483,27 @@ STAGE PLANS:
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 609 Data size: 108402 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(_col1)
keys: _col0 (type: string)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint), _col0 (type: string)
sort order: ++
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 4
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -847,23 +847,42 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Map 1 <- Map 4 (BROADCAST_EDGE)
- Reducer 2 <- Map 1 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+ Map 2 <- Map 1 (BROADCAST_EDGE)
+ Reducer 3 <- Map 2 (SIMPLE_EDGE)
+ Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: s1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 500 Data size: 43500 Basic stats: COMPLETE Column stats: COMPLETE
+ Execution mode: llap
+ LLAP IO: no inputs
+ Map 2
+ Map Operator Tree:
+ TableScan
+ alias: s2
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ Select Operator
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -872,68 +891,48 @@ STAGE PLANS:
1 _col0 (type: string)
outputColumnNames: _col1, _col2
input vertices:
- 1 Map 4
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ 0 Map 1
+ Statistics: Num rows: 1219 Data size: 216982 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
keys: _col1 (type: string), _col2 (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 609 Data size: 108402 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 609 Data size: 108402 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: s2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: string)
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 2
+ Reducer 3
Execution mode: llap
Reduce Operator Tree:
Group By Operator
keys: KEY._col0 (type: string), KEY._col1 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 275 Data size: 2921 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 609 Data size: 108402 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(_col1)
keys: _col0 (type: string)
mode: complete
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint), _col0 (type: string)
sort order: ++
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
- Reducer 3
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
+ Reducer 4
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey1 (type: string), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 137 Data size: 1455 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1319,21 +1318,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -1341,21 +1340,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -1363,21 +1362,21 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
LLAP IO: no inputs
@@ -1389,11 +1388,11 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 11
Execution mode: llap
@@ -1401,12 +1400,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 2
Execution mode: llap
@@ -1416,11 +1415,11 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
sort order: +
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 3
Execution mode: llap
@@ -1428,12 +1427,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Reducer 4
Execution mode: llap
@@ -1447,15 +1446,15 @@ STAGE PLANS:
1 _col0 (type: string)
2 _col0 (type: string)
outputColumnNames: _col0, _col1, _col3, _col4, _col5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 13464 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col3 (type: bigint), _col1 (type: bigint), _col4 (type: string), _col5 (type: bigint)
outputColumnNames: _col0, _col1, _col3, _col4, _col5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 19380 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 19380 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col3 (type: bigint), _col4 (type: string), _col5 (type: bigint)
Reducer 5
Execution mode: llap
@@ -1463,10 +1462,10 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint), KEY.reducesinkkey0 (type: string), VALUE._col1 (type: bigint), VALUE._col2 (type: string), VALUE._col3 (type: bigint)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 19380 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 19380 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -1479,14 +1478,14 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col1 > 1) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 6460 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col1 (type: bigint)
sort order: +
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 6460 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: string)
Reducer 8
Execution mode: llap
@@ -1494,12 +1493,12 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: string), KEY.reducesinkkey0 (type: bigint)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 6460 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 6460 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Stage: Stage-0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/nonmr_fetch_threshold.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/nonmr_fetch_threshold.q.out b/ql/src/test/results/clientpositive/llap/nonmr_fetch_threshold.q.out
new file mode 100644
index 0000000..b45e4db
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/nonmr_fetch_threshold.q.out
@@ -0,0 +1,205 @@
+PREHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:10
+ Limit [LIM_3]
+ Number of rows:10
+ Select Operator [SEL_2]
+ Output:["_col0","_col1","_col2","_col3"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:10
+ Limit [LIM_2]
+ Number of rows:10
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:10
+ Limit [LIM_3]
+ Number of rows:10
+ Select Operator [SEL_2]
+ Output:["_col0","_col1","_col2","_col3"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:10
+ Limit [LIM_2]
+ Number of rows:10
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: -- Scans without limit (should be Fetch task now)
+explain select concat(key, value) from src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Scans without limit (should be Fetch task now)
+explain select concat(key, value) from src
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: -- from HIVE-7397, limit + partition pruning filter
+explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10
+PREHOOK: type: QUERY
+POSTHOOK: query: -- from HIVE-7397, limit + partition pruning filter
+explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:10
+ Limit [LIM_3]
+ Number of rows:10
+ Select Operator [SEL_2]
+ Output:["_col0","_col1","_col2","_col3"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:10
+ Limit [LIM_2]
+ Number of rows:10
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: -- Scans without limit (should not be Fetch task now)
+explain select concat(key, value) from src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Scans without limit (should not be Fetch task now)
+explain select concat(key, value) from src
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_2]
+ Select Operator [SEL_1] (rows=500 width=184)
+ Output:["_col0"]
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+
+PREHOOK: query: -- Simple Scans without limit (will be Fetch task now)
+explain select key, value from src
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Simple Scans without limit (will be Fetch task now)
+explain select key, value from src
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: explain select key from src
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select key from src
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0"]
+ TableScan [TS_0]
+ Output:["key"]
+
+PREHOOK: query: explain select * from src
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select * from src
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
+
+PREHOOK: query: explain select key,1 from src
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select key,1 from src
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key"]
+
+PREHOOK: query: explain select cast(key as char(20)),1 from src
+PREHOOK: type: QUERY
+POSTHOOK: query: explain select cast(key as char(20)),1 from src
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key"]
+
[26/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/explainuser_4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_4.q.out b/ql/src/test/results/clientpositive/llap/explainuser_4.q.out
new file mode 100644
index 0000000..4ea1488
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/explainuser_4.q.out
@@ -0,0 +1,482 @@
+PREHOOK: query: -- First try with regular mergejoin
+explain
+select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+PREHOOK: type: QUERY
+POSTHOOK: query: -- First try with regular mergejoin
+explain
+select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_12]
+ Select Operator [SEL_11] (rows=7286 width=620)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_10]
+ Merge Join Operator [MERGEJOIN_17] (rows=7286 width=620)
+ Conds:RS_6._col2=RS_7._col2(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=6144 width=251)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_15] (rows=6144 width=251)
+ predicate:cint BETWEEN 1000000 AND 3000000
+ TableScan [TS_0] (rows=12288 width=251)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:COMPLETE,Output:["ctinyint","csmallint","cint","cbigint","cfloat","cdouble","cstring1","cstring2","ctimestamp1","ctimestamp2","cboolean1","cboolean2"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ PartitionCols:_col2
+ Select Operator [SEL_5] (rows=3424 width=251)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_16] (rows=3424 width=251)
+ predicate:(cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=251)
+ default@alltypesorc,b,Tbl:COMPLETE,Col:COMPLETE,Output:["ctinyint","csmallint","cint","cbigint","cfloat","cdouble","cstring1","cstring2","ctimestamp1","ctimestamp2","cboolean1","cboolean2"]
+
+PREHOOK: query: select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+11 NULL 1000828 1531084669 11.0 NULL wM316f6NqGIkoP388j3F6 poWQQo3Upvt3Wh 1969-12-31 16:00:02.351 NULL false true 11 NULL 1000828 1531084669 11.0 NULL wM316f6NqGIkoP388j3F6 poWQQo3Upvt3Wh 1969-12-31 16:00:02.351 NULL false true
+NULL -3799 1248059 1864027286 NULL -3799.0 Uhps6mMh3IfHB3j7yH62K 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:54.622 false true NULL -3799 1248059 1864027286 NULL -3799.0 Uhps6mMh3IfHB3j7yH62K 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:54.622 false true
+NULL 10782 1286921 1864027286 NULL 10782.0 ODLrXI8882q8LS8 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:52.138 true true NULL 10782 1286921 1864027286 NULL 10782.0 ODLrXI8882q8LS8 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:52.138 true true
+NULL -13036 1288927 -1645852809 NULL -13036.0 yinBY725P7V2 xH7445Rals48VOulSyR5F NULL 1969-12-31 16:00:00.763 true false NULL -13036 1288927 -1645852809 NULL -13036.0 yinBY725P7V2 xH7445Rals48VOulSyR5F NULL 1969-12-31 16:00:00.763 true false
+11 NULL 1310786 -413875656 11.0 NULL W0rvA4H1xn0xMG4uk0 8yVVjG 1969-12-31 16:00:02.351 NULL false true 11 NULL 1310786 -413875656 11.0 NULL W0rvA4H1xn0xMG4uk0 8yVVjG 1969-12-31 16:00:02.351 NULL false true
+-51 NULL 2089466 -240556350 -51.0 NULL cXX24dH7tblSj46j2g C31eea0wrHHqvj 1969-12-31 16:00:08.451 NULL true true -51 NULL 2089466 -240556350 -51.0 NULL cXX24dH7tblSj46j2g C31eea0wrHHqvj 1969-12-31 16:00:08.451 NULL true true
+NULL -8915 2101183 1864027286 NULL -8915.0 x7By66525 4KWs6gw7lv2WYd66P NULL 1969-12-31 16:00:05.831 false true NULL -8915 2101183 1864027286 NULL -8915.0 x7By66525 4KWs6gw7lv2WYd66P NULL 1969-12-31 16:00:05.831 false true
+8 NULL 2229621 -381406148 8.0 NULL q7onkS7QRPh5ghOK oKb0bi 1969-12-31 16:00:15.892 NULL true false 8 NULL 2229621 -381406148 8.0 NULL q7onkS7QRPh5ghOK oKb0bi 1969-12-31 16:00:15.892 NULL true false
+8 NULL 2433892 -1611863517 8.0 NULL 674ILv3V2TxFqXP6wSbL VLprkK2XfX 1969-12-31 16:00:15.892 NULL false true 8 NULL 2433892 -1611863517 8.0 NULL 674ILv3V2TxFqXP6wSbL VLprkK2XfX 1969-12-31 16:00:15.892 NULL false true
+-51 NULL 2949963 -1580871111 -51.0 NULL 0K68k3bdl7jO7 TPPAu 1969-12-31 16:00:08.451 NULL true false -51 NULL 2949963 -1580871111 -51.0 NULL 0K68k3bdl7jO7 TPPAu 1969-12-31 16:00:08.451 NULL true false
+PREHOOK: query: explain
+select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_14]
+ Group By Operator [GBY_12] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(VALUE._col0)"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ Group By Operator [GBY_10] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count()"]
+ Merge Join Operator [MERGEJOIN_19] (rows=7286 width=8)
+ Conds:RS_6._col0=RS_7._col0(Inner)
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=6144 width=2)
+ Output:["_col0"]
+ Filter Operator [FIL_17] (rows=6144 width=2)
+ predicate:cint BETWEEN 1000000 AND 3000000
+ TableScan [TS_0] (rows=12288 width=2)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:COMPLETE,Output:["cint"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=3424 width=8)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=3424 width=8)
+ predicate:(cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=8)
+ default@alltypesorc,b,Tbl:COMPLETE,Col:COMPLETE,Output:["cint","cbigint"]
+
+PREHOOK: query: select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+10
+PREHOOK: query: explain
+select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 4 llap
+ File Output Operator [FS_16]
+ Select Operator [SEL_15] (rows=2765 width=12)
+ Output:["_col0","_col1"]
+ <-Reducer 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_14]
+ Group By Operator [GBY_12] (rows=2765 width=12)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Group By Operator [GBY_10] (rows=2765 width=12)
+ Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
+ Merge Join Operator [MERGEJOIN_21] (rows=7286 width=4)
+ Conds:RS_6._col1=RS_7._col0(Inner),Output:["_col0"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ PartitionCols:_col1
+ Select Operator [SEL_2] (rows=6144 width=5)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_19] (rows=6144 width=5)
+ predicate:cint BETWEEN 1000000 AND 3000000
+ TableScan [TS_0] (rows=12288 width=5)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:COMPLETE,Output:["csmallint","cint"]
+ <-Map 5 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=3424 width=8)
+ Output:["_col0"]
+ Filter Operator [FIL_20] (rows=3424 width=8)
+ predicate:(cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=8)
+ default@alltypesorc,b,Tbl:COMPLETE,Col:COMPLETE,Output:["cint","cbigint"]
+
+PREHOOK: query: select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+-13036 1
+-8915 1
+-3799 1
+10782 1
+NULL 6
+PREHOOK: query: -- Try with dynamically partitioned hashjoin
+explain
+select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Try with dynamically partitioned hashjoin
+explain
+select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Map 4 (CUSTOM_SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_12]
+ Select Operator [SEL_11] (rows=6758 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_10]
+ Map Join Operator [MAPJOIN_17] (rows=6758 width=215)
+ Conds:RS_6.KEY.reducesinkkey0=RS_7.KEY.reducesinkkey0(Inner),HybridGraceHashJoin:true,Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Map 4 [CUSTOM_SIMPLE_EDGE] llap
+ PARTITION_ONLY_SHUFFLE [RS_7]
+ PartitionCols:_col2
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_16] (rows=6144 width=215)
+ predicate:(cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,b,Tbl:COMPLETE,Col:NONE,Output:["ctinyint","csmallint","cint","cbigint","cfloat","cdouble","cstring1","cstring2","ctimestamp1","ctimestamp2","cboolean1","cboolean2"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE] llap
+ PARTITION_ONLY_SHUFFLE [RS_6]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_15] (rows=6144 width=215)
+ predicate:cint BETWEEN 1000000 AND 3000000
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["ctinyint","csmallint","cint","cbigint","cfloat","cdouble","cstring1","cstring2","ctimestamp1","ctimestamp2","cboolean1","cboolean2"]
+
+PREHOOK: query: select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+11 NULL 1000828 1531084669 11.0 NULL wM316f6NqGIkoP388j3F6 poWQQo3Upvt3Wh 1969-12-31 16:00:02.351 NULL false true 11 NULL 1000828 1531084669 11.0 NULL wM316f6NqGIkoP388j3F6 poWQQo3Upvt3Wh 1969-12-31 16:00:02.351 NULL false true
+NULL -3799 1248059 1864027286 NULL -3799.0 Uhps6mMh3IfHB3j7yH62K 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:54.622 false true NULL -3799 1248059 1864027286 NULL -3799.0 Uhps6mMh3IfHB3j7yH62K 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:54.622 false true
+NULL 10782 1286921 1864027286 NULL 10782.0 ODLrXI8882q8LS8 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:52.138 true true NULL 10782 1286921 1864027286 NULL 10782.0 ODLrXI8882q8LS8 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:52.138 true true
+NULL -13036 1288927 -1645852809 NULL -13036.0 yinBY725P7V2 xH7445Rals48VOulSyR5F NULL 1969-12-31 16:00:00.763 true false NULL -13036 1288927 -1645852809 NULL -13036.0 yinBY725P7V2 xH7445Rals48VOulSyR5F NULL 1969-12-31 16:00:00.763 true false
+11 NULL 1310786 -413875656 11.0 NULL W0rvA4H1xn0xMG4uk0 8yVVjG 1969-12-31 16:00:02.351 NULL false true 11 NULL 1310786 -413875656 11.0 NULL W0rvA4H1xn0xMG4uk0 8yVVjG 1969-12-31 16:00:02.351 NULL false true
+-51 NULL 2089466 -240556350 -51.0 NULL cXX24dH7tblSj46j2g C31eea0wrHHqvj 1969-12-31 16:00:08.451 NULL true true -51 NULL 2089466 -240556350 -51.0 NULL cXX24dH7tblSj46j2g C31eea0wrHHqvj 1969-12-31 16:00:08.451 NULL true true
+NULL -8915 2101183 1864027286 NULL -8915.0 x7By66525 4KWs6gw7lv2WYd66P NULL 1969-12-31 16:00:05.831 false true NULL -8915 2101183 1864027286 NULL -8915.0 x7By66525 4KWs6gw7lv2WYd66P NULL 1969-12-31 16:00:05.831 false true
+8 NULL 2229621 -381406148 8.0 NULL q7onkS7QRPh5ghOK oKb0bi 1969-12-31 16:00:15.892 NULL true false 8 NULL 2229621 -381406148 8.0 NULL q7onkS7QRPh5ghOK oKb0bi 1969-12-31 16:00:15.892 NULL true false
+8 NULL 2433892 -1611863517 8.0 NULL 674ILv3V2TxFqXP6wSbL VLprkK2XfX 1969-12-31 16:00:15.892 NULL false true 8 NULL 2433892 -1611863517 8.0 NULL 674ILv3V2TxFqXP6wSbL VLprkK2XfX 1969-12-31 16:00:15.892 NULL false true
+-51 NULL 2949963 -1580871111 -51.0 NULL 0K68k3bdl7jO7 TPPAu 1969-12-31 16:00:08.451 NULL true false -51 NULL 2949963 -1580871111 -51.0 NULL 0K68k3bdl7jO7 TPPAu 1969-12-31 16:00:08.451 NULL true false
+PREHOOK: query: explain
+select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Map 4 (CUSTOM_SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3 llap
+ File Output Operator [FS_14]
+ Group By Operator [GBY_12] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(VALUE._col0)"]
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ Group By Operator [GBY_10] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count()"]
+ Map Join Operator [MAPJOIN_19] (rows=6758 width=215)
+ Conds:RS_6.KEY.reducesinkkey0=RS_7.KEY.reducesinkkey0(Inner),HybridGraceHashJoin:true
+ <-Map 4 [CUSTOM_SIMPLE_EDGE] llap
+ PARTITION_ONLY_SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=6144 width=215)
+ predicate:(cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,b,Tbl:COMPLETE,Col:NONE,Output:["cint","cbigint"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE] llap
+ PARTITION_ONLY_SHUFFLE [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_17] (rows=6144 width=215)
+ predicate:cint BETWEEN 1000000 AND 3000000
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["cint"]
+
+PREHOOK: query: select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+10
+PREHOOK: query: explain
+select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Map 5 (CUSTOM_SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 4 llap
+ File Output Operator [FS_16]
+ Select Operator [SEL_15] (rows=3379 width=215)
+ Output:["_col0","_col1"]
+ <-Reducer 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_14]
+ Group By Operator [GBY_12] (rows=3379 width=215)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Group By Operator [GBY_10] (rows=6758 width=215)
+ Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
+ Map Join Operator [MAPJOIN_21] (rows=6758 width=215)
+ Conds:RS_6.KEY.reducesinkkey0=RS_7.KEY.reducesinkkey0(Inner),HybridGraceHashJoin:true,Output:["_col0"]
+ <-Map 5 [CUSTOM_SIMPLE_EDGE] llap
+ PARTITION_ONLY_SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_20] (rows=6144 width=215)
+ predicate:(cint is not null and cbigint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,b,Tbl:COMPLETE,Col:NONE,Output:["cint","cbigint"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE] llap
+ PARTITION_ONLY_SHUFFLE [RS_6]
+ PartitionCols:_col1
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_19] (rows=6144 width=215)
+ predicate:cint BETWEEN 1000000 AND 3000000
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["csmallint","cint"]
+
+PREHOOK: query: select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+-8915 1
+-3799 1
+10782 1
+-13036 1
+NULL 6
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/filter_join_breaktask.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/filter_join_breaktask.q.out b/ql/src/test/results/clientpositive/llap/filter_join_breaktask.q.out
index b8e738c..284ffb9 100644
--- a/ql/src/test/results/clientpositive/llap/filter_join_breaktask.q.out
+++ b/ql/src/test/results/clientpositive/llap/filter_join_breaktask.q.out
@@ -32,108 +32,6 @@ SELECT f.key, g.value
FROM filter_join_breaktask f JOIN filter_join_breaktask m ON( f.key = m.key AND f.ds='2008-04-08' AND m.ds='2008-04-08' AND f.key is not null)
JOIN filter_join_breaktask g ON(g.value = m.value AND g.ds='2008-04-08' AND m.ds='2008-04-08' AND m.value is not null AND m.value !='')
POSTHOOK: type: QUERY
-ABSTRACT SYNTAX TREE:
-
-TOK_QUERY
- TOK_FROM
- TOK_JOIN
- TOK_JOIN
- TOK_TABREF
- TOK_TABNAME
- filter_join_breaktask
- f
- TOK_TABREF
- TOK_TABNAME
- filter_join_breaktask
- m
- AND
- AND
- AND
- =
- .
- TOK_TABLE_OR_COL
- f
- key
- .
- TOK_TABLE_OR_COL
- m
- key
- =
- .
- TOK_TABLE_OR_COL
- f
- ds
- '2008-04-08'
- =
- .
- TOK_TABLE_OR_COL
- m
- ds
- '2008-04-08'
- TOK_FUNCTION
- TOK_ISNOTNULL
- .
- TOK_TABLE_OR_COL
- f
- key
- TOK_TABREF
- TOK_TABNAME
- filter_join_breaktask
- g
- AND
- AND
- AND
- AND
- =
- .
- TOK_TABLE_OR_COL
- g
- value
- .
- TOK_TABLE_OR_COL
- m
- value
- =
- .
- TOK_TABLE_OR_COL
- g
- ds
- '2008-04-08'
- =
- .
- TOK_TABLE_OR_COL
- m
- ds
- '2008-04-08'
- TOK_FUNCTION
- TOK_ISNOTNULL
- .
- TOK_TABLE_OR_COL
- m
- value
- !=
- .
- TOK_TABLE_OR_COL
- m
- value
- ''
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- .
- TOK_TABLE_OR_COL
- f
- key
- TOK_SELEXPR
- .
- TOK_TABLE_OR_COL
- g
- value
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
@@ -141,6 +39,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE)
@@ -150,20 +49,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: f
- Statistics: Num rows: 25 Data size: 211 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
predicate: key is not null (type: boolean)
- Statistics: Num rows: 13 Data size: 109 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 13 Data size: 109 Basic stats: COMPLETE Column stats: NONE
- tag: 0
- auto parallelism: true
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ tag: 0
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -175,7 +80,7 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns key,value
columns.comments
@@ -218,21 +123,27 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: m
- Statistics: Num rows: 25 Data size: 211 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
- predicate: ((key is not null and value is not null) and (value <> '')) (type: boolean)
- Statistics: Num rows: 7 Data size: 59 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: key (type: int)
- sort order: +
- Map-reduce partition columns: key (type: int)
- Statistics: Num rows: 7 Data size: 59 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- value expressions: value (type: string)
- auto parallelism: true
+ predicate: ((value <> '') and key is not null) (type: boolean)
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int), value (type: string)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: int)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: int)
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ value expressions: _col1 (type: string)
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -244,7 +155,7 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns key,value
columns.comments
@@ -287,20 +198,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: g
- Statistics: Num rows: 25 Data size: 211 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Filter Operator
isSamplingPred: false
- predicate: ((value <> '') and value is not null) (type: boolean)
- Statistics: Num rows: 13 Data size: 109 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: value (type: string)
- sort order: +
- Map-reduce partition columns: value (type: string)
- Statistics: Num rows: 13 Data size: 109 Basic stats: COMPLETE Column stats: NONE
- tag: 1
- auto parallelism: true
+ predicate: (value <> '') (type: boolean)
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: value (type: string)
+ outputColumnNames: _col0
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ null sort order: a
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
+ Statistics: Num rows: 25 Data size: 411 Basic stats: COMPLETE Column stats: NONE
+ tag: 1
+ auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -312,7 +229,7 @@ STAGE PLANS:
partition values:
ds 2008-04-08
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
bucket_count -1
columns key,value
columns.comments
@@ -359,16 +276,17 @@ STAGE PLANS:
condition map:
Inner Join 0 to 1
keys:
- 0 key (type: int)
- 1 key (type: int)
- outputColumnNames: _col0, _col7
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0, _col3
Position of Big Table: 0
- Statistics: Num rows: 14 Data size: 119 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 27 Data size: 452 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
- key expressions: _col7 (type: string)
+ key expressions: _col3 (type: string)
+ null sort order: a
sort order: +
- Map-reduce partition columns: _col7 (type: string)
- Statistics: Num rows: 14 Data size: 119 Basic stats: COMPLETE Column stats: NONE
+ Map-reduce partition columns: _col3 (type: string)
+ Statistics: Num rows: 27 Data size: 452 Basic stats: COMPLETE Column stats: NONE
tag: 0
value expressions: _col0 (type: int)
auto parallelism: true
@@ -380,30 +298,31 @@ STAGE PLANS:
condition map:
Inner Join 0 to 1
keys:
- 0 _col7 (type: string)
- 1 value (type: string)
- outputColumnNames: _col0, _col13
+ 0 _col3 (type: string)
+ 1 _col0 (type: string)
+ outputColumnNames: _col0, _col5
Position of Big Table: 0
- Statistics: Num rows: 15 Data size: 130 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 497 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: int), _col13 (type: string)
+ expressions: _col0 (type: int), _col5 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 15 Data size: 130 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 497 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 15 Data size: 130 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 29 Data size: 497 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
properties:
columns _col0,_col1
columns.types int:string
escape.delim \
hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
serialization.format 1
serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/groupby1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/groupby1.q.out b/ql/src/test/results/clientpositive/llap/groupby1.q.out
index 1323a73..ba0a09a 100644
--- a/ql/src/test/results/clientpositive/llap/groupby1.q.out
+++ b/ql/src/test/results/clientpositive/llap/groupby1.q.out
@@ -25,6 +25,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -34,18 +35,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), substr(value, 5) (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: rand() (type: double)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -54,29 +56,29 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: partial1
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 47500 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0)
keys: KEY._col0 (type: string)
mode: final
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: UDFToInteger(_col0) (type: int), _col1 (type: double)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 2460 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 2460 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/groupby2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/groupby2.q.out b/ql/src/test/results/clientpositive/llap/groupby2.q.out
index 94d8c81..8286589 100644
--- a/ql/src/test/results/clientpositive/llap/groupby2.q.out
+++ b/ql/src/test/results/clientpositive/llap/groupby2.q.out
@@ -23,6 +23,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -32,17 +33,18 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: substr(key, 1, 1) (type: string), substr(value, 5) (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -51,29 +53,29 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: partial1
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 100000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 100000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint), _col2 (type: double)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0), sum(VALUE._col1)
keys: KEY._col0 (type: string)
mode: final
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 41000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), UDFToInteger(_col1) (type: int), concat(_col0, _col2) (type: string)
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 76260 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 76260 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/groupby3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/groupby3.q.out b/ql/src/test/results/clientpositive/llap/groupby3.q.out
index 75f0c36..be98fd6 100644
--- a/ql/src/test/results/clientpositive/llap/groupby3.q.out
+++ b/ql/src/test/results/clientpositive/llap/groupby3.q.out
@@ -41,6 +41,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -50,17 +51,18 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: substr(value, 5) (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 45500 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -68,26 +70,26 @@ STAGE PLANS:
aggregations: sum(KEY._col0:0._col0), avg(KEY._col0:0._col0), avg(DISTINCT KEY._col0:0._col0), max(KEY._col0:0._col0), min(KEY._col0:0._col0), std(KEY._col0:0._col0), stddev_samp(KEY._col0:0._col0), variance(KEY._col0:0._col0), var_samp(KEY._col0:0._col0)
mode: partial1
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 1 Data size: 176 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 1208 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 176 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 1208 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col0 (type: double), _col1 (type: struct<count:bigint,sum:double,input:string>), _col2 (type: struct<count:bigint,sum:double,input:string>), _col3 (type: string), _col4 (type: string), _col5 (type: struct<count:bigint,sum:double,variance:double>), _col6 (type: struct<count:bigint,sum:double,variance:double>), _col7 (type: struct<count:bigint,sum:double,variance:double>), _col8 (type: struct<count:bigint,sum:double,variance:double>)
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: sum(VALUE._col0), avg(VALUE._col1), avg(VALUE._col2), max(VALUE._col3), min(VALUE._col4), std(VALUE._col5), stddev_samp(VALUE._col6), variance(VALUE._col7), var_samp(VALUE._col8)
mode: final
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 1 Data size: 224 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 424 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: double), _col1 (type: double), _col2 (type: double), UDFToDouble(_col3) (type: double), UDFToDouble(_col4) (type: double), _col5 (type: double), _col6 (type: double), _col7 (type: double), _col8 (type: double)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
- Statistics: Num rows: 1 Data size: 224 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 224 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 1 Data size: 72 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/having.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/having.q.out b/ql/src/test/results/clientpositive/llap/having.q.out
index 026fdbf..9438910 100644
--- a/ql/src/test/results/clientpositive/llap/having.q.out
+++ b/ql/src/test/results/clientpositive/llap/having.q.out
@@ -11,6 +11,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -19,50 +20,51 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: bigint)
outputColumnNames: _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 1640 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col1 > 3) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 544 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col1 (type: bigint)
outputColumnNames: _col0
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 544 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 544 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -100,6 +102,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -108,38 +111,39 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (UDFToDouble(key) <> 302.0) (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -475,6 +479,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -483,46 +488,47 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col1 > 'val_255') (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 18428 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 5916 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 5916 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -749,6 +755,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -757,45 +764,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (UDFToDouble(key) > 300.0) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 69 Data size: 18699 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 69 Data size: 18699 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 69 Data size: 18699 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col1 > 'val_255') (type: boolean)
- Statistics: Num rows: 27 Data size: 286 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 23 Data size: 6233 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string)
outputColumnNames: _col0
- Statistics: Num rows: 27 Data size: 286 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 23 Data size: 2001 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 27 Data size: 286 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 23 Data size: 2001 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -948,6 +956,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -956,42 +965,43 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: max(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 55555 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col1 > 'val_255') (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 18428 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 18428 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1218,6 +1228,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
#### A masked pattern was here ####
@@ -1226,42 +1237,43 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: key, value
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count(value)
keys: key (type: string)
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: bigint)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: count(VALUE._col0)
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 205 Data size: 19475 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (_col1 >= 4) (type: boolean)
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 6460 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 83 Data size: 881 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 68 Data size: 6460 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
[05/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_smb_empty.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_smb_empty.q.out b/ql/src/test/results/clientpositive/llap/tez_smb_empty.q.out
new file mode 100644
index 0000000..8333037
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/tez_smb_empty.q.out
@@ -0,0 +1,918 @@
+PREHOOK: query: -- SORT_QUERY_RESULTS
+
+CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@srcbucket_mapjoin
+POSTHOOK: query: -- SORT_QUERY_RESULTS
+
+CREATE TABLE srcbucket_mapjoin(key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@srcbucket_mapjoin
+PREHOOK: query: CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@tab_part
+POSTHOOK: query: CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@tab_part
+PREHOOK: query: CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@srcbucket_mapjoin_part
+POSTHOOK: query: CREATE TABLE srcbucket_mapjoin_part (key int, value string) partitioned by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@srcbucket_mapjoin_part
+PREHOOK: query: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08')
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@srcbucket_mapjoin
+POSTHOOK: query: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08')
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@srcbucket_mapjoin
+POSTHOOK: Output: default@srcbucket_mapjoin@ds=2008-04-08
+PREHOOK: query: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08')
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@srcbucket_mapjoin@ds=2008-04-08
+POSTHOOK: query: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin partition(ds='2008-04-08')
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@srcbucket_mapjoin@ds=2008-04-08
+PREHOOK: query: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08')
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@srcbucket_mapjoin_part
+POSTHOOK: query: load data local inpath '../../data/files/srcbucket20.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08')
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@srcbucket_mapjoin_part
+POSTHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08
+PREHOOK: query: load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08')
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08
+POSTHOOK: query: load data local inpath '../../data/files/srcbucket21.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08')
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08
+PREHOOK: query: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08')
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08
+POSTHOOK: query: load data local inpath '../../data/files/srcbucket22.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08')
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08
+PREHOOK: query: load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08')
+PREHOOK: type: LOAD
+#### A masked pattern was here ####
+PREHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08
+POSTHOOK: query: load data local inpath '../../data/files/srcbucket23.txt' INTO TABLE srcbucket_mapjoin_part partition(ds='2008-04-08')
+POSTHOOK: type: LOAD
+#### A masked pattern was here ####
+POSTHOOK: Output: default@srcbucket_mapjoin_part@ds=2008-04-08
+PREHOOK: query: insert overwrite table tab_part partition (ds='2008-04-08')
+select key,value from srcbucket_mapjoin_part
+PREHOOK: type: QUERY
+PREHOOK: Input: default@srcbucket_mapjoin_part
+PREHOOK: Input: default@srcbucket_mapjoin_part@ds=2008-04-08
+PREHOOK: Output: default@tab_part@ds=2008-04-08
+POSTHOOK: query: insert overwrite table tab_part partition (ds='2008-04-08')
+select key,value from srcbucket_mapjoin_part
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@srcbucket_mapjoin_part
+POSTHOOK: Input: default@srcbucket_mapjoin_part@ds=2008-04-08
+POSTHOOK: Output: default@tab_part@ds=2008-04-08
+POSTHOOK: Lineage: tab_part PARTITION(ds=2008-04-08).key SIMPLE [(srcbucket_mapjoin_part)srcbucket_mapjoin_part.FieldSchema(name:key, type:int, comment:null), ]
+POSTHOOK: Lineage: tab_part PARTITION(ds=2008-04-08).value SIMPLE [(srcbucket_mapjoin_part)srcbucket_mapjoin_part.FieldSchema(name:value, type:string, comment:null), ]
+PREHOOK: query: CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@tab
+POSTHOOK: query: CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@tab
+PREHOOK: query: insert overwrite table tab partition (ds='2008-04-08')
+select key,value from srcbucket_mapjoin
+PREHOOK: type: QUERY
+PREHOOK: Input: default@srcbucket_mapjoin
+PREHOOK: Input: default@srcbucket_mapjoin@ds=2008-04-08
+PREHOOK: Output: default@tab@ds=2008-04-08
+POSTHOOK: query: insert overwrite table tab partition (ds='2008-04-08')
+select key,value from srcbucket_mapjoin
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@srcbucket_mapjoin
+POSTHOOK: Input: default@srcbucket_mapjoin@ds=2008-04-08
+POSTHOOK: Output: default@tab@ds=2008-04-08
+POSTHOOK: Lineage: tab PARTITION(ds=2008-04-08).key SIMPLE [(srcbucket_mapjoin)srcbucket_mapjoin.FieldSchema(name:key, type:int, comment:null), ]
+POSTHOOK: Lineage: tab PARTITION(ds=2008-04-08).value SIMPLE [(srcbucket_mapjoin)srcbucket_mapjoin.FieldSchema(name:value, type:string, comment:null), ]
+PREHOOK: query: CREATE TABLE empty(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@empty
+POSTHOOK: query: CREATE TABLE empty(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@empty
+PREHOOK: query: explain
+select count(*) from tab s1 join empty s3 on s1.key=s3.key
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select count(*) from tab s1 join empty s3 on s1.key=s3.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: s3
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: s1
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from tab s1 join empty s3 on s1.key=s3.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@empty
+PREHOOK: Input: default@tab
+PREHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from tab s1 join empty s3 on s1.key=s3.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@empty
+POSTHOOK: Input: default@tab
+POSTHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+0
+PREHOOK: query: explain
+select * from tab s1 left outer join empty s3 on s1.key=s3.key
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select * from tab s1 left outer join empty s3 on s1.key=s3.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: s3
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 184 Basic stats: COMPLETE Column stats: PARTIAL
+ Map Operator Tree:
+ TableScan
+ alias: s1
+ Statistics: Num rows: 242 Data size: 47094 Basic stats: COMPLETE Column stats: PARTIAL
+ Select Operator
+ expressions: key (type: int), value (type: string), ds (type: string)
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 242 Data size: 44528 Basic stats: COMPLETE Column stats: PARTIAL
+ Merge Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 242 Data size: 139392 Basic stats: COMPLETE Column stats: PARTIAL
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 242 Data size: 139392 Basic stats: COMPLETE Column stats: PARTIAL
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Execution mode: llap
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select * from tab s1 left outer join empty s3 on s1.key=s3.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@empty
+PREHOOK: Input: default@tab
+PREHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+POSTHOOK: query: select * from tab s1 left outer join empty s3 on s1.key=s3.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@empty
+POSTHOOK: Input: default@tab
+POSTHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+0 val_0 2008-04-08 NULL NULL NULL
+0 val_0 2008-04-08 NULL NULL NULL
+0 val_0 2008-04-08 NULL NULL NULL
+103 val_103 2008-04-08 NULL NULL NULL
+103 val_103 2008-04-08 NULL NULL NULL
+105 val_105 2008-04-08 NULL NULL NULL
+11 val_11 2008-04-08 NULL NULL NULL
+114 val_114 2008-04-08 NULL NULL NULL
+116 val_116 2008-04-08 NULL NULL NULL
+118 val_118 2008-04-08 NULL NULL NULL
+118 val_118 2008-04-08 NULL NULL NULL
+125 val_125 2008-04-08 NULL NULL NULL
+125 val_125 2008-04-08 NULL NULL NULL
+129 val_129 2008-04-08 NULL NULL NULL
+129 val_129 2008-04-08 NULL NULL NULL
+134 val_134 2008-04-08 NULL NULL NULL
+134 val_134 2008-04-08 NULL NULL NULL
+136 val_136 2008-04-08 NULL NULL NULL
+138 val_138 2008-04-08 NULL NULL NULL
+138 val_138 2008-04-08 NULL NULL NULL
+138 val_138 2008-04-08 NULL NULL NULL
+138 val_138 2008-04-08 NULL NULL NULL
+143 val_143 2008-04-08 NULL NULL NULL
+145 val_145 2008-04-08 NULL NULL NULL
+149 val_149 2008-04-08 NULL NULL NULL
+149 val_149 2008-04-08 NULL NULL NULL
+15 val_15 2008-04-08 NULL NULL NULL
+15 val_15 2008-04-08 NULL NULL NULL
+150 val_150 2008-04-08 NULL NULL NULL
+152 val_152 2008-04-08 NULL NULL NULL
+152 val_152 2008-04-08 NULL NULL NULL
+156 val_156 2008-04-08 NULL NULL NULL
+158 val_158 2008-04-08 NULL NULL NULL
+163 val_163 2008-04-08 NULL NULL NULL
+165 val_165 2008-04-08 NULL NULL NULL
+165 val_165 2008-04-08 NULL NULL NULL
+167 val_167 2008-04-08 NULL NULL NULL
+167 val_167 2008-04-08 NULL NULL NULL
+167 val_167 2008-04-08 NULL NULL NULL
+169 val_169 2008-04-08 NULL NULL NULL
+169 val_169 2008-04-08 NULL NULL NULL
+169 val_169 2008-04-08 NULL NULL NULL
+169 val_169 2008-04-08 NULL NULL NULL
+17 val_17 2008-04-08 NULL NULL NULL
+170 val_170 2008-04-08 NULL NULL NULL
+172 val_172 2008-04-08 NULL NULL NULL
+172 val_172 2008-04-08 NULL NULL NULL
+174 val_174 2008-04-08 NULL NULL NULL
+174 val_174 2008-04-08 NULL NULL NULL
+176 val_176 2008-04-08 NULL NULL NULL
+176 val_176 2008-04-08 NULL NULL NULL
+178 val_178 2008-04-08 NULL NULL NULL
+181 val_181 2008-04-08 NULL NULL NULL
+183 val_183 2008-04-08 NULL NULL NULL
+187 val_187 2008-04-08 NULL NULL NULL
+187 val_187 2008-04-08 NULL NULL NULL
+187 val_187 2008-04-08 NULL NULL NULL
+189 val_189 2008-04-08 NULL NULL NULL
+19 val_19 2008-04-08 NULL NULL NULL
+190 val_190 2008-04-08 NULL NULL NULL
+192 val_192 2008-04-08 NULL NULL NULL
+194 val_194 2008-04-08 NULL NULL NULL
+196 val_196 2008-04-08 NULL NULL NULL
+2 val_2 2008-04-08 NULL NULL NULL
+20 val_20 2008-04-08 NULL NULL NULL
+200 val_200 2008-04-08 NULL NULL NULL
+200 val_200 2008-04-08 NULL NULL NULL
+202 val_202 2008-04-08 NULL NULL NULL
+208 val_208 2008-04-08 NULL NULL NULL
+208 val_208 2008-04-08 NULL NULL NULL
+208 val_208 2008-04-08 NULL NULL NULL
+213 val_213 2008-04-08 NULL NULL NULL
+213 val_213 2008-04-08 NULL NULL NULL
+217 val_217 2008-04-08 NULL NULL NULL
+217 val_217 2008-04-08 NULL NULL NULL
+219 val_219 2008-04-08 NULL NULL NULL
+219 val_219 2008-04-08 NULL NULL NULL
+222 val_222 2008-04-08 NULL NULL NULL
+224 val_224 2008-04-08 NULL NULL NULL
+224 val_224 2008-04-08 NULL NULL NULL
+226 val_226 2008-04-08 NULL NULL NULL
+228 val_228 2008-04-08 NULL NULL NULL
+233 val_233 2008-04-08 NULL NULL NULL
+233 val_233 2008-04-08 NULL NULL NULL
+235 val_235 2008-04-08 NULL NULL NULL
+237 val_237 2008-04-08 NULL NULL NULL
+237 val_237 2008-04-08 NULL NULL NULL
+239 val_239 2008-04-08 NULL NULL NULL
+239 val_239 2008-04-08 NULL NULL NULL
+24 val_24 2008-04-08 NULL NULL NULL
+24 val_24 2008-04-08 NULL NULL NULL
+242 val_242 2008-04-08 NULL NULL NULL
+242 val_242 2008-04-08 NULL NULL NULL
+244 val_244 2008-04-08 NULL NULL NULL
+248 val_248 2008-04-08 NULL NULL NULL
+255 val_255 2008-04-08 NULL NULL NULL
+255 val_255 2008-04-08 NULL NULL NULL
+257 val_257 2008-04-08 NULL NULL NULL
+26 val_26 2008-04-08 NULL NULL NULL
+26 val_26 2008-04-08 NULL NULL NULL
+260 val_260 2008-04-08 NULL NULL NULL
+262 val_262 2008-04-08 NULL NULL NULL
+266 val_266 2008-04-08 NULL NULL NULL
+273 val_273 2008-04-08 NULL NULL NULL
+273 val_273 2008-04-08 NULL NULL NULL
+273 val_273 2008-04-08 NULL NULL NULL
+275 val_275 2008-04-08 NULL NULL NULL
+277 val_277 2008-04-08 NULL NULL NULL
+277 val_277 2008-04-08 NULL NULL NULL
+277 val_277 2008-04-08 NULL NULL NULL
+277 val_277 2008-04-08 NULL NULL NULL
+28 val_28 2008-04-08 NULL NULL NULL
+280 val_280 2008-04-08 NULL NULL NULL
+280 val_280 2008-04-08 NULL NULL NULL
+282 val_282 2008-04-08 NULL NULL NULL
+282 val_282 2008-04-08 NULL NULL NULL
+284 val_284 2008-04-08 NULL NULL NULL
+286 val_286 2008-04-08 NULL NULL NULL
+288 val_288 2008-04-08 NULL NULL NULL
+288 val_288 2008-04-08 NULL NULL NULL
+291 val_291 2008-04-08 NULL NULL NULL
+305 val_305 2008-04-08 NULL NULL NULL
+307 val_307 2008-04-08 NULL NULL NULL
+307 val_307 2008-04-08 NULL NULL NULL
+309 val_309 2008-04-08 NULL NULL NULL
+309 val_309 2008-04-08 NULL NULL NULL
+310 val_310 2008-04-08 NULL NULL NULL
+316 val_316 2008-04-08 NULL NULL NULL
+316 val_316 2008-04-08 NULL NULL NULL
+316 val_316 2008-04-08 NULL NULL NULL
+318 val_318 2008-04-08 NULL NULL NULL
+318 val_318 2008-04-08 NULL NULL NULL
+318 val_318 2008-04-08 NULL NULL NULL
+321 val_321 2008-04-08 NULL NULL NULL
+321 val_321 2008-04-08 NULL NULL NULL
+323 val_323 2008-04-08 NULL NULL NULL
+325 val_325 2008-04-08 NULL NULL NULL
+325 val_325 2008-04-08 NULL NULL NULL
+327 val_327 2008-04-08 NULL NULL NULL
+327 val_327 2008-04-08 NULL NULL NULL
+327 val_327 2008-04-08 NULL NULL NULL
+33 val_33 2008-04-08 NULL NULL NULL
+332 val_332 2008-04-08 NULL NULL NULL
+336 val_336 2008-04-08 NULL NULL NULL
+338 val_338 2008-04-08 NULL NULL NULL
+341 val_341 2008-04-08 NULL NULL NULL
+345 val_345 2008-04-08 NULL NULL NULL
+35 val_35 2008-04-08 NULL NULL NULL
+35 val_35 2008-04-08 NULL NULL NULL
+35 val_35 2008-04-08 NULL NULL NULL
+356 val_356 2008-04-08 NULL NULL NULL
+365 val_365 2008-04-08 NULL NULL NULL
+367 val_367 2008-04-08 NULL NULL NULL
+367 val_367 2008-04-08 NULL NULL NULL
+369 val_369 2008-04-08 NULL NULL NULL
+369 val_369 2008-04-08 NULL NULL NULL
+369 val_369 2008-04-08 NULL NULL NULL
+37 val_37 2008-04-08 NULL NULL NULL
+37 val_37 2008-04-08 NULL NULL NULL
+374 val_374 2008-04-08 NULL NULL NULL
+378 val_378 2008-04-08 NULL NULL NULL
+389 val_389 2008-04-08 NULL NULL NULL
+392 val_392 2008-04-08 NULL NULL NULL
+394 val_394 2008-04-08 NULL NULL NULL
+396 val_396 2008-04-08 NULL NULL NULL
+396 val_396 2008-04-08 NULL NULL NULL
+396 val_396 2008-04-08 NULL NULL NULL
+4 val_4 2008-04-08 NULL NULL NULL
+400 val_400 2008-04-08 NULL NULL NULL
+402 val_402 2008-04-08 NULL NULL NULL
+404 val_404 2008-04-08 NULL NULL NULL
+404 val_404 2008-04-08 NULL NULL NULL
+406 val_406 2008-04-08 NULL NULL NULL
+406 val_406 2008-04-08 NULL NULL NULL
+406 val_406 2008-04-08 NULL NULL NULL
+406 val_406 2008-04-08 NULL NULL NULL
+411 val_411 2008-04-08 NULL NULL NULL
+413 val_413 2008-04-08 NULL NULL NULL
+413 val_413 2008-04-08 NULL NULL NULL
+417 val_417 2008-04-08 NULL NULL NULL
+417 val_417 2008-04-08 NULL NULL NULL
+417 val_417 2008-04-08 NULL NULL NULL
+419 val_419 2008-04-08 NULL NULL NULL
+42 val_42 2008-04-08 NULL NULL NULL
+42 val_42 2008-04-08 NULL NULL NULL
+424 val_424 2008-04-08 NULL NULL NULL
+424 val_424 2008-04-08 NULL NULL NULL
+431 val_431 2008-04-08 NULL NULL NULL
+431 val_431 2008-04-08 NULL NULL NULL
+431 val_431 2008-04-08 NULL NULL NULL
+435 val_435 2008-04-08 NULL NULL NULL
+437 val_437 2008-04-08 NULL NULL NULL
+439 val_439 2008-04-08 NULL NULL NULL
+439 val_439 2008-04-08 NULL NULL NULL
+44 val_44 2008-04-08 NULL NULL NULL
+444 val_444 2008-04-08 NULL NULL NULL
+446 val_446 2008-04-08 NULL NULL NULL
+448 val_448 2008-04-08 NULL NULL NULL
+453 val_453 2008-04-08 NULL NULL NULL
+455 val_455 2008-04-08 NULL NULL NULL
+457 val_457 2008-04-08 NULL NULL NULL
+459 val_459 2008-04-08 NULL NULL NULL
+459 val_459 2008-04-08 NULL NULL NULL
+460 val_460 2008-04-08 NULL NULL NULL
+462 val_462 2008-04-08 NULL NULL NULL
+462 val_462 2008-04-08 NULL NULL NULL
+466 val_466 2008-04-08 NULL NULL NULL
+466 val_466 2008-04-08 NULL NULL NULL
+466 val_466 2008-04-08 NULL NULL NULL
+468 val_468 2008-04-08 NULL NULL NULL
+468 val_468 2008-04-08 NULL NULL NULL
+468 val_468 2008-04-08 NULL NULL NULL
+468 val_468 2008-04-08 NULL NULL NULL
+475 val_475 2008-04-08 NULL NULL NULL
+477 val_477 2008-04-08 NULL NULL NULL
+479 val_479 2008-04-08 NULL NULL NULL
+480 val_480 2008-04-08 NULL NULL NULL
+480 val_480 2008-04-08 NULL NULL NULL
+480 val_480 2008-04-08 NULL NULL NULL
+482 val_482 2008-04-08 NULL NULL NULL
+484 val_484 2008-04-08 NULL NULL NULL
+491 val_491 2008-04-08 NULL NULL NULL
+493 val_493 2008-04-08 NULL NULL NULL
+495 val_495 2008-04-08 NULL NULL NULL
+497 val_497 2008-04-08 NULL NULL NULL
+51 val_51 2008-04-08 NULL NULL NULL
+51 val_51 2008-04-08 NULL NULL NULL
+53 val_53 2008-04-08 NULL NULL NULL
+57 val_57 2008-04-08 NULL NULL NULL
+64 val_64 2008-04-08 NULL NULL NULL
+66 val_66 2008-04-08 NULL NULL NULL
+77 val_77 2008-04-08 NULL NULL NULL
+8 val_8 2008-04-08 NULL NULL NULL
+80 val_80 2008-04-08 NULL NULL NULL
+82 val_82 2008-04-08 NULL NULL NULL
+84 val_84 2008-04-08 NULL NULL NULL
+84 val_84 2008-04-08 NULL NULL NULL
+86 val_86 2008-04-08 NULL NULL NULL
+95 val_95 2008-04-08 NULL NULL NULL
+95 val_95 2008-04-08 NULL NULL NULL
+97 val_97 2008-04-08 NULL NULL NULL
+97 val_97 2008-04-08 NULL NULL NULL
+PREHOOK: query: explain
+select count(*) from tab s1 left outer join tab s2 on s1.key=s2.key join empty s3 on s1.key = s3.key
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select count(*) from tab s1 left outer join tab s2 on s1.key=s2.key join empty s3 on s1.key = s3.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: s2
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: s3
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: s1
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ Inner Join 0 to 2
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ 2 _col0 (type: int)
+ Statistics: Num rows: 532 Data size: 9904 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from tab s1 left outer join tab s2 on s1.key=s2.key join empty s3 on s1.key = s3.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@empty
+PREHOOK: Input: default@tab
+PREHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from tab s1 left outer join tab s2 on s1.key=s2.key join empty s3 on s1.key = s3.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@empty
+POSTHOOK: Input: default@tab
+POSTHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+0
+PREHOOK: query: explain
+select count(*) from tab s1 left outer join empty s2 on s1.key=s2.key join tab s3 on s1.key = s3.key
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select count(*) from tab s1 left outer join empty s2 on s1.key=s2.key join tab s3 on s1.key = s3.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: s2
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: s3
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: s1
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Left Outer Join0 to 1
+ Inner Join 0 to 2
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ 2 _col0 (type: int)
+ Statistics: Num rows: 532 Data size: 9904 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from tab s1 left outer join empty s2 on s1.key=s2.key join tab s3 on s1.key = s3.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@empty
+PREHOOK: Input: default@tab
+PREHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from tab s1 left outer join empty s2 on s1.key=s2.key join tab s3 on s1.key = s3.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@empty
+POSTHOOK: Input: default@tab
+POSTHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+480
+PREHOOK: query: explain
+select count(*) from empty s1 join empty s3 on s1.key=s3.key
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select count(*) from empty s1 join empty s3 on s1.key=s3.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: s3
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: s1
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from empty s1 join empty s3 on s1.key=s3.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@empty
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from empty s1 join empty s3 on s1.key=s3.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@empty
+#### A masked pattern was here ####
+0
+PREHOOK: query: explain
+select count(*) from empty s1 join tab s3 on s1.key=s3.key
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select count(*) from empty s1 join tab s3 on s1.key=s3.key
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Tez
+#### A masked pattern was here ####
+ Edges:
+ Reducer 2 <- Map 1 (SIMPLE_EDGE)
+#### A masked pattern was here ####
+ Vertices:
+ Map 1
+ Map Operator Tree:
+ TableScan
+ alias: s3
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
+ Map Operator Tree:
+ TableScan
+ alias: s1
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Filter Operator
+ predicate: key is not null (type: boolean)
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: key (type: int)
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Merge Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 _col0 (type: int)
+ 1 _col0 (type: int)
+ Statistics: Num rows: 266 Data size: 4952 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: count()
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: bigint)
+ Execution mode: llap
+ Reducer 2
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: count(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
+PREHOOK: query: select count(*) from empty s1 join tab s3 on s1.key=s3.key
+PREHOOK: type: QUERY
+PREHOOK: Input: default@empty
+PREHOOK: Input: default@tab
+PREHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+POSTHOOK: query: select count(*) from empty s1 join tab s3 on s1.key=s3.key
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@empty
+POSTHOOK: Input: default@tab
+POSTHOOK: Input: default@tab@ds=2008-04-08
+#### A masked pattern was here ####
+0
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/tez_smb_main.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/tez_smb_main.q.out b/ql/src/test/results/clientpositive/llap/tez_smb_main.q.out
index 813dba6..2d6b1a9 100644
--- a/ql/src/test/results/clientpositive/llap/tez_smb_main.q.out
+++ b/ql/src/test/results/clientpositive/llap/tez_smb_main.q.out
@@ -20,19 +20,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -40,19 +40,19 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 25 Data size: 191 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 25 Data size: 4375 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: string)
Execution mode: llap
LLAP IO: no inputs
@@ -66,10 +66,10 @@ STAGE PLANS:
0 _col0 (type: string)
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 21180 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 60 Data size: 21180 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -263,33 +263,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -298,7 +298,7 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
@@ -374,33 +374,33 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: a
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 242 Data size: 4502 Basic stats: COMPLETE Column stats: NONE
Execution mode: llap
LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: b
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: key is not null (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: key (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -409,7 +409,7 @@ STAGE PLANS:
1 _col0 (type: int)
input vertices:
0 Map 1
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 550 Data size: 10243 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count()
mode: hash
[50/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
Posted by pr...@apache.org.
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/auto_join29.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/auto_join29.q.out b/ql/src/test/results/clientpositive/llap/auto_join29.q.out
index 13b3ec4..94134c5 100644
--- a/ql/src/test/results/clientpositive/llap/auto_join29.q.out
+++ b/ql/src/test/results/clientpositive/llap/auto_join29.q.out
@@ -15,6 +15,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -24,41 +25,44 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key > 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -75,28 +79,28 @@ STAGE PLANS:
1 key (type: string)
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -626,6 +630,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -635,44 +640,47 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key > 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > 10) and (key < 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -689,28 +697,28 @@ STAGE PLANS:
1 key (type: string)
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 322 Data size: 171948 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 322 Data size: 171948 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 322 Data size: 171948 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 322 Data size: 171948 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 322 Data size: 171948 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1240,6 +1248,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1249,44 +1258,47 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1303,28 +1315,28 @@ STAGE PLANS:
1 key (type: string)
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 974 Data size: 520116 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 974 Data size: 520116 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 974 Data size: 520116 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 974 Data size: 520116 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 974 Data size: 520116 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -1866,6 +1878,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -1875,41 +1888,44 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -1926,28 +1942,28 @@ STAGE PLANS:
1 key (type: string)
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 987 Data size: 527058 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2489,6 +2505,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -2498,47 +2515,50 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > 10) and (key < 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key < 10) and (key > 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > 10) and (key < 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -2551,28 +2571,28 @@ STAGE PLANS:
1 key (type: string)
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -2602,6 +2622,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -2611,44 +2632,47 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > 10) and (key < 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key < 10) and (key > 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 5
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -2665,28 +2689,28 @@ STAGE PLANS:
1 key (type: string)
2 key (type: string)
outputColumnNames: _col0, _col1, _col5, _col6, _col10, _col11
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -3216,6 +3240,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE), Map 4 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -3225,7 +3250,7 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Left Outer Join0 to 1
@@ -3242,60 +3267,62 @@ STAGE PLANS:
input vertices:
1 Map 3
2 Map 4
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key < 10) and (key > 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > 10) and (key < 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 320 Data size: 170880 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -3325,6 +3352,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 2 <- Map 1 (BROADCAST_EDGE), Map 4 (BROADCAST_EDGE)
Reducer 3 <- Map 2 (SIMPLE_EDGE)
@@ -3334,25 +3362,26 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 2
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Right Outer Join0 to 1
@@ -3369,45 +3398,46 @@ STAGE PLANS:
input vertices:
0 Map 1
2 Map 4
- Statistics: Num rows: 365 Data size: 3878 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 960 Data size: 512640 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 365 Data size: 3878 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 960 Data size: 512640 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 365 Data size: 3878 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 960 Data size: 512640 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (key < 10) (type: boolean)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 166 Data size: 29548 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 365 Data size: 3878 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 960 Data size: 512640 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 365 Data size: 3878 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 960 Data size: 512640 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
@@ -3459,6 +3489,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Map 1 <- Map 3 (BROADCAST_EDGE), Map 4 (BROADCAST_EDGE)
Reducer 2 <- Map 1 (SIMPLE_EDGE)
@@ -3468,10 +3499,10 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > 10) and (key < 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -3484,60 +3515,62 @@ STAGE PLANS:
input vertices:
1 Map 3
2 Map 4
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
- HybridGraceHashJoin: true
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: _col0 (type: string), _col1 (type: string), _col5 (type: string), _col6 (type: string), _col10 (type: string), _col11 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: string)
sort order: ++++++
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
+ LLAP IO: no inputs
Map 3
Map Operator Tree:
TableScan
alias: src2
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key < 10) and (key > 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Map 4
Map Operator Tree:
TableScan
alias: src3
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: ((key > 10) and (key < 10)) (type: boolean)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: key (type: string)
sort order: +
Map-reduce partition columns: key (type: string)
- Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 55 Data size: 9790 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: value (type: string)
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string), KEY.reducesinkkey2 (type: string), KEY.reducesinkkey3 (type: string), KEY.reducesinkkey4 (type: string), KEY.reducesinkkey5 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 121 Data size: 1284 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 314 Data size: 167676 Basic stats: COMPLETE Column stats: COMPLETE
table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0