You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by jc...@apache.org on 2018/05/05 15:41:10 UTC
hive git commit: HIVE-19396 : HiveOperation is incorrectly set for
analyze statement (Ashutosh Chauhan via Zoltan Haindrich) (addendum)
Repository: hive
Updated Branches:
refs/heads/master 0fc67416e -> 0d517bf22
HIVE-19396 : HiveOperation is incorrectly set for analyze statement (Ashutosh Chauhan via Zoltan Haindrich) (addendum)
Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/0d517bf2
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/0d517bf2
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/0d517bf2
Branch: refs/heads/master
Commit: 0d517bf224480b271159e01d4b1bef8ae40738a1
Parents: 0fc6741
Author: Jesus Camacho Rodriguez <jc...@apache.org>
Authored: Sat May 5 08:40:54 2018 -0700
Committer: Jesus Camacho Rodriguez <jc...@apache.org>
Committed: Sat May 5 08:40:54 2018 -0700
----------------------------------------------------------------------
.../materialized_view_create_rewrite_5.q.out | 173 ++-----
...ized_view_create_rewrite_rebuild_dummy.q.out | 485 ++++---------------
...alized_view_create_rewrite_time_window.q.out | 77 ++-
3 files changed, 195 insertions(+), 540 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hive/blob/0d517bf2/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out b/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out
index c292baa..6a29852 100644
--- a/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out
+++ b/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_5.q.out
@@ -178,12 +178,12 @@ STAGE PLANS:
Select Operator
expressions: a (type: int)
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: may be used (ACID table)
Reducer 2
@@ -277,17 +277,17 @@ STAGE PLANS:
alias: cmv_basetable_2
Statistics: Num rows: 3 Data size: 348 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: ((c > 10) and a is not null) (type: boolean)
+ predicate: ((ROW__ID.writeid > 1) and (c > 10) and a is not null) (type: boolean)
Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: a (type: int), c (type: decimal(10,2))
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: decimal(10,2))
Execution mode: llap
LLAP IO: may be used (ACID table)
@@ -398,12 +398,9 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MATERIALIZED_VIEW
Table Parameters:
- COLUMN_STATS_ACCURATE {\"COLUMN_STATS\":{\"a\":\"true\",\"c\":\"true\"}}
bucketing_version 2
- numFiles 1
- numRows 5
- rawDataSize 0
- totalSize 761
+ numFiles 2
+ totalSize 1053
transactional true
transactional_properties default
#### A masked pattern was here ####
@@ -436,79 +433,21 @@ FROM cmv_basetable join cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
WHERE cmv_basetable_2.c > 10.10
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: cmv_basetable
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: a is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Map 3
- Map Operator Tree:
- TableScan
- alias: cmv_basetable_2
- Statistics: Num rows: 3 Data size: 348 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: ((c > 10.1) and a is not null) (type: boolean)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: default.cmv_mat_view
+ Filter Operator
+ predicate: (c > 10.1) (type: boolean)
+ Select Operator
+ expressions: a (type: int)
+ outputColumnNames: _col0
+ ListSink
PREHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -516,6 +455,7 @@ WHERE cmv_basetable_2.c > 10.10
PREHOOK: type: QUERY
PREHOOK: Input: default@cmv_basetable
PREHOOK: Input: default@cmv_basetable_2
+PREHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
POSTHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -523,6 +463,7 @@ WHERE cmv_basetable_2.c > 10.10
POSTHOOK: type: QUERY
POSTHOOK: Input: default@cmv_basetable
POSTHOOK: Input: default@cmv_basetable_2
+POSTHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
1
3
@@ -699,79 +640,21 @@ FROM cmv_basetable join cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
WHERE cmv_basetable_2.c > 10.10
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: cmv_basetable
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: a is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Map 3
- Map Operator Tree:
- TableScan
- alias: cmv_basetable_2
- Statistics: Num rows: 60 Data size: 6960 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: ((c > 10.1) and a is not null) (type: boolean)
- Statistics: Num rows: 20 Data size: 2320 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 20 Data size: 2320 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 20 Data size: 2320 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 33 Data size: 132 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 33 Data size: 132 Basic stats: COMPLETE Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: default.cmv_mat_view
+ Filter Operator
+ predicate: (c > 10.1) (type: boolean)
+ Select Operator
+ expressions: a (type: int)
+ outputColumnNames: _col0
+ ListSink
PREHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -779,6 +662,7 @@ WHERE cmv_basetable_2.c > 10.10
PREHOOK: type: QUERY
PREHOOK: Input: default@cmv_basetable
PREHOOK: Input: default@cmv_basetable_2
+PREHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
POSTHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -786,6 +670,7 @@ WHERE cmv_basetable_2.c > 10.10
POSTHOOK: type: QUERY
POSTHOOK: Input: default@cmv_basetable
POSTHOOK: Input: default@cmv_basetable_2
+POSTHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
2
2
http://git-wip-us.apache.org/repos/asf/hive/blob/0d517bf2/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_rebuild_dummy.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_rebuild_dummy.q.out b/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_rebuild_dummy.q.out
index 8445aed..cca7d9f 100644
--- a/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_rebuild_dummy.q.out
+++ b/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_rebuild_dummy.q.out
@@ -244,103 +244,21 @@ WHERE cmv_basetable_2.c > 10.10
GROUP BY cmv_basetable.a, cmv_basetable_2.c
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: cmv_basetable
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: a is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Map 4
- Map Operator Tree:
- TableScan
- alias: cmv_basetable_2
- Statistics: Num rows: 2 Data size: 232 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: ((c > 10.1) and a is not null) (type: boolean)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int), c (type: decimal(10,2))
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col1 (type: decimal(10,2))
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- outputColumnNames: _col0, _col2
- Statistics: Num rows: 3 Data size: 348 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: _col0 (type: int), _col2 (type: decimal(10,2))
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: decimal(10,2))
- sort order: ++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: decimal(10,2))
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- keys: KEY._col0 (type: int), KEY._col1 (type: decimal(10,2))
- mode: mergepartial
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: default.cmv_mat_view
+ Filter Operator
+ predicate: (c > 10.1) (type: boolean)
+ Select Operator
+ expressions: a (type: int)
+ outputColumnNames: _col0
+ ListSink
PREHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -349,6 +267,7 @@ GROUP BY cmv_basetable.a, cmv_basetable_2.c
PREHOOK: type: QUERY
PREHOOK: Input: default@cmv_basetable
PREHOOK: Input: default@cmv_basetable_2
+PREHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
POSTHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -357,6 +276,7 @@ GROUP BY cmv_basetable.a, cmv_basetable_2.c
POSTHOOK: type: QUERY
POSTHOOK: Input: default@cmv_basetable
POSTHOOK: Input: default@cmv_basetable_2
+POSTHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
1
3
@@ -532,9 +452,11 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
- Reducer 4 <- Reducer 3 (CUSTOM_SIMPLE_EDGE)
+ Map 8 <- Union 4 (CONTAINS)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Union 4 (CONTAINS)
+ Reducer 5 <- Union 4 (SIMPLE_EDGE)
+ Reducer 6 <- Reducer 5 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -556,26 +478,47 @@ STAGE PLANS:
Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: may be used (ACID table)
- Map 5
+ Map 7
Map Operator Tree:
TableScan
alias: cmv_basetable_2
Statistics: Num rows: 3 Data size: 348 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: ((c > 10) and a is not null) (type: boolean)
+ predicate: ((ROW__ID.writeid > 1) and (c > 10) and a is not null) (type: boolean)
Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: a (type: int), c (type: decimal(10,2))
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: decimal(10,2))
Execution mode: llap
LLAP IO: may be used (ACID table)
+ Map 8
+ Map Operator Tree:
+ TableScan
+ alias: default.cmv_mat_view
+ Statistics: Num rows: 2 Data size: 232 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: a (type: int), c (type: decimal(10,2))
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 232 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: _col0 (type: int), _col1 (type: decimal(10,2))
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: decimal(10,2))
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: decimal(10,2))
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
+ Execution mode: llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -605,9 +548,27 @@ STAGE PLANS:
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: _col0 (type: int), _col1 (type: decimal(10,2))
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: decimal(10,2))
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: decimal(10,2))
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
+ Reducer 5
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: int), KEY._col1 (type: decimal(10,2))
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
@@ -616,31 +577,33 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: int), _col1 (type: decimal(10,2))
outputColumnNames: a, c
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: compute_stats(a, 'hll'), compute_stats(c, 'hll')
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 1056 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 1056 Basic stats: COMPLETE Column stats: PARTIAL
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 1056 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 1056 Basic stats: COMPLETE Column stats: PARTIAL
value expressions: _col0 (type: struct<columntype:string,min:bigint,max:bigint,countnulls:bigint,bitvector:binary>), _col1 (type: struct<columntype:string,min:decimal(10,2),max:decimal(10,2),countnulls:bigint,bitvector:binary>)
- Reducer 4
+ Reducer 6
Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: compute_stats(VALUE._col0), compute_stats(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 1088 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 1088 Basic stats: COMPLETE Column stats: PARTIAL
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 1088 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 1088 Basic stats: COMPLETE Column stats: PARTIAL
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Union 4
+ Vertex: Union 4
Stage: Stage-2
Dependency Collection
@@ -670,14 +633,16 @@ PREHOOK: query: ALTER MATERIALIZED VIEW cmv_mat_view REBUILD
PREHOOK: type: QUERY
PREHOOK: Input: default@cmv_basetable
PREHOOK: Input: default@cmv_basetable_2
+PREHOOK: Input: default@cmv_mat_view
PREHOOK: Output: default@cmv_mat_view
POSTHOOK: query: ALTER MATERIALIZED VIEW cmv_mat_view REBUILD
POSTHOOK: type: QUERY
POSTHOOK: Input: default@cmv_basetable
POSTHOOK: Input: default@cmv_basetable_2
+POSTHOOK: Input: default@cmv_mat_view
POSTHOOK: Output: default@cmv_mat_view
-POSTHOOK: Lineage: cmv_mat_view.a SIMPLE [(cmv_basetable)cmv_basetable.FieldSchema(name:a, type:int, comment:null), ]
-POSTHOOK: Lineage: cmv_mat_view.c SIMPLE [(cmv_basetable_2)cmv_basetable_2.FieldSchema(name:c, type:decimal(10,2), comment:null), ]
+POSTHOOK: Lineage: cmv_mat_view.a EXPRESSION [(cmv_basetable)cmv_basetable.FieldSchema(name:a, type:int, comment:null), (cmv_mat_view)default.cmv_mat_view.FieldSchema(name:a, type:int, comment:null), ]
+POSTHOOK: Lineage: cmv_mat_view.c EXPRESSION [(cmv_basetable_2)cmv_basetable_2.FieldSchema(name:c, type:decimal(10,2), comment:null), (cmv_mat_view)default.cmv_mat_view.FieldSchema(name:c, type:decimal(10,2), comment:null), ]
PREHOOK: query: EXPLAIN
SELECT cmv_basetable.a
FROM cmv_basetable join cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -691,103 +656,21 @@ WHERE cmv_basetable_2.c > 10.10
GROUP BY cmv_basetable.a, cmv_basetable_2.c
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: cmv_basetable
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: a is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Map 4
- Map Operator Tree:
- TableScan
- alias: cmv_basetable_2
- Statistics: Num rows: 3 Data size: 348 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: ((c > 10.1) and a is not null) (type: boolean)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int), c (type: decimal(10,2))
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col1 (type: decimal(10,2))
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- outputColumnNames: _col0, _col2
- Statistics: Num rows: 2 Data size: 232 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: _col0 (type: int), _col2 (type: decimal(10,2))
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: decimal(10,2))
- sort order: ++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: decimal(10,2))
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- keys: KEY._col0 (type: int), KEY._col1 (type: decimal(10,2))
- mode: mergepartial
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: default.cmv_mat_view
+ Filter Operator
+ predicate: (c > 10.1) (type: boolean)
+ Select Operator
+ expressions: a (type: int)
+ outputColumnNames: _col0
+ ListSink
PREHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -796,6 +679,7 @@ GROUP BY cmv_basetable.a, cmv_basetable_2.c
PREHOOK: type: QUERY
PREHOOK: Input: default@cmv_basetable
PREHOOK: Input: default@cmv_basetable_2
+PREHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
POSTHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -804,6 +688,7 @@ GROUP BY cmv_basetable.a, cmv_basetable_2.c
POSTHOOK: type: QUERY
POSTHOOK: Input: default@cmv_basetable
POSTHOOK: Input: default@cmv_basetable_2
+POSTHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
1
3
@@ -969,103 +854,21 @@ WHERE cmv_basetable_2.c > 10.10
GROUP BY cmv_basetable.a, cmv_basetable_2.c
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: cmv_basetable
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: a is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Map 4
- Map Operator Tree:
- TableScan
- alias: cmv_basetable_2
- Statistics: Num rows: 42 Data size: 4872 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: ((c > 10.1) and a is not null) (type: boolean)
- Statistics: Num rows: 14 Data size: 1624 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int), c (type: decimal(10,2))
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 14 Data size: 1624 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 14 Data size: 1624 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col1 (type: decimal(10,2))
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- outputColumnNames: _col0, _col2
- Statistics: Num rows: 23 Data size: 2668 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: _col0 (type: int), _col2 (type: decimal(10,2))
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: decimal(10,2))
- sort order: ++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: decimal(10,2))
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- keys: KEY._col0 (type: int), KEY._col1 (type: decimal(10,2))
- mode: mergepartial
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: default.cmv_mat_view
+ Filter Operator
+ predicate: (c > 10.1) (type: boolean)
+ Select Operator
+ expressions: a (type: int)
+ outputColumnNames: _col0
+ ListSink
PREHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -1074,6 +877,7 @@ GROUP BY cmv_basetable.a, cmv_basetable_2.c
PREHOOK: type: QUERY
PREHOOK: Input: default@cmv_basetable
PREHOOK: Input: default@cmv_basetable_2
+PREHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
POSTHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -1082,6 +886,7 @@ GROUP BY cmv_basetable.a, cmv_basetable_2.c
POSTHOOK: type: QUERY
POSTHOOK: Input: default@cmv_basetable
POSTHOOK: Input: default@cmv_basetable_2
+POSTHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
1
PREHOOK: query: create table cmv_irrelevant_table (a int, b varchar(256), c decimal(10,2), d int) stored as orc TBLPROPERTIES ('transactional'='true')
@@ -1131,103 +936,21 @@ WHERE cmv_basetable_2.c > 10.10
GROUP BY cmv_basetable.a, cmv_basetable_2.c
POSTHOOK: type: QUERY
STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+ Stage-0 is a root stage
STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: cmv_basetable
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: a is not null (type: boolean)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Map 4
- Map Operator Tree:
- TableScan
- alias: cmv_basetable_2
- Statistics: Num rows: 42 Data size: 4872 Basic stats: COMPLETE Column stats: COMPLETE
- Filter Operator
- predicate: ((c > 10.1) and a is not null) (type: boolean)
- Statistics: Num rows: 14 Data size: 1624 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: a (type: int), c (type: decimal(10,2))
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 14 Data size: 1624 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int)
- sort order: +
- Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 14 Data size: 1624 Basic stats: COMPLETE Column stats: COMPLETE
- value expressions: _col1 (type: decimal(10,2))
- Execution mode: llap
- LLAP IO: may be used (ACID table)
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: int)
- 1 _col0 (type: int)
- outputColumnNames: _col0, _col2
- Statistics: Num rows: 23 Data size: 2668 Basic stats: COMPLETE Column stats: COMPLETE
- Group By Operator
- keys: _col0 (type: int), _col2 (type: decimal(10,2))
- mode: hash
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reduce Output Operator
- key expressions: _col0 (type: int), _col1 (type: decimal(10,2))
- sort order: ++
- Map-reduce partition columns: _col0 (type: int), _col1 (type: decimal(10,2))
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Group By Operator
- keys: KEY._col0 (type: int), KEY._col1 (type: decimal(10,2))
- mode: mergepartial
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
- Select Operator
- expressions: _col0 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
- ListSink
+ TableScan
+ alias: default.cmv_mat_view
+ Filter Operator
+ predicate: (c > 10.1) (type: boolean)
+ Select Operator
+ expressions: a (type: int)
+ outputColumnNames: _col0
+ ListSink
PREHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -1236,6 +959,7 @@ GROUP BY cmv_basetable.a, cmv_basetable_2.c
PREHOOK: type: QUERY
PREHOOK: Input: default@cmv_basetable
PREHOOK: Input: default@cmv_basetable_2
+PREHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
POSTHOOK: query: SELECT cmv_basetable.a
FROM cmv_basetable JOIN cmv_basetable_2 ON (cmv_basetable.a = cmv_basetable_2.a)
@@ -1244,6 +968,7 @@ GROUP BY cmv_basetable.a, cmv_basetable_2.c
POSTHOOK: type: QUERY
POSTHOOK: Input: default@cmv_basetable
POSTHOOK: Input: default@cmv_basetable_2
+POSTHOOK: Input: default@cmv_mat_view
#### A masked pattern was here ####
1
PREHOOK: query: drop materialized view cmv_mat_view
http://git-wip-us.apache.org/repos/asf/hive/blob/0d517bf2/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_time_window.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_time_window.q.out b/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_time_window.q.out
index 097da95..eb32704 100644
--- a/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_time_window.q.out
+++ b/ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_time_window.q.out
@@ -566,9 +566,11 @@ STAGE PLANS:
Tez
#### A masked pattern was here ####
Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
- Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
- Reducer 4 <- Reducer 3 (CUSTOM_SIMPLE_EDGE)
+ Map 8 <- Union 4 (CONTAINS)
+ Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 7 (SIMPLE_EDGE)
+ Reducer 3 <- Reducer 2 (SIMPLE_EDGE), Union 4 (CONTAINS)
+ Reducer 5 <- Union 4 (SIMPLE_EDGE)
+ Reducer 6 <- Reducer 5 (CUSTOM_SIMPLE_EDGE)
#### A masked pattern was here ####
Vertices:
Map 1
@@ -590,26 +592,47 @@ STAGE PLANS:
Statistics: Num rows: 5 Data size: 20 Basic stats: COMPLETE Column stats: COMPLETE
Execution mode: llap
LLAP IO: may be used (ACID table)
- Map 5
+ Map 7
Map Operator Tree:
TableScan
alias: cmv_basetable_2
Statistics: Num rows: 3 Data size: 348 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
- predicate: ((c > 10) and a is not null) (type: boolean)
+ predicate: ((ROW__ID.writeid > 1) and (c > 10) and a is not null) (type: boolean)
Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: a (type: int), c (type: decimal(10,2))
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
Map-reduce partition columns: _col0 (type: int)
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 192 Basic stats: COMPLETE Column stats: COMPLETE
value expressions: _col1 (type: decimal(10,2))
Execution mode: llap
LLAP IO: may be used (ACID table)
+ Map 8
+ Map Operator Tree:
+ TableScan
+ alias: default.cmv_mat_view
+ Statistics: Num rows: 2 Data size: 232 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: a (type: int), c (type: decimal(10,2))
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 2 Data size: 232 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ keys: _col0 (type: int), _col1 (type: decimal(10,2))
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: decimal(10,2))
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: decimal(10,2))
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
+ Execution mode: llap
+ LLAP IO: all inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
@@ -639,9 +662,27 @@ STAGE PLANS:
mode: mergepartial
outputColumnNames: _col0, _col1
Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Group By Operator
+ keys: _col0 (type: int), _col1 (type: decimal(10,2))
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
+ Reduce Output Operator
+ key expressions: _col0 (type: int), _col1 (type: decimal(10,2))
+ sort order: ++
+ Map-reduce partition columns: _col0 (type: int), _col1 (type: decimal(10,2))
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
+ Reducer 5
+ Execution mode: llap
+ Reduce Operator Tree:
+ Group By Operator
+ keys: KEY._col0 (type: int), KEY._col1 (type: decimal(10,2))
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
table:
input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
@@ -650,31 +691,33 @@ STAGE PLANS:
Select Operator
expressions: _col0 (type: int), _col1 (type: decimal(10,2))
outputColumnNames: a, c
- Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 116 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: compute_stats(a, 'hll'), compute_stats(c, 'hll')
mode: hash
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 1056 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 1056 Basic stats: COMPLETE Column stats: PARTIAL
Reduce Output Operator
sort order:
- Statistics: Num rows: 1 Data size: 1056 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 1056 Basic stats: COMPLETE Column stats: PARTIAL
value expressions: _col0 (type: struct<columntype:string,min:bigint,max:bigint,countnulls:bigint,bitvector:binary>), _col1 (type: struct<columntype:string,min:decimal(10,2),max:decimal(10,2),countnulls:bigint,bitvector:binary>)
- Reducer 4
+ Reducer 6
Execution mode: llap
Reduce Operator Tree:
Group By Operator
aggregations: compute_stats(VALUE._col0), compute_stats(VALUE._col1)
mode: mergepartial
outputColumnNames: _col0, _col1
- Statistics: Num rows: 1 Data size: 1088 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 1088 Basic stats: COMPLETE Column stats: PARTIAL
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 1088 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 1088 Basic stats: COMPLETE Column stats: PARTIAL
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ Union 4
+ Vertex: Union 4
Stage: Stage-2
Dependency Collection
@@ -704,14 +747,16 @@ PREHOOK: query: ALTER MATERIALIZED VIEW cmv_mat_view REBUILD
PREHOOK: type: QUERY
PREHOOK: Input: default@cmv_basetable
PREHOOK: Input: default@cmv_basetable_2
+PREHOOK: Input: default@cmv_mat_view
PREHOOK: Output: default@cmv_mat_view
POSTHOOK: query: ALTER MATERIALIZED VIEW cmv_mat_view REBUILD
POSTHOOK: type: QUERY
POSTHOOK: Input: default@cmv_basetable
POSTHOOK: Input: default@cmv_basetable_2
+POSTHOOK: Input: default@cmv_mat_view
POSTHOOK: Output: default@cmv_mat_view
-POSTHOOK: Lineage: cmv_mat_view.a SIMPLE [(cmv_basetable)cmv_basetable.FieldSchema(name:a, type:int, comment:null), ]
-POSTHOOK: Lineage: cmv_mat_view.c SIMPLE [(cmv_basetable_2)cmv_basetable_2.FieldSchema(name:c, type:decimal(10,2), comment:null), ]
+POSTHOOK: Lineage: cmv_mat_view.a EXPRESSION [(cmv_basetable)cmv_basetable.FieldSchema(name:a, type:int, comment:null), (cmv_mat_view)default.cmv_mat_view.FieldSchema(name:a, type:int, comment:null), ]
+POSTHOOK: Lineage: cmv_mat_view.c EXPRESSION [(cmv_basetable_2)cmv_basetable_2.FieldSchema(name:c, type:decimal(10,2), comment:null), (cmv_mat_view)default.cmv_mat_view.FieldSchema(name:c, type:decimal(10,2), comment:null), ]
PREHOOK: query: DESCRIBE FORMATTED cmv_mat_view
PREHOOK: type: DESCTABLE
PREHOOK: Input: default@cmv_mat_view