You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by pr...@apache.org on 2016/08/23 18:46:05 UTC
[34/51] [partial] hive git commit: HIVE-14502: Convert MiniTez tests
to MiniLlap tests (Prasanth Jayachandran reviewed by Siddharth Seth)
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/ctas.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/ctas.q.out b/ql/src/test/results/clientpositive/llap/ctas.q.out
index 93fc978..f48d431 100644
--- a/ql/src/test/results/clientpositive/llap/ctas.q.out
+++ b/ql/src/test/results/clientpositive/llap/ctas.q.out
@@ -34,6 +34,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -43,43 +44,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -117,6 +121,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_CTAS1
+POSTHOOK: Lineage: nzhang_ctas1.k SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas1.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: select * from nzhang_CTAS1
PREHOOK: type: QUERY
PREHOOK: Input: default@nzhang_ctas1
@@ -153,7 +159,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 96
@@ -184,6 +190,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -193,43 +200,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -267,6 +277,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas2
+POSTHOOK: Lineage: nzhang_ctas2.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas2.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: select * from nzhang_ctas2
PREHOOK: type: QUERY
PREHOOK: Input: default@nzhang_ctas2
@@ -303,7 +315,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 96
@@ -334,6 +346,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -343,43 +356,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: (key / 2) (type: double), concat(value, '_con') (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 96000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 96000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: double), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 96000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: double), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: double), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1920 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat
output format: org.apache.hadoop.hive.ql.io.RCFileOutputFormat
@@ -417,6 +433,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas3
+POSTHOOK: Lineage: nzhang_ctas3.conb EXPRESSION [(src)src.FieldSchema(name:value, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas3.half_key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]
PREHOOK: query: select * from nzhang_ctas3
PREHOOK: type: QUERY
PREHOOK: Input: default@nzhang_ctas3
@@ -453,7 +471,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 120
@@ -517,7 +535,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 120
@@ -548,6 +566,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -557,43 +576,46 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: llap
+ LLAP IO: no inputs
Reducer 2
Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -632,6 +654,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas4
+POSTHOOK: Lineage: nzhang_ctas4.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas4.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: select * from nzhang_ctas4
PREHOOK: type: QUERY
PREHOOK: Input: default@nzhang_ctas4
@@ -668,7 +692,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 10
rawDataSize 96
@@ -690,47 +714,6 @@ PREHOOK: query: explain extended create table nzhang_ctas5 row format delimited
PREHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: query: explain extended create table nzhang_ctas5 row format delimited fields terminated by ',' lines terminated by '\012' stored as textfile as select key, value from src sort by key, value limit 10
POSTHOOK: type: CREATETABLE_AS_SELECT
-ABSTRACT SYNTAX TREE:
-
-TOK_CREATETABLE
- TOK_TABNAME
- nzhang_ctas5
- TOK_LIKETABLE
- TOK_TABLEROWFORMAT
- TOK_SERDEPROPS
- TOK_TABLEROWFORMATFIELD
- ','
- TOK_TABLEROWFORMATLINES
- '\012'
- TOK_FILEFORMAT_GENERIC
- textfile
- TOK_QUERY
- TOK_FROM
- TOK_TABREF
- TOK_TABNAME
- src
- TOK_INSERT
- TOK_DESTINATION
- TOK_DIR
- TOK_TMP_FILE
- TOK_SELECT
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- key
- TOK_SELEXPR
- TOK_TABLE_OR_COL
- value
- TOK_SORTBY
- TOK_TABSORTCOLNAMEASC
- TOK_TABLE_OR_COL
- key
- TOK_TABSORTCOLNAMEASC
- TOK_TABLE_OR_COL
- value
- TOK_LIMIT
- 10
-
-
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-2 depends on stages: Stage-1
@@ -741,6 +724,7 @@ STAGE DEPENDENCIES:
STAGE PLANS:
Stage: Stage-1
Tez
+#### A masked pattern was here ####
Edges:
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
@@ -750,19 +734,23 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
GatherStats: false
Select Operator
expressions: key (type: string), value (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
+ null sort order: aa
sort order: ++
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
+ TopN: 10
+ TopN Hash Memory Usage: 0.1
auto parallelism: true
Execution mode: llap
+ LLAP IO: no inputs
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -772,7 +760,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -792,7 +780,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE true
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
columns key,value
columns.comments 'default','default'
@@ -819,33 +807,36 @@ STAGE PLANS:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 500 Data size: 89000 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Reduce Output Operator
key expressions: _col0 (type: string), _col1 (type: string)
+ null sort order: aa
sort order: ++
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
tag: -1
+ TopN: 10
+ TopN Hash Memory Usage: 0.1
auto parallelism: false
Reducer 3
- Execution mode: uber
+ Execution mode: llap
Needs Tagging: false
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
Limit
Number of rows: 10
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
File Output Operator
compressed: false
GlobalTableId: 1
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 10 Data size: 1780 Basic stats: COMPLETE Column stats: COMPLETE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
@@ -900,6 +891,8 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas5
+POSTHOOK: Lineage: nzhang_ctas5.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ]
+POSTHOOK: Lineage: nzhang_ctas5.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ]
PREHOOK: query: create table nzhang_ctas6 (key string, `to` string)
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
@@ -928,3 +921,5 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@nzhang_ctas6
POSTHOOK: Output: database:default
POSTHOOK: Output: default@nzhang_ctas7
+POSTHOOK: Lineage: nzhang_ctas7.key SIMPLE [(nzhang_ctas6)nzhang_ctas6.FieldSchema(name:key, type:string, comment:null), ]
+POSTHOOK: Lineage: nzhang_ctas7.to SIMPLE [(nzhang_ctas6)nzhang_ctas6.FieldSchema(name:to, type:string, comment:null), ]
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_3.q.out b/ql/src/test/results/clientpositive/llap/cte_3.q.out
index 82f6414..27f0ab1 100644
--- a/ql/src/test/results/clientpositive/llap/cte_3.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_3.q.out
@@ -8,59 +8,30 @@ with q1 as ( select key from src where key = '5')
select *
from q1
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-0 depends on stages: Stage-1
- Stage-3 is a root stage
-
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- TableScan
- alias: q1
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- ListSink
+Plan optimized by CBO.
+
+Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=85)
+ Output:["_col0"]
+ Filter Operator [FIL_4] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
+Stage-3
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_6]
+ Output:["_col0"]
+ TableScan [TS_5]
+ Output:["key"]
PREHOOK: query: with q1 as ( select key from src where key = '5')
select *
@@ -93,59 +64,30 @@ explain
with q1 as ( select key from src where key = '5')
select * from (select key from q1) a
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-0 depends on stages: Stage-1
- Stage-3 is a root stage
-
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- TableScan
- alias: q1
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- ListSink
+Plan optimized by CBO.
+
+Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=85)
+ Output:["_col0"]
+ Filter Operator [FIL_4] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
+Stage-3
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_6]
+ Output:["_col0"]
+ TableScan [TS_5]
+ Output:["key"]
PREHOOK: query: with q1 as ( select key from src where key = '5')
select * from (select key from q1) a
@@ -178,98 +120,45 @@ with q1 as ( select key from q2 where key = '5'),
q2 as ( select key from src where key = '5')
select * from (select key from q1) a
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-5 depends on stages: Stage-4
- Stage-3 depends on stages: Stage-4
- Stage-0 depends on stages: Stage-1
- Stage-6 is a root stage
-
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q2
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: q2
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-5
- Dependency Collection
-
- Stage: Stage-3
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-6
- Fetch Operator
- limit: -1
- Processor Tree:
- TableScan
- alias: q1
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- ListSink
+Plan optimized by CBO.
+
+Stage-5
+ Dependency Collection{}
+ Stage-4
+ Map 2 llap
+ File Output Operator [FS_8]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_7] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_9] (rows=1 width=0)
+ predicate:(key = '5')
+ TableScan [TS_5] (rows=1 width=0)
+ default@q2,q2,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q2"}
+ Select Operator [SEL_2] (rows=2 width=85)
+ Output:["_col0"]
+ Filter Operator [FIL_4] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
+Stage-3
+ Move Operator
+ Please refer to the previous Stage-4
+Stage-6
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_11]
+ Output:["_col0"]
+ TableScan [TS_10]
+ Output:["key"]
PREHOOK: query: with q1 as ( select key from q2 where key = '5'),
q2 as ( select key from src where key = '5')
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_5.q.out b/ql/src/test/results/clientpositive/llap/cte_5.q.out
index b69f73d..48e7aef 100644
--- a/ql/src/test/results/clientpositive/llap/cte_5.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_5.q.out
@@ -76,75 +76,35 @@ select a.colnum
from mydb.q1 as a join q1 as b
on a.colnum=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (UDFToDouble(colnum) = 5.0) (type: boolean)
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 3
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 250 Data size: 3656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: 5 (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 250 Data size: 3656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 3656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
- Stage: Stage-0
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=2 width=89)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_13] (rows=2 width=89)
+ Conds:(Inner)
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ Select Operator [SEL_2] (rows=1 width=3)
+ Filter Operator [FIL_11] (rows=1 width=3)
+ predicate:(UDFToDouble(colnum) = 5.0)
+ TableScan [TS_0] (rows=1 width=3)
+ mydb@q1,a,Tbl:COMPLETE,Col:NONE,Output:["colnum"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ Select Operator [SEL_5] (rows=2 width=85)
+ Filter Operator [FIL_12] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_3] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
Warning: Shuffle Join MERGEJOIN[13][tables = [$hdt$_0, $hdt$_1]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: with q1 as (select * from src where key= '5')
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out
index 7e20214..081a138 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_1.q.out
@@ -11,73 +11,33 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 3
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
- Stage: Stage-0
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=4 width=85)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_13] (rows=4 width=8)
+ Conds:(Inner)
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ Select Operator [SEL_2] (rows=2 width=85)
+ Filter Operator [FIL_11] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ Select Operator [SEL_5] (rows=2 width=85)
+ Filter Operator [FIL_12] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_3] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out
index 7e20214..081a138 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_2.q.out
@@ -11,73 +11,33 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-0 depends on stages: Stage-1
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 3
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- sort order:
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 2
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0
- 1
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 62500 Data size: 1390500 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
- Stage: Stage-0
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=4 width=85)
+ Output:["_col0"]
+ Merge Join Operator [MERGEJOIN_13] (rows=4 width=8)
+ Conds:(Inner)
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ Select Operator [SEL_2] (rows=2 width=85)
+ Filter Operator [FIL_11] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ Select Operator [SEL_5] (rows=2 width=85)
+ Filter Operator [FIL_12] (rows=2 width=87)
+ predicate:(key = '5')
+ TableScan [TS_3] (rows=500 width=87)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key"]
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out
index 415492a..cb88f0c 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_3.q.out
@@ -10,117 +10,50 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-4
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
+Vertex dependency in root stage
+Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: string)
- 1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
-
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-3
+ Fetch Operator
+ limit:-1
+ Stage-4
+ Reducer 3 llap
+ File Output Operator [FS_15]
+ Merge Join Operator [MERGEJOIN_20] (rows=1 width=0)
+ Conds:RS_11._col0=RS_12._col0(Inner),Output:["_col0"]
+ <-Map 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Select Operator [SEL_7] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_5] (rows=1 width=0)
+ default@q1,a,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col0
+ Select Operator [SEL_10] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_8] (rows=1 width=0)
+ default@q1,b,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=176)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_4] (rows=2 width=178)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out
index 5c2fa59..526d80f 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_4.q.out
@@ -51,119 +51,52 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-4
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: string)
- 1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
+Vertex dependency in root stage
+Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-3
+ Fetch Operator
+ limit:-1
+ Stage-4
+ Reducer 3 llap
+ File Output Operator [FS_15]
+ Merge Join Operator [MERGEJOIN_20] (rows=1 width=0)
+ Conds:RS_11._col0=RS_12._col0(Inner),Output:["_col0"]
+ <-Map 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Select Operator [SEL_7] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_5] (rows=1 width=0)
+ default@q1,a,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col0
+ Select Operator [SEL_10] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_8] (rows=1 width=0)
+ default@q1,b,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=176)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_4] (rows=2 width=178)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
PREHOOK: query: with q1 as (select * from src where key= '5')
select a.key
@@ -268,119 +201,52 @@ select a.key
from q1 a join q1 b
on a.key=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-4
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 _col0 (type: string)
- 1 _col0 (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
+Vertex dependency in root stage
+Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-3
+ Fetch Operator
+ limit:-1
+ Stage-4
+ Reducer 3 llap
+ File Output Operator [FS_15]
+ Merge Join Operator [MERGEJOIN_20] (rows=1 width=0)
+ Conds:RS_11._col0=RS_12._col0(Inner),Output:["_col0"]
+ <-Map 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Select Operator [SEL_7] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_5] (rows=1 width=0)
+ default@q1,a,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:_col0
+ Select Operator [SEL_10] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_8] (rows=1 width=0)
+ default@q1,b,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=176)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_4] (rows=2 width=178)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
PREHOOK: query: with q1 as (select * from src where key= '5')
select a.key
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out b/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out
index ee5bac1..95c65e4 100644
--- a/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out
+++ b/ql/src/test/results/clientpositive/llap/cte_mat_5.q.out
@@ -75,120 +75,52 @@ select a.colnum
from mydb.q1 as a join q1 as b
on a.colnum=b.key
POSTHOOK: type: QUERY
-STAGE DEPENDENCIES:
- Stage-1 is a root stage
- Stage-2 depends on stages: Stage-1
- Stage-4 depends on stages: Stage-2, Stage-0
- Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-4
+Plan optimized by CBO.
-STAGE PLANS:
- Stage: Stage-1
- Tez
-#### A masked pattern was here ####
- Vertices:
- Map 1
- Map Operator Tree:
- TableScan
- alias: src
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: (key = '5') (type: boolean)
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: '5' (type: string), value (type: string)
- outputColumnNames: _col0, _col1
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.TextInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- name: default.q1
- Execution mode: llap
- LLAP IO: no inputs
-
- Stage: Stage-2
- Dependency Collection
-
- Stage: Stage-4
- Tez
-#### A masked pattern was here ####
- Edges:
- Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
-#### A masked pattern was here ####
- Vertices:
- Map 2
- Map Operator Tree:
- TableScan
- alias: a
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Filter Operator
- predicate: colnum is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: colnum (type: int)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(_col0) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(_col0) (type: double)
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col0 (type: int)
- Execution mode: llap
- LLAP IO: no inputs
- Map 4
- Map Operator Tree:
- TableScan
- alias: b
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Filter Operator
- predicate: key is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: key (type: string)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Reduce Output Operator
- key expressions: UDFToDouble(_col0) (type: double)
- sort order: +
- Map-reduce partition columns: UDFToDouble(_col0) (type: double)
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Execution mode: llap
- LLAP IO: no inputs
- Reducer 3
- Execution mode: llap
- Reduce Operator Tree:
- Merge Join Operator
- condition map:
- Inner Join 0 to 1
- keys:
- 0 UDFToDouble(_col0) (type: double)
- 1 UDFToDouble(_col0) (type: double)
- outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
-
- Stage: Stage-0
- Move Operator
- files:
- hdfs directory: true
-#### A masked pattern was here ####
+Vertex dependency in root stage
+Reducer 3 <- Map 2 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
- Stage: Stage-3
- Fetch Operator
- limit: -1
- Processor Tree:
- ListSink
+Stage-3
+ Fetch Operator
+ limit:-1
+ Stage-4
+ Reducer 3 llap
+ File Output Operator [FS_15]
+ Merge Join Operator [MERGEJOIN_20] (rows=1 width=3)
+ Conds:RS_11.UDFToDouble(_col0)=RS_12.UDFToDouble(_col0)(Inner),Output:["_col0"]
+ <-Map 2 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_11]
+ PartitionCols:UDFToDouble(_col0)
+ Select Operator [SEL_7] (rows=1 width=3)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=1 width=3)
+ predicate:colnum is not null
+ TableScan [TS_5] (rows=1 width=3)
+ mydb@q1,a,Tbl:COMPLETE,Col:NONE,Output:["colnum"]
+ <-Map 4 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_12]
+ PartitionCols:UDFToDouble(_col0)
+ Select Operator [SEL_10] (rows=1 width=0)
+ Output:["_col0"]
+ Filter Operator [FIL_19] (rows=1 width=0)
+ predicate:key is not null
+ TableScan [TS_8] (rows=1 width=0)
+ default@q1,b,Tbl:PARTIAL,Col:NONE,Output:["key"]
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1 llap
+ File Output Operator [FS_3]
+ table:{"name:":"default.q1"}
+ Select Operator [SEL_2] (rows=2 width=176)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_4] (rows=2 width=178)
+ predicate:(key = '5')
+ TableScan [TS_0] (rows=500 width=178)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
PREHOOK: query: with q1 as (select * from src where key= '5')
select a.colnum
http://git-wip-us.apache.org/repos/asf/hive/blob/5f01dc42/ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out b/ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out
new file mode 100644
index 0000000..9b52994
--- /dev/null
+++ b/ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out
@@ -0,0 +1,140 @@
+PREHOOK: query: create table testdeci2(
+id int,
+amount decimal(10,3),
+sales_tax decimal(10,3),
+item string)
+#### A masked pattern was here ####
+PREHOOK: type: CREATETABLE
+#### A masked pattern was here ####
+PREHOOK: Output: database:default
+PREHOOK: Output: default@testdeci2
+POSTHOOK: query: create table testdeci2(
+id int,
+amount decimal(10,3),
+sales_tax decimal(10,3),
+item string)
+#### A masked pattern was here ####
+POSTHOOK: type: CREATETABLE
+#### A masked pattern was here ####
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@testdeci2
+PREHOOK: query: insert into table testdeci2 values(1,12.123,12345.123,'desk1'),(2,123.123,1234.123,'desk2')
+PREHOOK: type: QUERY
+PREHOOK: Input: default@values__tmp__table__1
+PREHOOK: Output: default@testdeci2
+POSTHOOK: query: insert into table testdeci2 values(1,12.123,12345.123,'desk1'),(2,123.123,1234.123,'desk2')
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@values__tmp__table__1
+POSTHOOK: Output: default@testdeci2
+POSTHOOK: Lineage: testdeci2.amount EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col2, type:string, comment:), ]
+POSTHOOK: Lineage: testdeci2.id EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col1, type:string, comment:), ]
+POSTHOOK: Lineage: testdeci2.item SIMPLE [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col4, type:string, comment:), ]
+POSTHOOK: Lineage: testdeci2.sales_tax EXPRESSION [(values__tmp__table__1)values__tmp__table__1.FieldSchema(name:tmp_values_col3, type:string, comment:), ]
+PREHOOK: query: describe formatted testdeci2
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@testdeci2
+POSTHOOK: query: describe formatted testdeci2
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@testdeci2
+# col_name data_type comment
+
+id int
+amount decimal(10,3)
+sales_tax decimal(10,3)
+item string
+
+# Detailed Table Information
+Database: default
+#### A masked pattern was here ####
+Retention: 0
+#### A masked pattern was here ####
+Table Type: MANAGED_TABLE
+Table Parameters:
+ numFiles 1
+ totalSize 578
+#### A masked pattern was here ####
+
+# Storage Information
+SerDe Library: org.apache.hadoop.hive.ql.io.orc.OrcSerde
+InputFormat: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
+OutputFormat: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
+Compressed: No
+Num Buckets: -1
+Bucket Columns: []
+Sort Columns: []
+Storage Desc Params:
+ serialization.format 1
+#### A masked pattern was here ####
+PREHOOK: query: describe formatted testdeci2 amount
+PREHOOK: type: DESCTABLE
+PREHOOK: Input: default@testdeci2
+POSTHOOK: query: describe formatted testdeci2 amount
+POSTHOOK: type: DESCTABLE
+POSTHOOK: Input: default@testdeci2
+# col_name data_type min max num_nulls distinct_count avg_col_len max_col_len num_trues num_falses comment
+
+amount decimal(10,3) from deserializer
+PREHOOK: query: analyze table testdeci2 compute statistics for columns
+PREHOOK: type: QUERY
+PREHOOK: Input: default@testdeci2
+#### A masked pattern was here ####
+POSTHOOK: query: analyze table testdeci2 compute statistics for columns
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@testdeci2
+#### A masked pattern was here ####
+PREHOOK: query: analyze table testdeci2 compute statistics for columns
+PREHOOK: type: QUERY
+PREHOOK: Input: default@testdeci2
+#### A masked pattern was here ####
+POSTHOOK: query: analyze table testdeci2 compute statistics for columns
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@testdeci2
+#### A masked pattern was here ####
+PREHOOK: query: explain
+select s.id,
+coalesce(d.amount,0) as sales,
+coalesce(d.sales_tax,0) as tax
+from testdeci2 s join testdeci2 d
+on s.item=d.item and d.id=2
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select s.id,
+coalesce(d.amount,0) as sales,
+coalesce(d.sales_tax,0) as tax
+from testdeci2 s join testdeci2 d
+on s.item=d.item and d.id=2
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 llap
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=5 width=228)
+ Output:["_col0","_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_15] (rows=5 width=228)
+ Conds:RS_6._col1=RS_7._col3(Inner),Output:["_col0","_col3","_col4"]
+ <-Map 1 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_6]
+ PartitionCols:_col1
+ Select Operator [SEL_2] (rows=5 width=88)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_13] (rows=5 width=88)
+ predicate:item is not null
+ TableScan [TS_0] (rows=5 width=88)
+ default@testdeci2,s,Tbl:COMPLETE,Col:COMPLETE,Output:["id","item"]
+ <-Map 3 [SIMPLE_EDGE] llap
+ SHUFFLE [RS_7]
+ PartitionCols:_col3
+ Select Operator [SEL_5] (rows=1 width=312)
+ Output:["_col1","_col2","_col3"]
+ Filter Operator [FIL_14] (rows=1 width=312)
+ predicate:((id = 2) and item is not null)
+ TableScan [TS_3] (rows=1 width=312)
+ default@testdeci2,d,Tbl:COMPLETE,Col:COMPLETE,Output:["id","amount","sales_tax","item"]
+