You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by px...@apache.org on 2016/01/29 20:06:05 UTC
[03/33] hive git commit: HIVE-12709: further improve user level
explain (Pengcheng Xiong, reviewed by Laljo John Pullokkaran)
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/explainuser_3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/explainuser_3.q.out b/ql/src/test/results/clientpositive/tez/explainuser_3.q.out
index e2db163..3f948a5 100644
--- a/ql/src/test/results/clientpositive/tez/explainuser_3.q.out
+++ b/ql/src/test/results/clientpositive/tez/explainuser_3.q.out
@@ -26,29 +26,19 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 2 vectorized
- File Output Operator [FS_8]
- compressed:false
- Statistics:Num rows: 10 Data size: 1704 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [OP_7]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 10 Data size: 1704 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE] vectorized
- Reduce Output Operator [RS_6]
- key expressions:_col0 (type: int), _col1 (type: string)
- sort order:++
- Statistics:Num rows: 10 Data size: 1704 Basic stats: COMPLETE Column stats: NONE
- Select Operator [OP_5]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 10 Data size: 1704 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- ACID table:true
- alias:acid_vectorized
- Statistics:Num rows: 10 Data size: 1704 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 vectorized
+ File Output Operator [FS_8]
+ Select Operator [OP_7] (rows=10 width=170)
+ Output:["_col0","_col1"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_6]
+ Select Operator [OP_5] (rows=10 width=170)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=10 width=170)
+ default@acid_vectorized,acid_vectorized, ACID table,Tbl:COMPLETE,Col:NONE,Output:["a","b"]
PREHOOK: query: explain select key, value
FROM srcpart LATERAL VIEW explode(array(1,2,3)) myTable AS myCol
@@ -59,56 +49,41 @@ POSTHOOK: type: QUERY
Plan not optimized by CBO.
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Map 1
- File Output Operator [FS_7]
- compressed:false
- Statistics:Num rows: 4000 Data size: 42496 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_6]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 4000 Data size: 42496 Basic stats: COMPLETE Column stats: NONE
- Lateral View Join Operator [LVJ_5]
- outputColumnNames:["_col0","_col1","_col7"]
- Statistics:Num rows: 4000 Data size: 42496 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_2]
- outputColumnNames:["key","value"]
- Statistics:Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
- Lateral View Forward [LVF_1]
- Statistics:Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:srcpart
- Statistics:Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
- File Output Operator [FS_7]
- compressed:false
- Statistics:Num rows: 4000 Data size: 42496 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_6]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 4000 Data size: 42496 Basic stats: COMPLETE Column stats: NONE
- Lateral View Join Operator [LVJ_5]
- outputColumnNames:["_col0","_col1","_col7"]
- Statistics:Num rows: 4000 Data size: 42496 Basic stats: COMPLETE Column stats: NONE
- UDTF Operator [UDTF_4]
- function name:explode
- Statistics:Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
- Select Operator [SEL_3]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
- Please refer to the previous Lateral View Forward [LVF_1]
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Map 1
+ File Output Operator [FS_7]
+ Select Operator [SEL_6] (rows=4000 width=10)
+ Output:["_col0","_col1"]
+ Lateral View Join Operator [LVJ_5] (rows=4000 width=10)
+ Output:["_col0","_col1","_col7"]
+ Select Operator [SEL_2] (rows=2000 width=10)
+ Output:["key","value"]
+ Lateral View Forward [LVF_1] (rows=2000 width=10)
+ TableScan [TS_0] (rows=2000 width=10)
+ default@srcpart,srcpart,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
+ File Output Operator [FS_7]
+ Select Operator [SEL_6] (rows=4000 width=10)
+ Output:["_col0","_col1"]
+ Lateral View Join Operator [LVJ_5] (rows=4000 width=10)
+ Output:["_col0","_col1","_col7"]
+ UDTF Operator [UDTF_4] (rows=2000 width=10)
+ function name:explode
+ Select Operator [SEL_3] (rows=2000 width=10)
+ Output:["_col0"]
+ Please refer to the previous Lateral View Forward [LVF_1]
PREHOOK: query: explain show tables
PREHOOK: type: SHOWTABLES
POSTHOOK: query: explain show tables
POSTHOOK: type: SHOWTABLES
Stage-1
- Fetch Operator
- limit:-1
- Stage-0
- Show Table Operator:
- database name:default
+ Fetch Operator
+ limit:-1
+ Stage-0
+ Show Table Operator:
+ database name:default
#### A masked pattern was here ####
PREHOOK: type: CREATEDATABASE
@@ -128,9 +103,9 @@ PREHOOK: type: DESCDATABASE
POSTHOOK: query: explain describe database extended newDB
POSTHOOK: type: DESCDATABASE
Stage-1
- Fetch Operator
- limit:-1
- Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-0
PREHOOK: query: describe database extended newDB
PREHOOK: type: DESCDATABASE
@@ -164,18 +139,16 @@ PREHOOK: type: ALTERTABLE_RENAME
POSTHOOK: query: explain alter table tab rename to newName
POSTHOOK: type: ALTERTABLE_RENAME
Stage-0
- Alter Table Operator:
- new name:newDB.newName
- old name:newDB.tab
- type:rename
+ Alter Table Operator:
+ new name:newDB.newName,old name:newDB.tab,type:rename
PREHOOK: query: explain drop table tab
PREHOOK: type: DROPTABLE
POSTHOOK: query: explain drop table tab
POSTHOOK: type: DROPTABLE
Stage-0
- Drop Table Operator:
- table:tab
+ Drop Table Operator:
+ table:tab
PREHOOK: query: drop table tab
PREHOOK: type: DROPTABLE
@@ -210,12 +183,11 @@ PREHOOK: type: QUERY
POSTHOOK: query: explain analyze table src compute statistics
POSTHOOK: type: QUERY
Stage-2
- Stats-Aggr Operator
- Stage-0
- Map 1
- TableScan [TS_0]
- alias:src
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: COMPLETE
+ Stats-Aggr Operator
+ Stage-0
+ Map 1
+ TableScan [TS_0] (rows=500 width=10)
+ default@src,src,Tbl:COMPLETE,Col:COMPLETE
PREHOOK: query: explain analyze table src compute statistics for columns
PREHOOK: type: QUERY
@@ -225,32 +197,20 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-2
- Column Stats Work{}
- Stage-0
- Reducer 2
- File Output Operator [FS_6]
- compressed:false
- Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [GBY_4]
- | aggregations:["compute_stats(VALUE._col0)","compute_stats(VALUE._col1)"]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_3]
- sort order:
- Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- value expressions:_col0 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:string,numbitvectors:int>), _col1 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:string,numbitvectors:int>)
- Group By Operator [GBY_2]
- aggregations:["compute_stats(key, 16)","compute_stats(value, 16)"]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator [SEL_1]
- outputColumnNames:["key","value"]
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Column Stats Work{}
+ Stage-0
+ Reducer 2
+ File Output Operator [FS_6]
+ Group By Operator [GBY_4] (rows=1 width=0)
+ Output:["_col0","_col1"],aggregations:["compute_stats(VALUE._col0)","compute_stats(VALUE._col1)"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_3]
+ Group By Operator [GBY_2] (rows=1 width=0)
+ Output:["_col0","_col1"],aggregations:["compute_stats(key, 16)","compute_stats(value, 16)"]
+ Select Operator [SEL_1] (rows=500 width=10)
+ Output:["key","value"]
+ TableScan [TS_0] (rows=500 width=10)
+ default@src,src,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
PREHOOK: query: explain
CREATE TEMPORARY MACRO SIGMOID (x DOUBLE) 1.0 / (1.0 + EXP(-x))
@@ -273,14 +233,13 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:1
- Limit [LIM_2]
- Number of rows:1
- Select Operator [SEL_1]
- outputColumnNames:["_col0"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:1
+ Limit [LIM_2]
+ Number of rows:1
+ Select Operator [SEL_1]
+ Output:["_col0"]
+ TableScan [TS_0]
PREHOOK: query: explain DROP TEMPORARY MACRO SIGMOID
PREHOOK: type: DROPMACRO
@@ -301,30 +260,23 @@ POSTHOOK: type: CREATETABLE_AS_SELECT
Plan optimized by CBO.
Stage-3
- Stats-Aggr Operator
- Stage-4
- Create Table Operator:
- columns:["key string","value string"]
- input format:org.apache.hadoop.mapred.TextInputFormat
- name:default.src_autho_test
- output format:org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat
- Stage-2
- Dependency Collection{}
- Stage-1
- Map 1
- File Output Operator [FS_2]
- compressed:false
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","name:":"default.src_autho_test"}
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Stage-0
- Move Operator
- Please refer to the previous Stage-1
+ Stats-Aggr Operator
+ Stage-4
+ Create Table Operator:
+ name:default.src_autho_test
+ Stage-2
+ Dependency Collection{}
+ Stage-1
+ Map 1
+ File Output Operator [FS_2]
+ table:{"name:":"default.src_autho_test"}
+ Select Operator [SEL_1] (rows=500 width=10)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=500 width=10)
+ default@src,src,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
+ Stage-0
+ Move Operator
+ Please refer to the previous Stage-1
PREHOOK: query: create table src_autho_test as select * from src
PREHOOK: type: CREATETABLE_AS_SELECT
@@ -353,18 +305,18 @@ PREHOOK: type: SHOW_GRANT
POSTHOOK: query: explain show grant user hive_test_user on table src_autho_test
POSTHOOK: type: SHOW_GRANT
Stage-1
- Fetch Operator
- limit:-1
- Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-0
PREHOOK: query: explain show grant user hive_test_user on table src_autho_test(key)
PREHOOK: type: SHOW_GRANT
POSTHOOK: query: explain show grant user hive_test_user on table src_autho_test(key)
POSTHOOK: type: SHOW_GRANT
Stage-1
- Fetch Operator
- limit:-1
- Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-0
PREHOOK: query: select key from src_autho_test order by key limit 20
PREHOOK: type: QUERY
@@ -441,9 +393,9 @@ PREHOOK: type: SHOW_ROLE_GRANT
POSTHOOK: query: explain show role grant user hive_test_user
POSTHOOK: type: SHOW_ROLE_GRANT
Stage-1
- Fetch Operator
- limit:-1
- Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-0
PREHOOK: query: explain drop role sRc_roLE
PREHOOK: type: DROPROLE
@@ -468,8 +420,8 @@ PREHOOK: type: DROPVIEW
POSTHOOK: query: explain drop view v
POSTHOOK: type: DROPVIEW
Stage-0
- Drop Table Operator:
- table:v
+ Drop Table Operator:
+ table:v
PREHOOK: query: explain create view v as with cte as (select * from src order by key limit 5)
select * from cte
@@ -480,9 +432,8 @@ POSTHOOK: type: CREATEVIEW
Plan not optimized by CBO.
Stage-0
- Create View Operator:
- name:default.v
- original text:with cte as (select * from src order by key limit 5)
+ Create View Operator:
+ name:default.v,original text:with cte as (select * from src order by key limit 5)
select * from cte
PREHOOK: query: explain with cte as (select * from src order by key limit 5)
@@ -497,32 +448,21 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:5
- Stage-1
- Reducer 2 vectorized
- File Output Operator [FS_8]
- compressed:false
- Statistics:Num rows: 5 Data size: 50 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Limit [LIM_7]
- Number of rows:5
- Statistics:Num rows: 5 Data size: 50 Basic stats: COMPLETE Column stats: NONE
- Select Operator [OP_6]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE]
- Reduce Output Operator [RS_2]
- key expressions:_col0 (type: string)
- sort order:+
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col1 (type: string)
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:5
+ Stage-1
+ Reducer 2 vectorized
+ File Output Operator [FS_8]
+ Limit [LIM_7] (rows=5 width=10)
+ Number of rows:5
+ Select Operator [OP_6] (rows=500 width=10)
+ Output:["_col0","_col1"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_2]
+ Select Operator [SEL_1] (rows=500 width=10)
+ Output:["_col0","_col1"]
+ TableScan [TS_0] (rows=500 width=10)
+ default@src,src,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
PREHOOK: query: create table orc_merge5 (userid bigint, string1 string, subtype double, decimal1 decimal, ts timestamp) stored as orc
PREHOOK: type: CREATETABLE
@@ -547,41 +487,34 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-3
- Stats-Aggr Operator
- Stage-0
- Move Operator
- table:{"input format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.orc_merge5"}
- Stage-2
- Dependency Collection{}
- Stage-5(CONDITIONAL)
- Move Operator
- Stage-8(CONDITIONAL CHILD TASKS: Stage-5, Stage-4, Stage-6)
- Conditional Operator
- Stage-1
- Map 1 vectorized
- File Output Operator [FS_10]
- compressed:false
- Statistics:Num rows: 306 Data size: 82044 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.hive.ql.io.orc.OrcInputFormat","output format:":"org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat","serde:":"org.apache.hadoop.hive.ql.io.orc.OrcSerde","name:":"default.orc_merge5"}
- Select Operator [OP_9]
- outputColumnNames:["_col0","_col1","_col2","_col3","_col4"]
- Statistics:Num rows: 306 Data size: 82044 Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_8]
- predicate:(userid <= 13) (type: boolean)
- Statistics:Num rows: 306 Data size: 82044 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:orc_merge5
- Statistics:Num rows: 919 Data size: 246402 Basic stats: COMPLETE Column stats: NONE
- Stage-4(CONDITIONAL)
- File Merge
- ORC File Merge Operator [OFM_7]
- Please refer to the previous Stage-8(CONDITIONAL CHILD TASKS: Stage-5, Stage-4, Stage-6)
- Stage-7
- Move Operator
- Stage-6(CONDITIONAL)
- File Merge
- ORC File Merge Operator [OFM_7]
- Please refer to the previous Stage-8(CONDITIONAL CHILD TASKS: Stage-5, Stage-4, Stage-6)
+ Stats-Aggr Operator
+ Stage-0
+ Move Operator
+ table:{"name:":"default.orc_merge5"}
+ Stage-2
+ Dependency Collection{}
+ Stage-5(CONDITIONAL)
+ Move Operator
+ Stage-8(CONDITIONAL CHILD TASKS: Stage-5, Stage-4, Stage-6)
+ Conditional Operator
+ Stage-1
+ Map 1 vectorized
+ File Output Operator [FS_10]
+ table:{"name:":"default.orc_merge5"}
+ Select Operator [OP_9] (rows=306 width=268)
+ Output:["_col0","_col1","_col2","_col3","_col4"]
+ Filter Operator [FIL_8] (rows=306 width=268)
+ predicate:(userid <= 13)
+ TableScan [TS_0] (rows=919 width=268)
+ default@orc_merge5,orc_merge5,Tbl:COMPLETE,Col:NONE,Output:["userid","string1","subtype","decimal1","ts"]
+ Stage-4(CONDITIONAL)
+ File Merge
+ Please refer to the previous Stage-8(CONDITIONAL CHILD TASKS: Stage-5, Stage-4, Stage-6)
+ Stage-7
+ Move Operator
+ Stage-6(CONDITIONAL)
+ File Merge
+ Please refer to the previous Stage-8(CONDITIONAL CHILD TASKS: Stage-5, Stage-4, Stage-6)
PREHOOK: query: drop table orc_merge5
PREHOOK: type: DROPTABLE
@@ -715,47 +648,28 @@ Vertex dependency in root stage
Map 2 <- Map 1 (CUSTOM_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Map 2
- File Output Operator [FS_10]
- compressed:false
- Statistics:Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_9]
- outputColumnNames:["_col0","_col1","_col2"]
- Statistics:Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Map Join Operator [MAPJOIN_15]
- | BucketMapJoin:true
- | condition map:[{"":"Inner Join 0 to 1"}]
- | HybridGraceHashJoin:true
- | keys:{"Map 1":"_col0 (type: int)","Map 2":"_col0 (type: int)"}
- | outputColumnNames:["_col0","_col1","_col3"]
- | Statistics:Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [CUSTOM_EDGE]
- | Reduce Output Operator [RS_6]
- | key expressions:_col0 (type: int)
- | Map-reduce partition columns:_col0 (type: int)
- | sort order:+
- | Statistics:Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
- | value expressions:_col1 (type: string)
- | Select Operator [SEL_2]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
- | Filter Operator [FIL_13]
- | predicate:key is not null (type: boolean)
- | Statistics:Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
- | TableScan [TS_0]
- | alias:a
- | Statistics:Num rows: 242 Data size: 2566 Basic stats: COMPLETE Column stats: NONE
- |<-Select Operator [SEL_5]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_14]
- predicate:key is not null (type: boolean)
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_3]
- alias:b
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Map 2
+ File Output Operator [FS_10]
+ Select Operator [SEL_9] (rows=550 width=10)
+ Output:["_col0","_col1","_col2"]
+ Map Join Operator [MAPJOIN_15] (rows=550 width=10)
+ BucketMapJoin:true,Conds:RS_6._col0=SEL_5._col0(Inner),HybridGraceHashJoin:true,Output:["_col0","_col1","_col3"]
+ <-Map 1 [CUSTOM_EDGE]
+ MULTICAST [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=242 width=10)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_13] (rows=242 width=10)
+ predicate:key is not null
+ TableScan [TS_0] (rows=242 width=10)
+ default@tab,a,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
+ <-Select Operator [SEL_5] (rows=500 width=10)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_14] (rows=500 width=10)
+ predicate:key is not null
+ TableScan [TS_3] (rows=500 width=10)
+ default@tab_part,b,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/explainuser_4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/explainuser_4.q.out b/ql/src/test/results/clientpositive/tez/explainuser_4.q.out
new file mode 100644
index 0000000..3a3e17d
--- /dev/null
+++ b/ql/src/test/results/clientpositive/tez/explainuser_4.q.out
@@ -0,0 +1,482 @@
+PREHOOK: query: -- First try with regular mergejoin
+explain
+select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+PREHOOK: type: QUERY
+POSTHOOK: query: -- First try with regular mergejoin
+explain
+select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3
+ File Output Operator [FS_12]
+ Select Operator [SEL_11] (rows=6758 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_10]
+ Merge Join Operator [MERGEJOIN_17] (rows=6758 width=215)
+ Conds:RS_6._col2=RS_7._col2(Inner),Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_6]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_15] (rows=6144 width=215)
+ predicate:(cint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["ctinyint","csmallint","cint","cbigint","cfloat","cdouble","cstring1","cstring2","ctimestamp1","ctimestamp2","cboolean1","cboolean2"]
+ <-Map 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_7]
+ PartitionCols:_col2
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_16] (rows=6144 width=215)
+ predicate:((cint is not null and cbigint is not null) and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["ctinyint","csmallint","cint","cbigint","cfloat","cdouble","cstring1","cstring2","ctimestamp1","ctimestamp2","cboolean1","cboolean2"]
+
+PREHOOK: query: select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+11 NULL 1000828 1531084669 11.0 NULL wM316f6NqGIkoP388j3F6 poWQQo3Upvt3Wh 1969-12-31 16:00:02.351 NULL false true 11 NULL 1000828 1531084669 11.0 NULL wM316f6NqGIkoP388j3F6 poWQQo3Upvt3Wh 1969-12-31 16:00:02.351 NULL false true
+NULL -3799 1248059 1864027286 NULL -3799.0 Uhps6mMh3IfHB3j7yH62K 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:54.622 false true NULL -3799 1248059 1864027286 NULL -3799.0 Uhps6mMh3IfHB3j7yH62K 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:54.622 false true
+NULL 10782 1286921 1864027286 NULL 10782.0 ODLrXI8882q8LS8 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:52.138 true true NULL 10782 1286921 1864027286 NULL 10782.0 ODLrXI8882q8LS8 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:52.138 true true
+NULL -13036 1288927 -1645852809 NULL -13036.0 yinBY725P7V2 xH7445Rals48VOulSyR5F NULL 1969-12-31 16:00:00.763 true false NULL -13036 1288927 -1645852809 NULL -13036.0 yinBY725P7V2 xH7445Rals48VOulSyR5F NULL 1969-12-31 16:00:00.763 true false
+11 NULL 1310786 -413875656 11.0 NULL W0rvA4H1xn0xMG4uk0 8yVVjG 1969-12-31 16:00:02.351 NULL false true 11 NULL 1310786 -413875656 11.0 NULL W0rvA4H1xn0xMG4uk0 8yVVjG 1969-12-31 16:00:02.351 NULL false true
+-51 NULL 2089466 -240556350 -51.0 NULL cXX24dH7tblSj46j2g C31eea0wrHHqvj 1969-12-31 16:00:08.451 NULL true true -51 NULL 2089466 -240556350 -51.0 NULL cXX24dH7tblSj46j2g C31eea0wrHHqvj 1969-12-31 16:00:08.451 NULL true true
+NULL -8915 2101183 1864027286 NULL -8915.0 x7By66525 4KWs6gw7lv2WYd66P NULL 1969-12-31 16:00:05.831 false true NULL -8915 2101183 1864027286 NULL -8915.0 x7By66525 4KWs6gw7lv2WYd66P NULL 1969-12-31 16:00:05.831 false true
+8 NULL 2229621 -381406148 8.0 NULL q7onkS7QRPh5ghOK oKb0bi 1969-12-31 16:00:15.892 NULL true false 8 NULL 2229621 -381406148 8.0 NULL q7onkS7QRPh5ghOK oKb0bi 1969-12-31 16:00:15.892 NULL true false
+8 NULL 2433892 -1611863517 8.0 NULL 674ILv3V2TxFqXP6wSbL VLprkK2XfX 1969-12-31 16:00:15.892 NULL false true 8 NULL 2433892 -1611863517 8.0 NULL 674ILv3V2TxFqXP6wSbL VLprkK2XfX 1969-12-31 16:00:15.892 NULL false true
+-51 NULL 2949963 -1580871111 -51.0 NULL 0K68k3bdl7jO7 TPPAu 1969-12-31 16:00:08.451 NULL true false -51 NULL 2949963 -1580871111 -51.0 NULL 0K68k3bdl7jO7 TPPAu 1969-12-31 16:00:08.451 NULL true false
+PREHOOK: query: explain
+select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3
+ File Output Operator [FS_14]
+ Group By Operator [GBY_12] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(VALUE._col0)"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_11]
+ Group By Operator [GBY_10] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count()"]
+ Merge Join Operator [MERGEJOIN_19] (rows=6758 width=215)
+ Conds:RS_6._col0=RS_7._col0(Inner)
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_17] (rows=6144 width=215)
+ predicate:(cint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["cint"]
+ <-Map 4 [SIMPLE_EDGE]
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=6144 width=215)
+ predicate:((cint is not null and cbigint is not null) and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["cint","cbigint"]
+
+PREHOOK: query: select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+10
+PREHOOK: query: explain
+select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 4
+ File Output Operator [FS_16]
+ Select Operator [SEL_15] (rows=3379 width=215)
+ Output:["_col0","_col1"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_14]
+ Group By Operator [GBY_12] (rows=3379 width=215)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Group By Operator [GBY_10] (rows=6758 width=215)
+ Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
+ Merge Join Operator [MERGEJOIN_21] (rows=6758 width=215)
+ Conds:RS_6._col1=RS_7._col0(Inner),Output:["_col0"]
+ <-Map 1 [SIMPLE_EDGE]
+ SHUFFLE [RS_6]
+ PartitionCols:_col1
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_19] (rows=6144 width=215)
+ predicate:(cint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["csmallint","cint"]
+ <-Map 5 [SIMPLE_EDGE]
+ SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_20] (rows=6144 width=215)
+ predicate:((cint is not null and cbigint is not null) and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["cint","cbigint"]
+
+PREHOOK: query: select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+-13036 1
+-8915 1
+-3799 1
+10782 1
+NULL 6
+PREHOOK: query: -- Try with dynamically partitioned hashjoin
+explain
+select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+PREHOOK: type: QUERY
+POSTHOOK: query: -- Try with dynamically partitioned hashjoin
+explain
+select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Map 4 (CUSTOM_SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3
+ File Output Operator [FS_12]
+ Select Operator [SEL_11] (rows=6758 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_10]
+ Map Join Operator [MAPJOIN_17] (rows=6758 width=215)
+ Conds:RS_6.KEY.reducesinkkey0=RS_7.KEY.reducesinkkey0(Inner),HybridGraceHashJoin:true,Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11","_col12","_col13","_col14","_col15","_col16","_col17","_col18","_col19","_col20","_col21","_col22","_col23"]
+ <-Map 4 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_7]
+ PartitionCols:_col2
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_16] (rows=6144 width=215)
+ predicate:((cint is not null and cbigint is not null) and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["ctinyint","csmallint","cint","cbigint","cfloat","cdouble","cstring1","cstring2","ctimestamp1","ctimestamp2","cboolean1","cboolean2"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_6]
+ PartitionCols:_col2
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0","_col1","_col2","_col3","_col4","_col5","_col6","_col7","_col8","_col9","_col10","_col11"]
+ Filter Operator [FIL_15] (rows=6144 width=215)
+ predicate:(cint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["ctinyint","csmallint","cint","cbigint","cfloat","cdouble","cstring1","cstring2","ctimestamp1","ctimestamp2","cboolean1","cboolean2"]
+
+PREHOOK: query: select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ *
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+order by a.cint
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+11 NULL 1000828 1531084669 11.0 NULL wM316f6NqGIkoP388j3F6 poWQQo3Upvt3Wh 1969-12-31 16:00:02.351 NULL false true 11 NULL 1000828 1531084669 11.0 NULL wM316f6NqGIkoP388j3F6 poWQQo3Upvt3Wh 1969-12-31 16:00:02.351 NULL false true
+NULL -3799 1248059 1864027286 NULL -3799.0 Uhps6mMh3IfHB3j7yH62K 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:54.622 false true NULL -3799 1248059 1864027286 NULL -3799.0 Uhps6mMh3IfHB3j7yH62K 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:54.622 false true
+NULL 10782 1286921 1864027286 NULL 10782.0 ODLrXI8882q8LS8 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:52.138 true true NULL 10782 1286921 1864027286 NULL 10782.0 ODLrXI8882q8LS8 4KWs6gw7lv2WYd66P NULL 1969-12-31 15:59:52.138 true true
+NULL -13036 1288927 -1645852809 NULL -13036.0 yinBY725P7V2 xH7445Rals48VOulSyR5F NULL 1969-12-31 16:00:00.763 true false NULL -13036 1288927 -1645852809 NULL -13036.0 yinBY725P7V2 xH7445Rals48VOulSyR5F NULL 1969-12-31 16:00:00.763 true false
+11 NULL 1310786 -413875656 11.0 NULL W0rvA4H1xn0xMG4uk0 8yVVjG 1969-12-31 16:00:02.351 NULL false true 11 NULL 1310786 -413875656 11.0 NULL W0rvA4H1xn0xMG4uk0 8yVVjG 1969-12-31 16:00:02.351 NULL false true
+-51 NULL 2089466 -240556350 -51.0 NULL cXX24dH7tblSj46j2g C31eea0wrHHqvj 1969-12-31 16:00:08.451 NULL true true -51 NULL 2089466 -240556350 -51.0 NULL cXX24dH7tblSj46j2g C31eea0wrHHqvj 1969-12-31 16:00:08.451 NULL true true
+NULL -8915 2101183 1864027286 NULL -8915.0 x7By66525 4KWs6gw7lv2WYd66P NULL 1969-12-31 16:00:05.831 false true NULL -8915 2101183 1864027286 NULL -8915.0 x7By66525 4KWs6gw7lv2WYd66P NULL 1969-12-31 16:00:05.831 false true
+8 NULL 2229621 -381406148 8.0 NULL q7onkS7QRPh5ghOK oKb0bi 1969-12-31 16:00:15.892 NULL true false 8 NULL 2229621 -381406148 8.0 NULL q7onkS7QRPh5ghOK oKb0bi 1969-12-31 16:00:15.892 NULL true false
+8 NULL 2433892 -1611863517 8.0 NULL 674ILv3V2TxFqXP6wSbL VLprkK2XfX 1969-12-31 16:00:15.892 NULL false true 8 NULL 2433892 -1611863517 8.0 NULL 674ILv3V2TxFqXP6wSbL VLprkK2XfX 1969-12-31 16:00:15.892 NULL false true
+-51 NULL 2949963 -1580871111 -51.0 NULL 0K68k3bdl7jO7 TPPAu 1969-12-31 16:00:08.451 NULL true false -51 NULL 2949963 -1580871111 -51.0 NULL 0K68k3bdl7jO7 TPPAu 1969-12-31 16:00:08.451 NULL true false
+PREHOOK: query: explain
+select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Map 4 (CUSTOM_SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 3
+ File Output Operator [FS_14]
+ Group By Operator [GBY_12] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count(VALUE._col0)"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_11]
+ Group By Operator [GBY_10] (rows=1 width=8)
+ Output:["_col0"],aggregations:["count()"]
+ Map Join Operator [MAPJOIN_19] (rows=6758 width=215)
+ Conds:RS_6.KEY.reducesinkkey0=RS_7.KEY.reducesinkkey0(Inner),HybridGraceHashJoin:true
+ <-Map 4 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_18] (rows=6144 width=215)
+ predicate:((cint is not null and cbigint is not null) and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["cint","cbigint"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_6]
+ PartitionCols:_col0
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_17] (rows=6144 width=215)
+ predicate:(cint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["cint"]
+
+PREHOOK: query: select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ count(*)
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+10
+PREHOOK: query: explain
+select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+PREHOOK: type: QUERY
+POSTHOOK: query: explain
+select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+POSTHOOK: type: QUERY
+Plan optimized by CBO.
+
+Vertex dependency in root stage
+Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE), Map 5 (CUSTOM_SIMPLE_EDGE)
+Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
+Reducer 4 <- Reducer 3 (SIMPLE_EDGE)
+
+Stage-0
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 4
+ File Output Operator [FS_16]
+ Select Operator [SEL_15] (rows=3379 width=215)
+ Output:["_col0","_col1"]
+ <-Reducer 3 [SIMPLE_EDGE]
+ SHUFFLE [RS_14]
+ Group By Operator [GBY_12] (rows=3379 width=215)
+ Output:["_col0","_col1"],aggregations:["count(VALUE._col0)"],keys:KEY._col0
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_11]
+ PartitionCols:_col0
+ Group By Operator [GBY_10] (rows=6758 width=215)
+ Output:["_col0","_col1"],aggregations:["count()"],keys:_col0
+ Map Join Operator [MAPJOIN_21] (rows=6758 width=215)
+ Conds:RS_6.KEY.reducesinkkey0=RS_7.KEY.reducesinkkey0(Inner),HybridGraceHashJoin:true,Output:["_col0"]
+ <-Map 5 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_7]
+ PartitionCols:_col0
+ Select Operator [SEL_5] (rows=6144 width=215)
+ Output:["_col0"]
+ Filter Operator [FIL_20] (rows=6144 width=215)
+ predicate:((cint is not null and cbigint is not null) and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_3] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["cint","cbigint"]
+ <-Map 1 [CUSTOM_SIMPLE_EDGE]
+ PARTITION_ONLY_SHUFFLE [RS_6]
+ PartitionCols:_col1
+ Select Operator [SEL_2] (rows=6144 width=215)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_19] (rows=6144 width=215)
+ predicate:(cint is not null and cint BETWEEN 1000000 AND 3000000)
+ TableScan [TS_0] (rows=12288 width=215)
+ default@alltypesorc,a,Tbl:COMPLETE,Col:NONE,Output:["csmallint","cint"]
+
+PREHOOK: query: select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+PREHOOK: type: QUERY
+PREHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+POSTHOOK: query: select
+ a.csmallint, count(*) c1
+from alltypesorc a join alltypesorc b on a.cint = b.cint
+where
+ a.cint between 1000000 and 3000000 and b.cbigint is not null
+group by a.csmallint
+order by c1
+POSTHOOK: type: QUERY
+POSTHOOK: Input: default@alltypesorc
+#### A masked pattern was here ####
+-8915 1
+-3799 1
+10782 1
+-13036 1
+NULL 6
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/nonmr_fetch_threshold.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/nonmr_fetch_threshold.q.out b/ql/src/test/results/clientpositive/tez/nonmr_fetch_threshold.q.out
index d6dd09f..c66f85d 100644
--- a/ql/src/test/results/clientpositive/tez/nonmr_fetch_threshold.q.out
+++ b/ql/src/test/results/clientpositive/tez/nonmr_fetch_threshold.q.out
@@ -5,14 +5,14 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:10
- Limit [LIM_3]
- Number of rows:10
- Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0]
- alias:srcpart
+ Fetch Operator
+ limit:10
+ Limit [LIM_3]
+ Number of rows:10
+ Select Operator [SEL_2]
+ Output:["_col0","_col1","_col2","_col3"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
PREHOOK: type: QUERY
@@ -21,14 +21,14 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:10
- Limit [LIM_2]
- Number of rows:10
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:10
+ Limit [LIM_2]
+ Number of rows:10
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10
PREHOOK: type: QUERY
@@ -37,14 +37,14 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:10
- Limit [LIM_3]
- Number of rows:10
- Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0]
- alias:srcpart
+ Fetch Operator
+ limit:10
+ Limit [LIM_3]
+ Number of rows:10
+ Select Operator [SEL_2]
+ Output:["_col0","_col1","_col2","_col3"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
PREHOOK: type: QUERY
@@ -53,14 +53,14 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:10
- Limit [LIM_2]
- Number of rows:10
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:10
+ Limit [LIM_2]
+ Number of rows:10
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: -- Scans without limit (should be Fetch task now)
explain select concat(key, value) from src
@@ -71,12 +71,12 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:-1
- Select Operator [SEL_1]
- outputColumnNames:["_col0"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: -- from HIVE-7397, limit + partition pruning filter
explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10
@@ -87,14 +87,14 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:10
- Limit [LIM_3]
- Number of rows:10
- Select Operator [SEL_2]
- outputColumnNames:["_col0","_col1","_col2","_col3"]
- TableScan [TS_0]
- alias:srcpart
+ Fetch Operator
+ limit:10
+ Limit [LIM_3]
+ Number of rows:10
+ Select Operator [SEL_2]
+ Output:["_col0","_col1","_col2","_col3"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: explain select cast(key as int) * 10, upper(value) from src limit 10
PREHOOK: type: QUERY
@@ -103,14 +103,14 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:10
- Limit [LIM_2]
- Number of rows:10
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:10
+ Limit [LIM_2]
+ Number of rows:10
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: -- Scans without limit (should not be Fetch task now)
explain select concat(key, value) from src
@@ -121,20 +121,15 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Map 1
- File Output Operator [FS_2]
- compressed:false
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Select Operator [SEL_1]
- outputColumnNames:["_col0"]
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:src
- Statistics:Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Map 1
+ File Output Operator [FS_2]
+ Select Operator [SEL_1] (rows=500 width=10)
+ Output:["_col0"]
+ TableScan [TS_0] (rows=500 width=10)
+ default@src,src,Tbl:COMPLETE,Col:NONE,Output:["key","value"]
PREHOOK: query: -- Simple Scans without limit (will be Fetch task now)
explain select key, value from src
@@ -145,12 +140,12 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:-1
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: explain select key from src
PREHOOK: type: QUERY
@@ -159,12 +154,12 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:-1
- Select Operator [SEL_1]
- outputColumnNames:["_col0"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0"]
+ TableScan [TS_0]
+ Output:["key"]
PREHOOK: query: explain select * from src
PREHOOK: type: QUERY
@@ -173,12 +168,12 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:-1
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key","value"]
PREHOOK: query: explain select key,1 from src
PREHOOK: type: QUERY
@@ -187,12 +182,12 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:-1
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key"]
PREHOOK: query: explain select cast(key as char(20)),1 from src
PREHOOK: type: QUERY
@@ -201,10 +196,10 @@ POSTHOOK: type: QUERY
Plan optimized by CBO.
Stage-0
- Fetch Operator
- limit:-1
- Select Operator [SEL_1]
- outputColumnNames:["_col0","_col1"]
- TableScan [TS_0]
- alias:src
+ Fetch Operator
+ limit:-1
+ Select Operator [SEL_1]
+ Output:["_col0","_col1"]
+ TableScan [TS_0]
+ Output:["key"]
http://git-wip-us.apache.org/repos/asf/hive/blob/be9735e1/ql/src/test/results/clientpositive/tez/vector_aggregate_without_gby.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/vector_aggregate_without_gby.q.out b/ql/src/test/results/clientpositive/tez/vector_aggregate_without_gby.q.out
index 1d84e3b..55a3842 100644
--- a/ql/src/test/results/clientpositive/tez/vector_aggregate_without_gby.q.out
+++ b/ql/src/test/results/clientpositive/tez/vector_aggregate_without_gby.q.out
@@ -43,36 +43,23 @@ Vertex dependency in root stage
Reducer 2 <- Map 1 (SIMPLE_EDGE)
Stage-0
- Fetch Operator
- limit:-1
- Stage-1
- Reducer 2 vectorized
- File Output Operator [FS_7]
- compressed:false
- Statistics:Num rows: 1 Data size: 88 Basic stats: COMPLETE Column stats: NONE
- table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}
- Group By Operator [OP_12]
- | aggregations:["max(VALUE._col0)","max(VALUE._col1)"]
- | outputColumnNames:["_col0","_col1"]
- | Statistics:Num rows: 1 Data size: 88 Basic stats: COMPLETE Column stats: NONE
- |<-Map 1 [SIMPLE_EDGE] vectorized
- Reduce Output Operator [RS_4]
- sort order:
- Statistics:Num rows: 1 Data size: 88 Basic stats: COMPLETE Column stats: NONE
- value expressions:_col0 (type: int), _col1 (type: string)
- Group By Operator [OP_11]
- aggregations:["max(dt)","max(greg_dt)"]
- outputColumnNames:["_col0","_col1"]
- Statistics:Num rows: 1 Data size: 88 Basic stats: COMPLETE Column stats: NONE
- Select Operator [OP_10]
- outputColumnNames:["dt","greg_dt"]
- Statistics:Num rows: 3 Data size: 306 Basic stats: COMPLETE Column stats: NONE
- Filter Operator [FIL_9]
- predicate:(id = 5) (type: boolean)
- Statistics:Num rows: 3 Data size: 306 Basic stats: COMPLETE Column stats: NONE
- TableScan [TS_0]
- alias:testvec
- Statistics:Num rows: 7 Data size: 714 Basic stats: COMPLETE Column stats: NONE
+ Fetch Operator
+ limit:-1
+ Stage-1
+ Reducer 2 vectorized
+ File Output Operator [FS_7]
+ Group By Operator [OP_12] (rows=1 width=88)
+ Output:["_col0","_col1"],aggregations:["max(VALUE._col0)","max(VALUE._col1)"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_4]
+ Group By Operator [OP_11] (rows=1 width=88)
+ Output:["_col0","_col1"],aggregations:["max(dt)","max(greg_dt)"]
+ Select Operator [OP_10] (rows=3 width=102)
+ Output:["dt","greg_dt"]
+ Filter Operator [FIL_9] (rows=3 width=102)
+ predicate:(id = 5)
+ TableScan [TS_0] (rows=7 width=102)
+ default@testvec,testvec,Tbl:COMPLETE,Col:NONE,Output:["id","dt","greg_dt"]
PREHOOK: query: select max(dt), max(greg_dt) from testvec where id=5
PREHOOK: type: QUERY