You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by jc...@apache.org on 2019/01/04 00:24:01 UTC
[02/35] hive git commit: HIVE-16957: Support CTAS for auto gather
column stats (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out b/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out
index fa9103f..4013b36 100644
--- a/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out
+++ b/ql/src/test/results/clientpositive/llap/vectorized_mapjoin3.q.out
@@ -126,7 +126,7 @@ STAGE PLANS:
TableScan
alias: t1
filterExpr: (decimal0801_col is not null and int_col_1 is not null) (type: boolean)
- Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
TableScan Vectorization:
native: true
vectorizationSchemaColumns: [0:decimal0801_col:decimal(8,1)/DECIMAL_64, 1:int_col_1:int, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
@@ -136,7 +136,7 @@ STAGE PLANS:
native: true
predicateExpression: FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:decimal(8,1)/DECIMAL_64), SelectColumnIsNotNull(col 1:int))
predicate: (decimal0801_col is not null and int_col_1 is not null) (type: boolean)
- Statistics: Num rows: 4 Data size: 464 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: decimal0801_col (type: decimal(8,1)), int_col_1 (type: int)
outputColumnNames: _col0, _col1
@@ -144,7 +144,7 @@ STAGE PLANS:
className: VectorSelectOperator
native: true
projectedOutputColumnNums: [0, 1]
- Statistics: Num rows: 4 Data size: 464 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -165,13 +165,13 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Reducer 3
- Statistics: Num rows: 1 Data size: 112 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 638 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
File Sink Vectorization:
className: VectorFileSinkOperator
native: false
- Statistics: Num rows: 1 Data size: 112 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 638 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -197,7 +197,7 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tt1
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
TableScan Vectorization:
native: true
vectorizationSchemaColumns: [0:int_col_0:int, 1:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
@@ -208,7 +208,7 @@ STAGE PLANS:
className: VectorSelectOperator
native: true
projectedOutputColumnNums: [0]
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: max(int_col_0)
Group By Vectorization:
@@ -220,7 +220,7 @@ STAGE PLANS:
projectedOutputColumnNums: [0]
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
Reduce Sink Vectorization:
@@ -228,7 +228,7 @@ STAGE PLANS:
native: true
nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
valueColumns: 0:int
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
@@ -274,14 +274,14 @@ STAGE PLANS:
projectedOutputColumnNums: [0]
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Filter Operator
Filter Vectorization:
className: VectorFilterOperator
native: true
predicateExpression: SelectColumnIsNotNull(col 0:int)
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
@@ -291,7 +291,7 @@ STAGE PLANS:
keyColumns: 0:int
native: true
nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Stage: Stage-0
Fetch Operator
@@ -337,7 +337,7 @@ STAGE PLANS:
TableScan
alias: t1
filterExpr: (int_col_1 is not null and decimal0801_col is not null) (type: boolean)
- Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
TableScan Vectorization:
native: true
vectorizationSchemaColumns: [0:decimal0801_col:decimal(8,1)/DECIMAL_64, 1:int_col_1:int, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
@@ -347,7 +347,7 @@ STAGE PLANS:
native: true
predicateExpression: FilterExprAndExpr(children: SelectColumnIsNotNull(col 1:int), SelectColumnIsNotNull(col 0:decimal(8,1)/DECIMAL_64))
predicate: (decimal0801_col is not null and int_col_1 is not null) (type: boolean)
- Statistics: Num rows: 4 Data size: 464 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: decimal0801_col (type: decimal(8,1)), int_col_1 (type: int)
outputColumnNames: _col0, _col1
@@ -355,7 +355,7 @@ STAGE PLANS:
className: VectorSelectOperator
native: true
projectedOutputColumnNums: [0, 1]
- Statistics: Num rows: 4 Data size: 464 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -376,13 +376,13 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Reducer 3
- Statistics: Num rows: 1 Data size: 112 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 638 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
File Sink Vectorization:
className: VectorFileSinkOperator
native: false
- Statistics: Num rows: 1 Data size: 112 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 638 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -408,7 +408,7 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tt1
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
TableScan Vectorization:
native: true
vectorizationSchemaColumns: [0:int_col_0:int, 1:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
@@ -419,7 +419,7 @@ STAGE PLANS:
className: VectorSelectOperator
native: true
projectedOutputColumnNums: [0]
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: max(int_col_0)
Group By Vectorization:
@@ -431,7 +431,7 @@ STAGE PLANS:
projectedOutputColumnNums: [0]
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
Reduce Sink Vectorization:
@@ -439,7 +439,7 @@ STAGE PLANS:
native: true
nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
valueColumns: 0:int
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
@@ -485,14 +485,14 @@ STAGE PLANS:
projectedOutputColumnNums: [0]
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Filter Operator
Filter Vectorization:
className: VectorFilterOperator
native: true
predicateExpression: SelectColumnIsNotNull(col 0:int)
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
@@ -502,7 +502,7 @@ STAGE PLANS:
keyColumns: 0:int
native: true
nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Stage: Stage-0
Fetch Operator
@@ -548,7 +548,7 @@ STAGE PLANS:
TableScan
alias: t1
filterExpr: (decimal0801_col is not null and int_col_1 is not null) (type: boolean)
- Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
TableScan Vectorization:
native: true
vectorizationSchemaColumns: [0:decimal0801_col:decimal(8,1)/DECIMAL_64, 1:int_col_1:int, 2:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
@@ -558,7 +558,7 @@ STAGE PLANS:
native: true
predicateExpression: FilterExprAndExpr(children: SelectColumnIsNotNull(col 0:decimal(8,1)/DECIMAL_64), SelectColumnIsNotNull(col 1:int))
predicate: (decimal0801_col is not null and int_col_1 is not null) (type: boolean)
- Statistics: Num rows: 4 Data size: 464 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: decimal0801_col (type: decimal(8,1)), int_col_1 (type: int)
outputColumnNames: _col0, _col1
@@ -566,7 +566,7 @@ STAGE PLANS:
className: VectorSelectOperator
native: true
projectedOutputColumnNums: [0, 1]
- Statistics: Num rows: 4 Data size: 464 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 580 Basic stats: COMPLETE Column stats: NONE
Map Join Operator
condition map:
Inner Join 0 to 1
@@ -587,13 +587,13 @@ STAGE PLANS:
outputColumnNames: _col0
input vertices:
1 Reducer 3
- Statistics: Num rows: 1 Data size: 112 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 638 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
File Sink Vectorization:
className: VectorFileSinkOperator
native: false
- Statistics: Num rows: 1 Data size: 112 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 5 Data size: 638 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
@@ -619,7 +619,7 @@ STAGE PLANS:
Map Operator Tree:
TableScan
alias: tt1
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
TableScan Vectorization:
native: true
vectorizationSchemaColumns: [0:int_col_0:int, 1:ROW__ID:struct<writeid:bigint,bucketid:int,rowid:bigint>]
@@ -630,7 +630,7 @@ STAGE PLANS:
className: VectorSelectOperator
native: true
projectedOutputColumnNums: [0]
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: max(int_col_0)
Group By Vectorization:
@@ -642,7 +642,7 @@ STAGE PLANS:
projectedOutputColumnNums: [0]
mode: hash
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
sort order:
Reduce Sink Vectorization:
@@ -650,7 +650,7 @@ STAGE PLANS:
native: true
nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
valueColumns: 0:int
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int)
Execution mode: vectorized, llap
LLAP IO: all inputs
@@ -696,14 +696,14 @@ STAGE PLANS:
projectedOutputColumnNums: [0]
mode: mergepartial
outputColumnNames: _col0
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Filter Operator
Filter Vectorization:
className: VectorFilterOperator
native: true
predicateExpression: SelectColumnIsNotNull(col 0:int)
predicate: _col0 is not null (type: boolean)
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: int)
sort order: +
@@ -713,7 +713,7 @@ STAGE PLANS:
keyColumns: 0:int
native: true
nativeConditionsMet: hive.vectorized.execution.reducesink.new.enabled IS true, hive.execution.engine tez IN [tez, spark] IS true, No PTF TopN IS true, No DISTINCT columns IS true, BinarySortableSerDe for keys IS true, LazyBinarySerDe for values IS true
- Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
+ Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE
Stage: Stage-0
Fetch Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/load_dyn_part14.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/load_dyn_part14.q.out b/ql/src/test/results/clientpositive/load_dyn_part14.q.out
index bc4d704..61718e6 100644
--- a/ql/src/test/results/clientpositive/load_dyn_part14.q.out
+++ b/ql/src/test/results/clientpositive/load_dyn_part14.q.out
@@ -73,6 +73,7 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized
Reduce Operator Tree:
Limit
@@ -265,6 +266,7 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized
Reduce Operator Tree:
Limit
@@ -295,6 +297,7 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
Execution mode: vectorized
Reduce Operator Tree:
Limit
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/masking_mv.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/masking_mv.q.out b/ql/src/test/results/clientpositive/masking_mv.q.out
index 3da49df..105e54b 100644
--- a/ql/src/test/results/clientpositive/masking_mv.q.out
+++ b/ql/src/test/results/clientpositive/masking_mv.q.out
@@ -57,7 +57,32 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.masking_test_view_n_mv
- Execution mode: vectorized
+ Select Operator
+ expressions: _col0 (type: int)
+ outputColumnNames: col1
+ Statistics: Num rows: 500 Data size: 30200 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: compute_stats(col1, 'hll')
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 424 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 424 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: struct<columntype:string,min:bigint,max:bigint,countnulls:bigint,bitvector:binary>)
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: compute_stats(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 440 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 440 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-7
Conditional Operator
@@ -86,6 +111,10 @@ STAGE PLANS:
Stage: Stage-2
Stats Work
Basic Stats Work:
+ Column Stats Desc:
+ Columns: key
+ Column Types: int
+ Table: default.masking_test_view_n_mv
Stage: Stage-9
Materialized View Work
@@ -138,7 +167,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MATERIALIZED_VIEW
Table Parameters:
- COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\"}}
bucketing_version 2
numFiles 1
numRows 500
@@ -695,7 +724,32 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.masking_test_view_n_mv_2
- Execution mode: vectorized
+ Select Operator
+ expressions: _col0 (type: int)
+ outputColumnNames: col1
+ Statistics: Num rows: 500 Data size: 30200 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: compute_stats(col1, 'hll')
+ mode: hash
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 424 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 424 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: struct<columntype:string,min:bigint,max:bigint,countnulls:bigint,bitvector:binary>)
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: compute_stats(VALUE._col0)
+ mode: mergepartial
+ outputColumnNames: _col0
+ Statistics: Num rows: 1 Data size: 440 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 440 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-7
Conditional Operator
@@ -724,6 +778,10 @@ STAGE PLANS:
Stage: Stage-2
Stats Work
Basic Stats Work:
+ Column Stats Desc:
+ Columns: key
+ Column Types: int
+ Table: default.masking_test_view_n_mv_2
Stage: Stage-9
Materialized View Work
@@ -776,7 +834,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MATERIALIZED_VIEW
Table Parameters:
- COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\"}}
bucketing_version 2
numFiles 1
numRows 500
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/merge3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/merge3.q.out b/ql/src/test/results/clientpositive/merge3.q.out
index fc115bd..fc8afcf 100644
--- a/ql/src/test/results/clientpositive/merge3.q.out
+++ b/ql/src/test/results/clientpositive/merge3.q.out
@@ -109,7 +109,22 @@ STAGE PLANS:
TotalFiles: 1
GatherStats: true
MultiFileSpray: false
- Execution mode: vectorized
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: col1, col2
+ Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: compute_stats(col1, 'hll'), compute_stats(col2, 'hll')
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ null sort order:
+ sort order:
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ tag: -1
+ value expressions: _col0 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>), _col1 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>)
+ auto parallelism: false
Path -> Alias:
#### A masked pattern was here ####
Path -> Partition:
@@ -119,7 +134,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
bucketing_version 2
column.name.delimiter ,
@@ -141,7 +156,7 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
- COLUMN_STATS_ACCURATE {"BASIC_STATS":"true"}
+ COLUMN_STATS_ACCURATE {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
bucket_count -1
bucketing_version 2
column.name.delimiter ,
@@ -163,6 +178,35 @@ STAGE PLANS:
name: default.merge_src
Truncated Path -> Alias:
/merge_src [merge_src]
+ Needs Tagging: false
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: compute_stats(VALUE._col0), compute_stats(VALUE._col1)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ GlobalTableId: 0
+#### A masked pattern was here ####
+ NumFilesPerFileSink: 1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+#### A masked pattern was here ####
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ properties:
+ columns _col0,_col1
+ columns.types struct<columntype:string,maxlength:bigint,avglength:double,countnulls:bigint,numdistinctvalues:bigint,ndvbitvector:binary>:struct<columntype:string,maxlength:bigint,avglength:double,countnulls:bigint,numdistinctvalues:bigint,ndvbitvector:binary>
+ escape.delim \
+ hive.serialization.extend.additional.nesting.levels true
+ serialization.escape.crlf true
+ serialization.format 1
+ serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+ TotalFiles: 1
+ GatherStats: false
+ MultiFileSpray: false
Stage: Stage-7
Conditional Operator
@@ -192,6 +236,11 @@ STAGE PLANS:
Stats Work
Basic Stats Work:
#### A masked pattern was here ####
+ Column Stats Desc:
+ Columns: key, value
+ Column Types: string, string
+ Table: default.merge_src2
+ Is Table Level Stats: true
Stage: Stage-3
Map Reduce
@@ -2346,7 +2395,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}}
bucketing_version 2
numFiles 1
numRows 2000
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/merge4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/merge4.q.out b/ql/src/test/results/clientpositive/merge4.q.out
index 621601d..1a5dc73 100644
--- a/ql/src/test/results/clientpositive/merge4.q.out
+++ b/ql/src/test/results/clientpositive/merge4.q.out
@@ -2915,6 +2915,7 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 1 Data size: 259 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
TableScan
alias: src
@@ -2931,6 +2932,7 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 1 Data size: 259 Basic stats: COMPLETE Column stats: COMPLETE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
Reduce Operator Tree:
Select Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/mm_default.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/mm_default.q.out b/ql/src/test/results/clientpositive/mm_default.q.out
index 5a85554..70519b7 100644
--- a/ql/src/test/results/clientpositive/mm_default.q.out
+++ b/ql/src/test/results/clientpositive/mm_default.q.out
@@ -180,7 +180,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\"}}
bucketing_version 2
numFiles 1
numRows 1
@@ -252,7 +252,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\"}}
bucketing_version 2
numFiles 1
numRows 1
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/named_column_join.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/named_column_join.q.out b/ql/src/test/results/clientpositive/named_column_join.q.out
index 5e94f88..22fd7f5 100644
--- a/ql/src/test/results/clientpositive/named_column_join.q.out
+++ b/ql/src/test/results/clientpositive/named_column_join.q.out
@@ -468,7 +468,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\"}}
bucketing_version 2
numFiles 1
numRows 13
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/nonmr_fetch.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/nonmr_fetch.q.out b/ql/src/test/results/clientpositive/nonmr_fetch.q.out
index ca1bf15..7c1de88 100644
--- a/ql/src/test/results/clientpositive/nonmr_fetch.q.out
+++ b/ql/src/test/results/clientpositive/nonmr_fetch.q.out
@@ -1300,8 +1300,9 @@ POSTHOOK: Output: default@srcx
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
- Stage-3 depends on stages: Stage-0
- Stage-2 depends on stages: Stage-3
+ Stage-4 depends on stages: Stage-0, Stage-3
+ Stage-2 depends on stages: Stage-4
+ Stage-3 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
@@ -1339,6 +1340,21 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.srcx
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: col1, col2
+ Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: compute_stats(col1, 'hll'), compute_stats(col2, 'hll')
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-0
Move Operator
@@ -1346,7 +1362,7 @@ STAGE PLANS:
hdfs directory: true
#### A masked pattern was here ####
- Stage: Stage-3
+ Stage: Stage-4
Create Table Operator:
Create Table
columns: key string, value string
@@ -1358,6 +1374,33 @@ STAGE PLANS:
Stage: Stage-2
Stats Work
Basic Stats Work:
+ Column Stats Desc:
+ Columns: key, value
+ Column Types: string, string
+ Table: default.srcx
+
+ Stage: Stage-3
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>), _col1 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>)
+ Execution mode: vectorized
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: compute_stats(VALUE._col0), compute_stats(VALUE._col1)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
PREHOOK: query: explain analyze table src compute statistics
PREHOOK: type: QUERY
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/nonreserved_keywords_insert_into1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/nonreserved_keywords_insert_into1.q.out b/ql/src/test/results/clientpositive/nonreserved_keywords_insert_into1.q.out
index b195ad2..9557caf 100644
--- a/ql/src/test/results/clientpositive/nonreserved_keywords_insert_into1.q.out
+++ b/ql/src/test/results/clientpositive/nonreserved_keywords_insert_into1.q.out
@@ -41,6 +41,7 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: vectorized
Reduce Operator Tree:
@@ -174,6 +175,7 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 100 Data size: 1000 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: vectorized
Reduce Operator Tree:
@@ -316,6 +318,7 @@ STAGE PLANS:
Reduce Output Operator
sort order:
Statistics: Num rows: 10 Data size: 100 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col0 (type: string), _col1 (type: string)
Execution mode: vectorized
Reduce Operator Tree:
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/nullformatCTAS.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/nullformatCTAS.q.out b/ql/src/test/results/clientpositive/nullformatCTAS.q.out
index 6fe3800..e6a7750 100644
--- a/ql/src/test/results/clientpositive/nullformatCTAS.q.out
+++ b/ql/src/test/results/clientpositive/nullformatCTAS.q.out
@@ -76,7 +76,32 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.null_tab3
- Execution mode: vectorized
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: col1, col2
+ Statistics: Num rows: 1 Data size: 1300 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: compute_stats(col1, 'hll'), compute_stats(col2, 'hll')
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>), _col1 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>)
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: compute_stats(VALUE._col0), compute_stats(VALUE._col1)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-7
Conditional Operator
@@ -105,6 +130,10 @@ STAGE PLANS:
Stage: Stage-2
Stats Work
Basic Stats Work:
+ Column Stats Desc:
+ Columns: a, b
+ Column Types: string, string
+ Table: default.null_tab3
Stage: Stage-3
Map Reduce
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/orc_createas1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/orc_createas1.q.out b/ql/src/test/results/clientpositive/orc_createas1.q.out
index dd1986c..1019f8a 100644
--- a/ql/src/test/results/clientpositive/orc_createas1.q.out
+++ b/ql/src/test/results/clientpositive/orc_createas1.q.out
@@ -88,6 +88,32 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_createas1b
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: col1, col2
+ Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: compute_stats(col1, 'hll'), compute_stats(col2, 'hll')
+ mode: hash
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>), _col1 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>)
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: compute_stats(VALUE._col0), compute_stats(VALUE._col1)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-7
Conditional Operator
@@ -116,6 +142,10 @@ STAGE PLANS:
Stage: Stage-2
Stats Work
Basic Stats Work:
+ Column Stats Desc:
+ Columns: key, value
+ Column Types: string, string
+ Table: default.orc_createas1b
Stage: Stage-3
Merge File Operator
@@ -269,6 +299,32 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde
name: default.orc_createas1c
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: string), _col2 (type: int)
+ outputColumnNames: col1, col2, col3
+ Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: compute_stats(col1, 'hll'), compute_stats(col2, 'hll'), compute_stats(col3, 'hll')
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 1288 Basic stats: COMPLETE Column stats: NONE
+ Reduce Output Operator
+ sort order:
+ Statistics: Num rows: 1 Data size: 1288 Basic stats: COMPLETE Column stats: NONE
+ value expressions: _col0 (type: struct<columntype:string,min:bigint,max:bigint,countnulls:bigint,bitvector:binary>), _col1 (type: struct<columntype:string,maxlength:bigint,sumlength:bigint,count:bigint,countnulls:bigint,bitvector:binary>), _col2 (type: struct<columntype:string,min:bigint,max:bigint,countnulls:bigint,bitvector:binary>)
+ Reduce Operator Tree:
+ Group By Operator
+ aggregations: compute_stats(VALUE._col0), compute_stats(VALUE._col1), compute_stats(VALUE._col2)
+ mode: mergepartial
+ outputColumnNames: _col0, _col1, _col2
+ Statistics: Num rows: 1 Data size: 1320 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 1320 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-7
Conditional Operator
@@ -297,6 +353,10 @@ STAGE PLANS:
Stage: Stage-2
Stats Work
Basic Stats Work:
+ Column Stats Desc:
+ Columns: key, value, part
+ Column Types: int, string, int
+ Table: default.orc_createas1c
Stage: Stage-3
Merge File Operator
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/parallel_orderby.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/parallel_orderby.q.out b/ql/src/test/results/clientpositive/parallel_orderby.q.out
index 05d570d..77685ea 100644
--- a/ql/src/test/results/clientpositive/parallel_orderby.q.out
+++ b/ql/src/test/results/clientpositive/parallel_orderby.q.out
@@ -69,6 +69,26 @@ STAGE PLANS:
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.total_ordered
+ Select Operator
+ expressions: _col0 (type: string), _col1 (type: string)
+ outputColumnNames: col1, col2
+ Statistics: Num rows: 1 Data size: 5600 Basic stats: COMPLETE Column stats: NONE
+ Group By Operator
+ aggregations: compute_stats(col1, 'hll'), compute_stats(col2, 'hll')
+ mode: complete
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: struct<columntype:string,maxlength:bigint,avglength:double,countnulls:bigint,numdistinctvalues:bigint,ndvbitvector:binary>), _col1 (type: struct<columntype:string,maxlength:bigint,avglength:double,countnulls:bigint,numdistinctvalues:bigint,ndvbitvector:binary>)
+ outputColumnNames: _col0, _col1
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 880 Basic stats: COMPLETE Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Move Operator
@@ -88,13 +108,16 @@ STAGE PLANS:
Stage: Stage-2
Stats Work
Basic Stats Work:
+ Column Stats Desc:
+ Columns: key, value
+ Column Types: string, string
+ Table: default.total_ordered
PREHOOK: query: create table total_ordered as select * from src5_n2 order by key, value
PREHOOK: type: CREATETABLE_AS_SELECT
PREHOOK: Input: default@src5_n2
PREHOOK: Output: database:default
PREHOOK: Output: default@total_ordered
-java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.LazyStruct cannot be cast to org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch
POSTHOOK: query: create table total_ordered as select * from src5_n2 order by key, value
POSTHOOK: type: CREATETABLE_AS_SELECT
POSTHOOK: Input: default@src5_n2
@@ -119,7 +142,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}}
bucketing_version 2
numFiles 1
numRows 48
@@ -230,7 +253,7 @@ Retention: 0
#### A masked pattern was here ####
Table Type: MANAGED_TABLE
Table Parameters:
- COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
+ COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"key\":\"true\",\"value\":\"true\"}}
bucketing_version 2
numFiles 1
numRows 48
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/perf/spark/query70.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/spark/query70.q.out b/ql/src/test/results/clientpositive/perf/spark/query70.q.out
index 56767af..d73e542 100644
--- a/ql/src/test/results/clientpositive/perf/spark/query70.q.out
+++ b/ql/src/test/results/clientpositive/perf/spark/query70.q.out
@@ -261,6 +261,7 @@ STAGE PLANS:
sort order: +
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 696954748 Data size: 61485550191 Basic stats: COMPLETE Column stats: NONE
+ TopN Hash Memory Usage: 0.1
value expressions: _col1 (type: decimal(17,2))
Reducer 11
Execution mode: vectorized
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out b/ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out
index 17e2f99..a49314c 100644
--- a/ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out
@@ -100,25 +100,25 @@ Stage-0
limit:-1
Stage-1
Reducer 6 vectorized
- File Output Operator [FS_170]
- Limit [LIM_169] (rows=1 width=492)
+ File Output Operator [FS_171]
+ Limit [LIM_170] (rows=1 width=492)
Number of rows:100
- Select Operator [SEL_168] (rows=1 width=492)
+ Select Operator [SEL_169] (rows=1 width=492)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_167]
- Select Operator [SEL_166] (rows=1 width=492)
+ SHUFFLE [RS_168]
+ Select Operator [SEL_167] (rows=1 width=492)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- PTF Operator [PTF_165] (rows=1 width=304)
+ PTF Operator [PTF_166] (rows=1 width=304)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"_col2 DESC NULLS LAST","partition by:":"(grouping(_col3, 1) + grouping(_col3, 0)), CASE WHEN ((grouping(_col3, 0) = 0)) THEN (_col0) ELSE (CAST( null AS STRING)) END"}]
- Select Operator [SEL_164] (rows=1 width=304)
+ Select Operator [SEL_165] (rows=1 width=304)
Output:["_col0","_col1","_col2","_col3"]
<-Reducer 4 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_163]
+ SHUFFLE [RS_164]
PartitionCols:(grouping(_col3, 1) + grouping(_col3, 0)), CASE WHEN ((grouping(_col3, 0) = 0)) THEN (_col0) ELSE (CAST( null AS STRING)) END
- Select Operator [SEL_162] (rows=1 width=304)
+ Select Operator [SEL_163] (rows=1 width=304)
Output:["_col0","_col1","_col2","_col3"]
- Group By Operator [GBY_161] (rows=1 width=304)
+ Group By Operator [GBY_162] (rows=1 width=304)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_50]
@@ -127,90 +127,92 @@ Stage-0
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col2)"],keys:_col0, _col1, 0L
Select Operator [SEL_47] (rows=1 width=296)
Output:["_col0","_col1","_col2"]
- Merge Join Operator [MERGEJOIN_136] (rows=1 width=296)
+ Merge Join Operator [MERGEJOIN_137] (rows=1 width=296)
Conds:RS_44._col1=RS_45._col0(Inner),Output:["_col2","_col6","_col7"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_44]
PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_132] (rows=91197860 width=85)
- Conds:RS_147._col0=RS_139._col0(Inner),Output:["_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_133] (rows=91197860 width=85)
+ Conds:RS_148._col0=RS_140._col0(Inner),Output:["_col1","_col2"]
<-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_139]
+ SHUFFLE [RS_140]
PartitionCols:_col0
- Select Operator [SEL_138] (rows=317 width=8)
+ Select Operator [SEL_139] (rows=317 width=8)
Output:["_col0"]
- Filter Operator [FIL_137] (rows=317 width=8)
+ Filter Operator [FIL_138] (rows=317 width=8)
predicate:d_month_seq BETWEEN 1212 AND 1223
TableScan [TS_3] (rows=73049 width=8)
default@date_dim,d1,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_month_seq"]
<-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_147]
+ SHUFFLE [RS_148]
PartitionCols:_col0
- Select Operator [SEL_146] (rows=525329897 width=114)
+ Select Operator [SEL_147] (rows=525329897 width=114)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_145] (rows=525329897 width=114)
+ Filter Operator [FIL_146] (rows=525329897 width=114)
predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_42_d1_d_date_sk_min) AND DynamicValue(RS_42_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_42_d1_d_date_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
TableScan [TS_0] (rows=575995635 width=114)
default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_store_sk","ss_net_profit"]
<-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_144]
- Group By Operator [GBY_143] (rows=1 width=12)
+ BROADCAST [RS_145]
+ Group By Operator [GBY_144] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
<-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_142]
- Group By Operator [GBY_141] (rows=1 width=12)
+ SHUFFLE [RS_143]
+ Group By Operator [GBY_142] (rows=1 width=12)
Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_140] (rows=317 width=4)
+ Select Operator [SEL_141] (rows=317 width=4)
Output:["_col0"]
- Please refer to the previous Select Operator [SEL_138]
+ Please refer to the previous Select Operator [SEL_139]
<-Reducer 10 [SIMPLE_EDGE]
SHUFFLE [RS_45]
PartitionCols:_col0
- Merge Join Operator [MERGEJOIN_135] (rows=556 width=188)
- Conds:RS_160._col2=RS_157._col0(Inner),Output:["_col0","_col1","_col2"]
+ Merge Join Operator [MERGEJOIN_136] (rows=556 width=188)
+ Conds:RS_161._col2=RS_158._col0(Inner),Output:["_col0","_col1","_col2"]
<-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_160]
+ SHUFFLE [RS_161]
PartitionCols:_col2
- Select Operator [SEL_159] (rows=1704 width=188)
+ Select Operator [SEL_160] (rows=1704 width=188)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_158] (rows=1704 width=188)
+ Filter Operator [FIL_159] (rows=1704 width=188)
predicate:s_state is not null
TableScan [TS_6] (rows=1704 width=188)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_county","s_state"]
<-Reducer 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_157]
+ SHUFFLE [RS_158]
PartitionCols:_col0
- Select Operator [SEL_156] (rows=16 width=86)
+ Select Operator [SEL_157] (rows=16 width=86)
Output:["_col0"]
- Filter Operator [FIL_155] (rows=16 width=198)
+ Filter Operator [FIL_156] (rows=16 width=198)
predicate:(rank_window_0 <= 5)
- PTF Operator [PTF_154] (rows=49 width=198)
+ PTF Operator [PTF_155] (rows=49 width=198)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"_col1 DESC NULLS LAST","partition by:":"_col0"}]
- Select Operator [SEL_153] (rows=49 width=198)
+ Select Operator [SEL_154] (rows=49 width=198)
Output:["_col0","_col1"]
<-Reducer 8 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_152]
+ SHUFFLE [RS_153]
PartitionCols:_col0
- Group By Operator [GBY_151] (rows=49 width=198)
+ Group By Operator [GBY_152] (rows=49 width=198)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
<-Reducer 7 [SIMPLE_EDGE]
SHUFFLE [RS_26]
PartitionCols:_col0
Group By Operator [GBY_25] (rows=2989 width=198)
Output:["_col0","_col1"],aggregations:["sum(_col2)"],keys:_col5
- Merge Join Operator [MERGEJOIN_134] (rows=91197860 width=168)
- Conds:RS_21._col1=RS_150._col0(Inner),Output:["_col2","_col5"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col1
- Please refer to the previous Merge Join Operator [MERGEJOIN_132]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_150]
- PartitionCols:_col0
- Select Operator [SEL_149] (rows=1704 width=90)
- Output:["_col0","_col1"]
- Filter Operator [FIL_148] (rows=1704 width=90)
- predicate:s_state is not null
- TableScan [TS_15] (rows=1704 width=90)
- default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_state"]
+ Top N Key Operator [TNK_87] (rows=91197860 width=168)
+ keys:_col5,sort order:+,top n:6
+ Merge Join Operator [MERGEJOIN_135] (rows=91197860 width=168)
+ Conds:RS_21._col1=RS_151._col0(Inner),Output:["_col2","_col5"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col1
+ Please refer to the previous Merge Join Operator [MERGEJOIN_133]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_151]
+ PartitionCols:_col0
+ Select Operator [SEL_150] (rows=1704 width=90)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_149] (rows=1704 width=90)
+ predicate:s_state is not null
+ TableScan [TS_15] (rows=1704 width=90)
+ default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_state"]
http://git-wip-us.apache.org/repos/asf/hive/blob/138b00ca/ql/src/test/results/clientpositive/perf/tez/query70.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/perf/tez/query70.q.out b/ql/src/test/results/clientpositive/perf/tez/query70.q.out
index 6d2528e..8d9d06d 100644
--- a/ql/src/test/results/clientpositive/perf/tez/query70.q.out
+++ b/ql/src/test/results/clientpositive/perf/tez/query70.q.out
@@ -100,25 +100,25 @@ Stage-0
limit:-1
Stage-1
Reducer 7 vectorized
- File Output Operator [FS_168]
- Limit [LIM_167] (rows=100 width=492)
+ File Output Operator [FS_169]
+ Limit [LIM_168] (rows=100 width=492)
Number of rows:100
- Select Operator [SEL_166] (rows=720 width=492)
+ Select Operator [SEL_167] (rows=720 width=492)
Output:["_col0","_col1","_col2","_col3","_col4"]
<-Reducer 6 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_165]
- Select Operator [SEL_164] (rows=720 width=492)
+ SHUFFLE [RS_166]
+ Select Operator [SEL_165] (rows=720 width=492)
Output:["_col0","_col1","_col2","_col3","_col4","_col5"]
- PTF Operator [PTF_163] (rows=720 width=304)
+ PTF Operator [PTF_164] (rows=720 width=304)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"_col2 DESC NULLS LAST","partition by:":"(grouping(_col3, 1) + grouping(_col3, 0)), CASE WHEN ((grouping(_col3, 0) = 0)) THEN (_col0) ELSE (CAST( null AS STRING)) END"}]
- Select Operator [SEL_162] (rows=720 width=304)
+ Select Operator [SEL_163] (rows=720 width=304)
Output:["_col0","_col1","_col2","_col3"]
<-Reducer 5 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_161]
+ SHUFFLE [RS_162]
PartitionCols:(grouping(_col3, 1) + grouping(_col3, 0)), CASE WHEN ((grouping(_col3, 0) = 0)) THEN (_col0) ELSE (CAST( null AS STRING)) END
- Select Operator [SEL_160] (rows=720 width=304)
+ Select Operator [SEL_161] (rows=720 width=304)
Output:["_col0","_col1","_col2","_col3"]
- Group By Operator [GBY_159] (rows=720 width=304)
+ Group By Operator [GBY_160] (rows=720 width=304)
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0, KEY._col1, KEY._col2
<-Reducer 4 [SIMPLE_EDGE]
SHUFFLE [RS_49]
@@ -127,89 +127,91 @@ Stage-0
Output:["_col0","_col1","_col2","_col3"],aggregations:["sum(_col2)"],keys:_col0, _col1, 0L
Select Operator [SEL_46] (rows=29778893 width=207)
Output:["_col0","_col1","_col2"]
- Merge Join Operator [MERGEJOIN_134] (rows=29778893 width=207)
- Conds:RS_43._col7=RS_158._col0(Inner),Output:["_col2","_col6","_col7"]
+ Merge Join Operator [MERGEJOIN_135] (rows=29778893 width=207)
+ Conds:RS_43._col7=RS_159._col0(Inner),Output:["_col2","_col6","_col7"]
<-Reducer 10 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_158]
+ SHUFFLE [RS_159]
PartitionCols:_col0
- Select Operator [SEL_157] (rows=16 width=86)
+ Select Operator [SEL_158] (rows=16 width=86)
Output:["_col0"]
- Filter Operator [FIL_156] (rows=16 width=198)
+ Filter Operator [FIL_157] (rows=16 width=198)
predicate:(rank_window_0 <= 5)
- PTF Operator [PTF_155] (rows=49 width=198)
+ PTF Operator [PTF_156] (rows=49 width=198)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"_col1 DESC NULLS LAST","partition by:":"_col0"}]
- Select Operator [SEL_154] (rows=49 width=198)
+ Select Operator [SEL_155] (rows=49 width=198)
Output:["_col0","_col1"]
<-Reducer 9 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_153]
+ SHUFFLE [RS_154]
PartitionCols:_col0
- Group By Operator [GBY_152] (rows=49 width=198)
+ Group By Operator [GBY_153] (rows=49 width=198)
Output:["_col0","_col1"],aggregations:["sum(VALUE._col0)"],keys:KEY._col0
<-Reducer 8 [SIMPLE_EDGE]
SHUFFLE [RS_26]
PartitionCols:_col0
Group By Operator [GBY_25] (rows=2989 width=198)
Output:["_col0","_col1"],aggregations:["sum(_col2)"],keys:_col5
- Merge Join Operator [MERGEJOIN_133] (rows=91197860 width=168)
- Conds:RS_21._col1=RS_151._col0(Inner),Output:["_col2","_col5"]
- <-Reducer 2 [SIMPLE_EDGE]
- SHUFFLE [RS_21]
- PartitionCols:_col1
- Merge Join Operator [MERGEJOIN_130] (rows=91197860 width=85)
- Conds:RS_145._col0=RS_137._col0(Inner),Output:["_col1","_col2"]
- <-Map 11 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_137]
- PartitionCols:_col0
- Select Operator [SEL_136] (rows=317 width=8)
- Output:["_col0"]
- Filter Operator [FIL_135] (rows=317 width=8)
- predicate:(d_date_sk is not null and d_month_seq BETWEEN 1212 AND 1223)
- TableScan [TS_3] (rows=73049 width=8)
- default@date_dim,d1,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_month_seq"]
- <-Map 1 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_145]
- PartitionCols:_col0
- Select Operator [SEL_144] (rows=525329897 width=114)
- Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_143] (rows=525329897 width=114)
- predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_38_d1_d_date_sk_min) AND DynamicValue(RS_38_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_38_d1_d_date_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
- TableScan [TS_0] (rows=575995635 width=114)
- default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_store_sk","ss_net_profit"]
- <-Reducer 12 [BROADCAST_EDGE] vectorized
- BROADCAST [RS_142]
- Group By Operator [GBY_141] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
- <-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
- SHUFFLE [RS_140]
- Group By Operator [GBY_139] (rows=1 width=12)
- Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
- Select Operator [SEL_138] (rows=317 width=4)
- Output:["_col0"]
- Please refer to the previous Select Operator [SEL_136]
- <-Map 14 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_151]
- PartitionCols:_col0
- Select Operator [SEL_150] (rows=1704 width=90)
- Output:["_col0","_col1"]
- Filter Operator [FIL_149] (rows=1704 width=90)
- predicate:(s_state is not null and s_store_sk is not null)
- TableScan [TS_15] (rows=1704 width=90)
- default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_state"]
+ Top N Key Operator [TNK_85] (rows=91197860 width=168)
+ keys:_col5,sort order:+,top n:6
+ Merge Join Operator [MERGEJOIN_134] (rows=91197860 width=168)
+ Conds:RS_21._col1=RS_152._col0(Inner),Output:["_col2","_col5"]
+ <-Reducer 2 [SIMPLE_EDGE]
+ SHUFFLE [RS_21]
+ PartitionCols:_col1
+ Merge Join Operator [MERGEJOIN_131] (rows=91197860 width=85)
+ Conds:RS_146._col0=RS_138._col0(Inner),Output:["_col1","_col2"]
+ <-Map 11 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_138]
+ PartitionCols:_col0
+ Select Operator [SEL_137] (rows=317 width=8)
+ Output:["_col0"]
+ Filter Operator [FIL_136] (rows=317 width=8)
+ predicate:(d_date_sk is not null and d_month_seq BETWEEN 1212 AND 1223)
+ TableScan [TS_3] (rows=73049 width=8)
+ default@date_dim,d1,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_month_seq"]
+ <-Map 1 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_146]
+ PartitionCols:_col0
+ Select Operator [SEL_145] (rows=525329897 width=114)
+ Output:["_col0","_col1","_col2"]
+ Filter Operator [FIL_144] (rows=525329897 width=114)
+ predicate:((ss_sold_date_sk BETWEEN DynamicValue(RS_38_d1_d_date_sk_min) AND DynamicValue(RS_38_d1_d_date_sk_max) and in_bloom_filter(ss_sold_date_sk, DynamicValue(RS_38_d1_d_date_sk_bloom_filter))) and ss_sold_date_sk is not null and ss_store_sk is not null)
+ TableScan [TS_0] (rows=575995635 width=114)
+ default@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE,Output:["ss_sold_date_sk","ss_store_sk","ss_net_profit"]
+ <-Reducer 12 [BROADCAST_EDGE] vectorized
+ BROADCAST [RS_143]
+ Group By Operator [GBY_142] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(VALUE._col0)","max(VALUE._col1)","bloom_filter(VALUE._col2, expectedEntries=1000000)"]
+ <-Map 11 [CUSTOM_SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_141]
+ Group By Operator [GBY_140] (rows=1 width=12)
+ Output:["_col0","_col1","_col2"],aggregations:["min(_col0)","max(_col0)","bloom_filter(_col0, expectedEntries=1000000)"]
+ Select Operator [SEL_139] (rows=317 width=4)
+ Output:["_col0"]
+ Please refer to the previous Select Operator [SEL_137]
+ <-Map 14 [SIMPLE_EDGE] vectorized
+ SHUFFLE [RS_152]
+ PartitionCols:_col0
+ Select Operator [SEL_151] (rows=1704 width=90)
+ Output:["_col0","_col1"]
+ Filter Operator [FIL_150] (rows=1704 width=90)
+ predicate:(s_state is not null and s_store_sk is not null)
+ TableScan [TS_15] (rows=1704 width=90)
+ default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_state"]
<-Reducer 3 [SIMPLE_EDGE]
SHUFFLE [RS_43]
PartitionCols:_col7
- Merge Join Operator [MERGEJOIN_131] (rows=91197860 width=266)
- Conds:RS_40._col1=RS_148._col0(Inner),Output:["_col2","_col6","_col7"]
+ Merge Join Operator [MERGEJOIN_132] (rows=91197860 width=266)
+ Conds:RS_40._col1=RS_149._col0(Inner),Output:["_col2","_col6","_col7"]
<-Reducer 2 [SIMPLE_EDGE]
SHUFFLE [RS_40]
PartitionCols:_col1
- Please refer to the previous Merge Join Operator [MERGEJOIN_130]
+ Please refer to the previous Merge Join Operator [MERGEJOIN_131]
<-Map 13 [SIMPLE_EDGE] vectorized
- SHUFFLE [RS_148]
+ SHUFFLE [RS_149]
PartitionCols:_col0
- Select Operator [SEL_147] (rows=1704 width=188)
+ Select Operator [SEL_148] (rows=1704 width=188)
Output:["_col0","_col1","_col2"]
- Filter Operator [FIL_146] (rows=1704 width=188)
+ Filter Operator [FIL_147] (rows=1704 width=188)
predicate:(s_state is not null and s_store_sk is not null)
TableScan [TS_6] (rows=1704 width=188)
default@store,store,Tbl:COMPLETE,Col:COMPLETE,Output:["s_store_sk","s_county","s_state"]