You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by jc...@apache.org on 2015/10/08 14:34:04 UTC
[2/2] hive git commit: HIVE-9695: Redundant filter operator in
reducer Vertex when CBO is disabled (Jesus Camacho Rodriguez,
reviewed by Ashutosh Chauhan)
HIVE-9695: Redundant filter operator in reducer Vertex when CBO is disabled (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/48c420e5
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/48c420e5
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/48c420e5
Branch: refs/heads/master
Commit: 48c420e5516156761f584797210e496e54f3f5d5
Parents: 768db67
Author: Jesus Camacho Rodriguez <jc...@apache.org>
Authored: Tue Oct 6 17:14:24 2015 +0100
Committer: Jesus Camacho Rodriguez <jc...@apache.org>
Committed: Thu Oct 8 13:33:10 2015 +0100
----------------------------------------------------------------------
.../hadoop/hive/ql/ppd/OpProcFactory.java | 10 +-
ql/src/test/queries/clientpositive/join44.q | 12 +++
.../clientpositive/dynamic_rdd_cache.q.out | 28 +++---
ql/src/test/results/clientpositive/join44.q.out | 88 ++++++++++++++++++
.../join_cond_pushdown_unqual1.q.out | 12 +--
.../join_cond_pushdown_unqual3.q.out | 18 ++--
.../join_cond_pushdown_unqual4.q.out | 2 +-
.../results/clientpositive/pointlookup2.q.out | 12 +--
.../results/clientpositive/pointlookup3.q.out | 12 +--
.../spark/dynamic_rdd_cache.q.out | 28 +++---
.../spark/join_cond_pushdown_unqual1.q.out | 12 +--
.../spark/join_cond_pushdown_unqual3.q.out | 18 ++--
.../spark/join_cond_pushdown_unqual4.q.out | 2 +-
.../tez/dynamic_partition_pruning.q.out | 4 +-
.../tez/dynamic_partition_pruning_2.q.out | 54 +++++------
.../clientpositive/tez/explainuser_1.q.out | 96 ++++++++++----------
.../tez/vector_mr_diff_schema_alias.q.out | 2 +-
.../vectorized_dynamic_partition_pruning.q.out | 4 +-
.../vector_mr_diff_schema_alias.q.out | 2 +-
19 files changed, 258 insertions(+), 158 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java b/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java
index dbd021b..8566374 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java
@@ -424,15 +424,15 @@ public final class OpProcFactory {
}
return null;
}
+ logExpr(nd, ewi);
+ owi.putPrunedPreds((Operator<? extends OperatorDesc>) nd, ewi);
if (HiveConf.getBoolVar(owi.getParseContext().getConf(),
HiveConf.ConfVars.HIVEPPDREMOVEDUPLICATEFILTERS)) {
// add this filter for deletion, if it does not have non-final candidates
- if (ewi.getNonFinalCandidates().values().isEmpty()) {
- owi.addCandidateFilterOp((FilterOperator)op);
- }
+ owi.addCandidateFilterOp((FilterOperator)op);
+ Map<String, List<ExprNodeDesc>> residual = ewi.getResidualPredicates(true);
+ createFilter(op, residual, owi);
}
- logExpr(nd, ewi);
- owi.putPrunedPreds((Operator<? extends OperatorDesc>) nd, ewi);
}
// merge it with children predicates
boolean hasUnpushedPredicates = mergeWithChildrenPred(nd, owi, ewi, null);
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/queries/clientpositive/join44.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/join44.q b/ql/src/test/queries/clientpositive/join44.q
new file mode 100644
index 0000000..0111079
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/join44.q
@@ -0,0 +1,12 @@
+set hive.cbo.enable=false;
+
+-- SORT_QUERY_RESULTS
+
+CREATE TABLE mytable(val1 INT, val2 INT, val3 INT);
+
+EXPLAIN
+SELECT *
+FROM mytable src1, mytable src2
+WHERE src1.val1=src2.val1
+ AND src1.val2 between 2450816 and 2451500
+ AND src2.val2 between 2450816 and 2451500;
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out b/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out
index eeb5847..501a86f 100644
--- a/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out
+++ b/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out
@@ -1030,7 +1030,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col7, _col11, _col12, _col16
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
- predicate: ((_col1 = _col7) and (_col3 = _col11) and (_col0 = _col16)) (type: boolean)
+ predicate: (((_col1 = _col7) and (_col3 = _col11)) and (_col0 = _col16)) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col12 (type: string), _col11 (type: int), _col7 (type: int), 3 (type: int), _col2 (type: int)
@@ -1067,15 +1067,15 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: double), _col5 (type: double)
- outputColumnNames: _col1, _col2, _col3, _col4, _col5
+ expressions: _col1 (type: int), _col2 (type: int), _col4 (type: double), _col5 (type: double)
+ outputColumnNames: _col1, _col2, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: (CASE (_col5) WHEN (0) THEN (0) ELSE ((_col4 / _col5)) END > 1) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col5 (type: double), CASE (_col5) WHEN (0) THEN (null) ELSE ((_col4 / _col5)) END (type: double)
- outputColumnNames: _col1, _col2, _col3, _col5, _col6
+ expressions: _col1 (type: int), _col2 (type: int), _col5 (type: double), CASE (_col5) WHEN (0) THEN (null) ELSE ((_col4 / _col5)) END (type: double)
+ outputColumnNames: _col1, _col2, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: true
@@ -1093,14 +1093,14 @@ STAGE PLANS:
sort order: ++
Map-reduce partition columns: _col2 (type: int), _col1 (type: int)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- value expressions: _col3 (type: int), _col5 (type: double), _col6 (type: double)
+ value expressions: _col5 (type: double), _col6 (type: double)
TableScan
Reduce Output Operator
key expressions: _col2 (type: int), _col1 (type: int)
sort order: ++
Map-reduce partition columns: _col2 (type: int), _col1 (type: int)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- value expressions: _col3 (type: int), _col5 (type: double), _col6 (type: double)
+ value expressions: _col5 (type: double), _col6 (type: double)
Reduce Operator Tree:
Join Operator
condition map:
@@ -1108,10 +1108,10 @@ STAGE PLANS:
keys:
0 _col2 (type: int), _col1 (type: int)
1 _col2 (type: int), _col1 (type: int)
- outputColumnNames: _col1, _col2, _col3, _col5, _col6, _col8, _col9, _col10, _col12, _col13
+ outputColumnNames: _col1, _col2, _col5, _col6, _col8, _col9, _col12, _col13
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
- predicate: ((_col2 = _col9) and (_col1 = _col8) and (_col3 = 3) and (_col10 = 4)) (type: boolean)
+ predicate: ((_col2 = _col9) and (_col1 = _col8)) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col1 (type: int), _col2 (type: int), _col5 (type: double), _col6 (type: double), _col8 (type: int), _col9 (type: int), _col12 (type: double), _col13 (type: double)
@@ -1257,7 +1257,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col7, _col11, _col12, _col16
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
- predicate: ((_col1 = _col7) and (_col3 = _col11) and (_col0 = _col16)) (type: boolean)
+ predicate: (((_col1 = _col7) and (_col3 = _col11)) and (_col0 = _col16)) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col12 (type: string), _col11 (type: int), _col7 (type: int), 4 (type: int), _col2 (type: int)
@@ -1294,15 +1294,15 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: double), _col5 (type: double)
- outputColumnNames: _col1, _col2, _col3, _col4, _col5
+ expressions: _col1 (type: int), _col2 (type: int), _col4 (type: double), _col5 (type: double)
+ outputColumnNames: _col1, _col2, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: (CASE (_col5) WHEN (0) THEN (0) ELSE ((_col4 / _col5)) END > 1) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col5 (type: double), CASE (_col5) WHEN (0) THEN (null) ELSE ((_col4 / _col5)) END (type: double)
- outputColumnNames: _col1, _col2, _col3, _col5, _col6
+ expressions: _col1 (type: int), _col2 (type: int), _col5 (type: double), CASE (_col5) WHEN (0) THEN (null) ELSE ((_col4 / _col5)) END (type: double)
+ outputColumnNames: _col1, _col2, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
File Output Operator
compressed: true
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/join44.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/join44.q.out b/ql/src/test/results/clientpositive/join44.q.out
new file mode 100644
index 0000000..8598701
--- /dev/null
+++ b/ql/src/test/results/clientpositive/join44.q.out
@@ -0,0 +1,88 @@
+PREHOOK: query: -- SORT_QUERY_RESULTS
+
+CREATE TABLE mytable(val1 INT, val2 INT, val3 INT)
+PREHOOK: type: CREATETABLE
+PREHOOK: Output: database:default
+PREHOOK: Output: default@mytable
+POSTHOOK: query: -- SORT_QUERY_RESULTS
+
+CREATE TABLE mytable(val1 INT, val2 INT, val3 INT)
+POSTHOOK: type: CREATETABLE
+POSTHOOK: Output: database:default
+POSTHOOK: Output: default@mytable
+PREHOOK: query: EXPLAIN
+SELECT *
+FROM mytable src1, mytable src2
+WHERE src1.val1=src2.val1
+ AND src1.val2 between 2450816 and 2451500
+ AND src2.val2 between 2450816 and 2451500
+PREHOOK: type: QUERY
+POSTHOOK: query: EXPLAIN
+SELECT *
+FROM mytable src1, mytable src2
+WHERE src1.val1=src2.val1
+ AND src1.val2 between 2450816 and 2451500
+ AND src2.val2 between 2450816 and 2451500
+POSTHOOK: type: QUERY
+STAGE DEPENDENCIES:
+ Stage-1 is a root stage
+ Stage-0 depends on stages: Stage-1
+
+STAGE PLANS:
+ Stage: Stage-1
+ Map Reduce
+ Map Operator Tree:
+ TableScan
+ alias: src1
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Filter Operator
+ predicate: (val1 is not null and val2 BETWEEN 2450816 AND 2451500) (type: boolean)
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Reduce Output Operator
+ key expressions: val1 (type: int)
+ sort order: +
+ Map-reduce partition columns: val1 (type: int)
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ value expressions: val2 (type: int), val3 (type: int)
+ TableScan
+ alias: src2
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Filter Operator
+ predicate: (val1 is not null and val2 BETWEEN 2450816 AND 2451500) (type: boolean)
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Reduce Output Operator
+ key expressions: val1 (type: int)
+ sort order: +
+ Map-reduce partition columns: val1 (type: int)
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ value expressions: val2 (type: int), val3 (type: int)
+ Reduce Operator Tree:
+ Join Operator
+ condition map:
+ Inner Join 0 to 1
+ keys:
+ 0 val1 (type: int)
+ 1 val1 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col6, _col7, _col8
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Filter Operator
+ predicate: (_col0 = _col6) (type: boolean)
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ Select Operator
+ expressions: _col0 (type: int), _col1 (type: int), _col2 (type: int), _col6 (type: int), _col7 (type: int), _col8 (type: int)
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ File Output Operator
+ compressed: false
+ Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
+ table:
+ input format: org.apache.hadoop.mapred.TextInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
+
+ Stage: Stage-0
+ Fetch Operator
+ limit: -1
+ Processor Tree:
+ ListSink
+
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out b/ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out
index 597b75f..c1c2105 100644
--- a/ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out
+++ b/ql/src/test/results/clientpositive/join_cond_pushdown_unqual1.q.out
@@ -255,8 +255,8 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20
Statistics: Num rows: 28 Data size: 3461 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((_col12 + _col0) = _col0) and _col13 is not null) (type: boolean)
- Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((_col12 + _col0) = _col0) (type: boolean)
+ Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -272,7 +272,7 @@ STAGE PLANS:
key expressions: _col13 (type: string)
sort order: +
Map-reduce partition columns: _col13 (type: string)
- Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string)
TableScan
alias: p3
@@ -294,14 +294,14 @@ STAGE PLANS:
0 _col13 (type: string)
1 p3_name (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32
- Statistics: Num rows: 7 Data size: 951 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 15 Data size: 1903 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23, _col24, _col25, _col26
- Statistics: Num rows: 7 Data size: 951 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 15 Data size: 1903 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 7 Data size: 951 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 15 Data size: 1903 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out b/ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out
index 9b2da59..b0258b8 100644
--- a/ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out
+++ b/ql/src/test/results/clientpositive/join_cond_pushdown_unqual3.q.out
@@ -118,7 +118,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32
Statistics: Num rows: 28 Data size: 3460 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((_col1 = _col13) and (_col13 = _col25)) (type: boolean)
+ predicate: ((_col13 = _col25) and (_col1 = _col13)) (type: boolean)
Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string)
@@ -202,7 +202,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32
Statistics: Num rows: 28 Data size: 3460 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((_col13 = _col1) and (_col25 = _col13)) (type: boolean)
+ predicate: ((_col25 = _col13) and (_col13 = _col1)) (type: boolean)
Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string)
@@ -267,8 +267,8 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20
Statistics: Num rows: 28 Data size: 3461 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((_col12 + _col0) = _col0) and _col13 is not null) (type: boolean)
- Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((_col12 + _col0) = _col0) (type: boolean)
+ Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
table:
@@ -284,7 +284,7 @@ STAGE PLANS:
key expressions: _col13 (type: string)
sort order: +
Map-reduce partition columns: _col13 (type: string)
- Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string)
TableScan
alias: p3
@@ -306,17 +306,17 @@ STAGE PLANS:
0 _col13 (type: string)
1 p3_name (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32
- Statistics: Num rows: 7 Data size: 951 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 15 Data size: 1903 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((_col12 + _col0) = _col0) and (_col25 = _col13)) (type: boolean)
- Statistics: Num rows: 1 Data size: 135 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 380 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23, _col24, _col25, _col26
- Statistics: Num rows: 1 Data size: 135 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 380 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 135 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 380 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out b/ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out
index 6ff13e4..26db67e 100644
--- a/ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out
+++ b/ql/src/test/results/clientpositive/join_cond_pushdown_unqual4.q.out
@@ -282,7 +282,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32, _col36, _col37, _col38, _col39, _col40, _col41, _col42, _col43, _col44
Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((_col13 = _col25) and (_col0 = _col36) and (_col0 = _col12)) (type: boolean)
+ predicate: (((_col13 = _col25) and (_col0 = _col36)) and (_col0 = _col12)) (type: boolean)
Statistics: Num rows: 1 Data size: 123 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string), _col36 (type: int), _col37 (type: string), _col38 (type: string), _col39 (type: string), _col40 (type: string), _col41 (type: int), _col42 (type: string), _col43 (type: double), _col44 (type: string)
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/pointlookup2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/pointlookup2.q.out b/ql/src/test/results/clientpositive/pointlookup2.q.out
index 55edd90..700fbde 100644
--- a/ql/src/test/results/clientpositive/pointlookup2.q.out
+++ b/ql/src/test/results/clientpositive/pointlookup2.q.out
@@ -1128,12 +1128,12 @@ STAGE PLANS:
Statistics: Num rows: 44 Data size: 352 Basic stats: COMPLETE Column stats: NONE
Filter Operator
isSamplingPred: false
- predicate: ((_col2) IN ('2000-04-08', '2000-04-09') and (struct(_col7,_col2)) IN (const struct(1,'2000-04-08'), const struct(2,'2000-04-09'))) (type: boolean)
- Statistics: Num rows: 11 Data size: 88 Basic stats: COMPLETE Column stats: NONE
+ predicate: (struct(_col7,_col2)) IN (const struct(1,'2000-04-08'), const struct(2,'2000-04-09')) (type: boolean)
+ Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col6 (type: string), _col7 (type: int), _col8 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 11 Data size: 88 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
@@ -1160,7 +1160,7 @@ STAGE PLANS:
Reduce Output Operator
key expressions: _col4 (type: int), _col5 (type: string), _col2 (type: string)
sort order: +++
- Statistics: Num rows: 11 Data size: 88 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
tag: -1
value expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string)
auto parallelism: false
@@ -1194,13 +1194,13 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: int), VALUE._col1 (type: string), KEY.reducesinkkey2 (type: string), VALUE._col2 (type: string), KEY.reducesinkkey0 (type: int), KEY.reducesinkkey1 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
- Statistics: Num rows: 11 Data size: 88 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 11 Data size: 88 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 22 Data size: 176 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/pointlookup3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/pointlookup3.q.out b/ql/src/test/results/clientpositive/pointlookup3.q.out
index 4cfb97e..60a276b 100644
--- a/ql/src/test/results/clientpositive/pointlookup3.q.out
+++ b/ql/src/test/results/clientpositive/pointlookup3.q.out
@@ -1289,12 +1289,12 @@ STAGE PLANS:
Statistics: Num rows: 66 Data size: 528 Basic stats: COMPLETE Column stats: NONE
Filter Operator
isSamplingPred: false
- predicate: ((_col2) IN ('2000-04-08', '2000-04-09') and (struct(_col7,_col2)) IN (const struct(1,'2000-04-08'), const struct(2,'2000-04-09'))) (type: boolean)
- Statistics: Num rows: 16 Data size: 128 Basic stats: COMPLETE Column stats: NONE
+ predicate: (struct(_col7,_col2)) IN (const struct(1,'2000-04-08'), const struct(2,'2000-04-09')) (type: boolean)
+ Statistics: Num rows: 33 Data size: 264 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col7 (type: int), _col8 (type: string), _col9 (type: string), _col10 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
- Statistics: Num rows: 16 Data size: 128 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 33 Data size: 264 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
@@ -1321,7 +1321,7 @@ STAGE PLANS:
Reduce Output Operator
key expressions: _col4 (type: int), _col5 (type: string), _col2 (type: string)
sort order: +++
- Statistics: Num rows: 16 Data size: 128 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 33 Data size: 264 Basic stats: COMPLETE Column stats: NONE
tag: -1
value expressions: _col0 (type: int), _col1 (type: string), _col3 (type: string), _col6 (type: string), _col7 (type: string)
auto parallelism: false
@@ -1355,13 +1355,13 @@ STAGE PLANS:
Select Operator
expressions: VALUE._col0 (type: int), VALUE._col1 (type: string), KEY.reducesinkkey2 (type: string), VALUE._col2 (type: string), KEY.reducesinkkey0 (type: int), KEY.reducesinkkey1 (type: string), VALUE._col3 (type: string), VALUE._col4 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7
- Statistics: Num rows: 16 Data size: 128 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 33 Data size: 264 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
GlobalTableId: 0
#### A masked pattern was here ####
NumFilesPerFileSink: 1
- Statistics: Num rows: 16 Data size: 128 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 33 Data size: 264 Basic stats: COMPLETE Column stats: NONE
#### A masked pattern was here ####
table:
input format: org.apache.hadoop.mapred.TextInputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out b/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out
index 7e9a0f3..730a31f 100644
--- a/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out
+++ b/ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out
@@ -833,7 +833,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col7, _col11, _col12, _col16
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
- predicate: ((_col1 = _col7) and (_col3 = _col11) and (_col0 = _col16)) (type: boolean)
+ predicate: (((_col1 = _col7) and (_col3 = _col11)) and (_col0 = _col16)) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col12 (type: string), _col11 (type: int), _col7 (type: int), 4 (type: int), _col2 (type: int)
@@ -860,22 +860,22 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: double), _col5 (type: double)
- outputColumnNames: _col1, _col2, _col3, _col4, _col5
+ expressions: _col1 (type: int), _col2 (type: int), _col4 (type: double), _col5 (type: double)
+ outputColumnNames: _col1, _col2, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: (CASE (_col5) WHEN (0) THEN (0) ELSE ((_col4 / _col5)) END > 1) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col5 (type: double), CASE (_col5) WHEN (0) THEN (null) ELSE ((_col4 / _col5)) END (type: double)
- outputColumnNames: _col1, _col2, _col3, _col5, _col6
+ expressions: _col1 (type: int), _col2 (type: int), _col5 (type: double), CASE (_col5) WHEN (0) THEN (null) ELSE ((_col4 / _col5)) END (type: double)
+ outputColumnNames: _col1, _col2, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Reduce Output Operator
key expressions: _col2 (type: int), _col1 (type: int)
sort order: ++
Map-reduce partition columns: _col2 (type: int), _col1 (type: int)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- value expressions: _col3 (type: int), _col5 (type: double), _col6 (type: double)
+ value expressions: _col5 (type: double), _col6 (type: double)
Reducer 4
Reduce Operator Tree:
Join Operator
@@ -887,7 +887,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col7, _col11, _col12, _col16
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
- predicate: ((_col1 = _col7) and (_col3 = _col11) and (_col0 = _col16)) (type: boolean)
+ predicate: (((_col1 = _col7) and (_col3 = _col11)) and (_col0 = _col16)) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col12 (type: string), _col11 (type: int), _col7 (type: int), 3 (type: int), _col2 (type: int)
@@ -914,22 +914,22 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col4 (type: double), _col5 (type: double)
- outputColumnNames: _col1, _col2, _col3, _col4, _col5
+ expressions: _col1 (type: int), _col2 (type: int), _col4 (type: double), _col5 (type: double)
+ outputColumnNames: _col1, _col2, _col4, _col5
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
predicate: (CASE (_col5) WHEN (0) THEN (0) ELSE ((_col4 / _col5)) END > 1) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: _col1 (type: int), _col2 (type: int), _col3 (type: int), _col5 (type: double), CASE (_col5) WHEN (0) THEN (null) ELSE ((_col4 / _col5)) END (type: double)
- outputColumnNames: _col1, _col2, _col3, _col5, _col6
+ expressions: _col1 (type: int), _col2 (type: int), _col5 (type: double), CASE (_col5) WHEN (0) THEN (null) ELSE ((_col4 / _col5)) END (type: double)
+ outputColumnNames: _col1, _col2, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Reduce Output Operator
key expressions: _col2 (type: int), _col1 (type: int)
sort order: ++
Map-reduce partition columns: _col2 (type: int), _col1 (type: int)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- value expressions: _col3 (type: int), _col5 (type: double), _col6 (type: double)
+ value expressions: _col5 (type: double), _col6 (type: double)
Reducer 6
Reduce Operator Tree:
Join Operator
@@ -938,10 +938,10 @@ STAGE PLANS:
keys:
0 _col2 (type: int), _col1 (type: int)
1 _col2 (type: int), _col1 (type: int)
- outputColumnNames: _col1, _col2, _col3, _col5, _col6, _col8, _col9, _col10, _col12, _col13
+ outputColumnNames: _col1, _col2, _col5, _col6, _col8, _col9, _col12, _col13
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Filter Operator
- predicate: ((_col2 = _col9) and (_col1 = _col8) and (_col3 = 3) and (_col10 = 4)) (type: boolean)
+ predicate: ((_col2 = _col9) and (_col1 = _col8)) (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
expressions: _col1 (type: int), _col2 (type: int), _col5 (type: double), _col6 (type: double), _col8 (type: int), _col9 (type: int), _col12 (type: double), _col13 (type: double)
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual1.q.out b/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual1.q.out
index 43955fa..5a77830 100644
--- a/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual1.q.out
+++ b/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual1.q.out
@@ -297,13 +297,13 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20
Statistics: Num rows: 28 Data size: 3461 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((_col12 + _col0) = _col0) and _col13 is not null) (type: boolean)
- Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((_col12 + _col0) = _col0) (type: boolean)
+ Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col13 (type: string)
sort order: +
Map-reduce partition columns: _col13 (type: string)
- Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string)
Reducer 3
Reduce Operator Tree:
@@ -314,14 +314,14 @@ STAGE PLANS:
0 _col13 (type: string)
1 p3_name (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32
- Statistics: Num rows: 7 Data size: 951 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 15 Data size: 1903 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23, _col24, _col25, _col26
- Statistics: Num rows: 7 Data size: 951 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 15 Data size: 1903 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 7 Data size: 951 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 15 Data size: 1903 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual3.q.out b/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual3.q.out
index e0df992..180787b 100644
--- a/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual3.q.out
+++ b/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual3.q.out
@@ -128,7 +128,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32
Statistics: Num rows: 28 Data size: 3460 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((_col1 = _col13) and (_col13 = _col25)) (type: boolean)
+ predicate: ((_col13 = _col25) and (_col1 = _col13)) (type: boolean)
Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string)
@@ -222,7 +222,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32
Statistics: Num rows: 28 Data size: 3460 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((_col13 = _col1) and (_col25 = _col13)) (type: boolean)
+ predicate: ((_col25 = _col13) and (_col13 = _col1)) (type: boolean)
Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string)
@@ -309,13 +309,13 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20
Statistics: Num rows: 28 Data size: 3461 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: (((_col12 + _col0) = _col0) and _col13 is not null) (type: boolean)
- Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
+ predicate: ((_col12 + _col0) = _col0) (type: boolean)
+ Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col13 (type: string)
sort order: +
Map-reduce partition columns: _col13 (type: string)
- Statistics: Num rows: 7 Data size: 865 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
value expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string)
Reducer 3
Reduce Operator Tree:
@@ -326,17 +326,17 @@ STAGE PLANS:
0 _col13 (type: string)
1 p3_name (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32
- Statistics: Num rows: 7 Data size: 951 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 15 Data size: 1903 Basic stats: COMPLETE Column stats: NONE
Filter Operator
predicate: (((_col12 + _col0) = _col0) and (_col25 = _col13)) (type: boolean)
- Statistics: Num rows: 1 Data size: 135 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 380 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string)
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col21, _col22, _col23, _col24, _col25, _col26
- Statistics: Num rows: 1 Data size: 135 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 380 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 135 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 3 Data size: 380 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual4.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual4.q.out b/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual4.q.out
index b30f4f4..e16884c 100644
--- a/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual4.q.out
+++ b/ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual4.q.out
@@ -286,7 +286,7 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col12, _col13, _col14, _col15, _col16, _col17, _col18, _col19, _col20, _col24, _col25, _col26, _col27, _col28, _col29, _col30, _col31, _col32, _col36, _col37, _col38, _col39, _col40, _col41, _col42, _col43, _col44
Statistics: Num rows: 14 Data size: 1730 Basic stats: COMPLETE Column stats: NONE
Filter Operator
- predicate: ((_col13 = _col25) and (_col0 = _col36) and (_col0 = _col12)) (type: boolean)
+ predicate: (((_col13 = _col25) and (_col0 = _col36)) and (_col0 = _col12)) (type: boolean)
Statistics: Num rows: 1 Data size: 123 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col0 (type: int), _col1 (type: string), _col2 (type: string), _col3 (type: string), _col4 (type: string), _col5 (type: int), _col6 (type: string), _col7 (type: double), _col8 (type: string), _col12 (type: int), _col13 (type: string), _col14 (type: string), _col15 (type: string), _col16 (type: string), _col17 (type: int), _col18 (type: string), _col19 (type: double), _col20 (type: string), _col24 (type: int), _col25 (type: string), _col26 (type: string), _col27 (type: string), _col28 (type: string), _col29 (type: int), _col30 (type: string), _col31 (type: double), _col32 (type: string), _col36 (type: int), _col37 (type: string), _col38 (type: string), _col39 (type: string), _col40 (type: string), _col41 (type: int), _col42 (type: string), _col43 (type: double), _col44 (type: string)
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out b/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out
index bbc2e16..b6fa1ac 100644
--- a/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out
+++ b/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning.q.out
@@ -1902,7 +1902,7 @@ POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11
POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
#### A masked pattern was here ####
1000
-Warning: Shuffle Join MERGEJOIN[13][tables = [srcpart, srcpart_date_hour]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[14][tables = [srcpart, srcpart_date_hour]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: -- non-equi join
EXPLAIN select count(*) from srcpart, srcpart_date_hour where (srcpart_date_hour.`date` = '2008-04-08' and srcpart_date_hour.hour = 11) and (srcpart.ds = srcpart_date_hour.ds or srcpart.hr = srcpart_date_hour.hr)
PREHOOK: type: QUERY
@@ -1988,7 +1988,7 @@ STAGE PLANS:
Processor Tree:
ListSink
-Warning: Shuffle Join MERGEJOIN[13][tables = [srcpart, srcpart_date_hour]] in Stage 'Reducer 2' is a cross product
+Warning: Shuffle Join MERGEJOIN[14][tables = [srcpart, srcpart_date_hour]] in Stage 'Reducer 2' is a cross product
PREHOOK: query: select count(*) from srcpart, srcpart_date_hour where (srcpart_date_hour.`date` = '2008-04-08' and srcpart_date_hour.hour = 11) and (srcpart.ds = srcpart_date_hour.ds or srcpart.hr = srcpart_date_hour.hr)
PREHOOK: type: QUERY
PREHOOK: Input: default@srcpart
http://git-wip-us.apache.org/repos/asf/hive/blob/48c420e5/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out b/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out
index c8e9da4..430d5ad 100644
--- a/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out
+++ b/ql/src/test/results/clientpositive/tez/dynamic_partition_pruning_2.q.out
@@ -178,23 +178,23 @@ STAGE PLANS:
Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
HybridGraceHashJoin: true
Filter Operator
- predicate: ((_col1 = _col5) and (_col6) IN ('foo', 'bar')) (type: boolean)
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ predicate: (_col1 = _col5) (type: boolean)
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col6 (type: string), _col0 (type: decimal(10,0))
outputColumnNames: _col6, _col0
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), sum(_col0)
keys: _col6 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Map 4
Map Operator Tree:
@@ -233,21 +233,21 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Reducer 3
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint), VALUE._col1 (type: decimal(20,0))
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -340,23 +340,23 @@ STAGE PLANS:
Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
HybridGraceHashJoin: true
Filter Operator
- predicate: ((_col1 = _col5) and (_col6) IN ('foo', 'bar')) (type: boolean)
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ predicate: (_col1 = _col5) (type: boolean)
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col6 (type: string), _col0 (type: decimal(10,0))
outputColumnNames: _col6, _col0
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), sum(_col0)
keys: _col6 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Map 4
Map Operator Tree:
@@ -380,21 +380,21 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Reducer 3
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint), VALUE._col1 (type: decimal(20,0))
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
@@ -683,23 +683,23 @@ STAGE PLANS:
Statistics: Num rows: 9 Data size: 29 Basic stats: COMPLETE Column stats: NONE
HybridGraceHashJoin: true
Filter Operator
- predicate: ((_col1 = _col5) and (_col6) IN ('foo', 'bar')) (type: boolean)
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ predicate: (_col1 = _col5) (type: boolean)
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Select Operator
expressions: _col6 (type: string), _col0 (type: decimal(10,0))
outputColumnNames: _col6, _col0
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(), sum(_col0)
keys: _col6 (type: string)
mode: hash
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 4 Data size: 12 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Map 4
Map Operator Tree:
@@ -738,21 +738,21 @@ STAGE PLANS:
keys: KEY._col0 (type: string)
mode: mergepartial
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
Reduce Output Operator
key expressions: _col0 (type: string)
sort order: +
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Reducer 3
Reduce Operator Tree:
Select Operator
expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: bigint), VALUE._col1 (type: decimal(20,0))
outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
File Output Operator
compressed: false
- Statistics: Num rows: 1 Data size: 3 Basic stats: COMPLETE Column stats: NONE
+ Statistics: Num rows: 2 Data size: 6 Basic stats: COMPLETE Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat