You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by ha...@apache.org on 2017/04/08 03:58:15 UTC
[14/14] hive git commit: HIVE-14564 : Column Pruning generates out of
order columns in SelectOperator which cause ArrayIndexOutOfBoundsException.
(Zhihai Xu via Ashutosh Chauhan)
HIVE-14564 : Column Pruning generates out of order columns in SelectOperator which cause ArrayIndexOutOfBoundsException. (Zhihai Xu via Ashutosh Chauhan)
Signed-off-by: Ashutosh Chauhan <ha...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/78d8b337
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/78d8b337
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/78d8b337
Branch: refs/heads/master
Commit: 78d8b337778993a2e24abb18cfd70e0976bd93ad
Parents: ccc96b4
Author: Zhihai Xu <zh...@gmail.com>
Authored: Wed Aug 17 17:41:00 2016 -0700
Committer: Ashutosh Chauhan <ha...@apache.org>
Committed: Fri Apr 7 20:56:44 2017 -0700
----------------------------------------------------------------------
.../ql/optimizer/ColumnPrunerProcFactory.java | 15 +-
.../clientpositive/select_column_pruning.q | 4 +
.../clientpositive/autoColumnStats_8.q.out | 4 +-
.../clientpositive/columnstats_partlvl.q.out | 4 +-
.../clientpositive/columnstats_partlvl_dp.q.out | 16 +-
.../clientpositive/columnstats_tbllvl.q.out | 16 +-
.../results/clientpositive/complex_alias.q.out | 8 +-
.../clientpositive/constant_prop_3.q.out | 4 +-
.../clientpositive/correlationoptimizer13.q.out | 8 +-
.../results/clientpositive/decimal_udf.q.out | 12 +-
.../display_colstats_tbllvl.q.out | 8 +-
.../distinct_windowing_no_cbo.q.out | 4 +-
.../results/clientpositive/druid_basic2.q.out | 8 +-
.../clientpositive/dynamic_rdd_cache.q.out | 52 +-
.../results/clientpositive/except_all.q.out | 16 +-
.../test/results/clientpositive/groupby9.q.out | 8 +-
.../clientpositive/groupby_join_pushdown.q.out | 26 +-
.../clientpositive/groupby_position.q.out | 26 +-
.../test/results/clientpositive/having2.q.out | 86 +-
.../clientpositive/index_auto_update.q.out | 4 +-
.../limit_pushdown_negative.q.out | 4 +-
.../clientpositive/llap/bucket_groupby.q.out | 4 +-
.../llap/column_table_stats.q.out | 12 +-
.../llap/column_table_stats_orc.q.out | 8 +-
.../llap/dynamic_partition_pruning_2.q.out | 72 +-
.../clientpositive/llap/explainuser_1.q.out | 20 +-
.../clientpositive/llap/explainuser_2.q.out | 382 +++---
.../clientpositive/llap/lateral_view.q.out | 4 +-
.../clientpositive/llap/limit_pushdown.q.out | 4 +-
.../clientpositive/llap/limit_pushdown3.q.out | 4 +-
.../clientpositive/llap/llap_stats.q.out | 4 +-
.../llap/multi_insert_lateral_view.q.out | 92 +-
.../llap/offset_limit_ppd_optimizer.q.out | 4 +-
.../test/results/clientpositive/llap/ptf.q.out | 12 +-
.../clientpositive/llap/subquery_in.q.out | 76 +-
.../clientpositive/llap/subquery_multi.q.out | 200 ++--
.../clientpositive/llap/subquery_notin.q.out | 236 ++--
.../clientpositive/llap/subquery_scalar.q.out | 208 ++--
.../clientpositive/llap/subquery_select.q.out | 100 +-
.../llap/vector_decimal_aggregate.q.out | 12 +-
.../llap/vector_decimal_round_2.q.out | 6 +-
.../llap/vector_decimal_udf.q.out | 12 +-
.../clientpositive/llap/vector_groupby_3.q.out | 6 +-
.../llap/vector_groupby_reduce.q.out | 12 +-
.../clientpositive/llap/vector_interval_2.q.out | 14 +-
.../clientpositive/llap/vector_orderby_5.q.out | 6 +-
.../clientpositive/llap/vectorization_0.q.out | 6 +-
.../clientpositive/llap/vectorization_13.q.out | 12 +-
.../clientpositive/llap/vectorization_15.q.out | 4 +-
.../llap/vectorization_short_regress.q.out | 56 +-
.../llap/vectorized_parquet.q.out | 4 +-
.../llap/vectorized_parquet_types.q.out | 6 +-
.../clientpositive/llap/vectorized_ptf.q.out | 12 +-
.../clientpositive/llap/windowing_gby.q.out | 42 +-
.../clientpositive/multi_insert_gby3.q.out | 4 +-
.../clientpositive/multigroupby_singlemr.q.out | 4 +-
.../clientpositive/nested_column_pruning.q.out | 8 +-
..._non_dictionary_encoding_vectorization.q.out | 4 +-
.../parquet_types_vectorization.q.out | 4 +-
.../results/clientpositive/perf/query1.q.out | 126 +-
.../results/clientpositive/perf/query12.q.out | 2 +-
.../results/clientpositive/perf/query15.q.out | 106 +-
.../results/clientpositive/perf/query16.q.out | 48 +-
.../results/clientpositive/perf/query17.q.out | 214 ++--
.../results/clientpositive/perf/query18.q.out | 2 +-
.../results/clientpositive/perf/query19.q.out | 100 +-
.../results/clientpositive/perf/query20.q.out | 70 +-
.../results/clientpositive/perf/query22.q.out | 98 +-
.../results/clientpositive/perf/query25.q.out | 212 ++--
.../results/clientpositive/perf/query26.q.out | 126 +-
.../results/clientpositive/perf/query29.q.out | 212 ++--
.../results/clientpositive/perf/query3.q.out | 70 +-
.../results/clientpositive/perf/query30.q.out | 212 ++--
.../results/clientpositive/perf/query31.q.out | 420 ++++---
.../results/clientpositive/perf/query38.q.out | 210 ++--
.../results/clientpositive/perf/query39.q.out | 196 ++-
.../results/clientpositive/perf/query42.q.out | 70 +-
.../results/clientpositive/perf/query46.q.out | 126 +-
.../results/clientpositive/perf/query5.q.out | 302 +++--
.../results/clientpositive/perf/query51.q.out | 84 +-
.../results/clientpositive/perf/query52.q.out | 70 +-
.../results/clientpositive/perf/query54.q.out | 244 ++--
.../results/clientpositive/perf/query55.q.out | 70 +-
.../results/clientpositive/perf/query58.q.out | 378 +++---
.../results/clientpositive/perf/query6.q.out | 10 +-
.../results/clientpositive/perf/query64.q.out | 1116 +++++++++---------
.../results/clientpositive/perf/query65.q.out | 84 +-
.../results/clientpositive/perf/query66.q.out | 2 +-
.../results/clientpositive/perf/query67.q.out | 2 +-
.../results/clientpositive/perf/query68.q.out | 126 +-
.../results/clientpositive/perf/query69.q.out | 2 +-
.../results/clientpositive/perf/query7.q.out | 126 +-
.../results/clientpositive/perf/query70.q.out | 70 +-
.../results/clientpositive/perf/query71.q.out | 202 ++--
.../results/clientpositive/perf/query72.q.out | 4 +-
.../results/clientpositive/perf/query79.q.out | 98 +-
.../results/clientpositive/perf/query8.q.out | 222 ++--
.../results/clientpositive/perf/query81.q.out | 216 ++--
.../results/clientpositive/perf/query83.q.out | 462 ++++----
.../results/clientpositive/perf/query85.q.out | 234 ++--
.../results/clientpositive/perf/query87.q.out | 210 ++--
.../results/clientpositive/perf/query89.q.out | 98 +-
.../results/clientpositive/perf/query91.q.out | 178 ++-
.../results/clientpositive/perf/query92.q.out | 42 +-
.../results/clientpositive/perf/query97.q.out | 42 +-
.../results/clientpositive/perf/query98.q.out | 70 +-
.../test/results/clientpositive/ppd_gby.q.out | 48 +-
.../test/results/clientpositive/ppd_gby2.q.out | 48 +-
.../results/clientpositive/ppd_windowing1.q.out | 36 +-
.../results/clientpositive/ptfgroupbyjoin.q.out | 40 +-
.../reduce_deduplicate_extended2.q.out | 128 +-
.../clientpositive/select_column_pruning.q.out | 141 +++
.../spark/dynamic_rdd_cache.q.out | 48 +-
.../results/clientpositive/spark/groupby9.q.out | 8 +-
.../clientpositive/spark/groupby_position.q.out | 22 +-
.../clientpositive/spark/limit_pushdown.q.out | 4 +-
.../spark/multi_insert_gby3.q.out | 4 +-
.../spark/multi_insert_lateral_view.q.out | 92 +-
.../spark/multigroupby_singlemr.q.out | 4 +-
.../test/results/clientpositive/spark/ptf.q.out | 12 +-
.../clientpositive/spark/subquery_in.q.out | 72 +-
.../spark/vector_decimal_aggregate.q.out | 12 +-
.../clientpositive/spark/vector_groupby_3.q.out | 6 +-
.../clientpositive/spark/vector_orderby_5.q.out | 6 +-
.../clientpositive/spark/vectorization_0.q.out | 6 +-
.../clientpositive/spark/vectorization_13.q.out | 12 +-
.../clientpositive/spark/vectorization_15.q.out | 4 +-
.../spark/vectorization_short_regress.q.out | 56 +-
.../clientpositive/spark/vectorized_ptf.q.out | 12 +-
.../clientpositive/subquery_in_having.q.out | 32 +-
.../clientpositive/subquery_notexists.q.out | 20 +-
.../subquery_notexists_having.q.out | 24 +-
.../clientpositive/subquery_notin_having.q.out | 4 +-
.../subquery_unqualcolumnrefs.q.out | 4 +-
.../temp_table_display_colstats_tbllvl.q.out | 8 +-
.../tez/vectorization_limit.q.out | 52 +-
.../udaf_binarysetfunctions.q.out | 20 +-
.../vector_decimal_aggregate.q.out | 12 +-
.../clientpositive/vector_decimal_round_2.q.out | 6 +-
.../clientpositive/vector_groupby_3.q.out | 6 +-
.../clientpositive/vector_groupby_reduce.q.out | 12 +-
.../clientpositive/vector_orderby_5.q.out | 6 +-
.../clientpositive/vectorization_13.q.out | 12 +-
.../clientpositive/vectorization_15.q.out | 4 +-
.../clientpositive/vectorization_limit.q.out | 50 +-
.../vectorized_parquet_types.q.out | 6 +-
.../results/clientpositive/windowing_gby2.q.out | 34 +-
147 files changed, 5114 insertions(+), 5412 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
index 00ec03e..acafd73 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
@@ -810,11 +810,18 @@ public final class ColumnPrunerProcFactory {
ArrayList<String> newOutputColumnNames = new ArrayList<String>();
ArrayList<ColumnInfo> rs_oldsignature = op.getSchema().getSignature();
ArrayList<ColumnInfo> rs_newsignature = new ArrayList<ColumnInfo>();
+ // The pruning needs to preserve the order of columns in the input schema
+ Set<String> colNames = new HashSet<String>();
for (FieldNode col : cols) {
- int index = originalOutputColumnNames.indexOf(col.getFieldName());
- newOutputColumnNames.add(col.getFieldName());
- newColList.add(originalColList.get(index));
- rs_newsignature.add(rs_oldsignature.get(index));
+ colNames.add(col.getFieldName());
+ }
+ for (int i = 0; i < originalOutputColumnNames.size(); i++) {
+ String colName = originalOutputColumnNames.get(i);
+ if (colNames.contains(colName)) {
+ newOutputColumnNames.add(colName);
+ newColList.add(originalColList.get(i));
+ rs_newsignature.add(rs_oldsignature.get(i));
+ }
}
op.getSchema().setSignature(rs_newsignature);
conf.setColList(newColList);
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/queries/clientpositive/select_column_pruning.q
----------------------------------------------------------------------
diff --git a/ql/src/test/queries/clientpositive/select_column_pruning.q b/ql/src/test/queries/clientpositive/select_column_pruning.q
new file mode 100644
index 0000000..207cd32
--- /dev/null
+++ b/ql/src/test/queries/clientpositive/select_column_pruning.q
@@ -0,0 +1,4 @@
+CREATE TABLE lv_table1( c1 STRING, c2 ARRAY<INT>, c3 INT, c4 CHAR(1), c5 STRING, c6 STRING, c7 STRING, c8 STRING, c9 STRING, c10 STRING, c11 STRING);
+INSERT OVERWRITE TABLE lv_table1 SELECT 'abc ', array(1,2,3), 100, 't', 'test', 'test', 'test', 'test', 'test', 'test', 'test' FROM src;
+EXPLAIN SELECT * FROM lv_table1 LATERAL VIEW explode(array(1,2,3)) myTable AS myCol WHERE c3 = 100 SORT BY c1 ASC, myCol ASC LIMIT 1;
+SELECT * FROM lv_table1 LATERAL VIEW explode(array(1,2,3)) myTable AS myCol WHERE c3 = 100 SORT BY c1 ASC, myCol ASC LIMIT 1;
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/autoColumnStats_8.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/autoColumnStats_8.q.out b/ql/src/test/results/clientpositive/autoColumnStats_8.q.out
index 329a897..681d962 100644
--- a/ql/src/test/results/clientpositive/autoColumnStats_8.q.out
+++ b/ql/src/test/results/clientpositive/autoColumnStats_8.q.out
@@ -157,8 +157,8 @@ STAGE PLANS:
GatherStats: true
MultiFileSpray: false
Select Operator
- expressions: _col2 (type: string), _col0 (type: string), _col1 (type: string)
- outputColumnNames: hr, key, value
+ expressions: _col0 (type: string), _col1 (type: string), _col2 (type: string)
+ outputColumnNames: key, value, hr
Statistics: Num rows: 666 Data size: 7075 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(key, 16), compute_stats(value, 16)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/columnstats_partlvl.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/columnstats_partlvl.q.out b/ql/src/test/results/clientpositive/columnstats_partlvl.q.out
index c5bfafb..07d26e9 100644
--- a/ql/src/test/results/clientpositive/columnstats_partlvl.q.out
+++ b/ql/src/test/results/clientpositive/columnstats_partlvl.q.out
@@ -539,8 +539,8 @@ STAGE PLANS:
alias: employee_part
Statistics: Num rows: 2 Data size: 210 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: employeesalary (type: double), employeeid (type: int), employeename (type: string)
- outputColumnNames: employeesalary, employeeid, employeename
+ expressions: employeeid (type: int), employeename (type: string), employeesalary (type: double)
+ outputColumnNames: employeeid, employeename, employeesalary
Statistics: Num rows: 2 Data size: 210 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(employeeid, 16), compute_stats(employeename, 16)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out b/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out
index 08fb42e..468d2e7 100644
--- a/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out
+++ b/ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out
@@ -84,8 +84,8 @@ STAGE PLANS:
alias: employee_part
Statistics: Num rows: 1 Data size: 64 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: country (type: string), employeename (type: string), employeeid (type: int)
- outputColumnNames: country, employeename, employeeid
+ expressions: employeeid (type: int), employeename (type: string), country (type: string)
+ outputColumnNames: employeeid, employeename, country
Statistics: Num rows: 1 Data size: 64 Basic stats: PARTIAL Column stats: NONE
Group By Operator
aggregations: compute_stats(employeename, 16), compute_stats(employeeid, 16)
@@ -162,8 +162,8 @@ STAGE PLANS:
alias: employee_part
Statistics: Num rows: 42 Data size: 169 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: country (type: string), employeeid (type: int)
- outputColumnNames: country, employeeid
+ expressions: employeeid (type: int), country (type: string)
+ outputColumnNames: employeeid, country
Statistics: Num rows: 42 Data size: 169 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(employeeid, 16)
@@ -251,8 +251,8 @@ STAGE PLANS:
alias: employee_part
Statistics: Num rows: 116 Data size: 466 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: employeesalary (type: double), country (type: string), employeeid (type: int)
- outputColumnNames: employeesalary, country, employeeid
+ expressions: employeeid (type: int), employeesalary (type: double), country (type: string)
+ outputColumnNames: employeeid, employeesalary, country
Statistics: Num rows: 116 Data size: 466 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(employeeid, 16)
@@ -339,8 +339,8 @@ STAGE PLANS:
alias: employee_part
Statistics: Num rows: 2 Data size: 466 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: employeesalary (type: double), country (type: string), employeeid (type: int), employeename (type: string)
- outputColumnNames: employeesalary, country, employeeid, employeename
+ expressions: employeeid (type: int), employeename (type: string), employeesalary (type: double), country (type: string)
+ outputColumnNames: employeeid, employeename, employeesalary, country
Statistics: Num rows: 2 Data size: 466 Basic stats: PARTIAL Column stats: NONE
Group By Operator
aggregations: compute_stats(employeeid, 16), compute_stats(employeename, 16)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out b/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out
index e8aaf8a..462d4c1 100644
--- a/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out
+++ b/ql/src/test/results/clientpositive/columnstats_tbllvl.q.out
@@ -56,8 +56,8 @@ STAGE PLANS:
alias: uservisits_web_text_none
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float)
- outputColumnNames: sourceip, avgtimeonsite, adrevenue
+ expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int)
+ outputColumnNames: sourceip, adrevenue, avgtimeonsite
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16)
@@ -108,8 +108,8 @@ STAGE PLANS:
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Select Operator
- expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float)
- outputColumnNames: sourceip, avgtimeonsite, adrevenue
+ expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int)
+ outputColumnNames: sourceip, adrevenue, avgtimeonsite
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16)
@@ -478,8 +478,8 @@ STAGE PLANS:
alias: uservisits_in_dummy_db
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float)
- outputColumnNames: sourceip, avgtimeonsite, adrevenue
+ expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int)
+ outputColumnNames: sourceip, adrevenue, avgtimeonsite
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16)
@@ -530,8 +530,8 @@ STAGE PLANS:
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Select Operator
- expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float)
- outputColumnNames: sourceip, avgtimeonsite, adrevenue
+ expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int)
+ outputColumnNames: sourceip, adrevenue, avgtimeonsite
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/complex_alias.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/complex_alias.q.out b/ql/src/test/results/clientpositive/complex_alias.q.out
index 361bc8d..5182153 100644
--- a/ql/src/test/results/clientpositive/complex_alias.q.out
+++ b/ql/src/test/results/clientpositive/complex_alias.q.out
@@ -156,8 +156,8 @@ STAGE PLANS:
Union
Statistics: Num rows: 2 Data size: 34 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col2 (type: int), _col1 (type: string)
- outputColumnNames: _col2, _col1
+ expressions: _col1 (type: string), _col2 (type: int)
+ outputColumnNames: _col1, _col2
Statistics: Num rows: 2 Data size: 34 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col2 (type: int), _col1 (type: string)
@@ -182,8 +182,8 @@ STAGE PLANS:
Union
Statistics: Num rows: 2 Data size: 34 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col2 (type: int), _col1 (type: string)
- outputColumnNames: _col2, _col1
+ expressions: _col1 (type: string), _col2 (type: int)
+ outputColumnNames: _col1, _col2
Statistics: Num rows: 2 Data size: 34 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col2 (type: int), _col1 (type: string)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/constant_prop_3.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/constant_prop_3.q.out b/ql/src/test/results/clientpositive/constant_prop_3.q.out
index ecf6e57..7296131 100644
--- a/ql/src/test/results/clientpositive/constant_prop_3.q.out
+++ b/ql/src/test/results/clientpositive/constant_prop_3.q.out
@@ -272,8 +272,8 @@ STAGE PLANS:
predicate: ((_col6 = 0) or (_col9 is null and _col1 is not null and (_col7 >= _col6))) (type: boolean)
Statistics: Num rows: 1 Data size: 18 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col3 (type: string), _col4 (type: string), _col5 (type: int), _col1 (type: int)
- outputColumnNames: _col3, _col4, _col5, _col1
+ expressions: _col1 (type: int), _col3 (type: string), _col4 (type: string), _col5 (type: int)
+ outputColumnNames: _col1, _col3, _col4, _col5
Statistics: Num rows: 1 Data size: 18 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(DISTINCT _col1)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/correlationoptimizer13.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/correlationoptimizer13.q.out b/ql/src/test/results/clientpositive/correlationoptimizer13.q.out
index 0bd27b4..21219ca 100644
--- a/ql/src/test/results/clientpositive/correlationoptimizer13.q.out
+++ b/ql/src/test/results/clientpositive/correlationoptimizer13.q.out
@@ -52,8 +52,8 @@ STAGE PLANS:
predicate: ((c1 < 120) and c3 is not null) (type: boolean)
Statistics: Num rows: 342 Data size: 7639 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: c1 (type: int), c3 (type: string)
- outputColumnNames: _col1, _col0
+ expressions: c3 (type: string), c1 (type: int)
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 342 Data size: 7639 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(1)
@@ -74,8 +74,8 @@ STAGE PLANS:
predicate: ((c2 > 100) and (c1 < 120) and c3 is not null) (type: boolean)
Statistics: Num rows: 114 Data size: 2546 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: c1 (type: int), c3 (type: string)
- outputColumnNames: _col1, _col0
+ expressions: c3 (type: string), c1 (type: int)
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 114 Data size: 2546 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(1)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/decimal_udf.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/decimal_udf.q.out b/ql/src/test/results/clientpositive/decimal_udf.q.out
index 6e72f77..515fc7d 100644
--- a/ql/src/test/results/clientpositive/decimal_udf.q.out
+++ b/ql/src/test/results/clientpositive/decimal_udf.q.out
@@ -1278,8 +1278,8 @@ STAGE PLANS:
alias: decimal_udf
Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: value (type: int), key (type: decimal(20,10))
- outputColumnNames: value, key
+ expressions: key (type: decimal(20,10)), value (type: int)
+ outputColumnNames: key, value
Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(key), count(key), avg(key)
@@ -1849,8 +1849,8 @@ STAGE PLANS:
alias: decimal_udf
Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: value (type: int), key (type: decimal(20,10))
- outputColumnNames: value, key
+ expressions: key (type: decimal(20,10)), value (type: int)
+ outputColumnNames: key, value
Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: stddev(key), variance(key)
@@ -1926,8 +1926,8 @@ STAGE PLANS:
alias: decimal_udf
Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: value (type: int), key (type: decimal(20,10))
- outputColumnNames: value, key
+ expressions: key (type: decimal(20,10)), value (type: int)
+ outputColumnNames: key, value
Statistics: Num rows: 3 Data size: 359 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: stddev_samp(key), var_samp(key)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out b/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out
index f7d1400..7b49e4a 100644
--- a/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out
+++ b/ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out
@@ -72,8 +72,8 @@ STAGE PLANS:
alias: uservisits_web_text_none
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float)
- outputColumnNames: sourceip, avgtimeonsite, adrevenue
+ expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int)
+ outputColumnNames: sourceip, adrevenue, avgtimeonsite
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16)
@@ -124,8 +124,8 @@ STAGE PLANS:
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
GatherStats: false
Select Operator
- expressions: sourceip (type: string), avgtimeonsite (type: int), adrevenue (type: float)
- outputColumnNames: sourceip, avgtimeonsite, adrevenue
+ expressions: sourceip (type: string), adrevenue (type: float), avgtimeonsite (type: int)
+ outputColumnNames: sourceip, adrevenue, avgtimeonsite
Statistics: Num rows: 65 Data size: 7060 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: compute_stats(sourceip, 16), compute_stats(avgtimeonsite, 16), compute_stats(adrevenue, 16)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out b/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out
index 6be1467..aac939f 100644
--- a/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out
+++ b/ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out
@@ -684,8 +684,8 @@ STAGE PLANS:
isPivotResult: true
Statistics: Num rows: 127193 Data size: 1017544 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col1 (type: smallint), rank_window_0 (type: int)
- outputColumnNames: _col1, rank_window_0
+ expressions: rank_window_0 (type: int), _col1 (type: smallint)
+ outputColumnNames: rank_window_0, _col1
Statistics: Num rows: 127193 Data size: 1017544 Basic stats: COMPLETE Column stats: NONE
Group By Operator
keys: _col1 (type: smallint), rank_window_0 (type: int)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/druid_basic2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/druid_basic2.q.out b/ql/src/test/results/clientpositive/druid_basic2.q.out
index bc9410b..495ebbe 100644
--- a/ql/src/test/results/clientpositive/druid_basic2.q.out
+++ b/ql/src/test/results/clientpositive/druid_basic2.q.out
@@ -557,8 +557,8 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
GatherStats: false
Select Operator
- expressions: robot (type: string), language (type: string), __time (type: timestamp), added (type: float), delta (type: float)
- outputColumnNames: robot, language, __time, added, delta
+ expressions: __time (type: timestamp), robot (type: string), language (type: string), added (type: float), delta (type: float)
+ outputColumnNames: __time, robot, language, added, delta
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Group By Operator
aggregations: max(added), sum(delta)
@@ -771,8 +771,8 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
GatherStats: false
Select Operator
- expressions: robot (type: string), language (type: string), __time (type: timestamp), added (type: float), delta (type: float)
- outputColumnNames: robot, language, __time, added, delta
+ expressions: __time (type: timestamp), robot (type: string), language (type: string), added (type: float), delta (type: float)
+ outputColumnNames: __time, robot, language, added, delta
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Group By Operator
aggregations: max(added), sum(delta)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out b/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out
index cf31ec1..2abb819 100644
--- a/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out
+++ b/ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out
@@ -1033,22 +1033,18 @@ STAGE PLANS:
1 _col0 (type: int)
outputColumnNames: _col2, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: _col5 (type: int), _col4 (type: int), _col6 (type: string), _col2 (type: int)
- outputColumnNames: _col5, _col4, _col6, _col2
+ Group By Operator
+ aggregations: stddev_samp(_col2), avg(_col2)
+ keys: _col5 (type: int), _col4 (type: int), _col6 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Group By Operator
- aggregations: stddev_samp(_col2), avg(_col2)
- keys: _col5 (type: int), _col4 (type: int), _col6 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: true
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ File Output Operator
+ compressed: true
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-4
Map Reduce
@@ -1270,22 +1266,18 @@ STAGE PLANS:
1 _col0 (type: int)
outputColumnNames: _col2, _col4, _col5, _col6
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Select Operator
- expressions: _col5 (type: int), _col4 (type: int), _col6 (type: string), _col2 (type: int)
- outputColumnNames: _col5, _col4, _col6, _col2
+ Group By Operator
+ aggregations: stddev_samp(_col2), avg(_col2)
+ keys: _col5 (type: int), _col4 (type: int), _col6 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- Group By Operator
- aggregations: stddev_samp(_col2), avg(_col2)
- keys: _col5 (type: int), _col4 (type: int), _col6 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3, _col4
- Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
- File Output Operator
- compressed: true
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ File Output Operator
+ compressed: true
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-13
Map Reduce
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/except_all.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/except_all.q.out b/ql/src/test/results/clientpositive/except_all.q.out
index 5ab3c64..61c1967 100644
--- a/ql/src/test/results/clientpositive/except_all.q.out
+++ b/ql/src/test/results/clientpositive/except_all.q.out
@@ -260,8 +260,8 @@ STAGE PLANS:
Union
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: string), _col1 (type: string), (_col2 * _col3) (type: bigint), _col3 (type: bigint)
- outputColumnNames: _col0, _col1, _col4, _col3
+ expressions: _col0 (type: string), _col1 (type: string), _col3 (type: bigint), (_col2 * _col3) (type: bigint)
+ outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(_col4), sum(_col3)
@@ -279,8 +279,8 @@ STAGE PLANS:
Union
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: string), _col1 (type: string), (_col2 * _col3) (type: bigint), _col3 (type: bigint)
- outputColumnNames: _col0, _col1, _col4, _col3
+ expressions: _col0 (type: string), _col1 (type: string), _col3 (type: bigint), (_col2 * _col3) (type: bigint)
+ outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(_col4), sum(_col3)
@@ -437,8 +437,8 @@ STAGE PLANS:
Union
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: string), _col1 (type: string), (_col2 * _col3) (type: bigint), _col3 (type: bigint)
- outputColumnNames: _col0, _col1, _col4, _col3
+ expressions: _col0 (type: string), _col1 (type: string), _col3 (type: bigint), (_col2 * _col3) (type: bigint)
+ outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(_col4), sum(_col3)
@@ -456,8 +456,8 @@ STAGE PLANS:
Union
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: _col0 (type: string), _col1 (type: string), (_col2 * _col3) (type: bigint), _col3 (type: bigint)
- outputColumnNames: _col0, _col1, _col4, _col3
+ expressions: _col0 (type: string), _col1 (type: string), _col3 (type: bigint), (_col2 * _col3) (type: bigint)
+ outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(_col4), sum(_col3)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/groupby9.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/groupby9.q.out b/ql/src/test/results/clientpositive/groupby9.q.out
index f4f1163..15ea185 100644
--- a/ql/src/test/results/clientpositive/groupby9.q.out
+++ b/ql/src/test/results/clientpositive/groupby9.q.out
@@ -839,8 +839,8 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: value (type: string), key (type: string)
- outputColumnNames: value, key
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: key, value
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(DISTINCT substr(value, 5))
@@ -3193,8 +3193,8 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: value (type: string), key (type: string)
- outputColumnNames: value, key
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: key, value
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(DISTINCT substr(value, 5))
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out b/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out
index 70a6b0e..95d499a 100644
--- a/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out
+++ b/ql/src/test/results/clientpositive/groupby_join_pushdown.q.out
@@ -588,22 +588,18 @@ STAGE PLANS:
1 _col1 (type: int)
outputColumnNames: _col0, _col2, _col3
Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col0 (type: tinyint), _col3 (type: tinyint), _col2 (type: bigint)
- outputColumnNames: _col0, _col3, _col2
+ Group By Operator
+ aggregations: sum(_col2)
+ keys: _col0 (type: tinyint), _col3 (type: tinyint)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col2)
- keys: _col0 (type: tinyint), _col3 (type: tinyint)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
- Statistics: Num rows: 13516 Data size: 2906160 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-2
Map Reduce
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/groupby_position.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/groupby_position.q.out b/ql/src/test/results/clientpositive/groupby_position.q.out
index ff27253..3ded69c 100644
--- a/ql/src/test/results/clientpositive/groupby_position.q.out
+++ b/ql/src/test/results/clientpositive/groupby_position.q.out
@@ -246,22 +246,18 @@ STAGE PLANS:
Filter Operator
predicate: (key < 20) (type: boolean)
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: value (type: string), key (type: string)
- outputColumnNames: value, key
+ Group By Operator
+ aggregations: count(DISTINCT substr(value, 5))
+ keys: value (type: string), key (type: string), substr(value, 5) (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(DISTINCT substr(value, 5))
- keys: value (type: string), key (type: string), substr(value, 5) (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Reduce Operator Tree:
Group By Operator
aggregations: count(DISTINCT KEY._col1:0._col0)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/having2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/having2.q.out b/ql/src/test/results/clientpositive/having2.q.out
index 59151fa..83d367f 100644
--- a/ql/src/test/results/clientpositive/having2.q.out
+++ b/ql/src/test/results/clientpositive/having2.q.out
@@ -132,8 +132,8 @@ STAGE PLANS:
alias: s1
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: customer_name (type: string), customer_balance (type: double), order_quantity (type: double), discount (type: double)
- outputColumnNames: customer_name, customer_balance, order_quantity, discount
+ expressions: order_quantity (type: double), discount (type: double), customer_name (type: string), customer_balance (type: double)
+ outputColumnNames: order_quantity, discount, customer_name, customer_balance
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Group By Operator
aggregations: sum(customer_balance), sum(order_quantity), count(discount)
@@ -199,8 +199,8 @@ STAGE PLANS:
alias: s1
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Select Operator
- expressions: customer_name (type: string), customer_balance (type: double), order_quantity (type: double), discount (type: double)
- outputColumnNames: customer_name, customer_balance, order_quantity, discount
+ expressions: order_quantity (type: double), discount (type: double), customer_name (type: string), customer_balance (type: double)
+ outputColumnNames: order_quantity, discount, customer_name, customer_balance
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE
Group By Operator
aggregations: sum(customer_balance), sum(order_quantity), count(discount)
@@ -310,22 +310,18 @@ STAGE PLANS:
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4
Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: string), _col2 (type: double), _col0 (type: double), _col4 (type: string)
- outputColumnNames: _col1, _col2, _col0, _col4
+ Group By Operator
+ aggregations: sum(_col2), avg(_col0), count(_col4)
+ keys: _col1 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col2), avg(_col0), count(_col4)
- keys: _col1 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-2
Map Reduce
@@ -433,22 +429,18 @@ STAGE PLANS:
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4
Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: string), _col2 (type: double), _col0 (type: double), _col4 (type: string)
- outputColumnNames: _col1, _col2, _col0, _col4
+ Group By Operator
+ aggregations: sum(_col2), avg(_col0), count(_col4)
+ keys: _col1 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col2), avg(_col0), count(_col4)
- keys: _col1 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-2
Map Reduce
@@ -560,22 +552,18 @@ STAGE PLANS:
1 _col0 (type: string)
outputColumnNames: _col0, _col1, _col2, _col4
Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col1 (type: string), _col2 (type: double), _col0 (type: double), _col4 (type: string)
- outputColumnNames: _col1, _col2, _col0, _col4
+ Group By Operator
+ aggregations: sum(_col2), avg(_col0), count(_col4)
+ keys: _col1 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2, _col3
Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: sum(_col2), avg(_col0), count(_col4)
- keys: _col1 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 550 Data size: 5843 Basic stats: COMPLETE Column stats: NONE
- File Output Operator
- compressed: false
- table:
- input format: org.apache.hadoop.mapred.SequenceFileInputFormat
- output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
- serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
+ File Output Operator
+ compressed: false
+ table:
+ input format: org.apache.hadoop.mapred.SequenceFileInputFormat
+ output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
+ serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Stage: Stage-2
Map Reduce
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/index_auto_update.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/index_auto_update.q.out b/ql/src/test/results/clientpositive/index_auto_update.q.out
index d3d2ca3..e7bc069 100644
--- a/ql/src/test/results/clientpositive/index_auto_update.q.out
+++ b/ql/src/test/results/clientpositive/index_auto_update.q.out
@@ -98,8 +98,8 @@ STAGE PLANS:
TableScan
alias: temp
Select Operator
- expressions: key (type: string), INPUT__FILE__NAME (type: string), BLOCK__OFFSET__INSIDE__FILE (type: bigint)
- outputColumnNames: key, INPUT__FILE__NAME, BLOCK__OFFSET__INSIDE__FILE
+ expressions: key (type: string), BLOCK__OFFSET__INSIDE__FILE (type: bigint), INPUT__FILE__NAME (type: string)
+ outputColumnNames: key, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME
Group By Operator
aggregations: collect_set(BLOCK__OFFSET__INSIDE__FILE)
keys: key (type: string), INPUT__FILE__NAME (type: string)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out b/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out
index a92656b..35e6827 100644
--- a/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out
+++ b/ql/src/test/results/clientpositive/limit_pushdown_negative.q.out
@@ -84,8 +84,8 @@ STAGE PLANS:
alias: src
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: value (type: string), key (type: string)
- outputColumnNames: value, key
+ expressions: key (type: string), value (type: string)
+ outputColumnNames: key, value
Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: sum(key)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out b/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out
index cb7623e..4ac5ac9 100644
--- a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out
+++ b/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out
@@ -1569,8 +1569,8 @@ STAGE PLANS:
alias: clustergroupby
Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Select Operator
- expressions: key (type: string), value (type: string)
- outputColumnNames: _col1, _col0
+ expressions: value (type: string), key (type: string)
+ outputColumnNames: _col0, _col1
Statistics: Num rows: 500 Data size: 9312 Basic stats: COMPLETE Column stats: NONE
Group By Operator
aggregations: count(1)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/column_table_stats.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/column_table_stats.q.out b/ql/src/test/results/clientpositive/llap/column_table_stats.q.out
index e23d882..fb04ee8 100644
--- a/ql/src/test/results/clientpositive/llap/column_table_stats.q.out
+++ b/ql/src/test/results/clientpositive/llap/column_table_stats.q.out
@@ -336,8 +336,8 @@ STAGE PLANS:
Statistics Aggregation Key Prefix: default.spart/
GatherStats: true
Select Operator
- expressions: ds (type: string), hr (type: string), key (type: string), value (type: string)
- outputColumnNames: ds, hr, key, value
+ expressions: key (type: string), value (type: string), ds (type: string), hr (type: string)
+ outputColumnNames: key, value, ds, hr
Statistics: Num rows: 58 Data size: 32968 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: compute_stats(key, 16), compute_stats(value, 16)
@@ -753,8 +753,8 @@ STAGE PLANS:
Statistics Aggregation Key Prefix: default.spart/
GatherStats: true
Select Operator
- expressions: ds (type: string), hr (type: string), key (type: string), value (type: string)
- outputColumnNames: ds, hr, key, value
+ expressions: key (type: string), value (type: string), ds (type: string), hr (type: string)
+ outputColumnNames: key, value, ds, hr
Statistics: Num rows: 58 Data size: 32968 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: compute_stats(key, 16), compute_stats(value, 16)
@@ -1170,8 +1170,8 @@ STAGE PLANS:
Statistics Aggregation Key Prefix: default.spart/
GatherStats: true
Select Operator
- expressions: ds (type: string), key (type: string), value (type: string)
- outputColumnNames: ds, key, value
+ expressions: key (type: string), value (type: string), ds (type: string)
+ outputColumnNames: key, value, ds
Statistics: Num rows: 29 Data size: 11148 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: compute_stats(key, 16), compute_stats(value, 16)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out b/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out
index 5f35e08..d55cf30 100644
--- a/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out
+++ b/ql/src/test/results/clientpositive/llap/column_table_stats_orc.q.out
@@ -333,8 +333,8 @@ STAGE PLANS:
Statistics Aggregation Key Prefix: default.spart/
GatherStats: true
Select Operator
- expressions: ds (type: string), hr (type: string), key (type: string), value (type: string)
- outputColumnNames: ds, hr, key, value
+ expressions: key (type: string), value (type: string), ds (type: string), hr (type: string)
+ outputColumnNames: key, value, ds, hr
Statistics: Num rows: 2 Data size: 1076 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: compute_stats(key, 16), compute_stats(value, 16)
@@ -738,8 +738,8 @@ STAGE PLANS:
Statistics Aggregation Key Prefix: default.spart/
GatherStats: true
Select Operator
- expressions: ds (type: string), key (type: string), value (type: string)
- outputColumnNames: ds, key, value
+ expressions: key (type: string), value (type: string), ds (type: string)
+ outputColumnNames: key, value, ds
Statistics: Num rows: 1 Data size: 354 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: compute_stats(key, 16), compute_stats(value, 16)
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out
index 9dab93d..e78429f 100644
--- a/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out
+++ b/ql/src/test/results/clientpositive/llap/dynamic_partition_pruning_2.q.out
@@ -181,22 +181,18 @@ STAGE PLANS:
input vertices:
1 Map 4
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col3 (type: string), _col0 (type: decimal(10,0))
- outputColumnNames: _col3, _col0
+ Group By Operator
+ aggregations: count(), sum(_col0)
+ keys: _col3 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(), sum(_col0)
- keys: _col3 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
+ value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -354,22 +350,18 @@ STAGE PLANS:
input vertices:
1 Map 4
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col3 (type: string), _col0 (type: decimal(10,0))
- outputColumnNames: _col3, _col0
+ Group By Operator
+ aggregations: count(), sum(_col0)
+ keys: _col3 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(), sum(_col0)
- keys: _col3 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
+ value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Execution mode: llap
LLAP IO: no inputs
Map 4
@@ -727,22 +719,18 @@ STAGE PLANS:
input vertices:
1 Map 4
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Select Operator
- expressions: _col3 (type: string), _col0 (type: decimal(10,0))
- outputColumnNames: _col3, _col0
+ Group By Operator
+ aggregations: count(), sum(_col0)
+ keys: _col3 (type: string)
+ mode: hash
+ outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Group By Operator
- aggregations: count(), sum(_col0)
- keys: _col3 (type: string)
- mode: hash
- outputColumnNames: _col0, _col1, _col2
+ Reduce Output Operator
+ key expressions: _col0 (type: string)
+ sort order: +
+ Map-reduce partition columns: _col0 (type: string)
Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- Reduce Output Operator
- key expressions: _col0 (type: string)
- sort order: +
- Map-reduce partition columns: _col0 (type: string)
- Statistics: Num rows: 9 Data size: 39 Basic stats: COMPLETE Column stats: NONE
- value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
+ value expressions: _col1 (type: bigint), _col2 (type: decimal(20,0))
Execution mode: llap
LLAP IO: no inputs
Map 4
http://git-wip-us.apache.org/repos/asf/hive/blob/78d8b337/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
----------------------------------------------------------------------
diff --git a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
index 790efeb..a69f2ad 100644
--- a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
+++ b/ql/src/test/results/clientpositive/llap/explainuser_1.q.out
@@ -471,7 +471,7 @@ Stage-0
Group By Operator [GBY_32] (rows=1 width=20)
Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col6, _col2
Select Operator [SEL_31] (rows=1 width=16)
- Output:["_col6","_col2"]
+ Output:["_col2","_col6"]
Filter Operator [FIL_30] (rows=1 width=16)
predicate:(((_col1 > 0) or (_col6 >= 0)) and ((_col6 >= 1) or (_col2 >= 1)) and ((UDFToLong(_col6) + _col2) >= 0))
Merge Join Operator [MERGEJOIN_48] (rows=3 width=16)
@@ -642,7 +642,7 @@ Stage-0
Group By Operator [GBY_31] (rows=1 width=20)
Output:["_col0","_col1","_col2"],aggregations:["count()"],keys:_col6, _col2
Select Operator [SEL_30] (rows=1 width=20)
- Output:["_col6","_col2"]
+ Output:["_col2","_col6"]
Filter Operator [FIL_29] (rows=1 width=20)
predicate:(((_col1 + _col4) >= 0) and ((_col1 > 0) or (_col6 >= 0)) and ((_col6 >= 1) or (_col2 >= 1)) and ((UDFToLong(_col6) + _col2) >= 0))
Merge Join Operator [MERGEJOIN_42] (rows=4 width=20)
@@ -1841,12 +1841,10 @@ Stage-0
PartitionCols:_col0
Group By Operator [GBY_5] (rows=41 width=178)
Output:["_col0","_col1"],keys:value, key
- Select Operator [SEL_4] (rows=83 width=178)
- Output:["value","key"]
- Filter Operator [FIL_21] (rows=83 width=178)
- predicate:((value = value) and (value > 'val_2'))
- TableScan [TS_2] (rows=500 width=178)
- default@src_cbo,a,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
+ Filter Operator [FIL_21] (rows=83 width=178)
+ predicate:((value = value) and (value > 'val_2'))
+ TableScan [TS_2] (rows=500 width=178)
+ default@src_cbo,a,Tbl:COMPLETE,Col:COMPLETE,Output:["key","value"]
PREHOOK: query: explain select *
from src_cbo b
@@ -2424,7 +2422,7 @@ Stage-0
Group By Operator [GBY_5] (rows=1 width=114)
Output:["_col0","_col1","_col2"],aggregations:["count()","count(p_name)"],keys:p_mfgr
Select Operator [SEL_4] (rows=4 width=223)
- Output:["p_mfgr","p_name"]
+ Output:["p_name","p_mfgr"]
Filter Operator [FIL_29] (rows=4 width=223)
predicate:((p_size < 10) and (p_mfgr = p_mfgr))
TableScan [TS_2] (rows=26 width=223)
@@ -2645,7 +2643,7 @@ Stage-0
Group By Operator [GBY_14] (rows=1 width=24)
Output:["_col0","_col1","_col2"],aggregations:["count()","count(_col0)"],keys:_col1
Select Operator [SEL_12] (rows=1 width=114)
- Output:["_col1","_col0"]
+ Output:["_col0","_col1"]
Filter Operator [FIL_45] (rows=1 width=114)
predicate:((_col1 = _col1) and ((_col2 - _col1) > 600.0))
Group By Operator [GBY_10] (rows=5 width=114)
@@ -5041,7 +5039,7 @@ Stage-4
SHUFFLE [RS_14]
PartitionCols:_col2
Select Operator [SEL_13] (rows=26 width=491)
- Output:["_col1","_col2","_col5","sum_window_0"]
+ Output:["sum_window_0","_col1","_col2","_col5"]
PTF Operator [PTF_12] (rows=26 width=491)
Function definitions:[{},{"name:":"windowingtablefunction","order by:":"_col5 ASC NULLS FIRST","partition by:":"_col2"}]
Select Operator [SEL_11] (rows=26 width=491)