You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by pengcheng xiong <px...@hortonworks.com> on 2017/03/08 00:59:10 UTC
Re: Review Request 56140: Can't order by an unselected column
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/
-----------------------------------------------------------
(Updated March 8, 2017, 12:59 a.m.)
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description
-------
HIVE-15160
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java db8d46e
ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java e14f1cf
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0872e53
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f979c14
ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/56140/diff/5/
Changes: https://reviews.apache.org/r/56140/diff/4-5/
Testing
-------
Thanks,
pengcheng xiong
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
> On May 3, 2017, 4:24 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/cp_sel.q.out
> > Line 46 (original), 50 (patched)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1704004#file1704004line50>
> >
> > Is this expected?
> > Seems like this may generate wrong results since there might be multiple tasks for Reducers each of which emit 1 row. Limit in fetch operator is needed.
Yes, it is. In case of order by, only 1 reducer is used, so no need of another shuffle.
> On May 3, 2017, 4:24 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/vector_coalesce.q.out
> > Line 447 (original)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1704013#file1704013line461>
> >
> > No RS for order by.
This is actually improvement. The query is {SELECT cfloat, cbigint, coalesce(cfloat, cbigint, 0) as c
FROM alltypesorc
WHERE (cfloat IS NULL AND cbigint IS NULL)
ORDER BY cfloat, cbigint, c
LIMIT 10;}
You can see that, cfloat, cbigint, c are all nulls....
The op tree is like this
HiveSortLimit(offset=[0], fetch=[10])
HiveProject(cfloat=[$0], cbigint=[$1], c=[$2])
HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], dir0=[ASC-nulls-first], dir1=[ASC-nulls-first], dir2=[ASC-nulls-first])
HiveProject(cfloat=[$4], cbigint=[$3], c=[coalesce($4, $3, 0)], ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint1=[$3], cfloat1=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11], block__offset__inside__file=[$12], input__file__name=[$13], row__id=[$14])
HiveFilter(condition=[AND(IS NULL($4), IS NULL($3))])
HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc])
After running HiveProjectFilterPullUpConstantsRule and HiveReduceExpressionsRule, we get rid of the order by...
> On May 3, 2017, 4:24 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/vector_date_1.q.out
> > Lines 598-607 (original)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1704046#file1704046line598>
> >
> > This plan looks incorrect. For an order by there should necessarily be a RS in plan, otherwise we can get sorting in map only plan.
dt1 is constant.
> On May 3, 2017, 4:24 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out
> > Lines 580-588 (original)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1704049#file1704049line582>
> >
> > Order by must have RS.
order by constant value...
- pengcheng
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review173690
-----------------------------------------------------------
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by Ashutosh Chauhan <ha...@apache.org>.
> On May 3, 2017, 4:24 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/cp_sel.q.out
> > Line 46 (original), 50 (patched)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1704004#file1704004line50>
> >
> > Is this expected?
> > Seems like this may generate wrong results since there might be multiple tasks for Reducers each of which emit 1 row. Limit in fetch operator is needed.
>
> pengcheng xiong wrote:
> Yes, it is. In case of order by, only 1 reducer is used, so no need of another shuffle.
Correct. Whats the reason for this? Is that because we got rid of Order by on Calcite tree itself?
> On May 3, 2017, 4:24 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/llap/vector_coalesce.q.out
> > Line 447 (original)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1704013#file1704013line461>
> >
> > No RS for order by.
>
> pengcheng xiong wrote:
> This is actually improvement. The query is {SELECT cfloat, cbigint, coalesce(cfloat, cbigint, 0) as c
> FROM alltypesorc
> WHERE (cfloat IS NULL AND cbigint IS NULL)
> ORDER BY cfloat, cbigint, c
> LIMIT 10;}
> You can see that, cfloat, cbigint, c are all nulls....
>
> The op tree is like this
>
> HiveSortLimit(offset=[0], fetch=[10])
> HiveProject(cfloat=[$0], cbigint=[$1], c=[$2])
> HiveSortLimit(sort0=[$0], sort1=[$1], sort2=[$2], dir0=[ASC-nulls-first], dir1=[ASC-nulls-first], dir2=[ASC-nulls-first])
> HiveProject(cfloat=[$4], cbigint=[$3], c=[coalesce($4, $3, 0)], ctinyint=[$0], csmallint=[$1], cint=[$2], cbigint1=[$3], cfloat1=[$4], cdouble=[$5], cstring1=[$6], cstring2=[$7], ctimestamp1=[$8], ctimestamp2=[$9], cboolean1=[$10], cboolean2=[$11], block__offset__inside__file=[$12], input__file__name=[$13], row__id=[$14])
> HiveFilter(condition=[AND(IS NULL($4), IS NULL($3))])
> HiveTableScan(table=[[default.alltypesorc]], table:alias=[alltypesorc])
>
>
> After running HiveProjectFilterPullUpConstantsRule and HiveReduceExpressionsRule, we get rid of the order by...
There is limit also in query. After this change, limit is executed in map side (potentially multiple tasks) but since FetchOperator doesnt have a limit, limit may not be honored.
> On May 3, 2017, 4:24 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/vector_date_1.q.out
> > Lines 598-607 (original)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1704046#file1704046line598>
> >
> > This plan looks incorrect. For an order by there should necessarily be a RS in plan, otherwise we can get sorting in map only plan.
>
> pengcheng xiong wrote:
> dt1 is constant.
Is this change because we optimized away order by on calcite tree?
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review173690
-----------------------------------------------------------
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review173690
-----------------------------------------------------------
ql/src/test/results/clientpositive/cp_sel.q.out
Line 46 (original), 50 (patched)
<https://reviews.apache.org/r/56140/#comment246730>
Is this expected?
Seems like this may generate wrong results since there might be multiple tasks for Reducers each of which emit 1 row. Limit in fetch operator is needed.
ql/src/test/results/clientpositive/llap/vector_coalesce.q.out
Line 447 (original)
<https://reviews.apache.org/r/56140/#comment246737>
No RS for order by.
ql/src/test/results/clientpositive/vector_date_1.q.out
Lines 598-607 (original)
<https://reviews.apache.org/r/56140/#comment246732>
This plan looks incorrect. For an order by there should necessarily be a RS in plan, otherwise we can get sorting in map only plan.
ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out
Lines 580-588 (original)
<https://reviews.apache.org/r/56140/#comment246734>
Order by must have RS.
- Ashutosh Chauhan
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
> On May 18, 2017, 6:59 p.m., Ashutosh Chauhan wrote:
> > ql/src/test/queries/clientpositive/order_by_expr_1.q
> > Lines 42 (patched)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1704001#file1704001line42>
> >
> > Can you also modify perfclidriver queries to their original form which contained such examples? They are currently rewritten to avoid this missing feature.
Open a new jira to track this.
- pengcheng
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review175406
-----------------------------------------------------------
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review175406
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 3038 (patched)
<https://reviews.apache.org/r/56140/#comment248869>
Better comment:
selPair.getValue() is RR which only contains columns needed in result set. Extra columns needed by order by will be absent from it.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Line 3516 (original), 3551 (patched)
<https://reviews.apache.org/r/56140/#comment248867>
Also RowResolver in return type.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 3827 (patched)
<https://reviews.apache.org/r/56140/#comment248849>
Comment: We examine the order by in this query block and adds in column needed by order by in select list.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 3857-3859 (patched)
<https://reviews.apache.org/r/56140/#comment248868>
Message should be like: Column already present in RR. skipping.
Also this should be at trace level.
ql/src/test/queries/clientpositive/order_by_expr_1.q
Lines 42 (patched)
<https://reviews.apache.org/r/56140/#comment248870>
Can you also modify perfclidriver queries to their original form which contained such examples? They are currently rewritten to avoid this missing feature.
- Ashutosh Chauhan
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review175753
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 3093 (patched)
<https://reviews.apache.org/r/56140/#comment249106>
We can see
Stage: Stage-0
Fetch Operator
limit: 10
Processor Tree:
ListSink
if we run explain rather than explain vectorization. This means, we may need to modify explain vectorization to show more info. for fetch work.
And, if there is no limit 10, it will return 43 rows.
This means the result is correct.
- pengcheng xiong
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/
-----------------------------------------------------------
(Updated May 23, 2017, 9:38 p.m.)
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description
-------
HIVE-15160
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java fa96e94f64
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 35fc68a555
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f678d0b0a0
ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out fc0030965a
ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
ql/src/test/results/clientpositive/llap/explainuser_1.q.out f701cabffe
ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 0a8df615fd
ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 77062c737e
ql/src/test/results/clientpositive/llap/subquery_in.q.out d7fd29e194
ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 840210476b
ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 00bb50a5a5
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
ql/src/test/results/clientpositive/llap/vector_interval_1.q.out 8d4f12e203
ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out 1d14092408
ql/src/test/results/clientpositive/order3.q.out 898f7a8853
ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
ql/src/test/results/clientpositive/perf/query31.q.out 9e3dad472a
ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
ql/src/test/results/clientpositive/perf/query39.q.out dcf3cb264e
ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
ql/src/test/results/clientpositive/perf/query64.q.out 7f97e392e1
ql/src/test/results/clientpositive/perf/query66.q.out ec7b6af471
ql/src/test/results/clientpositive/perf/query70.q.out 55c1461da8
ql/src/test/results/clientpositive/perf/query75.q.out 0ecc9852ed
ql/src/test/results/clientpositive/perf/query81.q.out dfd46396b5
ql/src/test/results/clientpositive/perf/query85.q.out ba8659e8f2
ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out bcb50cfadc
ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
ql/src/test/results/clientpositive/spark/subquery_in.q.out 5e38938ad6
ql/src/test/results/clientpositive/vector_coalesce.q.out 87ab937abb
ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
ql/src/test/results/clientpositive/vector_decimal_round.q.out d92b6c241e
ql/src/test/results/clientpositive/vector_interval_1.q.out 2a398ae5d3
ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out b67231c8c4
ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
Diff: https://reviews.apache.org/r/56140/diff/9/
Changes: https://reviews.apache.org/r/56140/diff/8-9/
Testing
-------
Thanks,
pengcheng xiong
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
> On May 18, 2017, 12:29 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
> > Lines 4049-4059 (original)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1703997#file1703997line4137>
> >
> > Aren't all these restrictions still valid?
They are not valid any more as we generate a select before OB and a select after OB for all the cases. There is no need to fix the schema after OB anymore.
- pengcheng
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review175299
-----------------------------------------------------------
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
> On May 18, 2017, 12:29 a.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java
> > Lines 70-71 (original), 81-83 (patched)
> > <https://reviews.apache.org/r/56140/diff/8/?file=1703996#file1703996line81>
> >
> > This change looks correct. But don't understand why it was needed. Can you describe the need for it?
This is a bug exposed by this patch. The query is
{code}
create table s as select * from src limit 10;
set hive.optimize.limittranspose=true;
explain
select key from s a
union all
select key from s b
order by key
limit 5;
{code}
HiveProjectSortTransposeRule is triggered for
{code}
HiveProject(key=[$0])
HiveSortLimit(sort0=[$1], dir0=[ASC-nulls-first], offset=[0], fetch=[5])
{code}
when
{code}
if (map.getTarget(fc.getFieldIndex()) < 0) {
return;
}
{code}
is called, fc.getFieldIndex() is 1, but map is 0->0. Then it throws
org.apache.calcite.util.mapping.Mappings$NoElementException: source #1 has no target in mapping [size=1, sourceCount=2, targetCount=1, elements=[0:0]]
- pengcheng
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review175299
-----------------------------------------------------------
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/#review175299
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java
Lines 70-71 (original), 81-83 (patched)
<https://reviews.apache.org/r/56140/#comment248786>
This change looks correct. But don't understand why it was needed. Can you describe the need for it?
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Line 2870 (original), 2870 (patched)
<https://reviews.apache.org/r/56140/#comment248793>
Add comment about need to pass true for last argument.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 3093 (patched)
<https://reviews.apache.org/r/56140/#comment248789>
This log message is confusing. It should be at debug level and should be like, didn't find column in parent of order by, will look into its parent next.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 3214 (patched)
<https://reviews.apache.org/r/56140/#comment248790>
Add comment about we need to add select since order by schema may have more columns than result schema.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Line 3516 (original), 3551 (patched)
<https://reviews.apache.org/r/56140/#comment248787>
Add javadoc for this new boolean argument.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 3857-3859 (patched)
<https://reviews.apache.org/r/56140/#comment248788>
This log message is confusing, because we expect this to happen as there will be overlap in columns of select and order by.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Line 4047 (original), 4128 (patched)
<https://reviews.apache.org/r/56140/#comment248791>
Remove the comment.
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 4049-4059 (original)
<https://reviews.apache.org/r/56140/#comment248792>
Aren't all these restrictions still valid?
ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java
Lines 183-189 (patched)
<https://reviews.apache.org/r/56140/#comment248769>
This seems to undo rest of logic of the function, ie it skips the duplicate checking which is required. Whats the need of this change ?
ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java
Line 463 (original), 471-475 (patched)
<https://reviews.apache.org/r/56140/#comment248770>
Whats the need of deep-copy here?
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 1619 (patched)
<https://reviews.apache.org/r/56140/#comment248751>
Comment : If there are aggregations in order by, we need to remember them in qb.
- Ashutosh Chauhan
On May 1, 2017, 5:30 p.m., pengcheng xiong wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56140/
> -----------------------------------------------------------
>
> (Updated May 1, 2017, 5:30 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> HIVE-15160
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
> ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
> ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
> ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
> ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
> ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
> ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
> ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
> ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
> ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
> ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
> ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
> ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
> ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
> ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
> ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
> ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
> ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
> ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
> ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
> ql/src/test/results/clientpositive/order3.q.out 898f7a8853
> ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
> ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
> ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
> ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
> ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
> ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
> ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
> ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
> ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
> ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
> ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
> ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
> ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
> ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
> ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
> ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
> ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
> ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
> ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
> ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
> ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
> ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
> ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
> ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
> ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
> ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
> ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
>
>
> Diff: https://reviews.apache.org/r/56140/diff/8/
>
>
> Testing
> -------
>
>
> Thanks,
>
> pengcheng xiong
>
>
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/
-----------------------------------------------------------
(Updated May 1, 2017, 5:30 p.m.)
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description
-------
HIVE-15160
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 654f3b1772
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
ql/src/test/results/clientpositive/llap/bucket_groupby.q.out d724131fca
ql/src/test/results/clientpositive/llap/explainuser_1.q.out 584c3b5520
ql/src/test/results/clientpositive/llap/limit_pushdown.q.out dd54dd22a6
ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out 24645b6426
ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 83de1fbea1
ql/src/test/results/clientpositive/llap/vector_coalesce.q.out 578f849bdb
ql/src/test/results/clientpositive/llap/vector_date_1.q.out a4f1050c89
ql/src/test/results/clientpositive/llap/vector_decimal_2.q.out 144356c108
ql/src/test/results/clientpositive/llap/vector_decimal_round.q.out 8bd80cf860
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_grouping.q.out 5af9e61b0a
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out f731ceecdc
ql/src/test/results/clientpositive/llap/vector_interval_1.q.out debf5ab39e
ql/src/test/results/clientpositive/llap/vector_interval_arithmetic.q.out aadb6e72cd
ql/src/test/results/clientpositive/order3.q.out 898f7a8853
ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
ql/src/test/results/clientpositive/perf/query66.q.out 072bfee92b
ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out 6572511967
ql/src/test/results/clientpositive/spark/limit_pushdown.q.out ede0096c73
ql/src/test/results/clientpositive/spark/pcr.q.out 77ac020d07
ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
Diff: https://reviews.apache.org/r/56140/diff/8/
Changes: https://reviews.apache.org/r/56140/diff/7-8/
Testing
-------
Thanks,
pengcheng xiong
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/
-----------------------------------------------------------
(Updated April 29, 2017, 2:14 a.m.)
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description
-------
HIVE-15160
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java 1487ed4f8e
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortProjectRealignRule.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 1b054a7e24
ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 262dafb487
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java d39b8bdf7e
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0d9c
ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1abb25
ql/src/test/results/clientpositive/auto_join8.q.out ccbafba6b5
ql/src/test/results/clientpositive/auto_join_without_localtask.q.out 17a912ec13
ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd6a6
ql/src/test/results/clientpositive/druid_basic2.q.out 6177d56987
ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 2abb819558
ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 473d17a1bd
ql/src/test/results/clientpositive/join8.q.out c1035b4ca7
ql/src/test/results/clientpositive/list_bucket_query_oneskew_2.q.out 325d568fa0
ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 041621fe0c
ql/src/test/results/clientpositive/order3.q.out 898f7a8853
ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/pcr.q.out a1301fdf79
ql/src/test/results/clientpositive/perf/query31.q.out 3ed312d3e3
ql/src/test/results/clientpositive/perf/query36.q.out 57ab26acc6
ql/src/test/results/clientpositive/perf/query39.q.out 19472c4d5e
ql/src/test/results/clientpositive/perf/query42.q.out 3bebac3321
ql/src/test/results/clientpositive/perf/query52.q.out 74ecaf28ba
ql/src/test/results/clientpositive/perf/query64.q.out 6b42393aad
ql/src/test/results/clientpositive/perf/query70.q.out 8e42fac9c5
ql/src/test/results/clientpositive/perf/query75.q.out b1e236d325
ql/src/test/results/clientpositive/perf/query81.q.out a09d5c99b5
ql/src/test/results/clientpositive/perf/query85.q.out 168bcd2a4a
ql/src/test/results/clientpositive/perf/query86.q.out 734e6a480b
ql/src/test/results/clientpositive/perf/query89.q.out 66481f710b
ql/src/test/results/clientpositive/perf/query91.q.out e592bba8d9
ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74608
ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39fd15
ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03415
ql/src/test/results/clientpositive/ppd_vc.q.out 86ab427107
ql/src/test/results/clientpositive/ppd_windowing1.q.out ad57ba9ecc
ql/src/test/results/clientpositive/regex_col.q.out 70ec3636d6
ql/src/test/results/clientpositive/semijoin4.q.out d6117ed2cb
ql/src/test/results/clientpositive/smb_mapjoin_25.q.out f72c2a70d9
ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out 0a7a36f803
ql/src/test/results/clientpositive/timestamp.q.out 9d0ceef50f
ql/src/test/results/clientpositive/vector_coalesce.q.out f158236beb
ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6b1e
ql/src/test/results/clientpositive/vector_decimal_round.q.out de49c170cf
ql/src/test/results/clientpositive/vector_interval_1.q.out f53a2c2db5
ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 75250e30a4
ql/src/test/results/clientpositive/vector_null_projection.q.out 94aea2fa62
ql/src/test/results/clientpositive/view_alias.q.out 90bf28dd9b
ql/src/test/results/clientpositive/windowing_duplicate.q.out c7b6d4f93e
Diff: https://reviews.apache.org/r/56140/diff/7/
Changes: https://reviews.apache.org/r/56140/diff/6-7/
Testing
-------
Thanks,
pengcheng xiong
Re: Review Request 56140: Can't order by an unselected column
Posted by pengcheng xiong <px...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56140/
-----------------------------------------------------------
(Updated March 13, 2017, 6:34 a.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
new patch with cli driver golden file changes.
Repository: hive-git
Description
-------
HIVE-15160
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java fd19d99
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortProjectRealignRule.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java bf0a11b
ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java e14f1cf
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 0732207
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java f979c14
ql/src/test/queries/clientpositive/order_by_expr_1.q PRE-CREATION
ql/src/test/queries/clientpositive/order_by_expr_2.q PRE-CREATION
ql/src/test/results/clientpositive/auto_join_without_localtask.q.out 17a912e
ql/src/test/results/clientpositive/cp_sel.q.out 1778ccd
ql/src/test/results/clientpositive/druid_basic2.q.out bc9410b
ql/src/test/results/clientpositive/dynamic_rdd_cache.q.out 87166a7
ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 7faf278
ql/src/test/results/clientpositive/order3.q.out 898f7a8
ql/src/test/results/clientpositive/order_by_expr_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/order_by_expr_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/pcr.q.out a1301fd
ql/src/test/results/clientpositive/pointlookup2.q.out 3438c74
ql/src/test/results/clientpositive/pointlookup3.q.out 2c3e39f
ql/src/test/results/clientpositive/ppd_udf_case.q.out 7678d03
ql/src/test/results/clientpositive/ppd_vc.q.out 86ab427
ql/src/test/results/clientpositive/regex_col.q.out 70ec363
ql/src/test/results/clientpositive/vector_coalesce.q.out 6f74ec9
ql/src/test/results/clientpositive/vector_date_1.q.out c2389e6
ql/src/test/results/clientpositive/vector_decimal_round.q.out b1fd628
ql/src/test/results/clientpositive/vector_interval_1.q.out b7a8e5a
ql/src/test/results/clientpositive/vector_interval_arithmetic.q.out 4908fca
ql/src/test/results/clientpositive/vectorization_limit.q.out 17cdd51
ql/src/test/results/clientpositive/view_alias.q.out 90bf28d
Diff: https://reviews.apache.org/r/56140/diff/6/
Changes: https://reviews.apache.org/r/56140/diff/5-6/
Testing
-------
Thanks,
pengcheng xiong