You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jesús Camacho Rodríguez <jc...@hortonworks.com> on 2014/10/29 02:27:47 UTC
Review Request 27318: HIVE-8625: Some union queries result in plans
with many unions with CBO on
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27318/
-----------------------------------------------------------
Review request for hive.
Summary (updated)
-----------------
HIVE-8625: Some union queries result in plans with many unions with CBO on
Repository: hive-git
Description (updated)
-------
HIVE-8625: Some union queries result in plans with many unions with CBO on
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java 2bd40fa6782804804a3b02b312b9e2bfc3cd9e61
ql/src/java/org/apache/hadoop/hive/ql/exec/RowSchema.java 71cc7eb409adc86e5f729b9b8fd510f95aabb7d7
ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java 42b546bdb2bf21119e52e6a08480f13d5f2fc344
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 355bc1f814af3bf419e8a4b11a03de72c98c4bbd
ql/src/test/results/clientpositive/union29.q.out 41dde4e1ce5d0549a5501a35d45c06a506e695db
ql/src/test/results/clientpositive/union30.q.out b5065368d3f27a6036d681413f5b3d3bba0af062
ql/src/test/results/clientpositive/union34.q.out dc46ac7b6c62f1b97ed8f8e176e0797ca507e72e
ql/src/test/results/clientpositive/union_remove_10.q.out b078793cd216fd3a8aee059f7070706a42b25b5f
ql/src/test/results/clientpositive/union_remove_11.q.out 82d91a9d036b5c1a61debfe9258b84e8ae2b8f8e
Diff: https://reviews.apache.org/r/27318/diff/
Testing
-------
Thanks,
Jesús Camacho Rodríguez
Re: Review Request 27318: HIVE-8625: Some union queries result in
plans with many unions with CBO on
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27318/#review58992
-----------------------------------------------------------
Ship it!
Ship It!
- Ashutosh Chauhan
On Oct. 29, 2014, 4:06 a.m., Jesús Camacho Rodríguez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27318/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2014, 4:06 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> For some queries e.g. union9, union16, we are getting plans with many binary unions when CBO is on. The reason is that an identity select operator is introduced after each union by the CBO; thus, the method that was merging the unions when the translation of the AST into a logical plan was being done is not executed, as it does not detect there is another union operator.
>
> With the fix in the patch attached, the method will merge the two unions if (1) one is the child of the other one, or (2) if one is a descendant of the other one and there is an identity select operator between them
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java 2bd40fa6782804804a3b02b312b9e2bfc3cd9e61
> ql/src/java/org/apache/hadoop/hive/ql/exec/RowSchema.java 71cc7eb409adc86e5f729b9b8fd510f95aabb7d7
> ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java 42b546bdb2bf21119e52e6a08480f13d5f2fc344
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 355bc1f814af3bf419e8a4b11a03de72c98c4bbd
> ql/src/test/results/clientpositive/union29.q.out 41dde4e1ce5d0549a5501a35d45c06a506e695db
> ql/src/test/results/clientpositive/union30.q.out b5065368d3f27a6036d681413f5b3d3bba0af062
> ql/src/test/results/clientpositive/union34.q.out dc46ac7b6c62f1b97ed8f8e176e0797ca507e72e
> ql/src/test/results/clientpositive/union_remove_10.q.out b078793cd216fd3a8aee059f7070706a42b25b5f
> ql/src/test/results/clientpositive/union_remove_11.q.out 82d91a9d036b5c1a61debfe9258b84e8ae2b8f8e
>
> Diff: https://reviews.apache.org/r/27318/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jesús Camacho Rodríguez
>
>
Re: Review Request 27318: HIVE-8625: Some union queries result in
plans with many unions with CBO on
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27318/
-----------------------------------------------------------
(Updated Oct. 29, 2014, 4:06 a.m.)
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description (updated)
-------
For some queries e.g. union9, union16, we are getting plans with many binary unions when CBO is on. The reason is that an identity select operator is introduced after each union by the CBO; thus, the method that was merging the unions when the translation of the AST into a logical plan was being done is not executed, as it does not detect there is another union operator.
With the fix in the patch attached, the method will merge the two unions if (1) one is the child of the other one, or (2) if one is a descendant of the other one and there is an identity select operator between them
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java 2bd40fa6782804804a3b02b312b9e2bfc3cd9e61
ql/src/java/org/apache/hadoop/hive/ql/exec/RowSchema.java 71cc7eb409adc86e5f729b9b8fd510f95aabb7d7
ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java 42b546bdb2bf21119e52e6a08480f13d5f2fc344
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 355bc1f814af3bf419e8a4b11a03de72c98c4bbd
ql/src/test/results/clientpositive/union29.q.out 41dde4e1ce5d0549a5501a35d45c06a506e695db
ql/src/test/results/clientpositive/union30.q.out b5065368d3f27a6036d681413f5b3d3bba0af062
ql/src/test/results/clientpositive/union34.q.out dc46ac7b6c62f1b97ed8f8e176e0797ca507e72e
ql/src/test/results/clientpositive/union_remove_10.q.out b078793cd216fd3a8aee059f7070706a42b25b5f
ql/src/test/results/clientpositive/union_remove_11.q.out 82d91a9d036b5c1a61debfe9258b84e8ae2b8f8e
Diff: https://reviews.apache.org/r/27318/diff/
Testing (updated)
-------
Thanks,
Jesús Camacho Rodríguez
Re: Review Request 27318: HIVE-8625: Some union queries result in
plans with many unions with CBO on
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27318/
-----------------------------------------------------------
(Updated Oct. 29, 2014, 4:05 a.m.)
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description (updated)
-------
Fixing style
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java 2bd40fa6782804804a3b02b312b9e2bfc3cd9e61
ql/src/java/org/apache/hadoop/hive/ql/exec/RowSchema.java 71cc7eb409adc86e5f729b9b8fd510f95aabb7d7
ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java 42b546bdb2bf21119e52e6a08480f13d5f2fc344
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 355bc1f814af3bf419e8a4b11a03de72c98c4bbd
ql/src/test/results/clientpositive/union29.q.out 41dde4e1ce5d0549a5501a35d45c06a506e695db
ql/src/test/results/clientpositive/union30.q.out b5065368d3f27a6036d681413f5b3d3bba0af062
ql/src/test/results/clientpositive/union34.q.out dc46ac7b6c62f1b97ed8f8e176e0797ca507e72e
ql/src/test/results/clientpositive/union_remove_10.q.out b078793cd216fd3a8aee059f7070706a42b25b5f
ql/src/test/results/clientpositive/union_remove_11.q.out 82d91a9d036b5c1a61debfe9258b84e8ae2b8f8e
Diff: https://reviews.apache.org/r/27318/diff/
Testing
-------
For some queries e.g. union9, union16, we are getting plans with many binary unions when CBO is on. The reason is that an identity select operator is introduced after each union by the CBO; thus, the method that was merging the unions when the translation of the AST into a logical plan was being done is not executed, as it does not detect there is another union operator.
With the fix in the patch attached, the method will merge the two unions if (1) one is the child of the other one, or (2) if one is a descendant of the other one and there is an identity select operator between them.
Thanks,
Jesús Camacho Rodríguez
Re: Review Request 27318: HIVE-8625: Some union queries result in
plans with many unions with CBO on
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27318/
-----------------------------------------------------------
(Updated Oct. 29, 2014, 1:29 a.m.)
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description
-------
HIVE-8625: Some union queries result in plans with many unions with CBO on
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java 2bd40fa6782804804a3b02b312b9e2bfc3cd9e61
ql/src/java/org/apache/hadoop/hive/ql/exec/RowSchema.java 71cc7eb409adc86e5f729b9b8fd510f95aabb7d7
ql/src/java/org/apache/hadoop/hive/ql/exec/SelectOperator.java 42b546bdb2bf21119e52e6a08480f13d5f2fc344
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 355bc1f814af3bf419e8a4b11a03de72c98c4bbd
ql/src/test/results/clientpositive/union29.q.out 41dde4e1ce5d0549a5501a35d45c06a506e695db
ql/src/test/results/clientpositive/union30.q.out b5065368d3f27a6036d681413f5b3d3bba0af062
ql/src/test/results/clientpositive/union34.q.out dc46ac7b6c62f1b97ed8f8e176e0797ca507e72e
ql/src/test/results/clientpositive/union_remove_10.q.out b078793cd216fd3a8aee059f7070706a42b25b5f
ql/src/test/results/clientpositive/union_remove_11.q.out 82d91a9d036b5c1a61debfe9258b84e8ae2b8f8e
Diff: https://reviews.apache.org/r/27318/diff/
Testing (updated)
-------
For some queries e.g. union9, union16, we are getting plans with many binary unions when CBO is on. The reason is that an identity select operator is introduced after each union by the CBO; thus, the method that was merging the unions when the translation of the AST into a logical plan was being done is not executed, as it does not detect there is another union operator.
With the fix in the patch attached, the method will merge the two unions if (1) one is the child of the other one, or (2) if one is a descendant of the other one and there is an identity select operator between them.
Thanks,
Jesús Camacho Rodríguez