You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Krisztian Kasa <kk...@hortonworks.com> on 2020/05/18 18:31:02 UTC
Review Request 72526: HIVE-23493
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/
-----------------------------------------------------------
Review request for hive and Jesús Camacho Rodríguez.
Bugs: HIVE-23493
https://issues.apache.org/jira/browse/HIVE-23493
Repository: hive-git
Description
-------
Rewrite plan to join back tables with many projected columns joined multiple times
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f5ad3a882b
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 32ad4c1539
ql/src/test/queries/clientpositive/cardinality_preserving_join_opt.q PRE-CREATION
ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q11.q PRE-CREATION
ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q4.q PRE-CREATION
ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q74.q PRE-CREATION
ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q11.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q4.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q74.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/72526/diff/1/
Testing
-------
mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q,cardinality_preserving_join_opt_q4.q,cardinality_preserving_join_opt_q11.q,cardinality_preserving_join_opt_q74.q -pl itests/qtest -Pitests
Thanks,
Krisztian Kasa
Re: Review Request 72526: HIVE-23493
Posted by Krisztian Kasa <kk...@hortonworks.com>.
> On May 18, 2020, 11:45 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
> > Lines 183 (patched)
> > <https://reviews.apache.org/r/72526/diff/1/?file=2232534#file2232534line183>
> >
> > It seems you use `projectExpr.getIndex`. I think we should use the position in the `List<> projectExpressions` rather than the index.
These indices are not always the same:
```
create table if not exists customer
(
c_customer_sk bigint,
c_customer_id int,
c_first_name string,
c_last_name string
);
create table store_sales
(
ss_customer_sk int,
ss_customer_id int,
ss_quantity int,
ss_list_price float
);
select ss_customer_sk, ss_customer_id, c_last_name, ss_list_price, c_customer_sk, c_customer_id, c_first_name, ss_quantity
from store_sales ss
join customer c on ss.ss_customer_sk = c.c_customer_sk and ss_customer_id = c_customer_id;
```
In this case the third element in the project list is the `c_last_name` column from the `customer` table which is the 8th element in the rowtype of the input of the Project operator.
`((RexSlot)rootProject.getProjects().get(2)).getIndex()` is `8`
> On May 18, 2020, 11:45 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
> > Lines 269 (patched)
> > <https://reviews.apache.org/r/72526/diff/1/?file=2232534#file2232534line269>
> >
> > You can just use a Pair here.
That was my first intention but after a while I was totally lost what was `left` and `right` means in this context. Then I decided to introduced this `ProjectMapping` class with more describing field names.
> On May 18, 2020, 11:45 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
> > Lines 325 (patched)
> > <https://reviews.apache.org/r/72526/diff/1/?file=2232534#file2232534line325>
> >
> > Do you need to override this method?
I only add collecting the `HiveTableScan` instances in order to copy them when the Join operators created.
- Krisztian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/#review220811
-----------------------------------------------------------
On May 18, 2020, 6:31 p.m., Krisztian Kasa wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72526/
> -----------------------------------------------------------
>
> (Updated May 18, 2020, 6:31 p.m.)
>
>
> Review request for hive and Jesús Camacho Rodríguez.
>
>
> Bugs: HIVE-23493
> https://issues.apache.org/jira/browse/HIVE-23493
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite plan to join back tables with many projected columns joined multiple times
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f5ad3a882b
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 32ad4c1539
> ql/src/test/queries/clientpositive/cardinality_preserving_join_opt.q PRE-CREATION
> ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q11.q PRE-CREATION
> ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q4.q PRE-CREATION
> ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q74.q PRE-CREATION
> ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q11.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q4.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q74.q.out PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/72526/diff/1/
>
>
> Testing
> -------
>
> mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q,cardinality_preserving_join_opt_q4.q,cardinality_preserving_join_opt_q11.q,cardinality_preserving_join_opt_q74.q -pl itests/qtest -Pitests
>
>
> Thanks,
>
> Krisztian Kasa
>
>
Re: Review Request 72526: HIVE-23493
Posted by Krisztian Kasa <kk...@hortonworks.com>.
> On May 18, 2020, 11:45 p.m., Jesús Camacho Rodríguez wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
> > Lines 325 (patched)
> > <https://reviews.apache.org/r/72526/diff/1/?file=2232534#file2232534line325>
> >
> > Do you need to override this method?
>
> Krisztian Kasa wrote:
> I only add collecting the `HiveTableScan` instances in order to copy them when the Join operators created.
Removed the override and create the new HiveTableScan from RelOptHiveTable.
- Krisztian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/#review220811
-----------------------------------------------------------
On May 26, 2020, 6:29 p.m., Krisztian Kasa wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72526/
> -----------------------------------------------------------
>
> (Updated May 26, 2020, 6:29 p.m.)
>
>
> Review request for hive and Jesús Camacho Rodríguez.
>
>
> Bugs: HIVE-23493
> https://issues.apache.org/jira/browse/HIVE-23493
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite plan to join back tables with many projected columns joined multiple times
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8094d28f21
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinRule.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 377e8280e5
> ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query11.q.out 0136ee4bb5
> ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query4.q.out 987a0f348e
> ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query74.q.out 289e5d2569
>
>
> Diff: https://reviews.apache.org/r/72526/diff/4/
>
>
> Testing
> -------
>
> mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q,cardinality_preserving_join_opt_q4.q,cardinality_preserving_join_opt_q11.q,cardinality_preserving_join_opt_q74.q -pl itests/qtest -Pitests
>
>
> Thanks,
>
> Krisztian Kasa
>
>
Re: Review Request 72526: HIVE-23493
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/#review220811
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 1711 (patched)
<https://reviews.apache.org/r/72526/#comment309516>
Let's set it to false by default until the cost-based config is in place.
If you identify some tests that are affected by the optimization, move them to a new file and set it to true specifically for that file to have some test coverage.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 54 (patched)
<https://reviews.apache.org/r/72526/#comment309518>
private static final ?
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 68 (patched)
<https://reviews.apache.org/r/72526/#comment309517>
Do you need this? Even if it is not a Project, optimization should still be applicable: You just need to create input refs for _n_ fields, where _n_ is the number of fields in the root node.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 183 (patched)
<https://reviews.apache.org/r/72526/#comment309519>
It seems you use `projectExpr.getIndex`. I think we should use the position in the `List<> projectExpressions` rather than the index.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 230 (patched)
<https://reviews.apache.org/r/72526/#comment309520>
Use _RexUtil.composeConjunction_ so the AND is flattened.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 269 (patched)
<https://reviews.apache.org/r/72526/#comment309521>
You can just use a Pair here.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 325 (patched)
<https://reviews.apache.org/r/72526/#comment309522>
Do you need to override this method?
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 2396 (patched)
<https://reviews.apache.org/r/72526/#comment309523>
TODO: Encapsulate this within a rule.
ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q11.q
Lines 1 (patched)
<https://reviews.apache.org/r/72526/#comment309524>
These tests should be moved to Perf constraints driver before check-in. If there are constraints missing in the driver e.g. unique constraints, please add them.
- Jesús Camacho Rodríguez
On May 18, 2020, 6:31 p.m., Krisztian Kasa wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72526/
> -----------------------------------------------------------
>
> (Updated May 18, 2020, 6:31 p.m.)
>
>
> Review request for hive and Jesús Camacho Rodríguez.
>
>
> Bugs: HIVE-23493
> https://issues.apache.org/jira/browse/HIVE-23493
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite plan to join back tables with many projected columns joined multiple times
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java f5ad3a882b
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 32ad4c1539
> ql/src/test/queries/clientpositive/cardinality_preserving_join_opt.q PRE-CREATION
> ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q11.q PRE-CREATION
> ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q4.q PRE-CREATION
> ql/src/test/queries/clientpositive/cardinality_preserving_join_opt_q74.q PRE-CREATION
> ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q11.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q4.q.out PRE-CREATION
> ql/src/test/results/clientpositive/llap/cardinality_preserving_join_opt_q74.q.out PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/72526/diff/1/
>
>
> Testing
> -------
>
> mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q,cardinality_preserving_join_opt_q4.q,cardinality_preserving_join_opt_q11.q,cardinality_preserving_join_opt_q74.q -pl itests/qtest -Pitests
>
>
> Thanks,
>
> Krisztian Kasa
>
>
Re: Review Request 72526: HIVE-23493
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/#review220896
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 54 (patched)
<https://reviews.apache.org/r/72526/#comment309606>
Cool comment! :)
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 99 (patched)
<https://reviews.apache.org/r/72526/#comment309617>
Could you add some more comments within this method (even if they are short) just going over the overall workflow?
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 104 (patched)
<https://reviews.apache.org/r/72526/#comment309616>
What if root has multiple inputs? This may not be supported right now, but to ensure correctness, add a check on the number of inputs and bail out otherwise.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 129 (patched)
<https://reviews.apache.org/r/72526/#comment309614>
_SourceTable_ class name is a bit misleading. Is this another mapping? _SourceTableMapping_?
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 204 (patched)
<https://reviews.apache.org/r/72526/#comment309610>
One way to extend this to more complex expressions is to use a RexVisitor to gather the RexTableInputRef(s) in the expression.
Then if they all come from the same table, you execute logic similar to what you are doing now.
However, the _mapping_ will be from position to RexNode.
When you are checking whether the keys are present, you would need to check for the TABLE_INPUT_REF RexNodes in those targets.
Finally, a RexShuttle can be used to replace RexTableInputRefs by the corresponding RexInputRef after additional join.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 226 (patched)
<https://reviews.apache.org/r/72526/#comment309611>
It seems problematic do skip the _cast_ here. Can't this potentially produce incorrect results?
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 250 (patched)
<https://reviews.apache.org/r/72526/#comment309615>
Nit. Renaming RelNodes to _leftInput_ and _rightInput_ may make things more clear.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 283 (patched)
<https://reviews.apache.org/r/72526/#comment309612>
Doesn't it make things easier if we just keep a MultiMap here? It seems you could potentially get rid of the bit sets too.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java
Lines 284 (patched)
<https://reviews.apache.org/r/72526/#comment309608>
These bitsets as well as _targetFields_ below are not actually immutable (you are calling _set_ on them). Iirc the _set_ operation actually instantiates a new bitset under the hood and returns it, hence I believe this may not be correct (the actual bitset is not changing). Even if it does, it will not very efficient.
If you are keeping mutable bitsets, the better choice is probably to use the mutable BitSet.
Otherwise, the best option is to try to make this whole class inmmutable, e.g., use a ImmutableBitSet.builder in _getExpressionLineageOf_ and then pass an immutablebitset in the constructor.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinRule.java
Lines 35 (patched)
<https://reviews.apache.org/r/72526/#comment309607>
We should be able to reuse some of the code below if this rule extends _HiveRelFieldTrimmerRule_. You can have a trimmer instance in the super class, pass it in the constructor. Then this subclass will only pass the trimmer implementation while reusing rest of logic. Does it make sense?
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
Lines 2389 (patched)
<https://reviews.apache.org/r/72526/#comment309609>
Unless I am mistaken, this will execute _HiveCardinalityPreservingJoinRule_ at least twice.
Although maybe this is obvious, is _HiveProjectMerge_ actually needed?
If it is, a better choice is to use a different partial program for it.
- Jesús Camacho Rodríguez
On May 26, 2020, 6:29 p.m., Krisztian Kasa wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72526/
> -----------------------------------------------------------
>
> (Updated May 26, 2020, 6:29 p.m.)
>
>
> Review request for hive and Jesús Camacho Rodríguez.
>
>
> Bugs: HIVE-23493
> https://issues.apache.org/jira/browse/HIVE-23493
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Rewrite plan to join back tables with many projected columns joined multiple times
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8094d28f21
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinRule.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 377e8280e5
> ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query11.q.out 0136ee4bb5
> ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query4.q.out 987a0f348e
> ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query74.q.out 289e5d2569
>
>
> Diff: https://reviews.apache.org/r/72526/diff/4/
>
>
> Testing
> -------
>
> mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q,cardinality_preserving_join_opt_q4.q,cardinality_preserving_join_opt_q11.q,cardinality_preserving_join_opt_q74.q -pl itests/qtest -Pitests
>
>
> Thanks,
>
> Krisztian Kasa
>
>
Re: Review Request 72526: HIVE-23493
Posted by Krisztian Kasa <kk...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/
-----------------------------------------------------------
(Updated June 8, 2020, 12:52 p.m.)
Review request for hive and Jesús Camacho Rodríguez.
Bugs: HIVE-23493
https://issues.apache.org/jira/browse/HIVE-23493
Repository: hive-git
Description
-------
Rewrite plan to join back tables with many projected columns joined multiple times
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1464d6aa7a
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinRule.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFieldTrimmerRule.java 73ff1bccf2
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 377e8280e5
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query11.q.out 0136ee4bb5
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query4.q.out 987a0f348e
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query74.q.out 289e5d2569
ql/src/test/results/clientpositive/perf/tez/constraints/query11.q.out 7f9df5e8af
ql/src/test/results/clientpositive/perf/tez/constraints/query4.q.out 585f4d6b9c
ql/src/test/results/clientpositive/perf/tez/constraints/query74.q.out 39c76fc82c
Diff: https://reviews.apache.org/r/72526/diff/5/
Changes: https://reviews.apache.org/r/72526/diff/4-5/
Testing (updated)
-------
mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q -pl itests/qtest -Pitests
mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestTezPerfConstraintsCliDriver -Dqfile=cbo_query4.q,cbo_query11.q,cbo_query74.q,query4.q,query11.q,query74.q -pl itests/qtest -Pitests
Thanks,
Krisztian Kasa
Re: Review Request 72526: HIVE-23493
Posted by Krisztian Kasa <kk...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/
-----------------------------------------------------------
(Updated May 26, 2020, 6:29 p.m.)
Review request for hive and Jesús Camacho Rodríguez.
Bugs: HIVE-23493
https://issues.apache.org/jira/browse/HIVE-23493
Repository: hive-git
Description
-------
Rewrite plan to join back tables with many projected columns joined multiple times
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8094d28f21
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinRule.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 377e8280e5
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query11.q.out 0136ee4bb5
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query4.q.out 987a0f348e
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query74.q.out 289e5d2569
Diff: https://reviews.apache.org/r/72526/diff/4/
Changes: https://reviews.apache.org/r/72526/diff/3-4/
Testing
-------
mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q,cardinality_preserving_join_opt_q4.q,cardinality_preserving_join_opt_q11.q,cardinality_preserving_join_opt_q74.q -pl itests/qtest -Pitests
Thanks,
Krisztian Kasa
Re: Review Request 72526: HIVE-23493
Posted by Krisztian Kasa <kk...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/
-----------------------------------------------------------
(Updated May 20, 2020, 6:57 p.m.)
Review request for hive and Jesús Camacho Rodríguez.
Bugs: HIVE-23493
https://issues.apache.org/jira/browse/HIVE-23493
Repository: hive-git
Description
-------
Rewrite plan to join back tables with many projected columns joined multiple times
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5a39006d8a
data/scripts/q_perf_test_init_constraints.sql 3b3f503ee4
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinRule.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 32ad4c1539
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query74.q.out ee232fa4e3
Diff: https://reviews.apache.org/r/72526/diff/3/
Changes: https://reviews.apache.org/r/72526/diff/2-3/
Testing
-------
mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q,cardinality_preserving_join_opt_q4.q,cardinality_preserving_join_opt_q11.q,cardinality_preserving_join_opt_q74.q -pl itests/qtest -Pitests
Thanks,
Krisztian Kasa
Re: Review Request 72526: HIVE-23493
Posted by Krisztian Kasa <kk...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72526/
-----------------------------------------------------------
(Updated May 19, 2020, 4:14 p.m.)
Review request for hive and Jesús Camacho Rodríguez.
Bugs: HIVE-23493
https://issues.apache.org/jira/browse/HIVE-23493
Repository: hive-git
Description
-------
Rewrite plan to join back tables with many projected columns joined multiple times
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5a39006d8a
data/scripts/q_perf_test_init_constraints.sql 3b3f503ee4
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinOptimization.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveCardinalityPreservingJoinRule.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java 19ce3ea223
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 32ad4c1539
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query11.q.out 127003c78b
ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query74.q.out ee232fa4e3
Diff: https://reviews.apache.org/r/72526/diff/2/
Changes: https://reviews.apache.org/r/72526/diff/1-2/
Testing
-------
mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=cardinality_preserving_join_opt.q,cardinality_preserving_join_opt_q4.q,cardinality_preserving_join_opt_q11.q,cardinality_preserving_join_opt_q74.q -pl itests/qtest -Pitests
Thanks,
Krisztian Kasa