You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Deepak Jaiswal <dj...@hortonworks.com> on 2017/04/12 16:56:24 UTC
Review Request 58390: HIVE-16423
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58390/
-----------------------------------------------------------
Review request for hive, Jason Dere and Sergey Shelukhin.
Bugs: HIVE-16423
https://issues.apache.org/jira/browse/HIVE-16423
Repository: hive-git
Description
-------
De-duplicate semijoin branches and add hint to enforce semi join optimization
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 726522bd94
itests/src/test/resources/testconfiguration.properties fb85b9ece2
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java db6b05b7ac
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 838cc69cb6
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java 73a9b0f34d
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 79662ec5f1
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6
ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f
ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java bdc5bef8a9
ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 5f9ccc8ceb
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java abf531b259
ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java 18e4fbdc0a
ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342
ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java 71c731081f
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 2b84beb100
ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out a47ce6e583
ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/58390/diff/1/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 58390: HIVE-16423
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
> On April 18, 2017, 11:13 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
> > Line 224 (original), 224 (patched)
> > <https://reviews.apache.org/r/58390/diff/2/?file=1692285#file1692285line229>
> >
> > The semijoin hints do not work if the semijoin is generated from the result of a join.
> >
> > One thing that could be useful here is ExprNodeDescUtils.findColumnOrigin(), which should be able to trace the origin of a column to it's original table operator (assuming it's been implemented properly). This might be useful if we want the hints to work even with join results.
>
> Deepak Jaiswal wrote:
> Sounds promising. I will investigate.
Looks like this function is not sufficient in this case. We can certainly revisit this logic in future.
- Deepak
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58390/#review172266
-----------------------------------------------------------
On April 13, 2017, 9:18 p.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58390/
> -----------------------------------------------------------
>
> (Updated April 13, 2017, 9:18 p.m.)
>
>
> Review request for hive, Jason Dere and Sergey Shelukhin.
>
>
> Bugs: HIVE-16423
> https://issues.apache.org/jira/browse/HIVE-16423
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> De-duplicate semijoin branches and add hint to enforce semi join optimization
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 726522bd94
> itests/src/test/resources/testconfiguration.properties fb85b9ece2
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java db6b05b7ac
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 838cc69cb6
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java 73a9b0f34d
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 79662ec5f1
> ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6
> ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235
> ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f
> ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java e27e953d76
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 5f9ccc8ceb
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java abf531b259
> ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java 18e4fbdc0a
> ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342
> ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java 71c731081f
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 2b84beb100
> ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION
> ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58390/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 58390: HIVE-16423
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
> On April 18, 2017, 11:13 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
> > Lines 171 (patched)
> > <https://reviews.apache.org/r/58390/diff/2/?file=1692285#file1692285line176>
> >
> > It should be valid to remove the synthetic predicate here. The NOT NULL predicate is probably added to the filter due to it being an inner join.
> > Remove this comment/question.
Will do that.
> On April 18, 2017, 11:13 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
> > Line 224 (original), 224 (patched)
> > <https://reviews.apache.org/r/58390/diff/2/?file=1692285#file1692285line229>
> >
> > The semijoin hints do not work if the semijoin is generated from the result of a join.
> >
> > One thing that could be useful here is ExprNodeDescUtils.findColumnOrigin(), which should be able to trace the origin of a column to it's original table operator (assuming it's been implemented properly). This might be useful if we want the hints to work even with join results.
Sounds promising. I will investigate.
> On April 18, 2017, 11:13 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
> > Lines 225 (patched)
> > <https://reviews.apache.org/r/58390/diff/2/?file=1692285#file1692285line230>
> >
> > The current semijoin hint mechanism allows either a single column from a table, or potentially any column from a table. Would we be able to create a semijoin hint for more than one specific column in a table?
As of now, no. Incase of n-way join or join involving composite key, user can provide multiple hints.
> On April 18, 2017, 11:13 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
> > Lines 8124 (patched)
> > <https://reviews.apache.org/r/58390/diff/2/?file=1692292#file1692292line8124>
> >
> > The hint information is being added to the JoinDesc of the Join operator .. do we need to propagate this information if the join is replaced by a MapJoin in ConvertJoinMapJoin?
It should be. I will investigate.
- Deepak
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58390/#review172266
-----------------------------------------------------------
On April 13, 2017, 9:18 p.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58390/
> -----------------------------------------------------------
>
> (Updated April 13, 2017, 9:18 p.m.)
>
>
> Review request for hive, Jason Dere and Sergey Shelukhin.
>
>
> Bugs: HIVE-16423
> https://issues.apache.org/jira/browse/HIVE-16423
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> De-duplicate semijoin branches and add hint to enforce semi join optimization
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 726522bd94
> itests/src/test/resources/testconfiguration.properties fb85b9ece2
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java db6b05b7ac
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 838cc69cb6
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java 73a9b0f34d
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 79662ec5f1
> ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6
> ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235
> ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f
> ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java e27e953d76
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 5f9ccc8ceb
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java abf531b259
> ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java 18e4fbdc0a
> ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342
> ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java 71c731081f
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 2b84beb100
> ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION
> ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58390/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 58390: HIVE-16423
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58390/#review172266
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
Lines 171 (patched)
<https://reviews.apache.org/r/58390/#comment245366>
It should be valid to remove the synthetic predicate here. The NOT NULL predicate is probably added to the filter due to it being an inner join.
Remove this comment/question.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
Line 224 (original), 224 (patched)
<https://reviews.apache.org/r/58390/#comment245384>
The semijoin hints do not work if the semijoin is generated from the result of a join.
One thing that could be useful here is ExprNodeDescUtils.findColumnOrigin(), which should be able to trace the origin of a column to it's original table operator (assuming it's been implemented properly). This might be useful if we want the hints to work even with join results.
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java
Lines 225 (patched)
<https://reviews.apache.org/r/58390/#comment245381>
The current semijoin hint mechanism allows either a single column from a table, or potentially any column from a table. Would we be able to create a semijoin hint for more than one specific column in a table?
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 8124 (patched)
<https://reviews.apache.org/r/58390/#comment245382>
The hint information is being added to the JoinDesc of the Join operator .. do we need to propagate this information if the join is replaced by a MapJoin in ConvertJoinMapJoin?
- Jason Dere
On April 13, 2017, 9:18 p.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58390/
> -----------------------------------------------------------
>
> (Updated April 13, 2017, 9:18 p.m.)
>
>
> Review request for hive, Jason Dere and Sergey Shelukhin.
>
>
> Bugs: HIVE-16423
> https://issues.apache.org/jira/browse/HIVE-16423
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> De-duplicate semijoin branches and add hint to enforce semi join optimization
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 726522bd94
> itests/src/test/resources/testconfiguration.properties fb85b9ece2
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java db6b05b7ac
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 838cc69cb6
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java 73a9b0f34d
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 79662ec5f1
> ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6
> ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235
> ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f
> ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java e27e953d76
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 5f9ccc8ceb
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java abf531b259
> ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java 18e4fbdc0a
> ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342
> ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java 71c731081f
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 2b84beb100
> ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION
> ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58390/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 58390: HIVE-16423
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58390/#review172432
-----------------------------------------------------------
ql/src/test/results/clientpositive/llap/semijoin_hint.q.out
Lines 504 (patched)
<https://reviews.apache.org/r/58390/#comment245549>
The expectedEntries value does not match the value from the hint. I think you need to update the final aggregator with the hint value as well.
- Jason Dere
On April 19, 2017, 11:57 p.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58390/
> -----------------------------------------------------------
>
> (Updated April 19, 2017, 11:57 p.m.)
>
>
> Review request for hive, Jason Dere and Sergey Shelukhin.
>
>
> Bugs: HIVE-16423
> https://issues.apache.org/jira/browse/HIVE-16423
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> De-duplicate semijoin branches and add hint to enforce semi join optimization
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 420d35eb42
> itests/src/test/resources/testconfiguration.properties ed5ce9d384
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java db6b05b7ac
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 838cc69cb6
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java 73a9b0f34d
> ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 67ea05d7b1
> ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6
> ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235
> ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f
> ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 77fc35d36d
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 7caeb78e44
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java eaad988fc5
> ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java 18e4fbdc0a
> ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342
> ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java 71c731081f
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 2b84beb100
> ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION
> ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/58390/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 58390: HIVE-16423
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58390/
-----------------------------------------------------------
(Updated April 20, 2017, 12:22 a.m.)
Review request for hive, Jason Dere and Sergey Shelukhin.
Changes
-------
Update bloom filter size on final group by as well.
Bugs: HIVE-16423
https://issues.apache.org/jira/browse/HIVE-16423
Repository: hive-git
Description
-------
De-duplicate semijoin branches and add hint to enforce semi join optimization
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 420d35eb42
itests/src/test/resources/testconfiguration.properties ed5ce9d384
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java db6b05b7ac
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 838cc69cb6
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java 73a9b0f34d
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 67ea05d7b1
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6
ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f
ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 77fc35d36d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 7caeb78e44
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java eaad988fc5
ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java 18e4fbdc0a
ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342
ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java 71c731081f
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 2b84beb100
ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION
ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/58390/diff/4/
Changes: https://reviews.apache.org/r/58390/diff/3-4/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 58390: HIVE-16423
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58390/
-----------------------------------------------------------
(Updated April 19, 2017, 11:57 p.m.)
Review request for hive, Jason Dere and Sergey Shelukhin.
Changes
-------
Applied review comments and added some more tests.
Bugs: HIVE-16423
https://issues.apache.org/jira/browse/HIVE-16423
Repository: hive-git
Description
-------
De-duplicate semijoin branches and add hint to enforce semi join optimization
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 420d35eb42
itests/src/test/resources/testconfiguration.properties ed5ce9d384
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java db6b05b7ac
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 838cc69cb6
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java 73a9b0f34d
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 67ea05d7b1
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6
ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f
ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 77fc35d36d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 7caeb78e44
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java eaad988fc5
ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java 18e4fbdc0a
ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342
ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java 71c731081f
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 2b84beb100
ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION
ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/58390/diff/3/
Changes: https://reviews.apache.org/r/58390/diff/2-3/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 58390: HIVE-16423
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58390/
-----------------------------------------------------------
(Updated April 13, 2017, 9:18 p.m.)
Review request for hive, Jason Dere and Sergey Shelukhin.
Changes
-------
Segregated the patch to only focus on hints.
Bugs: HIVE-16423
https://issues.apache.org/jira/browse/HIVE-16423
Repository: hive-git
Description
-------
De-duplicate semijoin branches and add hint to enforce semi join optimization
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 726522bd94
itests/src/test/resources/testconfiguration.properties fb85b9ece2
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java db6b05b7ac
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java 838cc69cb6
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/HiveOpConverter.java 73a9b0f34d
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 79662ec5f1
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java d58f4474b6
ql/src/java/org/apache/hadoop/hive/ql/parse/HintParser.g 8e70a46235
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 3f9f76c57f
ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java ec76fb79c5
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java e27e953d76
ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinBranchInfo.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/SemiJoinHint.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 5f9ccc8ceb
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java abf531b259
ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDynamicListDesc.java 18e4fbdc0a
ql/src/java/org/apache/hadoop/hive/ql/plan/JoinDesc.java bcf3691342
ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java 71c731081f
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 2b84beb100
ql/src/test/queries/clientpositive/semijoin_hint.q PRE-CREATION
ql/src/test/results/clientpositive/llap/semijoin_hint.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/58390/diff/2/
Changes: https://reviews.apache.org/r/58390/diff/1-2/
Testing
-------
Thanks,
Deepak Jaiswal