You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Deepak Jaiswal <dj...@hortonworks.com> on 2017/03/20 21:59:01 UTC
Review Request 57794: HIVE-16260 : Remove parallel edges between
semijoin and mpajoin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57794/
-----------------------------------------------------------
Review request for hive and Jason Dere.
Bugs: HIVE-16260
https://issues.apache.org/jira/browse/HIVE-16260
Repository: hive-git
Description
-------
Remove parallel edges between semijoin and mpajoin.
Added some extra config to tweak toe bloomfilter size and disable semijoin if bigtable is small.
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 62908f9
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFBloomFilter.java 2139eae
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFBloomFilterMerge.java d2446d5
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java c513ee5
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 47b229f
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 788aace
ql/src/test/queries/clientpositive/dynamic_partition_pruning.q d28da6e
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q 5482cdb
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_2.q 2306395
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_3.q 01650f8
ql/src/test/queries/clientpositive/mergejoin.q eecd105
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out d32cb5c
Diff: https://reviews.apache.org/r/57794/diff/1/
Testing
-------
Thanks,
Deepak Jaiswal
Re: Review Request 57794: HIVE-16260 : Remove parallel edges between
semijoin and mpajoin
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57794/#review169582
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 2858 (patched)
<https://reviews.apache.org/r/57794/#comment241990>
This setting is part of HIVE-16154 and should not be in this patch.
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFBloomFilter.java
Line 78 (original), 80 (patched)
<https://reviews.apache.org/r/57794/#comment241994>
I don't like that minEntries/scale factor changes have to be added in 3 places. Can this just be added to GenericUDAFBloomFilter, and modify getExpectedEntries() to incorporate the minEntries/factor logic?
- Jason Dere
On March 20, 2017, 9:59 p.m., Deepak Jaiswal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57794/
> -----------------------------------------------------------
>
> (Updated March 20, 2017, 9:59 p.m.)
>
>
> Review request for hive and Jason Dere.
>
>
> Bugs: HIVE-16260
> https://issues.apache.org/jira/browse/HIVE-16260
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Remove parallel edges between semijoin and mpajoin.
> Added some extra config to tweak toe bloomfilter size and disable semijoin if bigtable is small.
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 62908f9
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFBloomFilter.java 2139eae
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFBloomFilterMerge.java d2446d5
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java c513ee5
> ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 47b229f
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 788aace
> ql/src/test/queries/clientpositive/dynamic_partition_pruning.q d28da6e
> ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q 5482cdb
> ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_2.q 2306395
> ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_3.q 01650f8
> ql/src/test/queries/clientpositive/mergejoin.q eecd105
> ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out d32cb5c
>
>
> Diff: https://reviews.apache.org/r/57794/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Deepak Jaiswal
>
>
Re: Review Request 57794: HIVE-16260 : Remove parallel edges between
semijoin and mpajoin
Posted by Deepak Jaiswal <dj...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57794/
-----------------------------------------------------------
(Updated March 21, 2017, 11:30 p.m.)
Review request for hive and Jason Dere.
Changes
-------
Implemented changes recommended.
Bugs: HIVE-16260
https://issues.apache.org/jira/browse/HIVE-16260
Repository: hive-git
Description
-------
Remove parallel edges between semijoin and mpajoin.
Added some extra config to tweak toe bloomfilter size and disable semijoin if bigtable is small.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 62908f9
ql/src/java/org/apache/hadoop/hive/ql/optimizer/DynamicPartitionPruningOptimization.java c513ee5
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 47b229f
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFBloomFilter.java 788aace
ql/src/test/queries/clientpositive/dynamic_partition_pruning.q d28da6e
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction.q 5482cdb
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_2.q 2306395
ql/src/test/queries/clientpositive/dynamic_semijoin_reduction_3.q 01650f8
ql/src/test/queries/clientpositive/mergejoin.q eecd105
ql/src/test/queries/clientpositive/vectorized_dynamic_partition_pruning.q d2ded71
ql/src/test/queries/clientpositive/vectorized_dynamic_semijoin_reduction.q 68b57ea
ql/src/test/queries/clientpositive/vectorized_dynamic_semijoin_reduction2.q be8e4af
ql/src/test/results/clientpositive/llap/dynamic_partition_pruning.q.out 96d998f
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out d32cb5c
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_2.q.out 5f75977
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_3.q.out c272fc1
ql/src/test/results/clientpositive/llap/mergejoin.q.out ae99e66
ql/src/test/results/clientpositive/llap/vectorized_dynamic_partition_pruning.q.out 0182a46
ql/src/test/results/clientpositive/llap/vectorized_dynamic_semijoin_reduction.q.out e62d913
ql/src/test/results/clientpositive/llap/vectorized_dynamic_semijoin_reduction2.q.out d9fd706
ql/src/test/results/clientpositive/tez/explainanalyze_3.q.out 32609eb
ql/src/test/results/clientpositive/tez/explainuser_3.q.out da52b0a
Diff: https://reviews.apache.org/r/57794/diff/2/
Changes: https://reviews.apache.org/r/57794/diff/1-2/
Testing
-------
Thanks,
Deepak Jaiswal